Virtual & augmented reality for biological microscope in experiment education
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329
·Article·
Virtual & augmented reality for biological microscope in
experiment education
Xiang ZHOU1,2, Liyu TANG1,2*, Ding LIN1,2, Wei HAN1,2
1. Key Laboratory of Spatial Data Mining & Information Sharing of MOE, Fuzhou University, Fuzhou 350108, China
2. National Engineering Research Center of Geospatial Information Technology, Fuzhou University, Fuzhou 350108, China
* Corresponding author, tangly@fzu.edu.cn
Received: 23 March 2020 Accepted: 15 June 2020
Supported by the National Key Research and Development Program of China (2018YFB1004905).
Citation: Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN. Virtual & augmented reality for biological microscope in
experiment education. Virtual Reality & Intelligent Hardware, 2020, 2(4): 316—329
DOI: 10.1016/j.vrih.2020.07.004
Abstract Background Mixed-reality technologies, including virtual reality (VR) and augmented
reality (AR) , are considered to be promising potential tools for science teaching and learning processes
that could foster positive emotions, motivate autonomous learning, and improve learning outcomes.
Methods In this study, a technology-aided biological microscope learning system based on VR/AR is
presented. The structure of the microscope is described in a detailed three-dimensional (3D) model, each
component being represented with their topological interrelationships and associations among them being
established. The interactive behavior of the model was specified, and a standard operating guide was
compiled. The motion control of components was simulated based on collision detection. Combined with
immersive VR equipment and AR technology, we developed a virtual microscope subsystem and a mobile
virtual microscope guidance system. Results The system consisted of a VR subsystem and an AR
subsystem. The focus of the VR subsystem was to simulate operating the microscope and associated
interactive behaviors that allowed users to observe and operate the components of the 3D microscope
model by means of natural interactions in an immersive scenario. The AR subsystem allowed participants
to use a mobile terminal that took a picture of a microscope from a textbook and then displayed the
structure and functions of the instrument, as well as the relevant operating guidance. This flexibly allowed
students to use the system before or after class without time and space constraints. The system allowed
users to switch between the VR and AR subsystems. Conclusions The system is useful for helping
learners (especially K-12 students) to recognize a microscope's structure and grasp the required operational
skills by simulating operations using an interactive process. In the future, such technology-assisted
education would be a successful learning platform in an open learning space.
Keywords Virtual reality; Augmented reality; Microscope; Operating guide; Experiment
1 Introduction
Various biological processes and micro-objects are invisible to the naked eye and to understand this
knowledge, learners should possess the ability of abstract thinking, which makes it difficult for children to
2096-5796/©Copyright 2020 Beijing Zhongke Journal Publishing Co. Ltd., Publishing services by Elsevier B.V. on behalf of KeAi Commu‐
nication Co. Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by/4.0/).
www.vr-ih.com
Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
learn science. Microscopes are considered a necessary essential instrument for observing and manipulating
the micro-world[1]. Therefore, students are required to operate a microscope practically in K-12 biology
education. Owing to the technological progress of virtual reality (VR) and augmented reality (AR) (e. g.,
imagination, interaction and immersion), over the last decade, VR/AR has emerged as an potentially
effective tool for helping students to improve their knowledge and practical skills through education[2],
especially, macro-layer experiments (e. g., three-dimensional simulation of the motion of celestial bodies)
and micro-layer experiments (e. g., macromolecular structures). Scholars have found that AR, or the
combination of VR and AR, can improve learning efficiency and cultivate positive emotions[3-6].
Furthermore, it was found that there is a positive correlation between presence and the perceived affective
quality of learning[7]. With the popularization of related hardware devices supporting VR and AR
applications, several VR and AR technologies have been applied to education, such as magnetic field
experiments[8,9], biology lab courses[10], and geometry courses[6]. With mobile learning becoming more and
more popular[11,12], there are positive trends toward integrating VR, AR and mobile technologies in
education especially in the era of future open educational spaces[13].
The AR technologies used in science education reside in two categories, image-based AR and location-
based AR[14]. Image-based AR (including marker and markerless technology), is conducive to developing
students' spatial abilities, practical skills, and their understanding of concepts through real-time interactive
simulation experiments[14-16]. For example, in teaching chemistry, image-based AR can be used to present
the micro components of substances in real-time and be combined with specific markers to complete
virtual interactive learning, which is conducive to students' intuitive understanding of chemistry[17] By
integrating image-based AR and mobile technology into physical experimental simulations, conveniently
and effectively to assist students in experimental operations by using ubiquitous mobile phones, students
will be enabled to operate experiments in real-time without being limited by experimental
instrumentation[18]. One of the most common image-based ARs in K-12 education is AR book, which
particularly focuses on the sciences for secondary high school students[13]. Another is a location-based AR
that is conducive to scientific inquiry and is usually combined with a geographical setting for learning, real-
time data, and needs a large space[14]. The application of VR technology in education improves
contextuality and the intuitive aspect of knowledge presentation with deep interactivity and strong
immersion. It offers students a rich and varied personalized learning environment and provides
opportunities for active exploration and interactive communication[19,20]. By building a virtual laboratory
and allowing students to conduct virtual experiments in a virtual environment, students are assisted in
remembering, understanding, and improving their ability to analyze and solve problems[21,22]. The platform
NOBOOK developed by Beijing Lebu Company involves physical, chemical, and biological virtual
experimental resources for secondary school education (www.nobook.com). Despite advances in VR/AR,
the creation of meaningful content consistent with domain knowledge using this technology is still
extremely challenging.
The conventional learning approach in biology laboratory courses usually involves teachers using a
laboratory manual or textbook to guide students on the use of laboratory equipment. In the last decade,
either VR or AR has been applied to microscope experiments[10,23]. An existing biology microscope
simulation system with a keyboard and mouse was used to manipulate a virtual microscope[23]. The
advantage was that simplified basic operations were quickly executed without needing additional
interactive equipment. The disadvantage was that the lack of realism in the experimental operations
resulted in students finding it difficult to understand the operating principles of the microscope. An
interactive app module ArBioLab developed using image-based AR technology involves a microscope
environment that complies with the basic biology laboratory manual[10]. ArBioLab's AR microscope
317
Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329 experiment resources are manipulated using mobile devices, these being a good platform for flexible future education, but there is no function designed to guide students to learn standard operational processes (e.g., text or voice). However, most of them are pure VR or pure AR, and without using head-mounted displays for human-computer interaction produced a weak sense of reality and a poor interactive experience. A study of previous research on virtual microscope simulation experiments confirms that image-based AR experiments can be combined successfully with mobile devices and be applied to textbooks to achieve real-time virtual cognitive learning but lacks the guidance for students to complete standard experimental operations. Desktop VR experiments can be completed with the user interface, but it does not simulate the natural interaction behavior of humans to operate the instrument accurately. This causes students to misunderstand the microscope's operating principles. In this study, immersive VR and image-based mobile AR were used to simulate the operation of microscopes. The image-based mobile AR technology was used to simulate the process of operating the microscope. Through interaction with the UI and audio support, students were guided to learn the standard operations of the experiment with the experimental material being only a textbook. Immersive VR technology was used to construct a self-guided three-dimensional virtual learning environment and simulate real microscope operations with an interactive VR device for students to undertake the experiment without actual experimental instruments. AR experiments can be used for real-time assisted learning of microscope use, while the VR environment was designed to simulate a real experiment to help students autonomously master the correct experimental operations with supplementary learning guidance. This demonstrated empirically that using modern technology to increase students' interest in autonomous learning is also an effective auxiliary method for experimental education. 2 Biological microscope recognition and experiment simulation 2.1 Key knowledge points of microscope operation As a required biology knowledge point for secondary school students, they should understand the main structure and functions of the optical microscope, and correctly master the experimental operation skills. The steps for operating the optical microscope mainly include taking out and setting up the microscope, adjusting the light, making observations, and cleaning the microscope, amongst which adjusting the light and observation are the key steps. While adjusting the light processes, the order of steps is as follows: raising the lens tube to the highest position; aligning the low magnification objective lens with the light aperture; adjusting the mirror to maximize the amount of light emitted through the aperture. During the observation processes, the key operating steps are as follows: installing the slide; adjusting the distance between the objective lens and the slide for the objective lens to be close to the slide; focusing the eyepiece. 2.2 A conceptual model of microscope simulation To provide sufficient intuitive support for learning how to operate a microscope, VR and AR technologies were combined to develop a simulated operational system. In this work, certain components of a microscope model are described in a detailed three-dimensional model, and an entire microscope structure was generated according to these topological interrelationships. They were the chief virtual objects operating in the VR and AR scenarios. We specified the motion interrelations among components during the operation processes and the interactive operation during the simulation process. Figure 1 illustrates the approach of the microscope operation simulation. 318
Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
Figure 1 Schemes of microscope experiment simulation.
A created three-dimensional microscope model was the interactive object for the virtual experiment. The
three-dimensional model closely approximated a microscope's physical form and the topological
interrelationship between each component was constructed using 3ds Max. Autodesk 3ds Max was chosen
as the 3D modeling software to build the 3D virtual microscope, because in comparison with other
modeling platforms, it has the efficient modeling methods, effects, and rendering quality required for fine
modeling. The optical microscope had twelve main components: eyepiece, objective lens, lens tube,
nosepiece, arm, stage, stage clips, aperture, coarse focus knob, fine focus knob, mirror, and base. The three
objective lenses were mapped to have magnification powers of 4× , 10× , and 40× . Each component
embodied its functional properties.
3ds Max mainly includes the polygon, geometric, compound object, two-dimensional graphics, and
NURBS modeling methods. The characteristics of these modeling methods are shown in Table 1. Since the
microscope is a unit with multiple components that are either structurally regular or irregular, the model of
each component was completed individually by comprehensive utilization of the polygon modeling,
geometric modeling, and compound object modeling methods according to the characteristics required.
Finally, the component models were assembled into a three-dimensional microscope model.
To realize the simulated operation of a microscope, the relative motion between microscope components
should be defined. In this work, the motion interrelationships among the microscope components were
divided into the hierarchical relationship and control relationship. In the Unity3D development platform,
the motion direction and motion rate of the subclass components are consistent with those of the parent
component, but the motion of the subclass parts did not affect the motion state of the parent parts. The
319
Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329
Table 1 The modeling methods of Autodesk 3ds Max
Modeling method Characteristics
Polygon modeling Changes the geometric structure of the model by altering the arrangement of points, and is used to
add details to the model.
Geometric modeling Uses basic geometry to build a simple model, and model sizes can be adjusted by changing the size
parameters of the geometry.
Compound object modeling Combines multiple geometries into a new one, and the methods include connection, lofting, Boolean
operations, etc.
2D graphical modeling The two-dimensional shape composed of multiple splines is transformed into a three-dimensional
model by the editing commands such as extrude, rotate, etc.
NURBS modeling It is mainly used for surface modeling, and is suitable for building complex surfaces.
parent-child relationship between microscope components was set according to the standard experimental
procedures for microscopes. The main parent-child relationship was generated between the nosepiece and
the objective lens and among the lens tube, the eyepiece, and the nosepiece. A controlling relationship was
defined as arising when a moving component controlled the movement of other related components.
According to standard processes, during the experimental operation the vertical movement of the lens
barrel was controlled by rotating the coarse focusing knob and the converter controlled the motion state of
the objective lens. A text message described the operation of the warning key.
2.3 Optical adjustment simulation
When adjusting the light, the principles of light reflection should be considered. The basic principle is that
when the light is reflected, the incident light, the reflected light and the normal are on the same plane, and
the reflected light and the incident light are separated on both sides of the normal, and the reflection angle
is equal to the incident angle, as shown in Figure 2a. According to the mathematical model, the reflected
ray OB had to be calculated, where AO is the incident ray, OB is the reflected ray, the normal vector OP is
vertically halved AB, and ON is the unit normal vector. The calculation formula (1) of reflected light can
be obtained by calculating.
Figure 2 Calculation of light reflected from a surface.
OB = AO - 2∙( AO∙ ON )∙ ON (1)
In this study, the reference projection plane of the light source was defined, as shown in Figure 2b, as
being divided into 11×11 piece small blocks with the center of each block regarded as the incident point of
the light source. Thereby, the reflected light in the experiment was calculated in real-time according to the
mathematical model of light reflection. The light adjustment was completed on the assumption that the
320
Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
amount of reflected light entering the light-aperture was greater than 100.
2.4 VR based microscope operation
To improve experientially the virtual experiments in the VR experiment, HTC VIVE has the advantages of
an accurate 360-degree head-mounted display, realistic graphic display, directional audio, and high-
definition haptic feedback, it simulates natural interaction for users to interact with the virtual environment
and offers exciting virtual world experiences in a Table 2 The correlation between VR handheld controller
small real space. As technology develops, the and microscope operation
performance of such devices is improving and the Handle key name Microscope operation
cost is reducing; therefore, the HTC VIVE virtual Touchpad Rotating coarse/fine focus knob, nosepiece
reality device was used to operate the three- Holding button Pick up slide, Rotate reflector
dimensional microscope model. The association Menu button Choose different types of slides
between the VR handheld controller and Trigger button Click once, open the stage clips and click
twice, close the stage clips
microscope operation is shown in Table 2.
Collision detection is an important aspect of virtual reality systems. High-precision, real-time collision
detection technology plays a vital role in enhancing the practicability of 3D simulation. The hierarchical
bounding box algorithm[24] was used for microscope simulation. The collider was constructed for the
nosepiece, coarse focus knob, fine focus knob and other components such as the slide of the microscope
required for interaction. All colliders should match the shape of components of the microscope as much as
possible for collision detection to be accurate. This work took advantage of the built-in physics engine in
Unity3D precisely to achieve this goal. The coarse focus knob collider is shown in Figure 3. In the VR
experiment, the VR handheld controller collided with the microscope components to activate
corresponding operations, such as rotating the coarse focus screw and the mounting and demounting
procedures. In addition, the trigger was realized by the collider and was activated by the collision between
the colliders and this trigger event was used to detect whether each step of the microscope operation was
performed correctly and completed.
2.5 AR-based microscope instructions
In the AR experiment, we defined an image of a
microscope from a biological textbook or experim-
ental course as the real object, the 3D model of the
microscope and its components with properties
were virtual objects. Consequently, a database was
established for storing the learning materials, such
as the images of a microscope, the images of
observations using the microscope, the function
descriptions in text, and the audio instructions for
operating the microscope. We used the images
captured from the microscope laboratory course or
textbook for recognition. The virtual scenes
included a three-dimensional model of the
microscope and its interactive behavior. The AR- Figure 3 The screenshot of the coarse focus knob collider
based microscope interaction processes are shown highlighted in yellow.
321Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329
in Figure 4. User Interface (UI) controls were
defined to represent different magnifications of the
objective lenses (e. g., 4× , 10× , and 40× ) with
different types of slides and joysticks for
simulating the microscope's operation, and a
display window for the slide image. The
microscope operation experiment was conducted
Figure 4 Flow chart of AR-based microscope interaction.
on a mobile device by means of gesture interaction
or voice. Furthermore, the function of each component in the microscope was inquiry-based.
Image-based augmented reality technology was used to activate 3D microscope models and interaction
behaviors corresponding to the content of a biological textbook. Registration based on natural features is
popular for ease-of-use without adding abstract markers. The microscope image in the biological teaching
material was applied for identification. Information on the features of the recognition image was extracted.
Real-time binarizing was applied to the color images captured by the camera, and then feature information
for each image was extracted. Afterward, by employing the feature matching algorithm, the features in
each video stream image were matched to the features of the recognition image. After matching
successfully the transformation matrix between images was calculated in real-time. The registration
process is shown in Figure 5. The position of the camera's angle of view was obtained based on the
calculated transformation matrix, the video stream image taken by the camera was then, merged with the
three-dimensional virtual scene in real-time, with the real image being used as the background, and the
virtual scene being superimposed on it. Finally, the microscope could be operated on the mobile device
using the touch screen.
Figure 5 Flow chart of registration.
2.6 Microscope operating instructions
The microscope simulation was standardized according to the standard operation of a microscope, and the
guiding instructions that described all operational steps for the microscope experiment was designed.
Furthermore, this module could also record how learners manipulated the device while being tested. The
experimental guidance process is shown in Figure 6. To facilitate learners knowing whether their operation
processes were timeously correct, the guiding results were fed back via the UI in real-time. The "UI press"
is green when the operation is correct and is white when the operation is incorrect.
3 System implementation
The system's target groups are teachers and K-12 students. The teachers are comfortable in the knowledge,
that they can use the non-immersive VR subsystem to demonstrate the structure and how to manipulate the
instrument in a class. The students can use the system to understand the microscope's structure and acquire
the correct skills to complete virtual scientific experiments. The system provides a VR subsystem
(including a non-immersive VR system and an immersive one) and an AR subsystem. The system allows
learners to switch between the VR and AR subsystems according to requirements and conditions.
322Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
3.1 System structure
The VR/AR-based three-dimensional biological microscope simulation
and operational guidance system was oriented toward secondary school
education. The experimental system consisted of a VR-based and AR-
based microscope experiment and contained two learning units:
microscope structure and microscope operation. The system structure is
shown in Figure 7. The learning materials included images of a
physical biology microscope for recognition and 3D microscope
models with standardized properties and operational specifications.
The VR system was implemented using Unity3D the VRTK plug-in,
and the AR system was implemented using Unity3D with the Vuforia
SDK, and was compiled for mobile and tablet platforms or PCs.
Unity3D was used to write script code for the digital scenes, interactive
controls, and user interface. Since natural image recognition has been
integrated into Vuforia SDK, we used it to construct images as markers
to match to the corresponding 3D objects, detect the images, and then
link the corresponding images to the database.
The realization of the VR-based microscope operation simulation
was as follows: First, the topological motion interrelationships among
the various components of the microscope were set. Second, the Figure 6 Illustration of operating
operating guidance was compiled according to the microscope's guide process.
Figure 7 System structure.
323Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329
standard operating procedures. The system supported automatic evaluation of whether the user's operation
met the specifications. Specific experimental content included the simulation of microscope operation,
observation, 3D user interfaces, and image observation in the display window. In the VR experiment, the
immersive virtual reality device was used to interact with the microscope model. In the AR experiment, the
mobile devices were used to operate the microscope and display a 3D object obtained from the VR model.
The process sequence was as follows: First, using a mobile camera, learners took a photo of an image in a
biological textbook. Then the photo was recognized. Finally, this triggered the display of AR content,
including images and 3D models as overlays onto the screen. The system allows learners to switch to the
VR subsystem for operating the 3D models in immersive scenes.
3.2 Outcome
The VR/AR-based three-dimensional simulation and operation guidance system is convenient for selecting
an experimental subsystem using the UI shown in Figure 8a. Figure 8b and 8c show the UI and the VR-
based microscope experimental subsystem's operating scheme. The subsystem's functions mainly include
microscope recognition, operational guidance, image display, the interactive microscope, and the selection
of slides. The models can be manipulated by rotating, zooming, and through investigation promote
recognition of the structure and functions of the microscope, such as the objective lenses, coarse focus
knob, and fine focus knob. The system provides operating guidance to alleviate learner's cognitive burden.
The main operations included mounting slides and operating the microscope to observe the slide image in
Figure 8 Screenshots of virtual experiment system.
324Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
different objective lenses.
Figure 8d and 8e show a few screenshots of the AR-based microscope operation experiment subsystem.
Learners can operate the model and use the UI to control it, while simultaneously observing the results of
the operation using the mobile device. The learners could select the microscope's various components to
master their nomenclature and functions. For learning the interactive operation, when observing the onion
epidermal cell illustrated in Figure 8e, the power of the objective lenses can be dynamically switched (4×,
10× , 40× ) by linking to the existing cell observation images. The mobile AR subsystem is flexible for
students to use pre-class or after-school without being constrained by time and space.
4 Discussion and conclusions
The main objective of this study was to provide students with a new auxiliary learning method for a
microscope experiment using VR/AR technology. The virtual experiments were tested on students from the
perspective of learning effects, learning attitudes and usability, and were then discussed based on the
experimental results. Finally, future research for improvements was formulated based on the information
obtained from the experiment.
4.1 Experimental results and discussions
The experimental subjects were freshmen and senior junior-high school students, who operated the virtual
experiments. During the experiment, changes in the students' learning attitudes were recorded. After the
experiment, the students participating in the experiment were required to write out the microscope
experiment's operational steps. Figure 9 shows student participation in the experiment.
Figure 9 Student participation in the experiment.
(1) Discussion of learning effects
The freshmen were divided into two groups during the first learning experiment. One group studied the
content of the textbook and took the learning test, and then conducted the actual experiment and compared
them. The other group initially used the mobile AR experiments to assist in learning from a textbook and
recorded the learning effects then undertook the VR experiment and these learning results were tested and
lastly, they conducted the real-life operational experiments to check the learning effect. The experimental
environment settings for the senior and new students were similar.
According to the experimental results, in comparison with the traditional learning method, the AR-
assisted learning method had a positive impact on the new students' theoretical knowledge of new students'
the first experimental learning. After learning under the VR experimental conditions, basically, the students
had mastered the operational process being taught. Compared with students who learned without VR
325Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329 support, the new students' final practical operation experiment results were all satisfactory. For senior students, the auxiliary learning effects of AR were not significant, but after practicing under the VR experimental conditions, they could correctly operate the equipment and the learning effects were significantly better than those students without the VR learning experience. The experimental results showed that VR/AR had a significant effect on assisting new students' learning, and had a positive influence on helping the students to master experimental skills. After the students used the AR experiment to understand the experimental theoretical content of the textbook, the VR experiment effectively helped the students to learn independently the experimental method of operation and the learning effects were close to those of the actual experimental operation. For the senior students, the effects of AR-assisted learning were not significant, because they already had a grasp of the theoretical knowledge being tested. (2) Discussion of learning attitudes The students who used VR/AR-assisted learning had powers of concentration similar to the students without VR/AR-assisted learning, but autonomous learning requires more initiative and interest, and also promotes mutual learning and communication among students. Students who used AR-assisted learning thought that it helped them learn the theoretical aspects of microscope use and VR-assisted learning had a significant positive impact on the practical execution of the experiment. 4.2 Usability analysis of virtual experiment Based on the reviews of students participating in the experiment, there are suggestions for improving the utility of an auxiliary experimental learning system based on VR/AR. The following were the opinions compiled: (1) In the AR experiment, a video about the experiment could be added, which would assist students to preview and review the standard experimental process. (2) The key operations in the experiment could be emphasized in the form of text using AR, combined with the operation in the AR experiment thus enhancing the learning memory of students. (3) In VR experiments, the sense of experience would be closer to reality by adding some simulation content about biology laboratories in school. When an error occurred in the operation, a voice prompt could be combined with the operating guide for the error to be corrected timeously. 4.3 Conclusion Virtual reality and augmented reality were combined to develop an operation simulation system for understanding a microscope's structures and functions and mastering standard operating skills in secondary school biology education. The system provided a VR subsystem (including non-immersive and immersive VR systems) and an AR subsystem. This allowed learners to switch between the VR and AR subsystems according to their requirements and the device conditions. A 3D model of a microscope was built accurately and the relative motion among components was specified during the operational process. The microscope model components' hierarchical and control interrelationships were reasonably specified using Unity3D for simulating interactive behavior. The operating guide was defined and compiled. The VR subsystem focused on simulating the interactive behavior of a microscope for learners to practice microscope manipulation in an immersive scenario. The experimental results demonstrate that the learning effect of experimental operation based on immersive VR is similar to the actual operation of a microscope. 326
Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
Wearable devices allow students in junior high school to foster positive emotions and motivate self-
learning. The AR subsystem focused on assisting learners to understand the structure of a microscope and
become familiar with the experimental procedures before entering the laboratory. The objective is
simplicity and portability. From the analysis of the experimental results the mobile AR subsystem had a
positive effect on students' comprehension of theoretical subject matter, and without being constrained by
time and space allowed for pre-class or after-school use. VR and AR are potential aided-learning tools for
future educational experiences and learning environments across other K-12 educational contexts and in
open-learning spaces. In the future, such simulations would be optimized to enhance their realism.
4.4 Improvements and future work
This research has great potential and utility to help students develop knowledge of and comprehend basic
experimental theory and master basic experimental operations. Through empirical research, microscope
experimental learning tools based on VR/AR should be improved. We also proposed several aspects where
the research can be continued.
In terms of system design:
(1) To enrich the function of experiments based on VR and AR, it is necessary to emphasize the core
content for students accurately to learn key and difficult content.
(2) The user interface of learning tools will be improved to make it easier for students to use.
In terms of experimental design:
(1) Two single variable control groups were added. One group studied under the AR experimental
conditions, the other group studied under the VR experimental conditions, and analysis of the variances
between these groups has already encouraged research into other differences and advantages of AR and VR
in auxiliary experimental learning contexts.
(2) Measurement of the long-term learning effect of experimental learning based on VR/AR. Once the
experimental groups complete the process, the students' theoretical knowledge and mastery of the actual
experimental operation should be examined to test the long-term effectiveness of microscope experimental
learning based on VR/AR.
Acknowledgements
We thank Mr. Yifei Jin from Beijing Lebu Education Technology Company for providing images of plant
cells corresponding to the power of the various objective lenses as observed through a microscope.
References
1 Sigitov A, Hinkenjann A, Roth T. Towards VR-based systems for school experiments. Procedia Computer Science,
2013, 25: 201–210
DOI:10.1016/j.procs.2013.11.025
2 Ardiny H, Khanmirza E. The role of AR and VR technologies in education developments: opportunities and challenges.
In: 2018 6th RSI International Conference on Robotics and Mechatronics (IcRoM). Tehran, Iran, IEEE, 2018, 482–487
DOI:10.1109/icrom.2018.8657615
3 Stull A T, Gainer M J, Hegarty M. Learning by enacting: the role of embodiment in chemistry education. Learning and
Instruction, 2018, 55: 80–92
DOI:10.1016/j.learninstruc.2017.09.008
4 Harley J M, Lajoie S P, Tressel T, Jarrell A. Fostering positive emotions and history knowledge with location-based
augmented reality and tour-guide prompts. Learning and Instruction, 2018, 101163
327Virtual Reality & Intelligent Hardware 2020 Vol 2 Issue 4：316—329
DOI:10.1016/j.learninstruc.2018.09.001
5 Yip J, Wong S H, Yick K L, Chan K, Wong K H. Improving quality of teaching and learning in classes by using
augmented reality video. Computers & Education, 2019, 128: 88–101
DOI:10.1016/j.compedu.2018.09.014
6 Chang S C, Hwang G J. Impacts of an augmented reality-based flipped learning guiding approach on students' scientific
project performance and perceptions. Computers & Education, 2018, 125: 226–239
DOI:10.1016/j.compedu.2018.06.007
7 Kartiko I, Kavakli M, Cheng K. Learning science in a virtual reality application: the impacts of animated-virtual actors'
visual complexity. Computers & Education, 2010, 55(2): 881–891
DOI:10.1016/j.compedu.2010.03.019
8 Mannuß F, Rubel J, Wagner C, Bingel F, Hinkenjann A. Augmenting magnetic field lines for school experiments. In:
2011 10th IEEE International Symposium on Mixed and Augmented Reality. Basel, Switzerland, IEEE, 2011, 263–264
DOI:10.1109/ismar.2011.6143893
9 Liu Q T, Xu S X, Yu S F, Yang Y Y, Wu L J, Ba S. Design and implementation of an AR-based inquiry courseware:
magnetic field. In: 2019 International Symposium on Educational Technology (ISET). Hradec Kralove, Czech Republic,
IEEE, 2019, 134–138
DOI:10.1109/iset.2019.00036
10 Chang R C, Yu Z S. Using augmented reality technologies to enhance students' engagement and achievement in science
laboratories. International Journal of Distance Education Technologies, 2018, 16(4): 54–72
DOI:10.4018/ijdet.2018100104
11 Burden K, Kearney M. Future scenarios for mobile science learning. Research in Science Education, 2016, 46(2): 287–
308
DOI:10.1007/s11165-016-9514-1
12 Yang S X, Mei B, Yue X Y. Mobile augmented reality assisted chemical education: insights from elements 4D. Journal
of Chemical Education, 2018, 95(6): 1060–1062
DOI:10.1021/acs.jchemed.8b00017
13 Leahy S M, Holland C, Ward F. The digital frontier: envisioning future technologies impact on the classroom. Futures,
2019, 113: 102422
DOI:10.1016/j.futures.2019.04.009
14 Cheng K H, Tsai C C. Affordances of augmented reality in science learning: suggestions for future research. Journal of
Science Education and Technology, 2013, 22(4): 449–462
DOI:10.1007/s10956-012-9405-9
15 Brett E S, Reed R S. Using coordination classes to interpret conceptual change in astronomical thinking. In the
International Conference on Learning Sciences, 2004, 4: 634
DOI:10.1145/2858036.2858589
16 Andujar J M, Mejias A, Marquez M A. Augmented reality for the improvement of remote laboratories: an augmented
remote laboratory. IEEE Transactions on Education, 2011, 54(3): 492–500
DOI:10.1109/te.2010.2085047
17 Cai S, Wang X, Chiang F K. A case study of Augmented Reality simulation system application in a chemistry course.
Computers in Human Behavior, 2014, 37: 31–40
DOI:10.1016/j.chb.2014.04.018
18 Abu Bakar J A, Gopalan V, Zulkifli A N, Alwi A. Design and development of mobile augmented reality for physics
experiment. In: Communications in Computer and Information Science. Singapore, Springer Singapore, 2018, 47–58
DOI:10.1007/978-981-13-1628-9_5
19 McGrath D, Wegener M, McIntyre T J, Savage C, Williamson M. Student experiences of virtual reality: a case study in
learning special relativity. American Journal of Physics, 2010, 78(8): 862–868
DOI:10.1119/1.3431565
20 Ding N, Wang Y. Application of virtual reality in education: advantages and challenges. Modern Educational
328Xiang ZHOU et al: Virtual & augmented reality for biological microscope in experiment education
Technology, 2017, 27(2): 20–26
DOI:10.3969/j.issn.1009-8097.2017.02.003
21 Wolski R, Jagodziński P. Virtual laboratory: using a hand movement recognition system to improve the quality of
chemical education. British Journal of Educational Technology, 2019, 50(1): 218–231
DOI:10.1111/bjet.12563
22 Abreu P A, Carvalho K d L, RabeloV W H, Castro H C. Computational strategy for visualizing structures and teaching
biochemistry. Biochemistry and Molecular Biology Education, 2019, 47(1):76–84
DOI:10.1002/bmb.21199
23 Rui M A, Dong L Y, Chen X D, Wang J Z. Microscopic control system for biological experiment teaching based on 3D
simulation technology. Computer Technology and Development, 2011, 21(6): 208–211
DOI: 10.3969/j.issn.1673-629X.2011.06.055
24 Chang J W, Wang W P, Kim M S. Efficient collision detection using a dual OBB-sphere bounding volume hierarchy.
Computer-Aided Design, 2010, 42(1): 50–57
DOI:10.1016/j.cad.2009.04.010
329You can also read
