|
Alla Solyar, M.D. a,*, Hernando Cuellar, M.D.b, Babak Sadoughi, M.D.b ,
Todd R. Olson, Ph.D.c , Marvin P. Fried, M.D., F.A.C.S.a
aDepartment of Otolaryngology, Monte?ore Medical Center, Albert Einstein College of Medicine, Bronx, NY, USA;
bSurgical Simulation Center, Department of Otolaryngology, Monte?ore Medical Center, Bronx, NY, USA; CDepartment of Anatomy and Structural biology, Albert Einstein College of Medicine, Bronx, NY, USA
KEYWORDS:
Virtual reality;Anatomy education;Surgical simulation
Abstract
BACKGROUND: Virtual reality simulators provide an effective learning environment and are widely used. This study evaluated the Endoscopic Sinus Surgery Simulator (ES3; Lockheed Martin) as a tool for anatomic education.
METHODS: Two medical student groups (experimental, n=8; control, n=7) studied paranasal sinus anatomy using either the simulator or textbooks. Their knowledge was then tested on the identi?cation of anatomic structures on a view of the nasal cavities.
RESULTS: The mean scores were 9.4±0.5 and 5.1±3.0 out of 10 for the simulator and textbook groups, respectively (P=.009). Moreover, the simulator group completed the test in a signi?cantly shorter time, 5.9±1.1 versus 8.3±2.0 minutes (P=.021). A survey asking the students to rate their respective study modality did not materialize signi?cant differences.
CONCLUSION: The ES3 can be an effective tool in teaching sinonasal anatomy. This study may help shape the future of anatomic education and the development of modern educational tools. 2008 Elsevier Inc. All rights reserved.
Virtual reality is an important tool for medical and sur-1 gical training as well as education.The ndoscopic Sinus Surgery Simulator (ES3; Lockheed Martin, Bethesda, MD,USA), a virtual reality device developed to train individuals 2,3 on a virtual patient, has shown promising results in im-4 proving resident skills in sinus surgery. It also may be a valuable educational resource in the study of anatomy. This study evaluates the ES3 as an anatomy education tool for medical students.
Over the last few decades, the anatomy curriculum in the US medical schools has changed with less time and fewer trained anatomy faculty. Today, because medical schools are cutting back on anatomic education, many training programs believe that incoming residents are not adequately 5 prepared in gross anatomy. According to Cottam, who surveyed over 1,000 residency programs in family practice, diagnostic radiology, emergency medicine, and general surgery, 57% of residency programs indicated that incoming residents needed a refresher, whereas 14% reported that they were seriously lacking in the gross anatomic knowledge. Furthermore, 52% of general surgery residency program ssurveyed by Cottam felt that residents in 1999 were less prepared than they were in 1989.Thus,the continued devaluing of anatomic education in the curriculum of medical schools since Cottam's study makes it more importat than ever to implement enhanced methods of anatomy instruction. Virtual reality simulators are tools that can be used to increase the effciency and effectiveness of anatomic instruction at all levels of education. As novel training and teaching vehicles, they have gained much attention as effective modalities for surgical resident education. However,they can also be used to augment and enrich the teaching of anatomy because the hours devoted to the subject continue to be reduced, the number of trained anatomists decreases, and cadaver costs and scarcity arise.
Virtual reality application sprovide a 3-dimensional education alexperience in which dynamic views of structures and theirs patial relationships can be explored.There is a developing belief among educators that virtual reality tools are both engaging and effcacious in medical student education. 6Hoff-man and Vu 6 comment that new virtual reality system sallow training" without the risks and ethical concern stypically associated with using animal and human subjects.°They add that"those simulat edencounters,in combination with existing opportunities to work with realpatients,could increase the depth and breadth of learners' exposure to medical problems,ensure uniformity of training experiences,and enhance the acquisition of clinical skills".6 To further promote the use of simulators in anatomical education,Hariri et al7 evaluated a surgical simulator as a tool for learning clinical anatomy by testing and comparing shoulder anatomy knowledgein 2 groups of students who used either the simulator or textbooks to study anatomy.Their results indicate that the 2 groups were comparable in the knowledge of shoulder joint anatomy.Furthermore, students rated the simulator a more effective learning
tool over the textbook and indicated that they were more likely to use the simulator as al earning tool if it were available to them. 7 Although students in the simulator group performed comparably to students in the textbook group, both groups scored poo rmarks on the shoulder joint anatomy testing.
The Endoscopic Sinus Surgery Simulator (ES3), a virtual reality device developed by Lockheed Martin, was evaluated as a teaching and training tool. This computer-based device uses acquired computed tomography scan data to simulate sinus imagery and offers haptic force feedback in a 3-dimensional field.3,8 Endoscopic surgical tools are incorporated, and manipulation and dissection can be performed in a virtual setting. The simulator has 3 levels of difficulty: novice, intermediate, and advanced. The novice setting allows basic skills to be developed, whereas the intermediate setting adds paranasal sinus anatomy. The advanced level setting also uses virtual anatomy but unlike the intermediate level does not label anatomic landmarks. The present study evaluated the efficacy of the ES3 in teaching sinus anatomy to junior medical students.
Materials and Methods
After approval of the study by the Albert Einstein College of Medicine Institutional Review Board, 17 first-year medical students were recruited through electronic mail before the start of the 2004 anatomy class. Students in the class of 2008 were randomly assigned to either the experimental (simulator) group or the control (textbook) group. Two of the 9 students in the control group did not complete the study. Enrolled students from both groups were invited to a short introductory lecture where they viewed a video describing the ES3 and were given reading materials on sinus anatomy, consisting of chapters “Sinus Anatomy and Function”9 and “Endoscopic Sinus Surgery”10 from Bailey’s Head and Neck Surgery-Otolaryngology as well as images from the 2nd edition of Netter’s Atlas of Human Anatomy.11 Both groups then took a pretest consisting of identifications of anatomic structures from textbook images to establish a knowledge baseline. The purpose of this pretest was to establish that both groups were at the same level of knowledge about the anatomy of the paranasal sinuses. Both groups viewed an introductory video of the nasal cavity taken during an endoscopic procedure. Students in the control group were asked to study paranasal sinus anatomy from reading materials given previously as well as images from the 4th edition of Rohen’s Color Atlas of Anatomy,12 Stammberger’s Functional Endoscopic Sinus Surgery,13 Weir’s Imaging Atlas of Human Anatomy,14 and Atlas of Endoscopic Sinonasal Surgery. 15 No limit to study time was imposed on the control group, and they were allowed to study on their own just up to the time the simulator group completed its training. Students in the simulator group were asked not to use any other study
materials and were given supplementary reading material covering step-by-step description of the ES3 modes and description of tools used in the ES3.
The simulator group then underwent training on the ES3 with each student having three 1-hour training sessions
using the intermediate setting on the simulator (between 6-10 trials total). In this setting, users were asked to navigate through a series of hoops from the nasal vestibule to the nasopharynx and middle meatus, inject a vasoconstrictive agent into the middle turbinate in the first 3 trials, and perform a limited dissection of the proctor-defined structures such as the uncinate process, the ethmoid bulla, and the maxillary ostium. Anatomical structures were clearly labeled. After completion of the intermediate mode, students proceeded to the advanced setting where they performed 3 trials. The advanced setting is similar to the intermediate but lacks labeled anatomic landmarks (Fig. 1).
After the study group completed the ES3 training, both groups underwent a second nasal cavity endoscopic view videotape session. In these individual sessions, students were asked to identify the following structures: inferior turbinate, middle turbinate (before and after dissection), nasal septum, maxillary ostium, maxillary sinus, uncinate process, ethmoid bulla, nasopharynx, and Eustachian tube. Each session was scored on the number of structures correctly identified and time to completion. After completion of the test, students were asked to fill out 2 surveys. One survey questioned the extent of their studying. The second assessed students’ satisfaction level with their study modality on a Likert scale. In the latter survey, the students were also asked to grade the effectiveness of the study modality as a tool for learning anatomy, likelihood of future use if modality was available, ease of use, ease of learning, and image realism. Statistical analysis was performed by using a 2-tailed Student t test (SPSS 11.0 software; SPSS Inc, Chicago, IL).
Results
The simulator and the control group performed comparably on the pretest, scoring a mean of 6.9 ± 1.2 and 6.7 ± 1.5 correct answers out of 8, respectively (P= .749, Fig. 2), thus showing that the groups had similar baselines at the beginning of the experiment. During the videotape testing sessions, the simulator group performed significantly better than the control group, scoring a mean of 9.4 ±0.5 correct identifications out of 10, whereas the control group answered 5.1 ± 3.0 questions out of 10 correctly. This resulted in a statistically significant
difference between the 2 groups (P = .009, Fig. 3). These results also show that students in the simulator group performed more uniformly, whereas students in the control group had more scattered results. In addition to better performance during the videotape sessions, the students in the simulator group completed the test in a significantly shorter period of time, taking a mean of 5.9 ±1.1 minutes, whereas the control group needed a mean of 8.3 2.0 minutes to complete the test (P= .021, Fig. 4).
On average, the control group spent 78 minutes studying from the given materials before testing and used 68% of the given materials. The results of the Likert scale are shown in Table 1, encompassing mean results for students’ rating of the effectiveness of the study modality assigned to their group. The Likert scale ranges from 1 to 5, with 1 being the most negative response and 5 the most positive response. Students in the subject and control groups rated the effectiveness of their study modality to teach anatomy as 4.3±0.7 and 3.6±1.0 (P=
.143), respectively. Students in the simulator group rated the likelihood of use of modality, if available, as 4.3 ± 0.9, whereas students in the textbook group rated it at 3.9± 0.7 (P=.361). The simulator group ranked ease of use at 2.9±1.5, whereas the textbook-trained group reported ease of use for their study modality as 4.0 ±0.6 (P= .079). Ease of learning was rated at 3.9± 1.4 by the simulator group, whereas the textbook group rated it at 3.7±0.8 (P= .786). Finally, the simulator was rated as 3.9±1.0 for image realism, whereas the textbook method was rated at 3.4±1.0 (P= .397).
Comments
The results of this study show that the ES3 is a remarkably effective tool in helping medical students learn nasal
and paranasal sinus anatomy. Indeed, although both groups started out with similar levels of knowledge as shown by the comparable results of the pretest, students in the simulator group performed significantly better at the end of the study when they were asked to identify anatomic structures on the videotape. They also had less variability in the number of correct identifications within the group and were able to complete the test in a significantly shorter period of time.
The results of the Likert survey, asking students to rate the study modality, were unexpected in that students in the simulator-trained group and the textbook-trained group gave similar ratings to the simulator and textbook methods. The differences between the 2 groups’ responses were not significant. Students in the simulator group found their study modality more difficult to use, giving it a mean score of 2.9 ±1.5, whereas students in the textbook-trained group rated the ease of use of their modality at a mean score of 4.0± 0.6. Even though the P value (.079) for ease of use did not reach statistical significance, it came closer to the cutoff value than in any other category. The reason for a lower than expected rating of the simulator as a study modality may lie in the fact that students in the subject group were asked to learn the anatomy while performing endoscopic sinus surgery on the simulator, a task that entails a frustrating learning curve. The students assigned to the simulator group were actually learning much more than anatomy. Similarly, the advanced level of the ES3 is a very challenging task for novices, adding to further frustration and dislike of the modality, although the acceptance of the simulator does not seem compromised. One of our prior studies had clearly showed that, in general, medical students viewed a device such as the ES3 very positively as an addition to their educational tools.16
One of the limitations of this study was the lack of control over the amount of studying done by the textbook
group. It was decided not to control textbook group study time because of methodologic and logistic issues involved in monitoring students during 4 hours of study (approximate amount of time spent by experimental group with the simulator). The textbook group reported having studied for 78 minutes in average, ranging from 20 minutes to 2.5 hours, and used on average 68% of the given materials. The amount of time used to study did not correlate with test performance. The amount of learning time also posed a dilemma in the Hariri et al7 study in which students were controlled for time, with both simulator and textbook groups being given 10 minutes to study anatomy from their respective modalities. This may have been responsible for low scores in both groups. As Hariri et al7 pointed out, it might have been helpful to increase the duration of the study sessions or to implement multiple teaching sessions in their study. However, the concerns on the reliability of the outcome measurements are quite typical in this type of study design and often impossible to address. As pointed out by Nicholson et al17 who recently published one of the few other randomized controlled trials of a virtual reality tool for anatomy education, “the novelty of the 3-D model may have encouraged the intervention group to spend more time and to concentrate more on the material,” but “one can argue that, either way, the outcome is the same: better understanding of 3-D anatomical relationships.”
We also believe that our study could have benefited from time adjustments. Instead of four 1-hour sessions, students in the experimental group could be given one 40-minute session, and, instead of performing actual dissection, they could spend the time exploring the endoscopic view of the nasal cavities while familiarizing themselves with the paranasal anatomy with no or only limited dissection. Nnodim investigated learning of lower limb anatomy using a control group that studied via dissection and an experimental group that studied previously dissected specimens. On the paper and practical examinations that followed, the experimental group performed significantly better, thus suggesting that one does not need to dissect to perform well on this type of tag exam.18 One may hypothesize from Nnodim’s study that the inclusion of simulator-based anatomy learning exercises
might be a viable alternative to dissection as a basic anatomic learning tool because student performance on tag
examinations did not seem to benefit from actual participation in dissection. Subsequent changes to the experimental design that limits pretest study time from textbooks and eliminates cadaver-based study from the preparation of the experimental group may produce data that could be useful to test these hypotheses. Another advantage of such an adjustment in study design is a better “ease of use” score on the Likert scale questionnaire of the simulator-trained group. The simulator environment can more closely reflect not only cadaveric dissection but also applied clinical procedures.
Few studies of this nature have been done in the past. The most recent and most similar is that of Hariri et al,7
which showed that a surgical simulator is at least as effective as textbook images for learning anatomy and suggested that student learning was enhanced through increased motivation as shown in their Likert scale results. Our study, which was very similar in structure, showed that the endoscopic sinus surgery simulator was a more effective tool for learning paranasal sinus anatomy than the textbook images. The main difference between the 2 studies, and possibly a reason for the difference in the results, lay with the amount of time allocated for anatomy study. Alternatively, the fact that students in our experimental group performed actual dissection, even though they found it hard, may have helped them score better on the test. This, however, directly contradicts
the findings in Nnodim’s study, which showed that one does not need to dissect anatomic structures to score
well on laboratory tag examinations.18 In addition, the similarity of the images displayed on the simulator to the actual anatomic views used for the final test may be perceived as an advantage for the experimental group on the final test. However, during the study portion of the protocol, the control group had access to atlas images of live endocopic surgery that replicated the final test even more realistically.
Future studies should examine how the amount of time spent studying anatomy on the simulator influences test
scores and attempt to better define the role dissection plays in the simulator group performance on the final test. Our observations, nonetheless, strongly support the benefit of simulation technologies for anatomy education, and their potential contribution should further motivate medical educators to rapidly incorporate virtual reality simulation into their curricula.
Conclusion
Our findings suggest that the ES3 is an effective tool for teaching paranasal sinus anatomy, as shown with the students in the simulator group performing significantly better on the final test of identification of anatomic structures and time to completion when compared with the students in the textbook group. Moreover, clinical application of the anatomy is also taught. Students in both groups gave similar marks to both modalities, with no significant differences between the groups’ responses with respect to ability to teach anatomy, likelihood of future use, and overall ease of use.
Changes in medical curricula continue to diminish the importance placed on anatomic education as measured by
the amount of time being allocated for its teaching. This together with the national shortage of trained anatomists19 is leading many schools to investigate alternatives to the traditional methods of teaching anatomy. We have no doubt that virtual reality simulators will find an important place in teaching anatomy to both surgical residents and medical students. This will be increasingly possible as simulators develop better graphics, evolve to more fully reflect normal human variation and pathologies, and become more costeffective. The value and practical applications of these devices in both surgical dissection instruction and basic anatomic training will almost certainly increase over the coming years. Further investigation of simulator use in conjunction with other learning modalities will help shape the future of both anatomic education and the development of virtual reality simulators.20
References
1. Caversaccio M, Eichenberger A, Hausler R. Virtual simulator as a training tool for endonasal surgery. Am J Rhinol 2003;17:283–90.
2. Uribe JI, Ralph WM Jr, Glaser AY, Fried MP. Learning curves, acquisition, and retention of skills trained with the endoscopic sinus surgery simulator. Am J Rhinol 2004;18:87–92.
3. Edmond CV Jr, Heskamp D, Sluis D, et al. ENT endoscopic surgical training simulator. Stud Health Technol Inform 1997;39:518 –28.
4. Edmond CV Jr. Impact of the endoscopic sinus surgical simulator on operating room performance. Laryngoscope 2002;112:1148 –58.
5. Cottam WW. Adequacy of medical school gross anatomy education as perceived by certain postgraduate residency programs and anatomy course directors. Clin Anat 1999;12:55– 65.
6. Hoffman H, Vu D. Virtual reality: teaching tool of the twenty-first century? Acad Med 1997;72:1076–81.
7. Hariri S, Rawn C, Srivastava S, et al. Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 2004;38:896 –902.
8. Ackerman MJ, Spitzer VM, Scherzinger AL, Whitlock DG. The visible human data set: an image resource for anatomical visualization.Medinfo 1995;8:1195– 8.
9. Miller AJ, Amedee R. Sinus anatomy and function. In: Bailey BJ (ed):Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998.
10. Lanza DC, Kennedy DW. Endoscopic sinus surgery. In: Bailey BJ (ed): Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998.
11. Netter FH. Atlas of Human Anatomy (ed 2). Icon Learning Systems, A Division of Havas Medi Media, Teterboro, NJ, 1997.
12. Rohen JW, Yokochi C, Lütjen-Drecoll. Color Atlas of Anatomy (ed 4). Philadelphia, PA: F.K. Schattauer Verlagsgesellschaft GmbH and LippincottWilliams & Wilkins, 1998.
13. Stammberger H. Functional Endoscopic Sinus Surgery: The Messerklinger Technique. Philadelphia, PA: Decker, 1991.
14. Weir J, Abrahams PH. Imaging Atlas of Human Anatomy (ed 2). Mosby, 1998.
15. Mehta D. Atlas of Endoscopic Sinonasal Surgery. Philadelphia, PA:Lea & Febiger, 1993.
16. Glaser AY, Hall CB, Uribe SJ, Fried MP. Medical students’ attitudes toward the use of an endoscopic sinus surgery simulator as a training tool. Am J Rhinol 2006;20:177–9.
17. Nicholson DT, Chalk C, Funnell WR, Daniel SJ. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ 2006;40:1081–7.
18. Nnodim JO. Learning human anatomy: by dissection or from prosections? Med Educ 1990;24:389 –95.
19. McCuskey RS, Carmichael SW, Kirch DG. The importance of anatomy in health professions education and the shortage of qualified educators. Acad Med 2005;80:349 –51.
20. Marks SC Jr. Recovering the significance of 3-dimensional data in medical education and clinical practice. Clin Anat 2001;14:90 –1. |