|
Maya M. Hammoud, MD; Francis S. Nuthalapaty, MD; Alice R. Goepfert, MD; Petra M. Casey, MD; Sandra Emmons, MD; Eve L. Espey, MD; Joseph M. Kaczmarczyk, DO, MPH; Nadine T. Katz, MD; James J. Neutens, PhD; Edward G. Peskin, MD; for the Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee
From the Department of Obstetrics and Gynecology, Weill Cornell Medical College in Qatar, Doha, Qatar (Dr Hammoud); Department of Obstetrics and Gynecology, Greenville Hospital System, University Medical Center, Greenville, SC (Dr Nuthalapaty); Department of Obstetrics and Gynecology, University of Alabama at Birmingham, Birmingham, AL (Dr Goepfert); Department of Obstetrics and Gynecology, Mayo Medical School, Rochester,MN(Dr Casey); Department Obstetrics and Gynecology, Oregon Health and Sciences University, Portland, OR (Dr Emmons); Department of Obstetrics and Gynecology, University of New Mexico, Albuquerque,NM(Dr Espey); Department of Obstetrics and Gynecology, Uniformed Services University, Bethesda,MD(Dr Kaczmarczyk); Department of Obstetrics and Gynecology and Women’s Health, Albert Einstein College of Medicine, Bronx,
NY (Dr Katz); Department of Obstetrics and Gynecology, University of Tennessee–Knoxville, Knoxville, TN (Dr
Neutens); Department of Obstetrics and Gynecology, University of Massachusetts, Worcester,MA(Dr Peskin).
Received Dec. 31, 2007; accepted May 14, 2008 Reprints not available from the authors. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of Defense, the Department of Health and Human Services, or the US government.
0002-9378/$34.00
© 2008 Mosby, Inc. All rights reserved.
doi: 10.1016/j.ajog.2008.05.002
Simulation-based training (SBT) is becoming widely used in medical education to help residents and medical students develop good technical skills before they practice on real patients. SBT seems ideal because it provides a nonthreatening controlled environment for practice with immediate feedback and can include objective performance assessment. However, various forms of SBT and assessment often are being used with limited
evidence-based data to support their validity and reliability. In addition, although SBT with high-tech simulators is more sophisticated and attractive, this is not necessarily superior to SBT with low-tech (and lower cost) simulators. Therefore, understanding the types of surgical simulators and appropriate applications can help to ensure that this teaching and assessment modality is applied most effectively. This article summarizes the
key concepts that are needed to use surgical simulators effectively for teaching and assessment.
Key words: assessment, medical education, simulator, training
In this article, the seventh in the ongoing To the Point series produced by the Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee, we review the different types of surgical simulators that are available currently and summarize the key concepts that are needed to use surgical simulators effectively for teaching and assessment.
The goal of surgical simulator-based training (SBT) is to help trainees acquire and refine the cognitive and technical skills that are necessary to perform both simple and complex surgical procedures. Similar to applications in the military and in the airline industry, surgical SBT can be used to train and evaluate the trainee in complex decision-making, in time-sensitive and rare scenarios, and in skilled tasks. An excellent review of the history, rationale, and potential for SBT laboratories in obstetrics and gynecology was published by Macedonia et al in
2003.1 These authors focused mainly on training in obstetrics and described their curriculum at the Uniformed Services University of the Health Sciences that included SBT in procedures such as spontaneous vaginal delivery, perineal laceration repair, assisted vaginal breech delivery, and low forceps delivery.
With ongoing advances in surgical techniques, the content of surgical training in obstetrics and gynecology is becoming increasingly complex. For example, as compared with laparotomy, minimally invasive laparoscopic surgery requires the trainee to develop spatial relationship skills and associated psychomotor skills to be able to manipulate surgical instruments in a 3-dimensional operative field while looking at a 2-dimensional video screen. Therefore, effective training in obstetrics and gynecology requires skills acquisition in basic and complex surgical techniques and obstetric procedures. All of these skills potentially can be taught and assessed with SBT.
Types of surgical simulators
There are many types of simulators that are available for surgical skills training. Simulators can be broken down into 2 different groups: high fidelity and low fidelity. These models vary widely with respect to their level of fidelity or realism, as compared with a living human patient. The fidelity of a simulator is determined by the extent to which it provides realism through characteristics such as visual cues, tactile features, feedback capabilities,
and interaction with the trainee.
High-fidelity simulators use very realistic materials and equipment to represent the tasks that the trainee must perform. High-fidelity simulators provide the trainee with additional real-life cues to immerse them in a more realistic interactive scenario and environment. Trainees are challenged to apply correct interventions or maneuvers to complete a surgical procedure successfully. Low-fidelity simulators use materials and equipment that are less similar to what is used in the true surgical environment. They typically serve to practice isolated procedures such as knot tying or intravenous line insertion, instrument handling, and hand-eye coordination
used in laparoscopy.
A variety of intermediate level simulators are provided between the 2 extremes. Although high-fidelity simulators are more attractive to trainees, they are more expensive and may not be optimal for teaching more than basic skills. Wepresent more detailed descriptions of some low- and high-fidelity surgical simulators and their uses, advantages, and disadvantages (Table 1).
Low-fidelity simulators
Bench models. These simulators include any models that are static. There is a wide variety of models that are available for training of the medical student or the surgical resident. These include but are not limited to knot tying trainers, tissue models for practicing dissection and suturing, abdominal opening and closure trainers, episiotomy repair trainer, anal sphincter repair trainer, and urethral sling procedure trainer.
These models can be purchased directly from a commercial manufacturer2,3; some can be made. Examples are available as outlined in the CREOG (Council on Resident Education in Obstetrics and Gynecology) surgical
curriculum.4
Video box trainers. These simulators use real surgical instruments and equipment that includes cameras and video monitors to simulate various laparoscopic surgical skills and techniques. Box trainers typically have slits on the anterior surface for trocar insertions. The port holders are at waist level and are capable of accepting a camera and 2 instruments. The video output is visible on an operating room preview system monitor. Box trainers are an excellent mechanism to train for eye-hand coordination, camera handling, suturing techniques, grasping
mechanisms, point-to-point movement clip applying, and cutting. Many drills have been developed for use with these trainers that involve movement and coordination exercises with small objects. Suturing skills can also be practiced with standard sutures and needles. Perhaps 1 of the most important attributes of the box trainer is the sensory feedback that it provides to the trainee in terms of the feel of the instruments on the tissue surfaces
and the pressure of opening and closing the instruments.
Low-fidelity synthetic bench and video box models sacrifice realism for portability, lower costs, and the potential for repetitive use. However, it is important to note that intensive training that uses bench and video box trainers can improve not only technical skills but also translates into improved operative performance.5,6
High-fidelity simulators
Virtual-reality simulators. The virtualreality surgical simulators are the latest and most promising development in surgical simulation, especially in regards to laparoscopic simulations. Virtual reality is a technology that allows a user to interact with a computer-simulated environment, be it real or imagined. Most current virtual-reality environments are primarily visual experiences that are displayed either on a computer screen or through special stereoscopic displays. Haptic systems are those that incorporate tactile information; generally, this is in the form of force feedback.
Virtual-reality surgical simulators incorporate a number of unique features that enable them to provide a more believable practice environment and more focused assessment than low-fidelity box trainers. These simulators record and save objective data on individual performance on specific tasks, such as time taken to complete the task, economy of the hand motion, dexterity, and instrument path lengths. These data can be used by the educator to monitor the progress of the trainee or the data can be used by the trainee to monitor his or her own progress while practicing independently. In addition, the trainee can set the level of difficulty and move from 1
level to the next.
One of the most commonly used virtual- reality simulators is the LapSim system. 7 This simulator, which consists of laparoscopic instruments and a desktop computer, focuses on basic laparoscopic skills. The system has numerous modules that simulate tasks in the abdominal cavity, such as camera navigation, instrument navigation, coordination, grasping, cutting, clip applying, and suturing. Recent enhancements include complex diathermy and bowel manipulation. The simulator records data on 6-10 parameters per task that include timing and tissue damage. Performance on the LapSim has been shown to distinguish between novice and expert skill
levels.8 In addition, basic skills that are achieved by systematic training with a laparoscopic simulator such as LapSim can be transferred to the operating room.9
Procedural simulators. Procedural simulators are virtual-reality simulators that allow the trainee to perform an entire procedure and not just its component parts. Some of the available procedural simulators include gastrointestinal endoscopy, cholecystectomy, and ectopic pregnancy. The advantage of this type of simulator is that, in addition to aiding basic surgical skill acquisition, they also enhance knowledge and recognition of anatomy and the temporal sequence of the procedure. This combination of structured cognitive and manual skills application potentially transfers to the operating room.10
Animal models. Live animal surgery models have been popular for many years as a method of teaching, developing, and refining surgical techniques in both open and laparoscopic approaches.11,12 They are most desirable when a trainer is teaching complex laparoscopic techniques that are not best taught on patients for the first time.13 Although these models are considered to be of high fidelity, they are limited by availability, high costs, potential for transmission of infectious disease, and moral and ethical concerns.
Simulator use for training
The breadth of current simulator options presents educators with the challenge of deciding how to apply this technology to achieve the most effective simulation-based learning opportunities. The first step in the evaluation of any SBT is defining the capabilities of the simulator. The availability of surgical simulators builds on the long-standing tenet that basic surgical skills, such as knot tying and suturing, should be acquired outside the operating room. Simulators, however, have introduced the potential to dissect even the most complex procedure into its component parts and provide opportunities for task-specific practice. Ericsson14 acknowledged that the primary goal of practice is to improve some specific aspect of performance and that it should not be assumed
that this training is transferable to a clinical context or the operating room.
The next step in the evaluation of SBT is defining the relationship between the simulator and the actual simulation. A recent systematic review of surgical simulation evaluated 30 randomized controlled trials that were published before April 2005.15 Included trials were those that assessed any training technique with surgical simulators vs no simulator training and standard training and that reported measures of surgical task performance. The simulators that were used in the included studies were similar to those previously described. Although the quality of the randomized controlled trials that were included in the review was noted to be generally poor, the authors concluded that none of the methods of SBT has yet been shown to be superior to
other forms of surgical training. In general, SBT was superior to no training.
In an accompanying editorial, faculty members from the Departments of Surgery and Anesthesia at Stanford Univer sity School of Medicine cautioned educators and investigators to consider carefully the research questions when assessing outcomes of SBT.16 They assert that currently available surgical simulators are “tools that aim to train an isolated technical skill or set of skills.” On the other hand, simulation is a “set of techniques for re-creating aspects of the real world, typically to replace or amplify actual experiences”. In essence, surgical
simulators are only as effective as the simulation scenario in which they are used and how they are incorporated into the medical curriculum for residents and students. Simulation and simulators should be seen to be complementary tools that accelerate the learning curve and that enhance the real-world patient
encounters that remain the “cornerstone of medical education.”
In this context, once a commitment to SBT is made, educators should consider 4 criteria when designing, implementing, evaluating, or purchasing simulation-based training programs for procedural skills (as described by
Kneebone17):
1. Simulations should allow for sustained, deliberate practice within a safe environment, ensuring that recently acquired skills are consolidated within a defined curriculum that assures regular reinforcement.
2. Simulations should provide access to expert tutors when appropriate,assuring that such support fades
when no longer needed.
3. Simulations should map to real-life clinical experience, ensuring that learning supports the experience
that is gained within communities of actual practice.
4. Simulation-based learning environments should provide a supportive, motivational, and traineecentered
milieu that is constructive to learning.
The last 2 of these 4 criteria are especially important and worthy of emphasis. Actual clinical practice encompasses multiple competencies, which is a point that may be neglected when surgical simulation
exercises focus too narrowly on a specific surgical task. Therefore, when possible, simulation exercises should incorporate components such as communication and teamwork.
Using simulators for assessment
Any tool that is used for assessment of surgical skills must be both reliable and valid. Reliability refers to the ability of a test to generate the same results if repeated at multiple points in time. Validity refers to the ability of a test to measure what it was designed to measure. Furthermore, construct validity refers to whether the test can differentiate between different levels of experience in a particular factor (eg, whether a simulator can differentiate between a novice and an expert level). The traditional method for surgical assessment has been
quantification of procedural volume. Although procedural volume is considered an important benchmark for residency accreditation and is a key characteristic that is evaluated by applicants who consider programs for training, this method lacks both reliability and validity as a measure of surgical competency. In contrast,
simulators have the potential to provide a reliable and valid method of task-specific skills assessment when used
in combination with Objective Structured Assessment of Technical Skills (OSATS).
OSATS was introduced by Martin et al in 1997.18 This assessment tool is composed of several stations at which trainees perform procedures on simulators in fixed time periods. Task performance is assessed with checklists that are specific to the operation or task and a global rating scale. Martin et al showed that this
format is a reliable and valid method for the assessment of surgical skills and that bench model simulation gives equivalent results to the use of live animals for this test format. It appears that the task-specific checklists add little value to the assessment process above that obtained from global rating scales.19 These findings were confirmed in obstetric and gynecology resident training by investigators at the University of Washington in Seattle, who exposed their residents to a surgical laboratory curriculum using both inanimate and animal (porcine) models. The surgical skills that were evaluated by OSATS significantly improved over time both individually and as a cohort by resident year.20 These findings suggest that simulators are best used for assessment when they can be incorporated into an objective structured clinical examination and are used to assess individual performance and progress over time.
Unlike low-fidelity simulators combined with OSATS that require a significant investment of faculty resources to
administer, virtual-reality simulators can provide automatic “objective” assessment by recording metrics. A number of investigations regarding the reliability and validity of virtual-reality simulators for the assessment of surgical skills have confirmed that both the LapSimGyn VR and the minimally invasive surgical trainer–virtual reality systems confirm that these simulators provide an adequate means of psychomotor skills assessment for various laparoscopic procedures. 21-24 It remains to be determined, however, whether performance on these trainers translates into improved surgical competency in the operating room. Thus far, 2 randomized controlled
trials have demonstrated the benefits of using the minimally invasive surgical trainer–virtual reality for training
for a laparoscopic cholecystectomy. 25,26 Therefore, it appears that simulators can play a well-defined role in
surgical assessment and can be a useful adjunct to the global performance assessment that happens during the conduct of real procedures in the operating room. Table 2 presents a summary of the tools that are available for assessment of surgical skills with advantages and disadvantages of each, along with a comparison to the traditional method that is used for the assessment of competence, for clinical evaluation by direct observation
in the operating room, and for recording the number of surgical procedures that are performed.
The critical question for contemporary surgical educators to consider is whether the apprenticeship model for
surgical skills training is still adequate for training competent surgeons in today’s environment. Educational technology in the form of low- and high-fidelity surgical simulators provides an opportunity to supplement and even enhance surgical teaching and learning. This article has provided a summary of the key concepts that are needed to utilize surgical simulators effectively for teaching and assessment. However, simply providing access
to simulators is no guarantee that they will be used effectively for either training or assessment. Such interventions must be developed with a stepwise approach. One such approach is the ADDIE framework27:
analyze (analyze relevant trainee characteristics and tasks to be learned), design (define objectives and outcomes; select an instructional approach), develop (create the instructional materials), implement (deliver the instructional materials), evaluate (ensure that the instruction achieved the desired goal).
A recent qualitative systematic review summarized the 10 features that characterize effective, high-fidelity medical simulation. 28 Key among these features is the integration of the simulation into the broader educational program rather than offering it as an extraordinary activity. Additionally, it is important to keep in mind that there is no proven positive relationship between simulator fidelity and its effectiveness and that lower levels of fidelity
may reduce costs without compromising outcomes.29 Finally, a more robust assessment program for surgical skills potentially can be achieved with a combination of OSATS, performance assessments from virtual-reality simulators, and global rating scales from observation of a trainee operating on a real patient.
REFERENCES
1. Macedonia CR, Gherman RB, Satin AJ. Simulation laboratories for training in obstetrics & gynecology. Obstet Gynecol 2003:102:388-92.
2. Limbs and things medical simulation models. Last accessed: Dec. 11, 2007. Available at: www.golimbs.com.
3. Simulab Corporation. Last accessed: Dec.
11, 2007. Available at: www.simulab.com.
4. Council on Resident Education in Obstetrics and Gynecology surgical curriculum. Last accessed Dec. 11, 2007. Available at: www. acog.org/departments/dept_web.cfm?recno1.
5. Scott DJ, Bergen PC, Rege RV, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience?J Am Coll Surg 2000:191:272-83.
6. Coleman RL, Muller CV. Effects of a laboratory- based skills curriculum on laparoscopic proficiency: a randomized trial. Am J Obstet Gynecol 2002:186:836-42.
7. Immersion Medical. Last accessed: Dec 11, 2007. Available at: www.immersion.com/ medical/products/laparoscopy.
8. Aggarwal R, Grantcharov TP, Eriksen JR, etal. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006;244:310-4.
9. Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 2002;16:1324-8.
10. Aggarwal R, Tully A, Grantcharov T, et al. Virtual reality simulation training can improve technical skills during laparoscopic salpingectomy for ectopic pregnancy. BJOG 2006; 113:1382-7.
11. Woods JR Jr, Ansbacher R, Castro RJ, Marshall W, Trabal JF. Animal surgery: an adjunct to training in obstetrics and gynecology. Obstet Gynecol 1980;56:373-6.
12. Goff BA, Lentz GM, Lee DM, Mandel LS. Formal teaching of surgical skills in an obstetricgynecologic residency. Obstet Gynecol 1999; 93:785-90.
13. Scheeres DE, Mellinger JD, Brasser BA, Davis AT. Animate advanced laparoscopic courses improve resident operative performance. Am J Surg 2004;188:157-60.
14. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79(suppl):S70-81.
15. Sutherland LM, Middleton PF, Anthony A, etal. Surgical simulation: a systematic review. Ann Surg 2006;243:291-300.
16. Dutta S, Gaba D, Krummel TM. To simulate or not to simulate: what is the question? Ann Surg 2006;243:301-3.
17. Kneebone R. Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 2005;80:549-53.
18. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-8.
19. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998;73:993-7.
20. Lentz GM, Mandel LS, Goff BA. A six-year study of surgical teaching and skills evaluation for obstetric/gynecologic residents in porcine and inanimate surgical models. Am J Obstet Gynecol 2005;193:2056-61.
21. Larsen CR, Grantcharov T, Aggarwal R, et al. Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 2006;20:1460-6.
22. Gallagher AG, Smith CD, Bowers SP, et al. Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg 2003;197:479-88.
23. Grantcharov TP, Bardram L, Funch- Jensen P, Rosenberg J. Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg
2003;185:146-9.
24. Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor
skills: learning curves and reliability measures. Surg Endosc 2002;16:1746-52.
25. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, doubleblinded study. Ann Surg 2002;236:458-63.
26. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91:146-50.
27. Association of American Medical Colleges. Effective use of educational technology in medical education: colloquium on educational technology: recommendations and guidelines for medical educators. Washington (DC): Institute for Improving Medical Education; 2007.
28. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005; 271:10-28.
29. Grober ED, Hamstra SJ, Wanzel KR, et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg 2004;240:374-81. |