Virtual Reality and Augmented Reality in Plastic Surgery: A Review
---
paper_title: Virtual reality in surgery and medicine.

paper_content:

This report documents the state of development of enhanced and virtual reality-based systems in medicine. Virtual reality systems seek to simulate a surgical procedure in a computer-generated world in order to improve training. Enhanced reality systems seek to augment or enhance reality by providing improved imaging alternatives for specific patient data. Virtual reality represents a paradigm shift in the way we teach and evaluate the skills of medical personnel. Driving the development of virtual reality-based simulators is laparoscopic abdominal surgery, where there is a perceived need for better training techniques; within a year, systems will be fielded for second-year residency students. Further refinements over perhaps the next five years should allow surgeons to evaluate and practice new techniques in a simulator before using them on patients. Technical developments are rapidly improving the realism of these machines to an amazing degree, as well as bringing the price down to affordable levels. In the next five years, many new anatomical models, procedures, and skills are likely to become available on simulators. Enhanced reality systems are generally being developed to improve visualization of specific patient data. Three-dimensional (3-D) stereovision systems for endoscopic applications, head-mounted displays, and stereotactic image navigation systems are being fielded now, with neurosurgery and laparoscopic surgery being major driving influences. Over perhaps the next five years, enhanced and virtual reality systems are likely to merge. This will permit patient-specific images to be used on virtual reality simulators or computer-generated landscapes to be input into surgical visualization instruments. Percolating all around these activities are developments in robotics and telesurgery. An advanced information infrastructure eventually will permit remote physicians to share video, audio, medical records, and imaging data with local physicians in real time. Surgical robots are likely to be deployed for specific tasks in the operating room (OR) and to support telesurgery applications. Technical developments in robotics and motion control are key components of many virtual reality systems. Since almost all of the virtual reality and enhanced reality systems will be digitally based, they are also capable of being put "on-line" for tele-training, consulting, and even surgery. Advancements in virtual and enhanced reality systems will be driven in part by consumer applications of this technology. Many of the companies that will supply systems for medical applications are also working on commercial products. A big consumer hit can benefit the entire industry by increasing volumes and bringing down costs.(ABSTRACT TRUNCATED AT 400 WORDS)
---
paper_title: Overview: Virtual Reality in Medicine

paper_content:

96 800x600 Normal 0 14 false false false IT JA X-NONE /* Style Definitions */ ::: table.MsoNormalTable ::: {mso-style-name:"Tabella normale"; ::: mso-tstyle-rowband-size:0; ::: mso-tstyle-colband-size:0; ::: mso-style-noshow:yes; ::: mso-style-priority:99; ::: mso-style-parent:""; ::: mso-padding-alt:0cm 5.4pt 0cm 5.4pt; ::: mso-para-margin:0cm; ::: mso-para-margin-bottom:.0001pt; ::: mso-pagination:widow-orphan; ::: font-size:10.0pt; ::: font-family:Cambria;} ::: Background: Virtual Reality (VR) was defined as a collection of technological devices: “a computer capable of interactive 3D visualization, a head-mounted display and data gloves equipped with one or more position trackers”. Today, lots of scientists define VR as a simulation of the real world based on computer graphics, a three dimensional world in which communities of real people interact, create content, items and services, producing real economic value through e-Commerce. Objective: To report the results of a systematic review of articles and reviews published about the theme: “Virtual Reality in Medicine”. Methods: We used the search query string: “Virtual Reality”, “Metaverse”, “Second Life”, “Virtual World”, “Virtual Life” in order to find out how many articles were written about these themes. For the “Meta-review” we used only “Virtual Reality” AND “Review”. We searched the following databases: Psycinfo, Journal of Medical Internet Research, Isiknowledge till September 2011 and Pubmed till February 2012. We included any source published in either print format or on the Internet, available in all languages, and containing texts that define or attempt to define VR in explicit terms. Results: We retrieved 3,443 articles on Pubmed in 2012 and 8,237 on Isiknowledge in 2011. This large number of articles covered a wide range of themes, but showed no clear consensus about VR. We identified 4 general uses of VR in Medicine, and searched for the existing reviews about them. We found 364 reviews in 2011, although only 197 were pertinent to our aims: 1. Communication Interface (11 Reviews); 2. Medical Education (49 reviews); 3. Surgical Simulation (49 Reviews) and 4. Psychotherapy (88 Reviews). Conclusion: We found a large number of articles, but no clear consensus about the meaning of the term VR in Medicine. We found numerous articles published on these topics and many of them have been reviewed. We decided to group these reviews in 4 areas in order to provide a systematic overview of the subject matter, and to enable those interested to learn more about these particular topics.
---
paper_title: Mixed-reality simulation for orthognathic surgery

paper_content:

BackgroundMandibular motion tracking system (ManMoS) has been developed for orthognathic surgery. This article aimed to introduce the ManMoS and to examine the accuracy of this system.MethodsSkeletal and dental models are reconstructed in a virtual space from the DICOM data of three-dimensional computed tomography (3D-CT) recording and the STL data of 3D scanning, respectively. The ManMoS uniquely integrates the virtual dento-skeletal model with the real motion of the dental cast mounted on the simulator, using the reference splint. Positional change of the dental cast is tracked by using the 3D motion tracking equipment and reflects on the jaw position of the virtual model in real time, generating the mixed-reality surgical simulation. ManMoS was applied for two clinical cases having a facial asymmetry. In order to assess the accuracy of the ManMoS, the positional change of the lower dental arch was compared between the virtual and real models.ResultsWith the measurement data of the real lower dental cast as a reference, measurement error for the whole simulation system was less than 0.32 mm. In ManMoS, the skeletal and dental asymmetries were adequately diagnosed in three dimensions. Jaw repositioning was simulated with priority given to the skeletal correction rather than the occlusal correction. In two cases, facial asymmetry was successfully improved while a normal occlusal relationship was reconstructed. Positional change measured in the virtual model did not differ significantly from that in the real model.ConclusionsIt was suggested that the accuracy of the ManMoS was good enough for a clinical use. This surgical simulation system appears to meet clinical demands well and is an important facilitator of communication between orthodontists and surgeons.
---
paper_title: An Augmented Reality Haptic Training Simulator for Spinal Needle Procedures

paper_content:

This paper presents the prototype for an augmented reality haptic simulation system with potential for spinal needle insertion training. The proposed system is composed of a torso mannequin, a MicronTracker2 optical tracking system, a PHANToM haptic device, and a graphical user interface to provide visual feedback. The system allows users to perform simulated needle insertions on a physical mannequin overlaid with an augmented reality cutaway of patient anatomy. A tissue model based on a finite-element model provides force during the insertion. The system allows for training without the need for the presence of a trained clinician or access to live patients or cadavers. A pilot user study demonstrates the potential and functionality of the system.
---
paper_title: Virtual Reality Cerebral Aneurysm Clipping Simulation With Real-Time Haptic Feedback

paper_content:

Background ::: With the decrease in the number of cerebral aneurysms treated surgically and the increase of complexity of those treated surgically, there is a need for simulation-based tools to teach future neurosurgeons the operative techniques of aneurysm clipping.
---
paper_title: Computer based system for simulating spine surgery

paper_content:

A computer based system described herein provides preoperative simulations to verify burring and morphological operations in spine surgery. An efficient bur-tissue intersection computation method is used to simulate tissue burring by diverse shapes of burs for various cutting procedures in spine surgery. A volume manipulation is then used to calculate burred changes on volume data, provide high-quality burred tissue surfaces and check if any structure separate from the spine. Morphological simulations such as repositions, deletions and fusions can be implemented on the separate structures for spine decompression and morphology correction. The coated surface or flutes on a bur are pre-divided into small elements in which a tissue removal force on each element can be fast summated to obtain the burring force by predetermined transformation matrices. Examples of typical spine surgery modalities illustrate the effectiveness of the proposed methods and simulator.
---
paper_title: Real-Time Mandibular Angle Reduction Surgical Simulation With Haptic Rendering

paper_content:

Mandibular angle reduction is a popular and efficient procedure widely used to alter the facial contour. The primary surgical instruments, the reciprocating saw, and the round burr, employed in the surgery have a common feature: operating at a high speed. Generally, inexperienced surgeons need a long-time practice to learn how to minimize the risks caused by the uncontrolled contacts and cutting motions in manipulation of instruments with high-speed reciprocation or rotation. A virtual reality-based surgical simulator for the mandibular angle reduction was designed and implemented on a compute unified device architecture (CUDA)-based platform in this paper. High-fidelity visual and haptic feedbacks are provided to enhance the perception in a realistic virtual surgical environment. The impulse-based haptic models were employed to simulate the contact forces and torques on the instruments. It provides convincing haptic sensation for surgeons to control the instruments under different reciprocation or rotation velocities. The real-time methods for bone removal and reconstruction during surgical procedures have been proposed to support realistic visual feedbacks. The simulated contact forces were verified by comparing against the actual force data measured through the constructed mechanical platform. An empirical study based on the patient-specific data was conducted to evaluate the ability of the proposed system in training surgeons with various experiences. The results confirm the validity of our simulator.
---
paper_title: A Virtual Reality Based Simulation Environment for Orthopedic Surgery

paper_content:

Virtual reality simulators in medicine are becoming more widely used to train residents and interns. In this paper, we discuss the design of a Virtual Surgical Environment VSE for performing a class of orthopedic surgeries. The VSE focuses on providing training to residents in LISS plating based surgeries for addressing fractures of the femur. Using such virtual environments enables medical residents to develop a better understanding of the various surgical steps along with enabling them to practice specific surgical steps involving surgical instruments as well as other components. A brief discussion of the network based implementation is also provided; such an approach enables multiple users from different geographical locations to interact with an expert surgeon at another location. This network based approach was demonstrated as part of Next Generation Internet activities related to the GENI and US Ignite projects.
---
paper_title: Augmented reality patient-specific reconstruction plate design for pelvic and acetabular fracture surgery

paper_content:

Purpose ::: The objective of this work is to develop a preoperative reconstruction plate design system for unilateral pelvic and acetabular fracture reduction and internal fixation surgery, using computer graphics and augmented reality (AR) techniques, in order to respect the patient-specific morphology and to reduce surgical invasiveness, as well as to simplify the surgical procedure.
---
paper_title: Haptic computer-assisted patient-specific preoperative planning for orthopedic fractures surgery

paper_content:

PURPOSE ::: The aim of orthopedic trauma surgery is to restore the anatomy and function of displaced bone fragments to support osteosynthesis. For complex cases, including pelvic bone and multi-fragment femoral neck and distal radius fractures, preoperative planning with a CT scan is indicated. The planning consists of (1) fracture reduction-determining the locations and anatomical sites of origin of the fractured bone fragments and (2) fracture fixation-selecting and placing fixation screws and plates. The current bone fragment manipulation, hardware selection, and positioning processes based on 2D slices and a computer mouse are time-consuming and require a technician. ::: ::: ::: METHODS ::: We present a novel 3D haptic-based system for patient-specific preoperative planning of orthopedic fracture surgery based on CT scans. The system provides the surgeon with an interactive, intuitive, and comprehensive, planning tool that supports fracture reduction and fixation. Its unique features include: (1) two-hand haptic manipulation of 3D bone fragments and fixation hardware models; (2) 3D stereoscopic visualization and multiple viewing modes; (3) ligaments and pivot motion constraints to facilitate fracture reduction; (4) semiautomatic and automatic fracture reduction modes; and (5) interactive custom fixation plate creation to fit the bone morphology. ::: ::: ::: RESULTS ::: We evaluate our system with two experimental studies: (1) accuracy and repeatability of manual fracture reduction and (2) accuracy of our automatic virtual bone fracture reduction method. The surgeons achieved a mean accuracy of less than 1 mm for the manual reduction and 1.8 mm (std [Formula: see text] 1.1 mm) for the automatic reduction. ::: ::: ::: CONCLUSION ::: 3D haptic-based patient-specific preoperative planning of orthopedic fracture surgery from CT scans is useful and accurate and may have significant advantages for evaluating and planning complex fractures surgery.
---
paper_title: Mandible Reconstruction with 3D Virtual Planning

paper_content:

The fibula free flap has now become the most reliable and frequently used option for mandible reconstruction. Recently, three dimensional images and printing technologies are applied to mandibular reconstruction. We introduce our recent experience of mandibular reconstruction using three dimensionally planned fibula free flap in a patient with gunshot injury. The defect was virtually reconstructed with three-dimensional image. Because bone fragments are dislocated from original position, relocation was necessary. Fragments are virtually relocated to original position using mirror image of unaffected right side of the mandible. A medical rapid prototyping (MRP) model and cutting guide was made with 3D printer. Titanium reconstruction plate was adapted to the MRP model manually. 7 cm-sized fibula bone flap was designed on left lower leg. After dissection, proximal and distal margin of fibula flap was osteotomized by using three dimensional cutting guide. Segmentation was also done as planned. The fibula bone flap was attached to the inner side of the prebent reconstruction plate and fixed with screws. Postoperative evaluation was done by comparison between preoperative planning and surgical outcome. Although dislocated condyle is still not in ideal position, we can see that reconstruction was done as planned.
---
paper_title: High-fidelity haptic and visual rendering for patient-specific simulation of temporal bone surgery

paper_content:

AbstractMedical imaging techniques provide a wealth of information for surgical preparation, but it is still often the case that surgeons are examining three-dimensional pre-operative image data as a series of two-dimensional images. With recent advances in visual computing and interactive technologies, there is much opportunity to provide surgeons an ability to actively manipulate and interpret digital image data in a surgically meaningful way. This article describes the design and initial evaluation of a virtual surgical environment that supports patient-specific simulation of temporal bone surgery using pre-operative medical image data. Computational methods are presented that enable six degree-of-freedom haptic feedback during manipulation, and that simulate virtual dissection according to the mechanical principles of orthogonal cutting and abrasive wear. A highly efficient direct volume renderer simultaneously provides high-fidelity visual feedback during surgical manipulation of the virtual anatomy....
---
paper_title: Collaborative virtual environments for orthopedic surgery

paper_content:

Virtual reality simulators can be beneficial in increasing the quality of training while decreasing the time needed for training a specific skill. These simulators can provide a risk free environment which has the benefit of repetition and correction of the user's error. It has been years since the introduction of these simulators into the medical applications. The application of VR in orthopedic surgery, however, is a relatively new area of interest. In this paper, collaborative virtual reality based environments with the application in orthopedic surgery, with a focus on LISS (Less Invasive Stabilization System) plating, is presented. They can be used for training medical residents in orthopedic surgery.
---
paper_title: Percutaneous spinal fixation simulation with virtual reality and haptics.

paper_content:

BACKGROUND ::: In this study, we evaluated the use of a part-task simulator with 3-dimensional and haptic feedback as a training tool for percutaneous spinal needle placement. ::: ::: ::: OBJECTIVE ::: To evaluate the learning effectiveness in terms of entry point/target point accuracy of percutaneous spinal needle placement on a high-performance augmented-reality and haptic technology workstation with the ability to control the duration of computer-simulated fluoroscopic exposure, thereby simulating an actual situation. ::: ::: ::: METHODS ::: Sixty-three fellows and residents performed needle placement on the simulator. A virtual needle was percutaneously inserted into a virtual patient's thoracic spine derived from an actual patient computed tomography data set. ::: ::: ::: RESULTS ::: Ten of 126 needle placement attempts by 63 participants ended in failure for a failure rate of 7.93%. From all 126 needle insertions, the average error (15.69 vs 13.91), average fluoroscopy exposure (4.6 vs 3.92), and average individual performance score (32.39 vs 30.71) improved from the first to the second attempt. Performance accuracy yielded P = .04 from a 2-sample t test in which the rejected null hypothesis assumes no improvement in performance accuracy from the first to second attempt in the test session. ::: ::: ::: CONCLUSION ::: The experiments showed evidence (P = .04) of performance accuracy improvement from the first to the second percutaneous needle placement attempt. This result, combined with previous learning retention and/or face validity results of using the simulator for open thoracic pedicle screw placement and ventriculostomy catheter placement, supports the efficacy of augmented reality and haptics simulation as a learning tool.
---
paper_title: Virtual surgical system in reduction of maxillary fracture

paper_content:

Reduction of maxillary fracture is widely performed in cranial-maxillofacial surgeries, but it requires skilled and experienced surgeons. A virtual surgery system aiming on this kind of surgery is designed. CHAI 3D is used for rendering the haptic feedback. A multi-proxy algorithm is proposed to prevent the handle of operation tool stabbing into the virtual models and causing misjudgment. With the Geomagic haptic device, operators can manipulate one virtual model in the 3D virtual environment. This system can be used to train medical students or for preoperative planning of complicated surgeries.
---
paper_title: A haptics-assisted cranio-maxillofacial surgery planning system for restoring skeletal anatomy in complex trauma cases

paper_content:

Purpose Cranio-maxillofacial (CMF) surgery to restore normal skeletal anatomy in patients with serious trauma to the face can be both complex and time-consuming. But it is generally accepted that careful pre-operative planning leads to a better outcome with a higher degree of function and reduced morbidity in addition to reduced time in the operating room. However, today’s surgery planning systems are primitive, relying mostly on the user’s ability to plan complex tasks with a two-dimensional graphical interface. Methods A system for planning the restoration of skeletal anatomy in facial trauma patients using a virtual model derived from patient-specific CT data. The system combines stereo visualization with six degrees-of-freedom, high-fidelity haptic feedback that enables analysis, planning, and preoperative testing of alternative solutions for restoring bone fragments to their proper positions. The stereo display provides accurate visual spatial perception, and the haptics system provides intuitive haptic feedback when bone fragments are in contact as well as six degrees-of-freedom attraction forces for precise bone fragment alignment. Results A senior surgeon without prior experience of the system received 45 min of system training. Following the training session, he completed a virtual reconstruction in 22 min of a complex mandibular fracture with an adequately reduced result. Conclusion Preliminary testing with one surgeon indicates that our surgery planning system, which combines stereo visualization with sophisticated haptics, has the potential to become a powerful tool for CMF surgery planning. With little training, it allows a surgeon to complete a complex plan in a short amount of time.
---
paper_title: Virtual Reality Facial Contouring Surgery Simulator Based on CT Transversal Slices

paper_content:

In facial contouring surgery, surgeons operate to change facial bone morphology and obtain esthetic feminine face. Face appearance after surgery is required to evaluate; therefore, simulations for facial bone and soft tissue procedures are both required. This study develops a soft tissue manipulation method based on volume data to simulate face peeling and healing. Inte- grating with our reported bone surgical simulation methods, the proposed simulator can provide high quality illustrations for the simulations of various modalities of facial contouring surgery.
---
paper_title: A surgical simulator for planning and performing repair of cleft lips.

paper_content:

UNLABELLED ::: The objective of this project was to develop a computer-based surgical simulation system for planning and performing cleft lip repair. This system allows the user to interact with a virtual patient to perform the traditional steps of cleft-lip repair (rotation-advancement technique). ::: ::: ::: MATERIALS AND METHODS ::: The system interfaces to force-feedback (haptic) devices to track the user's motion and provide feedback during the procedure, while performing real-time soft-tissue simulation. An 11-day-old unilateral cleft lip, alveolus and palate patient was previously CT scanned for ancillary diagnostic purposes using standard imaging protocols and 1mm slices. High-resolution 3D meshes were automatically generated from this data using the ROVE software developed in-house. The resulting 3D meshes of bone and soft tissue were instilled with physical properties of soft tissues for purposes of simulation. Once these preprocessing steps were completed, the patient's bone and soft tissue data are presented on the computer screen in stereo and the user can freely view, rotate, and otherwise interact with the patient's data in real time. The user is prompted to select anatomical landmarks on the patient's data for preoperative planning purposes, then their locations are compared against that of a 'gold standard' and a score, derived from their deviation from that standard and time required, is generated. The user can then move a haptic stylus and guide the motion of the virtual cutting tool. The soft tissues can thus be incised using this virtual cutting tool, moved using virtual forceps, and fused in order to perform any of the major procedures for cleft lip repair. Real-time soft tissue deformation of the mesh realistically simulates normal tissues and haptic-rate (>1 kHz) force-feedback is provided. The surgical result of the procedure can then be immediately visualized and the entire training process can be repeated at will. A short evaluation study was also performed. Two groups (non-medical and plastic surgery residents) of six persons each performed the anatomical marking task of the simulator four times. ::: ::: ::: RESULTS ::: Results showed that the plastic surgery residents scored consistently better than the persons without medical background. Every person's score increased with practice, and the length of time needed to complete the 11 markings decreased. The data was compiled and showed which specific markers consistently took users the longest to identify as well as which locations were hardest to accurately mark. ::: ::: ::: CONCLUSION ::: These findings suggest that the simulator is a valuable training tool, giving residents a way to practice anatomical identification for cleft lip surgery without the risks associated with training on a live patient. Educators can also use the simulator to examine which markers are consistently problematic, and modify their training to address these needs.
---
paper_title: Haptics-assisted Virtual Planning of Bone, Soft Tissue, and Vessels in Fibula Osteocutaneous Free Flaps

paper_content:

Background: Virtual surgery planning has proven useful for reconstructing head and neck defects by fibula osteocutaneous free flaps (FOFF). Benefits include improved healing, function, and aestheti ...
---
paper_title: Application of an augmented reality tool for maxillary positioning in orthognathic surgery - a feasibility study.

paper_content:

BACKGROUND ::: An augmented reality tool for computer assisted surgery named X-Scope allows visual tracking of real anatomical structures in superposition with volume rendered CT or MRI scans and thus can be used for navigated translocation of bony segments. ::: ::: ::: METHODS ::: In a feasibility study X-Scope was used in orthognathic surgery to control the translocation of the maxilla after Le Fort I osteotomy within a bimaxillary procedure. The situation achieved was compared with the pre-operative situation by means of cephalometric analysis on lateral and frontal cephalograms. ::: ::: ::: RESULTS ::: The technique was successfully utilized in 5 patients. Maxillary positioning using X-Scope was accomplished accurately within a range of 1mm. The tool was used in all cases in addition to the usual intra-operative splints. A stand-alone application without conventional control does not yet seem reasonable. ::: ::: ::: CONCLUSION ::: Augmented reality tools like X-Scope may be helpful for controlling maxillary translocation in orthognathic surgery. The application to other interventions in cranio-maxillofacial surgery such as Le Fort III osteotomy, fronto-orbital advancement, and cranial vault reshaping or repair may also be considered.
---
paper_title: Video see‐through augmented reality for oral and maxillofacial surgery

paper_content:

BACKGROUND ::: Oral and maxillofacial surgery has not been benefitting from image guidance techniques owing to the limitations in image registration. ::: ::: ::: METHODS ::: A real-time markerless image registration method is proposed by integrating a shape matching method into a 2D tracking framework. The image registration is performed by matching the patient's teeth model with intraoperative video to obtain its pose. The resulting pose is used to overlay relevant models from the same CT space on the camera video for augmented reality. ::: ::: ::: RESULTS ::: The proposed system was evaluated on mandible/maxilla phantoms, a volunteer and clinical data. Experimental results show that the target overlay error is about 1 mm, and the frame rate of registration update yields 3-5 frames per second with a 4 K camera. ::: ::: ::: CONCLUSIONS ::: The significance of this work lies in its simplicity in clinical setting and the seamless integration into the current medical procedure with satisfactory response time and overlay accuracy. Copyright © 2016 John Wiley & Sons, Ltd.
---
paper_title: Mandibular angle split osteotomy based on a novel augmented reality navigation using specialized robot-assisted arms--A feasibility study.

paper_content:

PURPOSE ::: Augmented reality (AR) navigation, is a visible 3-dimensional display technology, that, when combined with robot-assisted surgery (RAS), allows precision and automation in operational procedures. In this study, we used an innovative, minimally invasive, simplified operative method to position the landmarks and specialized robot-assisted arms to apply in a rapid protyping (RP) model. This is the first report of the use of AR and RAS technology in craniomaxillofacial surgery. ::: ::: ::: METHOD ::: Five patients with prominent mandibular angle were randomly chosen for this feasibility study. We reconstructed the mandibular modules and created preoperational plans as semi-embedded and nail-fixation modules for an easy registration procedure. The left side of the mandibular modules comprised the experimental groups with use of a robot, and the right sides comprised the control groups without a robot. With AR Toolkits program tracking and display system applied, we carried out the operative plans and measured the error. ::: ::: ::: RESULTS ::: Both groups were successfully treated in this study, but the RAS was more accurate and stable. The average position and angle were significant (p < 0.01) between the 2 groups. ::: ::: ::: CONCLUSIONS ::: This study reports a novel augmented reality navigation with specialized robot-assisted arms for mandibular angle split osteotomy. AR and RAS can be helpful for patients undergoing craniomaxillofacial surgery.
---
paper_title: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

paper_content:

AIM ::: We present a newly designed, localiser-free, head-mounted system featuring augmented reality as an aid to maxillofacial bone surgery, and assess the potential utility of the device by conducting a feasibility study and validation. ::: ::: ::: METHODS ::: Our head-mounted wearable system facilitating augmented surgery was developed as a stand-alone, video-based, see-through device in which the visual features were adapted to facilitate maxillofacial bone surgery. We implement a strategy designed to present augmented reality information to the operating surgeon. LeFort1 osteotomy was chosen as the test procedure. The system is designed to exhibit virtual planning overlaying the details of a real patient. We implemented a method allowing performance of waferless, augmented-reality assisted bone repositioning. In vitro testing was conducted on a physical replica of a human skull, and the augmented reality system was used to perform LeFort1 maxillary repositioning. Surgical accuracy was measured with the aid of an optical navigation system that recorded the coordinates of three reference points (located in anterior, posterior right, and posterior left positions) on the repositioned maxilla. The outcomes were compared with those expected to be achievable in a three-dimensional environment. Data were derived using three levels of surgical planning, of increasing complexity, and for nine different operators with varying levels of surgical skill. ::: ::: ::: RESULTS ::: The mean error was 1.70 ± 0.51 mm. The axial errors were 0.89 ± 0.54 mm on the sagittal axis, 0.60 ± 0.20 mm on the frontal axis, and 1.06 ± 0.40 mm on the craniocaudal axis. The simplest plan was associated with a slightly lower mean error (1.58 ± 0.37 mm) compared with the more complex plans (medium: 1.82 ± 0.71 mm; difficult: 1.70 ± 0.45 mm). The mean error for the anterior reference point was lower (1.33 ± 0.58 mm) than those for both the posterior right (1.72 ± 0.24 mm) and posterior left points (2.05 ± 0.47 mm). No significant difference in terms of error was noticed among operators, despite variations in surgical experience. Feedback from surgeons was acceptable; all tests were completed within 15 min and the tool was considered to be both comfortable and usable in practice. ::: ::: ::: CONCLUSION ::: We used a new localiser-free, head-mounted, wearable, stereoscopic, video see-through display to develop a useful strategy affording surgeons access to augmented reality information. Our device appears to be accurate when used to assist in waferless maxillary repositioning. Our results suggest that the method can potentially be extended for use with many surgical procedures on the facial skeleton. Further, our positive results suggest that it would be appropriate to proceed to in vivo testing to assess surgical accuracy under real clinical conditions.
---
paper_title: Computer-assisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display.

paper_content:

There may well be a shift towards 3-dimensional orthognathic surgery when virtual surgical planning can be applied clinically. We present a computer-assisted protocol that uses surgical navigation supplemented by an interactive image-guided visualisation display (IGVD) to transfer virtual maxillary planning precisely. The aim of this study was to analyse its accuracy and versatility in vivo. The protocol consists of maxillofacial imaging, diagnosis, planning of virtual treatment, and intraoperative surgical transfer using an IGV display. The advantage of the interactive IGV display is that the virtually planned maxilla and its real position can be completely superimposed during operation through a video graphics array (VGA) camera, thereby augmenting the surgeon's 3-dimensional perception. Sixteen adult class III patients were treated with by bimaxillary osteotomy. Seven hard tissue variables were chosen to compare (ΔT1-T0) the virtual maxillary planning (T0) with the postoperative result (T1) using 3-dimensional cephalometry. Clinically acceptable precision for the surgical planning transfer of the maxilla (<0.35 mm) was seen in the anteroposterior and mediolateral angles, and in relation to the skull base (<0.35°), and marginal precision was seen in the orthogonal dimension (<0.64 mm). An interactive IGV display complemented surgical navigation, augmented virtual and real-time reality, and provided a precise technique of waferless stereotactic maxillary positioning, which may offer an alternative approach to the use of arbitrary splints and 2-dimensional orthognathic planning.
---
paper_title: An effective visualization technique for depth perception in augmented reality-based surgical navigation.

paper_content:

BACKGROUND ::: Depth perception is a major issue in augmented reality (AR)-based surgical navigation. We propose an AR and virtual reality (VR) switchable visualization system with distance information, and evaluate its performance in a surgical navigation set-up. ::: ::: ::: METHODS ::: To improve depth perception, seamless switching from AR to VR was implemented. In addition, the minimum distance between the tip of the surgical tool and the nearest organ was provided in real time. To evaluate the proposed techniques, five physicians and 20 non-medical volunteers participated in experiments. ::: ::: ::: RESULTS ::: Targeting error, time taken, and numbers of collisions were measured in simulation experiments. There was a statistically significant difference between a simple AR technique and the proposed technique. ::: ::: ::: CONCLUSIONS ::: We confirmed that depth perception in AR could be improved by the proposed seamless switching between AR and VR, and providing an indication of the minimum distance also facilitated the surgical tasks.
---
paper_title: A virtual training system for maxillofacial surgery using advanced haptic feedback and immersive workbench.

paper_content:

Background ::: ::: VR-based surgery simulation provides a cost-effective and efficient method to train novices. In this study, a virtual training system for maxillofacial surgery (VR-MFS), which aims mainly at the simulation of operations on mandible and maxilla, was developed and demonstrated. ::: ::: ::: ::: Methods ::: ::: The virtual models of the anatomic structures were reconstructed from CT data, and the virtual instruments were built from laser scanning data using reverse engineering technology. For collision detection, axis aligned bounding boxes (AABBs) were constructed for the anatomic models. Then, the simulation algorithms were developed, and the haptic force feedback was consequently calculated based on regression equations. Finally, the vivid 3D stereo effect was implemented with the use of an immersive workbench. ::: ::: ::: ::: Results ::: ::: A virtual training system for maxillofacial surgery was developed; in particular, the application for Le-Fort I osteotomy was implemented. The tactile, visual and aural effects were highly integrated, making the virtual surgical environment vivid and realistic. ::: ::: ::: ::: Conclusions ::: ::: The VR-MFS provides an effective approach in terms of helping novices to become familiar with maxillofacial surgery procedures. The same method can also be applied to other bone simulations. Copyright © 2013 John Wiley & Sons, Ltd.
---
paper_title: Impulse-Based Rendering Methods for Haptic Simulation of Bone-Burring.

paper_content:

Bone-burring is a common procedure in orthopedic, dental, and otologic surgeries. Virtual reality (VR)-based surgical simulations with both visual and haptic feedbacks provide novice surgeons with a feasible and safe way to practice their burring skill. However, creating realistic haptic interactions between a high-speed rotary burr and stiff bone is a challenging task. In this paper, we propose a novel interactive haptic bone-burring model based on impulse-based dynamics to simulate the contact forces, including resistant and frictional forces. In order to mimic the lateral and axial burring vibration forces, a 3D vibration model has been developed. A prototype haptic simulation system for the bone-burring procedure has been implemented to evaluate the proposed haptic rendering methods. Several experiments of force evaluations and task-oriented tests were conducted on the prototype system. The results demonstrate the validity and feasibility of the proposed methods.
---
paper_title: A virtual reality simulator for orthopedic basic skills: A design and validation study

paper_content:

Orthopedic drilling as a skill demands high levels of dexterity and expertise from the surgeon. It is a basic skill that is required in many orthopedic procedures. Inefficient drilling can be a source of avoidable medical errors that may lead to adverse events. It is hence important to train and evaluate residents in safe environments for this skill. This paper presents a virtual orthopedic drilling simulator that was designed to provide visiohaptic interaction with virtual bones. The simulation provides a realistic basic training environment for orthopedic surgeons. It contains modules to track and analyze movements of surgeons, in order to determine their surgical proficiency. The simulator was tested with senior surgeons, residents and medical students for validation purposes. Through the multi-tiered testing strategy it was shown that the simulator was able to produce a learning effect that transfers to real-world drilling. Further, objective measures of surgical performance were found to be able to differentiate between experts and novices.
---
paper_title: Development and validation of a surgical training simulator with haptic feedback for learning bone-sawing skill

paper_content:

Display Omitted We developed a visuo-haptic surgical training simulator for bone-sawing procedure.We validated the simulator's face validity and construct validity.This simulator was able to produce the effect of learning bone-sawing skill.This simulator could provide a training alternative for novices. ObjectiveBone sawing or cutting is widely used for bone removal processes in bone surgery. It is an essential skill that surgeons should execute with a high level of experience and sensitive force perception. Surgical training simulators, with virtual and haptic feedback functions, can offer a safe, repeatable and cost-effective alternative to traditional surgeries. In this research, we developed a surgical training simulator with virtual and haptic force feedback for maxillofacial surgery, and we validated the effects on the learning of bone-sawing skills through empirical evaluation. MethodsOmega.6 from Force Dimension was employed as the haptic device, and Display300 from SenseGraphices was used as the 3D stereo display. The voxel-based model was constructed using computed tomography (CT) images, and the virtual tools were built through reverse engineering. The multi-point collision detection method was applied for haptic rendering to test the 3D relationship between the virtual tool and the bone voxels. Bone-sawing procedures in maxillofacial surgery were simulated with a virtual environment and real-time haptic feedback. A total of 25 participants (16 novices and 9 experienced surgeons) were included in 2 groups to perform the bone-sawing simulation for assessing the construct validity. Each of the participants completed the same bone-sawing procedure at the predefined maxillary region six times. For each trial, the sawing operative time, the maximal acceleration, and the percentage of the haptic force exceeding the threshold were recorded and analysed to evaluate the validity. After six trials, all of the participants scored the simulator in terms of safe force learning, stable hand control and overall performance to confirm the face validity. Moreover, 10 novices in 2 groups indentified the transfer validity on rapid prototype skull models by comparing the operative time and the maximal acceleration. ResultsThe analysed results of construct validity showed that the two groups significantly reduced their sawing operative times after six trials. Regarding maximal acceleration, the curve significantly descended and reached a plateau after the fifth repetition (novices) or third repetition (surgeons). Regarding safe haptic force, the novices obviously reduced the percentage of the haptic force exceeding the threshold, with statistical significance after four trials, but the surgeons did not show a significant difference. Moreover, the subjectively scored results demonstrated that the proposed simulator was more helpful for the novices than for the experienced surgeons, with scores of 8.31 and 7.22, respectively, for their overall performance. The experimental results on skill transference showed that the experimental group performed bone-sawing operation in lower maximal acceleration than control group with a significant difference (p<0.05). These findings suggested that the simulator training had positive effects on real sawing. ConclusionsThe evaluation results proved the construct validity, face validity and the transfer validity of the simulator. These results indicated that this simulator was able to produce the effect of learning bone-sawing skill, and it could provide a training alternative for novices.
---
paper_title: A Virtual Reality System to Train Image Guided Placement of Kirschner-Wires for Distal Radius Fractures

paper_content:

We present the design, development and initial user testing of a virtual reality simulator to train orthopaedic surgeons in the optimal placement of K-wires for fixation of distal radius fractures. Our platform includes 5 DOF haptic feedback to recreate the manual skill aspects of the drilling process, a 3D view of the anatomy and a controllable x-ray image. Once complete, the user is given an overview of their performance compared with the ’ideal placement’ defined by an expert orthopaedic surgeon. The design goals based on analysis of the core steps in the procedure are presented, along with the technical implementation in terms of both haptic and graphical feedback. Preliminary user testing results are discussed, together with current limitations and planned future development.
---
paper_title: The Validity and Reliability of a Hybrid Reality Simulator for Wire Navigation in Orthopedic Surgery

paper_content:

Creating training simulators for orthopedic surgery presents unique and largely unexplored engineering challenges. One of these challenges is simulating both the large forces and subtle haptic feedback involved in drilling into bone. This paper involves a hybrid reality wire navigation simulator that combines a physical drill and plastic foam bone surrogate with 3-D graphics to mimic the radiographic imaging often used in a common surgical task. This paper describes experiments to test the simulator's face, content, and construct validity, as well as the precision of the motion-tracking system. The results suggest that the simulator has sufficient face validity to engage trainees and recreates the critical aspects of the wire navigation task. Experimental simulator trials reveal time and accuracy differentials between experts and novices, suggesting construct validity. The results also show that the tracking system is accurate within 3.6 mm.
---
paper_title: Comparison of cadaveric and isomorphic virtual haptic simulation in temporal bone training

paper_content:

Virtual surgery may improve learning and provides an opportunity for pre-operative surgical rehearsal. We describe a novel haptic temporal bone simulator specifically developed for multicore processing and improved visual realism. A position locking algorithm for enhanced drill-bone interaction and haptic fidelity is further employed. The simulation construct is evaluated against cadaveric education. A voxel-based simulator was designed for multicore architecture employing Marching Cubes and Laplacian smoothing to perform real-time haptic and graphic rendering of virtual bone. Ten Otolaryngology trainees dissected a cadaveric temporal bone (CTB) followed by a virtual isomorphic haptic model (VM) based on derivative microCT data. Participants rated 1) physical characteristics, 2) specific anatomic constructs, 3) usefulness in skill development and 4) perceived educational value. The survey instrument employed a Likert scale (1-7). Residents were equivocal about the physical properties of the VM, as cortical (3.2 ± 2.0) and trabecular (2.8 ± 1.6) bone drilling character was appraised as dissimilar to CTB. Overall similarity to cadaveric training was moderate (3.5 ± 1.8). Residents generally felt the VM was beneficial in skill development, rating it highest for translabyrinthine skull-base approaches (5.2 ± 1.3). The VM was considered an effective (5.4 ± 1.5) and accurate (5.7 ± 1.4) training tool which should be integrated into resident education (5.5 ± 1.4). The VM was thought to improve performance (5.3 ± 1.8) and confidence (5.3 ± 1.9) and was highly rated for anatomic learning (6.1 ± 1.9). Study participants found the VM to be a beneficial and effective platform for learning temporal bone anatomy and surgical techniques. They identify some concern with limited physical realism likely owing to the haptic device interface. This study is the first to compare isomorphic simulation in education. This significantly removes possible confounding features as the haptic simulation was based on derivative imaging.
---
paper_title: An Augmented Reality Haptic Training Simulator for Spinal Needle Procedures

paper_content:

This paper presents the prototype for an augmented reality haptic simulation system with potential for spinal needle insertion training. The proposed system is composed of a torso mannequin, a MicronTracker2 optical tracking system, a PHANToM haptic device, and a graphical user interface to provide visual feedback. The system allows users to perform simulated needle insertions on a physical mannequin overlaid with an augmented reality cutaway of patient anatomy. A tissue model based on a finite-element model provides force during the insertion. The system allows for training without the need for the presence of a trained clinician or access to live patients or cadavers. A pilot user study demonstrates the potential and functionality of the system.
---
paper_title: Virtual Reality Cerebral Aneurysm Clipping Simulation With Real-Time Haptic Feedback

paper_content:

Background ::: With the decrease in the number of cerebral aneurysms treated surgically and the increase of complexity of those treated surgically, there is a need for simulation-based tools to teach future neurosurgeons the operative techniques of aneurysm clipping.
---
paper_title: Computer based system for simulating spine surgery

paper_content:

A computer based system described herein provides preoperative simulations to verify burring and morphological operations in spine surgery. An efficient bur-tissue intersection computation method is used to simulate tissue burring by diverse shapes of burs for various cutting procedures in spine surgery. A volume manipulation is then used to calculate burred changes on volume data, provide high-quality burred tissue surfaces and check if any structure separate from the spine. Morphological simulations such as repositions, deletions and fusions can be implemented on the separate structures for spine decompression and morphology correction. The coated surface or flutes on a bur are pre-divided into small elements in which a tissue removal force on each element can be fast summated to obtain the burring force by predetermined transformation matrices. Examples of typical spine surgery modalities illustrate the effectiveness of the proposed methods and simulator.
---
paper_title: Three-Dimensional Display Technologies for Anatomical Education: A Literature Review

paper_content:

Anatomy is a foundational component of biological sciences and medical education and is important for a variety of clinical tasks. To augment current curriculum and improve students’ spatial knowledge of anatomy, many educators, anatomists, and researchers use three-dimensional (3D) visualization technologies. This article reviews 3D display technologies and their associated assessments for anatomical education. In the first segment, the review covers the general function of displays employing 3D techniques. The second segment of the review highlights the use and assessment of 3D technology in anatomical education, focusing on factors such as knowledge gains, student perceptions, and cognitive load. The review found 32 articles on the use of 3D displays in anatomical education and another 38 articles on the assessment of 3D displays. The review shows that the majority (74 %) of studies indicate that the use of 3D is beneficial for many tasks in anatomical education, and that student perceptions are positive toward the technology.
---
paper_title: Virtual Reality Cerebral Aneurysm Clipping Simulation With Real-Time Haptic Feedback

paper_content:

Background ::: With the decrease in the number of cerebral aneurysms treated surgically and the increase of complexity of those treated surgically, there is a need for simulation-based tools to teach future neurosurgeons the operative techniques of aneurysm clipping.
---
paper_title: Virtual Reality Technology

paper_content:

From the Publisher: ::: This in-depth review of current virtual reality technology and its applications provides a detailed analysis of the engineering, scientific and functional aspects of virtual reality systems and the fundamentals of VR modeling and programming. It also contains an exhaustive list of present and future VR applications in a number of diverse fields. Virtual Reality Technology is the first book to include a full chapter on force and tactile feedback and to discuss newer interface tools such as 3-D probes and cyberscopes. Supplemented with 23 color plates and more than 200 drawings and tables which illustrate the concepts described.
---

Title: Virtual Reality and Augmented Reality in Plastic Surgery: A Review

Section 1: INTRODUCTION
Description 1: This section provides a definition and historical context of VR/AR technologies, their applications in healthcare, and their potential in plastic surgery.

Section 2: METHODS
Description 2: This section details the systematic review process, search terms, and the inclusion criteria used to gather relevant publications and products in the study. 

Section 3: RESULTS
Description 3: This section summarizes the findings from the reviewed studies and products, categorized into surgical planning, navigation, and training, along with the devices used.

Section 4: Surgical planning
Description 4: This section discusses the use of VR/AR in patient-specific simulations during the preoperative phase and the use of various devices and technologies in surgical planning.

Section 5: Surgical navigation
Description 5: This section describes AR-based navigation systems that assist surgeons by displaying anatomical information and preoperative plans during surgery.

Section 6: Surgical training
Description 6: This section covers VR and AR systems used for surgical skill training and simulation, highlighting various devices and their applications in different surgical procedures.

Section 7: DISCUSSION
Description 7: This section provides a discussion of the main advantages of VR/AR in plastic surgery, including their potential future developments and impact on surgical practice.