A Survey of auditory display in image-guided interventions
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Touchless interaction with software in interventional radiology and surgery: a systematic literature review

paper_content:

PURPOSE ::: In this article, we systematically examine the current state of research of systems that focus on touchless human-computer interaction in operating rooms and interventional radiology suites. We further discuss the drawbacks of current solutions and underline promising technologies for future development. ::: ::: ::: METHODS ::: A systematic literature search of scientific papers that deal with touchless control of medical software in the immediate environment of the operation room and interventional radiology suite was performed. This includes methods for touchless gesture interaction, voice control and eye tracking. ::: ::: ::: RESULTS ::: Fifty-five research papers were identified and analyzed in detail including 33 journal publications. Most of the identified literature (62 %) deals with the control of medical image viewers. The others present interaction techniques for laparoscopic assistance (13 %), telerobotic assistance and operating room control (9 % each) as well as for robotic operating room assistance and intraoperative registration (3.5 % each). Only 8 systems (14.5 %) were tested in a real clinical environment, and 7 (12.7 %) were not evaluated at all. ::: ::: ::: CONCLUSION ::: In the last 10 years, many advancements have led to robust touchless interaction approaches. However, only a few have been systematically evaluated in real operating room settings. Further research is required to cope with current limitations of touchless software interfaces in clinical environments. The main challenges for future research are the improvement and evaluation of usability and intuitiveness of touchless human-computer interaction and the full integration into productive systems as well as the reduction of necessary interaction steps and further development of hands-free interaction.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Image-Guided Interventions: Technology Review and Clinical Applications

paper_content:

Image-guided interventions are medical procedures that use computer-based systems to provide virtual image overlays to help the physician precisely visualize and target the surgical site. This field has been greatly expanded by the advances in medical imaging and computing power over the past 20 years. This review begins with a historical overview and then describes the component technologies of tracking, registration, visualization, and software. Clinical applications in neurosurgery, orthopedics, and the cardiac and thoracoabdominal areas are discussed, together with a description of an evolving technology named Natural Orifice Transluminal Endoscopic Surgery (NOTES). As the trend toward minimally invasive procedures continues, image-guided interventions will play an important role in enabling new procedures, while improving the accuracy and success of existing approaches. Despite this promise, the role of image-guided systems must be validated by clinical trials facilitated by partnerships between scie...
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: Less is sometimes more: a comparison of distance-control and navigated-control concepts of image-guided navigation support for surgeons.

paper_content:

Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ...
---
paper_title: A Surgical Navigation System for Guiding Exact Cochleostomy Using Auditory Feedback: A Clinical Feasibility Study

paper_content:

In cochlear implantation (CI), the insertion of the electrode array into the appropriate compartment of the cochlea, the scala tympani, is important for an optimal hearing outcome. The current surgical technique for CI depends primarily on the surgeon's skills and experience level to achieve the correct placement of the electrode array, and the surgeon needs to confirm that the exact placement is achieved prior to completing the procedure. Thus, a surgical navigation system can help the surgeon to access the scala tympani without injuring important organs in the complex structure of the temporal bone. However, the use of a surgical microscope has restricted the effectiveness of the surgical navigation because it has been difficult to deliver the navigational information to the surgeon from outside of the surgeon's visual attention. We herein present a clinical feasibility study of an auditory feedback function developed as a computer-surgeon interface that can guide the surgeon to the preset cochleostomy location. As a result, the surgeon could confirm that the drilling point was correct, while keeping his or her eyes focused on the microscope. The proposed interface reduced the common frustration that surgeons experience when using surgical navigation during otologic surgeries.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Auditory support for navigated radiofrequency ablation

paper_content:

Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: Less is sometimes more: a comparison of distance-control and navigated-control concepts of image-guided navigation support for surgeons.

paper_content:

Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ...
---
paper_title: A Surgical Navigation System for Guiding Exact Cochleostomy Using Auditory Feedback: A Clinical Feasibility Study

paper_content:

In cochlear implantation (CI), the insertion of the electrode array into the appropriate compartment of the cochlea, the scala tympani, is important for an optimal hearing outcome. The current surgical technique for CI depends primarily on the surgeon's skills and experience level to achieve the correct placement of the electrode array, and the surgeon needs to confirm that the exact placement is achieved prior to completing the procedure. Thus, a surgical navigation system can help the surgeon to access the scala tympani without injuring important organs in the complex structure of the temporal bone. However, the use of a surgical microscope has restricted the effectiveness of the surgical navigation because it has been difficult to deliver the navigational information to the surgeon from outside of the surgeon's visual attention. We herein present a clinical feasibility study of an auditory feedback function developed as a computer-surgeon interface that can guide the surgeon to the preset cochleostomy location. As a result, the surgeon could confirm that the drilling point was correct, while keeping his or her eyes focused on the microscope. The proposed interface reduced the common frustration that surgeons experience when using surgical navigation during otologic surgeries.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Auditory support for navigated radiofrequency ablation

paper_content:

Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: Less is sometimes more: a comparison of distance-control and navigated-control concepts of image-guided navigation support for surgeons.

paper_content:

Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ...
---
paper_title: A Surgical Navigation System for Guiding Exact Cochleostomy Using Auditory Feedback: A Clinical Feasibility Study

paper_content:

In cochlear implantation (CI), the insertion of the electrode array into the appropriate compartment of the cochlea, the scala tympani, is important for an optimal hearing outcome. The current surgical technique for CI depends primarily on the surgeon's skills and experience level to achieve the correct placement of the electrode array, and the surgeon needs to confirm that the exact placement is achieved prior to completing the procedure. Thus, a surgical navigation system can help the surgeon to access the scala tympani without injuring important organs in the complex structure of the temporal bone. However, the use of a surgical microscope has restricted the effectiveness of the surgical navigation because it has been difficult to deliver the navigational information to the surgeon from outside of the surgeon's visual attention. We herein present a clinical feasibility study of an auditory feedback function developed as a computer-surgeon interface that can guide the surgeon to the preset cochleostomy location. As a result, the surgeon could confirm that the drilling point was correct, while keeping his or her eyes focused on the microscope. The proposed interface reduced the common frustration that surgeons experience when using surgical navigation during otologic surgeries.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Auditory support for navigated radiofrequency ablation

paper_content:

Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: The process of sonification design for guidance tasks

paper_content:

This article deals with the process of sonification design for guidance tasks. It presents several studies that aim at overcoming two major problems of sonification: the aesthetics of sound design and the lack of a general method. On the basis of these studies, it proposes some guidelines for the generalization of the sonification process. First, it introduces the need to disassociate data and display dimensions; then, it proposes a method to classify and evaluate sound strategies; finally, it introduces a method for the customization of the sound design. The whole process is based on the identification and the manipulation of particular sound morphologies.
---
paper_title: Using an auditory display to manage attention in a dual task, multiscreen environment

paper_content:

Spatialized sound technology is under consideration for use in future U. S. Navy watchstation systems as a technique to manage attention. In this study, we looked at whether spatialized sound would reduce head movements. The subjects used a simulated watchstation that had three displays, one forward and one on each side. A dual task paradigm was used that included a continuous tracking task in one window and an intermittent task in another window. These two windows were presented adjacent to each other in the center display or opposite each other on the side displays. Subjects performed the dual task with and without sound. Head turns were recorded manually and were found to be significantly fewer in number when sound was present. Further, when sound was present, subjects used its cessation as an indication of a successfully entered response. This aural feedback reduced head movements that would normally be made to confirm the successful data entry. Together with other results on reaction time and accuracy, these results provide persuasive support for the use of spatialized sound to direct attention.
---
paper_title: A Model for Interaction in Exploratory Sonification Displays

paper_content:

This paper presents a general model for sonification of large spatial data sets (e.g. seismic data, medical data) based on ideas from ecological acoustics. The model incorporates not only what we hear (the sounds), but also how we listen (the interaction). Metaphorically speaking the interpreter is walking along paths in areas of the data set, listening to locally and globally defined sound objects. The time aspects of sonification are given special attention, introducing the notion of temporalization. Some features of a preliminary Windows NT implementation are summarized.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Less is sometimes more: a comparison of distance-control and navigated-control concepts of image-guided navigation support for surgeons.

paper_content:

Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ...
---
paper_title: A Surgical Navigation System for Guiding Exact Cochleostomy Using Auditory Feedback: A Clinical Feasibility Study

paper_content:

In cochlear implantation (CI), the insertion of the electrode array into the appropriate compartment of the cochlea, the scala tympani, is important for an optimal hearing outcome. The current surgical technique for CI depends primarily on the surgeon's skills and experience level to achieve the correct placement of the electrode array, and the surgeon needs to confirm that the exact placement is achieved prior to completing the procedure. Thus, a surgical navigation system can help the surgeon to access the scala tympani without injuring important organs in the complex structure of the temporal bone. However, the use of a surgical microscope has restricted the effectiveness of the surgical navigation because it has been difficult to deliver the navigational information to the surgeon from outside of the surgeon's visual attention. We herein present a clinical feasibility study of an auditory feedback function developed as a computer-surgeon interface that can guide the surgeon to the preset cochleostomy location. As a result, the surgeon could confirm that the drilling point was correct, while keeping his or her eyes focused on the microscope. The proposed interface reduced the common frustration that surgeons experience when using surgical navigation during otologic surgeries.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: Touchless interaction with software in interventional radiology and surgery: a systematic literature review

paper_content:

PURPOSE ::: In this article, we systematically examine the current state of research of systems that focus on touchless human-computer interaction in operating rooms and interventional radiology suites. We further discuss the drawbacks of current solutions and underline promising technologies for future development. ::: ::: ::: METHODS ::: A systematic literature search of scientific papers that deal with touchless control of medical software in the immediate environment of the operation room and interventional radiology suite was performed. This includes methods for touchless gesture interaction, voice control and eye tracking. ::: ::: ::: RESULTS ::: Fifty-five research papers were identified and analyzed in detail including 33 journal publications. Most of the identified literature (62 %) deals with the control of medical image viewers. The others present interaction techniques for laparoscopic assistance (13 %), telerobotic assistance and operating room control (9 % each) as well as for robotic operating room assistance and intraoperative registration (3.5 % each). Only 8 systems (14.5 %) were tested in a real clinical environment, and 7 (12.7 %) were not evaluated at all. ::: ::: ::: CONCLUSION ::: In the last 10 years, many advancements have led to robust touchless interaction approaches. However, only a few have been systematically evaluated in real operating room settings. Further research is required to cope with current limitations of touchless software interfaces in clinical environments. The main challenges for future research are the improvement and evaluation of usability and intuitiveness of touchless human-computer interaction and the full integration into productive systems as well as the reduction of necessary interaction steps and further development of hands-free interaction.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Auditory support for navigated radiofrequency ablation

paper_content:

Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements.
---
paper_title: Augmented-reality visualizations guided by cognition: perceptual heuristics for combining visible and obscured information

paper_content:

One unique feature of mixed and augmented reality (MR/AR) systems is that hidden and occluded objects an be readily visualized. We call this specialized use of MR/AR, obscured information visualization (OIV). In this paper, we describe the beginning of a research program designed to develop such visualizations through the use of principles derived from perceptual psychology and cognitive science. In this paper we surveyed the cognitive science literature as it applies to such visualization tasks, described experimental questions derived from these cognitive principles, and generated general guidelines that can be used in designing future OIV systems (as well improving AR displays more generally). We also report the results from an experiment that utilized a functioning AR-OIV system: we found that in relative depth judgment, subjects reported rendered objects as being in front of real-world objects, except when additional occlusion and motion cues were presented together.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory support for navigated radiofrequency ablation

paper_content:

Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements.
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: NASA-Task Load Index (NASA-TLX); 20 years later

paper_content:

NASA-TLX is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards. The years of research that precede...
---
paper_title: Less is sometimes more: a comparison of distance-control and navigated-control concepts of image-guided navigation support for surgeons.

paper_content:

Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ...
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience.

paper_content:

In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures.
---
paper_title: A Surgical Navigation System for Guiding Exact Cochleostomy Using Auditory Feedback: A Clinical Feasibility Study

paper_content:

In cochlear implantation (CI), the insertion of the electrode array into the appropriate compartment of the cochlea, the scala tympani, is important for an optimal hearing outcome. The current surgical technique for CI depends primarily on the surgeon's skills and experience level to achieve the correct placement of the electrode array, and the surgeon needs to confirm that the exact placement is achieved prior to completing the procedure. Thus, a surgical navigation system can help the surgeon to access the scala tympani without injuring important organs in the complex structure of the temporal bone. However, the use of a surgical microscope has restricted the effectiveness of the surgical navigation because it has been difficult to deliver the navigational information to the surgeon from outside of the surgeon's visual attention. We herein present a clinical feasibility study of an auditory feedback function developed as a computer-surgeon interface that can guide the surgeon to the preset cochleostomy location. As a result, the surgeon could confirm that the drilling point was correct, while keeping his or her eyes focused on the microscope. The proposed interface reduced the common frustration that surgeons experience when using surgical navigation during otologic surgeries.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: Auditory support for resection guidance in navigated liver surgery

paper_content:

Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery

paper_content:

OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery.
---
paper_title: A Telerobotic System for Transnasal Surgery

paper_content:

Mechanics-based models of concentric tube continuum robots have recently achieved a level of sophistication that makes it possible to begin to apply these robots to a variety of real-world clinical scenarios. Endonasal skull base surgery is one such application, where their small diameter and tentacle like dexterity are particularly advantageous. In this paper we provide the medical motivation for an endonasal surgical robot featuring concentric tube manipulators, and describe our model-based design and teleoperation methods, as well as a complete system incorporating image-guidance. Experimental demonstrations using a laparoscopic training task, a cadaver reachability study, and a phantom tumor resection experiment illustrate that both novice and expert users can effectively teleoperate the system, and that skull base surgeons can use the robot to achieve their objectives in a realistic surgical scenario.
---
paper_title: The perceived urgency of auditory warning alarms used in the hospital operating room is inappropriate

paper_content:

PURPOSE ::: To examine the perceived urgency of 13 auditory warning alarms commonly occurring in the hospital operating room. ::: ::: ::: METHODS ::: Undergraduate students, who were naïve with respect to the clinical situation associated with the alarms, judged perceived urgency of each alarm on a ten-point scale. ::: ::: ::: RESULTS ::: The perceived urgency of the alarms was not consistent with the actual urgency of the clinical situation that triggers it. In addition, those alarms indicating patient condition were generally perceived as less urgent than those alarms indicating the operation of equipment. Of particular interest were three sets of alarms designed by equipment manufacturers to indicate specific priorities for action. Listeners did not perceive any differences in the urgency of the 'information only', 'medium' and 'high' priority alarms of two of the monitors with all judged as low to moderate in urgency. In contrast, the high priority alarm of the third monitor was judged as significantly more urgent than its low and medium urgency counterparts. ::: ::: ::: CONCLUSION ::: The alarms currently in use do not convey the intended sense of urgency to naïve listeners, and this holds even for two sets of alarms designed specifically by manufacturers to convey different levels of urgency.
---
paper_title: Intraoperative image-guided navigation system: development and applicability in 65 patients undergoing liver surgery

paper_content:

BACKGROUND ::: Image-guided systems have recently been introduced for their application in liver surgery. We aimed to identify and propose suitable indications for image-guided navigation systems in the domain of open oncologic liver surgery and, more specifically, in the setting of liver resection with and without microwave ablation. ::: ::: ::: METHOD ::: Retrospective analysis was conducted in patients undergoing liver resection with and without microwave ablation using an intraoperative image-guided stereotactic system during three stages of technological development (accuracy: 8.4 ± 4.4 mm in phase I and 8.4 ± 6.5 mm in phase II versus 4.5 ± 3.6 mm in phase III). It was evaluated, in which indications image-guided surgery was used according to the different stages of technical development. ::: ::: ::: RESULTS ::: Between 2009 and 2013, 65 patients underwent image-guided surgical treatment, resection alone (n = 38), ablation alone (n = 11), or a combination thereof (n = 16). With increasing accuracy of the system, image guidance was progressively used for atypical resections and combined microwave ablation and resection instead of formal liver resection (p < 0.0001). ::: ::: ::: CONCLUSION ::: Clinical application of image guidance is feasible, while its efficacy is subject to accuracy. The concept of image guidance has been shown to be increasingly efficient for selected indications in liver surgery. While accuracy of available technology is increasing pertaining to technological advancements, more and more previously untreatable scenarios such as multiple small, bilobar lesions and so-called vanishing lesions come within reach.
---
paper_title: Localized Intraoperative Virtual Endoscopy (LIVE) for Surgical Guidance in 16 Skull Base Patients

paper_content:

IMPORTANCE ::: Previous preclinical studies of localized intraoperative virtual endoscopy-image-guided surgery (LIVE-IGS) for skull base surgery suggest a potential clinical benefit. ::: ::: ::: OBJECTIVE ::: The first aim was to evaluate the registration accuracy of virtual endoscopy based on high-resolution magnetic resonance imaging under clinical conditions. The second aim was to implement and assess real-time proximity alerts for critical structures during skull base drilling. ::: ::: ::: DESIGN AND SETTING ::: Patients consecutively referred for sinus and skull base surgery were enrolled in this prospective case series. ::: ::: ::: PARTICIPANTS ::: Five patients were used to check registration accuracy and feasibility with the subsequent 11 patients being treated under LIVE-IGS conditions with presentation to the operating surgeon (phase 2). ::: ::: ::: INTERVENTION ::: Sixteen skull base patients were endoscopically operated on by using image-based navigation while LIVE-IGS was tested in a clinical setting. ::: ::: ::: MAIN OUTCOME AND MEASURES ::: Workload was quantitatively assessed using the validated National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire. ::: ::: ::: RESULTS ::: Real-time localization of the surgical drill was accurate to ~1 to 2 mm in all cases. The use of 3-mm proximity alert zones around the carotid arteries and optic nerve found regular clinical use, as the median minimum distance between the tracked drill and these structures was 1 mm (0.2-3.1 mm) and 0.6 mm (0.2-2.5 mm), respectively. No statistical differences were found in the NASA-TLX indicators for this experienced surgical cohort. ::: ::: ::: CONCLUSIONS AND RELEVANCE ::: Real-time proximity alerts with virtual endoscopic guidance was sufficiently accurate under clinical conditions. Further clinical evaluation is required to evaluate the potential surgical benefits, particularly for less experienced surgeons or for teaching purposes.
---
paper_title: Influence of music on operation theatre staff.

paper_content:

Background and Objective: The purpose of the study was to evaluate the perception of influence of music among surgeons, anesthesiologist and nurses in our hospital as well as to critically evaluate whether music can be used as an aid in improving the work efficiency of medical personnel in the operation theatre (OT). Materials and Methods: A prospective, questionnaire-based cross-sectional study was conducted. A total of 100 randomly selected subjects were interviewed, which included 44 surgeons, 25 anesthesiologists and 31 nurses. Statistical package for social sciences (SPSS) Windows Version 16 software was used for statistical evaluation. Results: Most of the OT medical personnel were found to be aware of the beneficial effects of music, with 87% consenting to the playing of music in the OT. It was also found that most participants agreed to have heard music on a regular basis in the OT, while 17% had heard it whenever they have been to the OT. Conclusions: Majority of the respondent's preferred playing music in the OT which helped them relax. It improved the cognitive function of the listeners and created a sense of well being among the people and elevated mood in them. Music helped in reducing the autonomic reactivity of theatre personnel in stressful surgeries allowing them to approach their surgeries in a more thoughtful and relaxed manner. Qualitative, objective and comprehensive effect of specific music types varied with different individuals. Music can aid in improving the work efficiency of medical personnel in the OT. The study has reinforced the beneficial effects of playing music in the OT outweighing its deleterious outcomes.
---
paper_title: Comparison and Evaluation of Sonification Strategies for Guidance Tasks

paper_content:

This article aims to reveal the efficiency of sonification strategies in terms of rapidity, precision and overshooting in the case of a one-dimensional guidance task. The sonification strategies are based on the four main perceptual attributes of a sound (i.e. pitch, loudness, duration/tempo and timbre) and classified with respect to the presence or not of one or several auditory references. Perceptual evaluations are used to display the strategies in a precision/rapidity space and enable prediction of user behavior for a chosen sonification strategy. The evaluation of sonification strategies constitutes a first step toward general guidelines for sound design in interactive multimedia systems that involve guidance issues.
---
paper_title: The process of sonification design for guidance tasks

paper_content:

This article deals with the process of sonification design for guidance tasks. It presents several studies that aim at overcoming two major problems of sonification: the aesthetics of sound design and the lack of a general method. On the basis of these studies, it proposes some guidelines for the generalization of the sonification process. First, it introduces the need to disassociate data and display dimensions; then, it proposes a method to classify and evaluate sound strategies; finally, it introduces a method for the customization of the sound design. The whole process is based on the identification and the manipulation of particular sound morphologies.
---
paper_title: Warning navigation system using real-time safe region monitoring for otologic surgery

paper_content:

Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.
---
paper_title: An overview of systems for CT- and MRI-guided percutaneous needle placement in the thorax and abdomen

paper_content:

BACKGROUND ::: Minimally invasive biopsies, drainages and therapies in the soft tissue organs of the thorax and abdomen are typically performed through a needle, which is inserted percutaneously to reach the target area. The conventional workflow for needle placement employs an iterative freehand technique. This article provides an overview of needle-placement systems developed to improve this method. ::: ::: ::: METHODS ::: An overview of systems for needle placement was assembled, including those found in scientific publications and patents, as well as those that are commercially available. The systems are categorized by function and tabulated. ::: ::: ::: RESULTS ::: Over 40 systems were identified, ranging from simple passive aids to fully actuated robots. ::: ::: ::: CONCLUSIONS ::: The overview shows a wide variety of developed systems with growing complexity. However, given that only a few systems have reached commercial availability, it is clear that the technical community is struggling to develop solutions that are adopted clinically. Copyright © 2014 John Wiley & Sons, Ltd.
---
paper_title: Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency

paper_content:

This paper presents an experimental study of the effects of individual sound parameters on perceived (psychoacoustic) urgency. Experimental Series 1 showed that fundamental frequency, harmonic series, amplitude envelope shape, and delayed harmonics all have clear and consistent effects on perceived urgency. Experimental Series 2 showed that temporal and melodic parameters such as speed, rhythm, pitch range, and melodic structure also have clear and consistent effects on perceived urgency. The final experiment tested a set of 13 auditory warnings generated by an application of the earlier experimental findings. The urgency rank ordering of this warning set was predicted, and the correlation between the predicted and the obtained order was highly significant. The results of these experiments have a widespread application in the improvement of existing auditory warning systems and the design of new systems, where the psychoacoustic and psychological appropriateness of warnings could be enhanced.
---
paper_title: Auditory feedback to support image-guided medical needle placement

paper_content:

Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle.
---
paper_title: Objective evaluation of the effect of noise on the performance of a complex laparoscopic task.

paper_content:

BACKGROUND ::: Noise in operating rooms has been found to be much higher than the recommended level of 45 dB. The aim of this study was to objectively evaluate the effect of noise and music on the performance of a complex surgical task. ::: ::: ::: METHODS ::: Twelve surgeons with varying experience in laparoscopic suturing undertook 3 sutures in a laparoscopic trainer under 3 conditions: quiet, noise at 80 to 85 dB, and music. Other than the test conditions, all other conditions were standardized. A validated motion analysis system was used to assess performance. The tasks were recorded by video and played back to 2 blinded observers who rated the surgeons' performance on a global rating scale by observing the tasks for accuracy, knot quality, and number of nonpurposeful movements. ::: ::: ::: RESULTS ::: Time taken for the tasks (P=.78), total number of movements (P=.78), total path length (P=.47), global score (P=.54), accuracy, and knot quality remained unchanged across the 3 conditions. The main study measures had a high test-retest reliability and internal consistency. No learning effect was seen across the 3 conditions. ::: ::: ::: CONCLUSIONS ::: Surgeons can effectively "block out" noise and music. This is probably due to the high levels of concentration required for the performance of a complex surgical task. Future research should focus on the effect of these conditions on communication in the operating room.
---
paper_title: Effect of sensory substitution on suture-manipulation forces for robotic surgical systems.

paper_content:

OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
---
paper_title: Surgical navigation using audio feedback.

paper_content:

Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware.
---
paper_title: Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality

paper_content:

Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task.
---
paper_title: Validation of Exposure Visualization and Audible Distance Emission for Navigated Temporal Bone Drilling in Phantoms

paper_content:

BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling.
---
paper_title: Image-Guided Interventions: Technology Review and Clinical Applications

paper_content:

Image-guided interventions are medical procedures that use computer-based systems to provide virtual image overlays to help the physician precisely visualize and target the surgical site. This field has been greatly expanded by the advances in medical imaging and computing power over the past 20 years. This review begins with a historical overview and then describes the component technologies of tracking, registration, visualization, and software. Clinical applications in neurosurgery, orthopedics, and the cardiac and thoracoabdominal areas are discussed, together with a description of an evolving technology named Natural Orifice Transluminal Endoscopic Surgery (NOTES). As the trend toward minimally invasive procedures continues, image-guided interventions will play an important role in enabling new procedures, while improving the accuracy and success of existing approaches. Despite this promise, the role of image-guided systems must be validated by clinical trials facilitated by partnerships between scie...
---

Title: A Survey of Auditory Display in Image-Guided Interventions

Section 1: Introduction
Description 1: Introduce the topic of image-guided interventions and the growing importance of auditory display in this field.

Section 2: Literature search
Description 2: Explain the methodology used to search for relevant literature, including search terms and databases.

Section 3: Eligibility criteria
Description 3: Describe the criteria used to include or exclude literature from the review.

Section 4: Data extraction
Description 4: Detail the process and categories used for extracting data from the selected articles.

Section 5: Results
Description 5: Summarize the findings of the literature review, including the number and type of articles reviewed.

Section 6: Interventional tasks supported by auditory display
Description 6: Discuss the various interventional tasks that are supported by auditory display as found in the literature.

Section 7: Clinical motivations for exploring auditory display
Description 7: Explore the motivations for developing auditory display systems for clinical use.

Section 8: Methods of auditory display for image-guided interventions
Description 8: Outline the different methods of auditory display used in image-guided interventions, such as alerts, auditory icons, and parameter-mapping models.

Section 9: Experimental designs and findings
Description 9: Describe the different experimental designs used in the evaluation of auditory display systems and summarize the findings.

Section 10: Discussion
Description 10: Provide a discussion on the state of the art, including benefits, drawbacks, and future research directions.

Section 11: Conclusion
Description 11: Conclude the review by summarizing the key points and potential for future development in auditory display for image-guided interventions.