Registration Technology of Augmented Reality in Oral Medicine: A Review
---
paper_title: Augmented reality: a review.

paper_content:

Augmented reality is a technology that overlays digital information on objects or places in the real world for the purpose of enhancing the user experience. It is not virtual reality, that is, the technology that creates a totally digital or computer created environment. Augmented reality, with its ability to combine reality and digital information, is being studied and implemented in medicine, marketing, museums, fashion, and numerous other areas. This article presents an overview of augmented reality, discussing what it is, how it works, its current implementations, and its potential impact on libraries.
---
paper_title: Pneumatic Actuation in Haptic-Enabled Medical Simulators: A Review

paper_content:

Haptic feedback is essential for achieving virtual and augmented reality (VR/AR) systems with high fidelity and realism. Haptic-enabled VR/AR systems offer a proficient environment in which to enhance the skills of medical practitioners. While pneumatic actuation is traditionally used in heavy automation industries, haptics researchers have utilized pneumatic techniques to produce various haptic effects, i.e., stiffness feedback and contour control, and these techniques have shown promising results in the medical domain. In this paper, we focus on the use of pneumatic-actuated haptic systems in VR/AR-based medical simulators. We begin with the taxonomy of the physical-virtual continuum and discuss the role of pneumatics in medical haptic systems. Furthermore, we propose the conceptual design architecture of a pneumatic haptic system. In addition, the systematic state-of-the-art role of pneumatic haptics in medical systems is presented for different categories. In this paper, we provide a systematic review and discuss the study of pneumatics to provide guidelines for the design and development of pneumatic haptic medical systems.
---
paper_title: Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR

paper_content:

Although Augmented Reality technology was first developed over forty years ago, there has been little survey work giving an overview of recent research in the field. This paper reviews the ten-year development of the work presented at the ISMAR conference and its predecessors with a particular focus on tracking, interaction and display research. It provides a roadmap for future augmented reality research which will be of great value to this relatively young field, and also for helping researchers decide which topics should be explored when they are beginning their own studies in the area.
---
paper_title: A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard

paper_content:

Shipbuilding companies are upgrading their inner workings in order to create Shipyards 4.0, where the principles of Industry 4.0 are paving the way to further digitalized and optimized processes in an integrated network. Among the different Industry 4.0 technologies, this paper focuses on augmented reality, whose application in the industrial field has led to the concept of industrial augmented reality (IAR). This paper first describes the basics of IAR and then carries out a thorough analysis of the latest IAR systems for industrial and shipbuilding applications. Then, in order to build a practical IAR system for shipyard workers, the main hardware and software solutions are compared. Finally, as a conclusion after reviewing all the aspects related to IAR for shipbuilding, it proposed an IAR system architecture that combines cloudlets and fog computing, which reduce latency response and accelerate rendering tasks while offloading compute intensive tasks from the cloud.
---
paper_title: The World Oral Health Report 2003: continuous improvement of oral health in the 21st century--the approach of the WHO Global Oral Health Programme.

paper_content:

Chronic diseases and injuries are the leading health problems in all but a few parts of the world. The rapidly changing disease patterns throughout the world are closely linked to changing lifestyles, which include diets rich in sugars, widespread use of tobacco, and increased consumption of alcohol. In addition to socio-environmental determinants, oral disease is highly related to these lifestyle factors, which are risks to most chronic diseases as well as protective factors such as appropriate exposure to fluoride and good oral hygiene. Oral diseases qualify as major public health problems owing to their high prevalence and incidence in all regions of the world, and as for all diseases, the greatest burden of oral diseases is on disadvantaged and socially marginalized populations. The severe impact in terms of pain and suffering, impairment of function and effect on quality of life must also be considered. Traditional treatment of oral diseases is extremely costly in several industrialized countries, and not feasible in most low-income and middle-income countries. The WHO Global Strategy for Prevention and Control of Noncommunicable Diseases, added to the common risk factor approach is a new strategy for managing prevention and control of oral diseases. The WHO Oral Health Programme has also strengthened its work for improved oral health globally through links with other technical programmes within the Department for Noncommunicable Disease Prevention and Health Promotion. The current oral health situation and development trends at global level are described and WHO strategies and approaches for better oral health in the 21st century are outlined.
---
paper_title: A non-rigid medical image registration method based on improved linear elastic model

paper_content:

Abstract Non-rigid medical image registration is an important research project of medical image processing; it is the basis of medical image fusion. Relative to rigid image, the deformation of non-rigid image is more serious and more complicated. According to the characteristics of non-rigid medical image deformation, this paper proposes an adaptive non-rigid medical image registration algorithm. Firstly, it is based on global registration; secondly, it is about extracting feature points of global registration image and the reference image, and then generating irregular triangle grid according to extracted feature points. Finally, local accurate image registration is achieved using the minimum potential energy as a similar measure. Experimental results show that relative to the traditional non-rigid registration algorithm, this algorithm not only ensures the registration accuracy but also enhances the robustness and anti-noise of registration algorithm.
---
paper_title: Epigenetics and its implications for oral health

paper_content:

Abstract Background Over the last decade, we have been able to understand the role of various genetic processes involved in the oral diseases. However, In past few years, much of stresses has been paid to understand the regulation of these genetic processes controlled by the epigenetic mechanisms. Epigenetic processes regulate the gene activity without altering underlying DNA sequences, through mechanisms including DNA methylation, histone modifications, and non-coding RNAs. These epigenetic processes are significantly associated with various oral health problems including periodontitis, oral cancer. Highlights Different strategies like Genome-wide DNA methylation studies, microRNA based profiling studies, next-generation RNA sequencing analyses have been proven to be helpful in establishing the relationship among these epigenetic processes and gene expression. A few studies also suggest an emerging association of various epigenetic biomarkers in oral cancer and periodontal diseases. Conclusion Epigenetic mechanisms are highly sensitive and reversible, thereby offers significant potential for their use to develop novel diagnostic tools, pharmacotherapies and personalized based treatment care. However, there is lack of integrated approaches combining epigenomics with proteomics and genomics comparing the usefulness of epigenetic biomarkers in oral health research.
---
paper_title: Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization

paper_content:

We compare two technological approaches to augmented reality for 3-D medical visualization: optical and video see-through devices. We provide a context to discuss the technology by reviewing several medical applications of augmented-reality re search efforts driven by real needs in the medical field, both in the United States and in Europe. We then discuss the issues for each approach, optical versus video, from both a technology and human-factor point of view. Finally, we point to potentially promising future developments of such devices including eye tracking and multifocus planes capabilities, as well as hybrid optical/video technology.
---
paper_title: Joint 3D Reconstruction of a Static Scene and Moving Objects

paper_content:

We present a technique for simultaneous 3D reconstruction of static regions and rigidly moving objects in a scene. An RGB-D frame is represented as a collection of features, which are points and planes. We classify the features into static and dynamic regions and grow separate maps, static and object maps, for each of them. To robustly classify the features in each frame, we fuse multiple RANSAC-based registration results obtained by registering different groups of the features to different maps, including (1) all the features to the static map, (2) all the features to each object map, and (3) subsets of the features, each forming a segment, to each object map. This multi-group registration approach is designed to overcome the following challenges: scenes can be dominated by static regions, making object tracking more difficult; and moving object might have larger pose variation between frames compared to the static regions. We show qualitative results from indoor scenes with objects in various shapes. The technique enables on-the-fly object model generation to be used for robotic manipulation.
---
paper_title: Heterogeneous Multi-View Information Fusion: Review of 3-D Reconstruction Methods and a New Registration with Uncertainty Modeling

paper_content:

We consider a multisensor network fusion framework for 3-D data registration using inertial planes, the underlying geometric relations, and transformation model uncertainties. We present a comprehensive review of 3-D reconstruction methods and registration techniques in terms of the underlying geometric relations and associated uncertainties in the registered images. The 3-D data registration and the scene reconstruction task using a set of multiview images are an essential goal of structure-from-motion algorithms that still remains challenging for many applications, such as surveillance, human motion and behavior modeling, virtual-reality, smart-rooms, health-care, teleconferencing, games, human–robot interaction, medical imaging, and scene understanding. We propose a framework to incorporate measurement uncertainties in the registered imagery, which is a critical issue to ensure the robustness of these applications but is often not addressed. In our test bed environment, a network of sensors is used where each physical node consists of a coupled camera and associated inertial sensor (IS)/inertial measurement unit. Each camera-IS node can be considered as a hybrid sensor or fusion-based virtual camera. The 3-D scene information is registered onto a set of virtual planes defined by the IS. The virtual registrations are based on using the homography calculated from 3-D orientation data provided by the IS. The uncertainty associated with each 3-D point projected onto the virtual planes is modeled using statistical geometry methods. Experimental results demonstrate the feasibility and effectiveness of the proposed approach for multiview reconstruction with sensor fusion.
---
paper_title: Lower Oral Health Literacy may Lead to Poorer Oral Health Outcomes

paper_content:

Article Title and Bibliographic Information Oral health literacy and associated oral conditions: A systematic review. Firmino RT, Ferreira FM, Palva SM, Granvlle-Garcia AF, Fraiz FC, Martins CC. JADA 2017; 148(8):604-13. Source of Funding Government and foundation. This study was supported by the Brazilian Ministry of Education Coordination for the Improvement of Higher Education Personnel, the Minas Gerais State Research Foundation, and the Brazilian National Council for Scientific and Technological Development. Type of Study/Design Systematic review.
---
paper_title: AR in VR: assessing surgical augmented reality visualizations in a steerable virtual reality environment

paper_content:

Augmented reality (AR) has emerged as a promising approach to support surgeries; however, its application in real world scenarios is still very limited. Besides sophisticated registration tasks that need to be solved, surgical AR visualizations have not been studied in a standardized and comparative manner. To foster the development of future AR applications, a steerable framework is urgently needed to rapidly evaluate new visualization techniques, explore their individual parameter spaces and define relevant application scenarios. Inspired by its beneficial usage in the automotive industry, the underlying concept of virtual reality (VR) is capable of transforming complex real environments into controllable virtual ones. We present an interactive VR framework, called Augmented Visualization Box (AVB), in which visualizations for AR can be systematically investigated without explicitly performing an error-prone registration. As use case, a virtual laparoscopic scenario with anatomical surface models was created in a computer game engine. In a study with eleven surgeons, we analyzed this VR setting under different environmental factors and its applicability for a quantitative assessment of different AR overlay concepts. According to the surgeons, the visual impression of the VR scene is mostly influenced by 2D surface details and lighting conditions. The AR evaluation shows that, depending on the visualization used and its capability to encode depth, 37% to 91% of the experts made wrong decisions, but were convinced of their correctness. These results show that surgeons have more confidence in their decisions, although they are wrong, when supported by AR visualizations. With AVB, intraoperative situations are realistically simulated to quantitatively benchmark current AR overlay methods. Successful surgical task execution in an AR system can only be facilitated if visualizations are customized toward the surgical task.
---
paper_title: Augmented reality (AR) and virtual reality (VR) applied in dentistry

paper_content:

The OSCE is a reliable evaluation method to estimate the preclinical examination of dental students. The most ideal assessment for OSCE is used the augmented reality simulator to evaluate. This literature review investigated a recently developed in virtual reality (VR) and augmented reality (AR) starting of the dental history to the progress of the dental skill. As result of the lacking of technology, it needs to depend on other device increasing the success rate and decreasing the risk of the surgery. The development of tracking unit changed the surgical and educational way. Clinical surgery is based on mature education. VR and AR simultaneously affected the skill of the training lesson and navigation system. Widely, the VR and AR not only applied in the dental training lesson and surgery, but also improved all field in our life.
---
paper_title: Knowledge, attitude and practice among Health Visitors in the United Kingdom toward children's oral health

paper_content:

OBJECTIVES ::: The purpose of this study was to determine knowledge, attitude, and practical behavior of health visitors regarding children's oral health in the United Kingdom (UK). ::: ::: ::: METHODS ::: A web-based self-administered survey with 18 closed and 2 open ended questions was distributed to a convenience sample of approximately 9,000 health visitors who were currently employed in the United Kingdom and a member of the Institute of Health Visiting. ::: ::: ::: RESULTS ::: A total of 1,088 health visitors completed the survey, resulting in a response rate of 12%. One-third of the health visitors reported that they had not received oral health training previously. Almost all agreed that oral health advice/promotion should be included in their routine health visiting contacts. Previous oral health training/education was associated with an increase in oral health knowledge; confidence in entering a discussion with parents/caregivers and willingness to be involved in dental referral process. ::: ::: ::: CONCLUSIONS ::: The results of our study support the need for health visitors to receive oral health training in oral health promotion including oral health risk assessment, guidance on evidence based up-to-date prevention measures, increasing the dental attendance prevalence at early stages and awareness of including specific oral health guidelines/fact sheets into their regular practice.
---
paper_title: Virtual reality and haptics for dental surgery: a personal review

paper_content:

Virtual reality and haptics are novel human-machine interaction technologies which have shown a good application potential in many fields such as medical, entertainment, manufacturing, and education areas in the last 20 years. Especially for dental surgery simulation and training, they provide a new and low-cost approach whereby dentists can practice procedures as many times as they want at no incremental cost and training can take place anywhere. This paper provides a comprehensive survey of virtual reality and haptics for dental surgery simulation and training. Some new ideas and recent research progress are investigated, the major research efforts and their typical systems are introduced, and the involved major research issues are summarized, and finally future trends and conclusions are discussed.
---
paper_title: Robust augmented reality registration method for localization of solid organs’ tumors using CT-derived virtual biomechanical model and fluorescent fiducials

paper_content:

BACKGROUND ::: Augmented reality (AR) is the fusion of computer-generated and real-time images. AR can be used in surgery as a navigation tool, by creating a patient-specific virtual model through 3D software manipulation of DICOM imaging (e.g., CT scan). The virtual model can be superimposed to real-time images enabling transparency visualization of internal anatomy and accurate localization of tumors. However, the 3D model is rigid and does not take into account inner structures' deformations. We present a concept of automated AR registration, while the organs undergo deformation during surgical manipulation, based on finite element modeling (FEM) coupled with optical imaging of fluorescent surface fiducials. ::: ::: ::: METHODS ::: Two 10 × 1 mm wires (pseudo-tumors) and six 10 × 0.9 mm fluorescent fiducials were placed in ex vivo porcine kidneys (n = 10). Biomechanical FEM-based models were generated from CT scan. Kidneys were deformed and the shape changes were identified by tracking the fiducials, using a near-infrared optical system. The changes were registered automatically with the virtual model, which was deformed accordingly. Accuracy of prediction of pseudo-tumors' location was evaluated with a CT scan in the deformed status (ground truth). In vivo: fluorescent fiducials were inserted under ultrasound guidance in the kidney of one pig, followed by a CT scan. The FEM-based virtual model was superimposed on laparoscopic images by automatic registration of the fiducials. ::: ::: ::: RESULTS ::: Biomechanical models were successfully generated and accurately superimposed on optical images. The mean measured distance between the estimated tumor by biomechanical propagation and the scanned tumor (ground truth) was 0.84 ± 0.42 mm. All fiducials were successfully placed in in vivo kidney and well visualized in near-infrared mode enabling accurate automatic registration of the virtual model on the laparoscopic images. ::: ::: ::: CONCLUSIONS ::: Our preliminary experiments showed the potential of a biomechanical model with fluorescent fiducials to propagate the deformation of solid organs' surface to their inner structures including tumors with good accuracy and automatized robust tracking.
---
paper_title: Improving the oral health of older people: the approach of the WHO Global Oral Health Programme.

paper_content:

The proportion of older people continues to grow worldwide, especially in developing countries. Non-communicable diseases are fast becoming the leading causes of disability and mortality, and in coming decades health and social policy-makers will face tremendous challenges posed by the rapidly changing burden of chronic diseases in old age. Chronic disease and most oral diseases share common risk factors. Globally, poor oral health amongst older people has been particularly evident in high levels of tooth loss, dental caries experience, and the prevalence rates of periodontal disease, xerostomia and oral precancer/cancer. The negative impact of poor oral conditions on the quality of life of older adults is an important public health issue, which must be addressed by policy-makers. The means for strengthening oral health programme implementation are available; the major challenge is therefore to translate knowledge into action programmes for the oral health of older people. The World Health Organization recommends that countries adopt certain strategies for improving the oral health of the elderly. National health authorities should develop policies and measurable goals and targets for oral health. National public health programmes should incorporate oral health promotion and disease prevention based on the common risk factors approach. Control of oral disease and illness in older adults should be strengthened through organization of affordable oral health services, which meet their needs. The needs for care are highest among disadvantaged, vulnerable groups in both developed and developing countries. In developing countries the challenges to provision of primary oral health care are particularly high because of a shortage of dental manpower. In developed countries reorientation of oral health services towards prevention should consider oral care needs of older people. Education and continuous training must ensure that oral health care providers have skills in and a profound understanding of the biomedical and psychosocial aspects of care for older people. Research for better oral health should not just focus on the biomedical and clinical aspects of oral health care; public health research needs to be strengthened particularly in developing countries. Operational research and efforts to translate science into practice are to be encouraged. WHO supports national capacity building in the oral health of older people through intercountry and interregional exchange of experiences.
---
paper_title: Augmented reality in dentistry: a current perspective

paper_content:

AbstractAugmentation reality technology offers virtual information in addition to that of the real environment and thus opens new possibilities in various fields. The medical applications of augmentation reality are generally concentrated on surgery types, including neurosurgery, laparoscopic surgery and plastic surgery. Augmentation reality technology is also widely used in medical education and training. In dentistry, oral and maxillofacial surgery is the primary area of use, where dental implant placement and orthognathic surgery are the most frequent applications. Recent technological advancements are enabling new applications of restorative dentistry, orthodontics and endodontics. This review briefly summarizes the history, definitions, features, and components of augmented reality technology and discusses its applications and future perspectives in dentistry.
---
paper_title: Smart Glasses for Neurosurgical Navigation by Augmented Reality.

paper_content:

BACKGROUND ::: Wearable devices with heads-up displays or smart glasses can overlay images onto the sight of the wearer. This technology has never been applied to surgical navigation. ::: ::: ::: OBJECTIVE ::: To assess the applicability and accuracy of smart glasses for augmented reality (AR)-based neurosurgical navigation. ::: ::: ::: METHODS ::: Smart glasses were applied to AR-based neurosurgical navigation. Three-dimensional computer graphics were created based on preoperative magnetic resonance images and visualized in see-through smart glasses. Optical markers were attached to the smart glasses and the patient's head for accurate navigation. Two motion capture cameras were used for registration and continuous monitoring of the location of the smart glasses in relation to the patient's head. After the accuracy was assessed with a phantom, this technique was applied in 2 patients with brain tumors located in the brain surface. ::: ::: ::: RESULTS ::: A stereoscopic view by image overlay through the smart glasses was successful in the phantom and in both patients. Hands-free neuronavigation inside the operative field was available from any angles and distances. The targeting error in the phantom measured in 75 points ranged from 0.2 to 8.1 mm (mean 3.1 ± 1.9 mm, median 2.7 mm). The intraoperative targeting error between the visualized and real locations in the 2 patients (measured in 40 points) ranged from 0.6 to 4.9 mm (mean 2.1 ± 1.1 mm, median 1.8 mm). ::: ::: ::: CONCLUSION ::: Smart glasses enabled AR-based neurosurgical navigation in a hands-free fashion. Stereoscopic computer graphics of targeted brain tumors corresponding to the surgical field were clearly visualized during surgery.
---
paper_title: Implementation of augmented reality in operative dentistry learning

paper_content:

Objective ::: To evaluate the efficacy of augmented reality (AR) in the gaining of knowledge and skills amongst dental students in the design of cavity preparations and analyse their degree of satisfaction. ::: ::: Material and methods ::: AR cavity models were prepared for use with computers and mobile devices. Forty-one students were divided into two groups (traditional teaching methods vs AR). Questionnaires were designed to evaluate knowledge and skills, with the administration of a satisfaction questionnaire for those using AR. The degree of compliance with the standards in cavity design was assessed. The Mann-Whitney U-test was used to compare knowledge and skills between the two groups, and the Wilcoxon test was applied to compare intragroup differences. The chi-square test in turn was used to compare the qualitative parameters of the cavity designs between the groups. Statistical significance was considered for P<.05 in all cases. ::: ::: Results ::: No significant differences were observed in level of knowledge before, immediately after or 6 months after teaching between the two groups (P>.05). Although the results corresponding to most of the studied skills parameters were better in the experimental group, significant differences (P<.05) were only founded for cavity depth and extent for Class I and divergence of the buccal and lingual walls for the Class II. The experience was rated as favourable or very favourable by 100% of the participants. The students showed preference for computers (60%) vs mobile devices (10%). ::: ::: Conclusions ::: The AR techniques favoured the gaining of knowledge and skills and were regarded as a useful tool by the students.
---
paper_title: Indoor Augmented Reality Using Deep Learning for Industry 4.0 Smart Factories

paper_content:

This paper proposes to design, develop and implement a fast and markerless mobile augmented reality system to achieve the registration for, the visualization of, and the interaction with machines in indoor smart factories with Industry 4.0 vision. A lightweight deep-learning image detection module based on MobileNets running on mobile devices is used to detect/recognize different machines and different portions of machines. Internet of Things (IoT) networking allows machines and sensors in machines to report data, such as machine settings and machine states, to the cloud-side server. Thus, augmented information associated with a machine portion can be derived from the server and superimposed with the portion image shown on the device display. Furthermore, interaction methods based on touch gestures and distance calculation are also implemented. A prototype system is developed and tested in a mechanical workshop for the purpose of validation and evaluation. The system is shown to achieve high detection accuracy, intuitive visualization, and unique interaction modes.
---
paper_title: A review of computer-aided oral and maxillofacial surgery: planning, simulation and navigation.

paper_content:

ABSTRACTIntroduction: Currently, oral and maxillofacial surgery (OMFS) still poses a significant challenge for surgeons due to the anatomic complexity and limited field of view of the oral cavity. With the great development of computer technologies, he computer-aided surgery has been widely used for minimizing the risks and improving the precision of surgery.Areas covered: The major goal of this paper is to provide a comprehensive reference source of current and future development of computer-aided OMFS including surgical planning, simulation and navigation for relevant researchers.Expert commentary: Compared with the traditional OMFS, computer-aided OMFS overcomes the disadvantage that the treatment on the region of anatomically complex maxillofacial depends almost exclusively on the experience of the surgeon.
---
paper_title: Mobile Augmented Reality Survey: From Where We Are to Where We Go

paper_content:

The boom in the capabilities and features of mobile devices, like smartphones, tablets, and wearables, combined with the ubiquitous and affordable Internet access and the advances in the areas of cooperative networking, computer vision, and mobile cloud computing transformed mobile augmented reality (MAR) from science fiction to a reality. Although mobile devices are more constrained computationalwise from traditional computers, they have a multitude of sensors that can be used to the development of more sophisticated MAR applications and can be assisted from remote servers for the execution of their intensive parts. In this paper, after introducing the reader to the basics of MAR, we present a categorization of the application fields together with some representative examples. Next, we introduce the reader to the user interface and experience in MAR applications and continue with the core system components of the MAR systems. After that, we discuss advances in tracking and registration, since their functionality is crucial to any MAR application and the network connectivity of the devices that run MAR applications together with its importance to the performance of the application. We continue with the importance of data management in MAR systems and the systems performance and sustainability, and before we conclude this survey, we present existing challenging problems.
---
paper_title: A 3D GIS-based interactive registration mechanism for outdoor augmented reality system

paper_content:

An interactive method is proposed for registration between virtual and real objects.We propose an outdoor augmented reality (AR) system, named ARGIS.The proposed ARGIS combines 3DGIS, HMD, GPS, IMU and interactive devices.ARGIS overcomes the drawback of traditional vision tracking method for registration.We developed a prototype and tested it on the campus of Peking University, China. Registering virtual objects with reference to their corresponding real-world objects plays a key role in augmented reality (AR) system. Although there have been a lot of work on using vision-based method to perform registration for indoor AR system, it is very difficult to apply such registration method for outdoor AR system due to the inability to modify the objects in outdoor environment and the huge range of working area. 3D Geographic Information System (GIS) is capable of providing an outdoor virtual geographic environment where users are located at, which may provide users with a corresponding virtual object for the one in the physical world. In this study, a 3D GIS-based registration mechanism is proposed for outdoor AR system. Specifically, an easy-use interactive method for precise registration was developed to improve the performance of the registration. To implement the registration mechanism, an outdoor AR system built upon 3D GIS was developed, named Augmented Reality Geographical Information System (ARGIS). ARGIS has the capability of performing precise registration in outdoor environment without using traditional vision tracking method, which thus enables users to arbitrarily manipulate the system. A prototype was developed to conduct experiment on the campus of Peking University, Beijing, China to test the proposed registration mechanism. The experiment shows that the developed registration mechanism is feasible and efficient in the outdoor environment. The ARGIS is expected to enrich the applications of outdoor AR system, including but not limited to underground facility mapping, emergency rescue and urban planning.
---
paper_title: Intraoperative augmented reality with heads-up displays in maxillofacial surgery: a systematic review of the literature and a classification of relevant technologies

paper_content:

Abstract Although the term augmented reality appears increasingly in published studies, the real-time, image-guided (so-called ‘hands-free’ and ‘heads-up’) surgery techniques are often confused with other virtual imaging procedures. A systematic review of the literature was conducted to classify augmented reality applications in the fields of maxillofacial surgery. Publications containing the terms ‘augmented reality’, ‘hybrid reality’, and ‘surgery’ were sought through a search of three medical databases, covering the years 1995–2018. Thirteen publications containing enough usable data to perform a comparative analysis of methods used and results obtained were identified. Five out of 13 described a method based on a hands-free and heads-up augmented reality approach using smart glasses or a headset combined with tracking. Most of the publications reported a minimum error of less than 1 mm between the virtual model and the patient. Augmented reality during surgery may be classified into four categories: heads-up guided surgery (type I) with tracking (Ia) or without tracking (Ib); guided surgery using a semi-transparent screen (type II); guided surgery based on the digital projection of images onto the patient (type III); and guided surgery based on the transfer of digital data to a monitor display (type IV).
---
paper_title: Augmented Reality with Virtual Cerebral Aneurysms: A Feasibility Study

paper_content:

Objective Advanced magnetic resonance imaging (MRI) acquisition methods such as 4D phase contrast magnetic resonance imaging (4D pcMRI) have found widespread applications in assessing hemodynamics in cerebral aneurysms. One of the goals is the identification of a hemodynamic scenario predictive of aneurysm rupture. Because of the technical complexity the 4D pcMRI technology, challenges exist in extracting and communicating the wealth of information contained in the MRI image data, in particular, to a multidisciplinary team of health professionals and basic scientists. Methods Here we introduce a new resource in the form of a commercially available optical see-through head-mounted display, which allows augmenting the real world by computer-generated virtual objects. Results An image processing algorithm was presented for integrating 3D surface models derived from 3D rotational angiography data, 3D time-of-flight MRI data, and 4D pcMRI data into the augmented reality environment. This new approach was successfully evaluated on blood flow data acquired before and after flow diverter treatment in 6 patients harboring a cerebral aneurysm. Conclusions Augmented reality may become an additional means for visualizing complex medical data, in particular complex flows as illustrated here.
---
paper_title: Combining intraoperative ultrasound brain shift correction and augmented reality visualizations: a pilot study of eight cases

paper_content:

We present our work investigating the feasibility of combining intraoperative ultrasound for brain shift correction and augmented reality (AR) visualization for intraoperative interpretation of patient-specific models in image-guided neurosurgery (IGNS) of brain tumors. We combine two imaging technologies for image-guided brain tumor neurosurgery. Throughout surgical interventions, AR was used to assess different surgical strategies using three-dimensional (3-D) patient-specific models of the patient’s cortex, vasculature, and lesion. Ultrasound imaging was acquired intraoperatively, and preoperative images and models were registered to the intraoperative data. The quality and reliability of the AR views were evaluated with both qualitative and quantitative metrics. A pilot study of eight patients demonstrates the feasible combination of these two technologies and their complementary features. In each case, the AR visualizations enabled the surgeon to accurately visualize the anatomy and pathology of interest for an extended period of the intervention. Inaccuracies associated with misregistration, brain shift, and AR were improved in all cases. These results demonstrate the potential of combining ultrasound-based registration with AR to become a useful tool for neurosurgeons to improve intraoperative patient-specific planning by improving the understanding of complex 3-D medical imaging data and prolonging the reliable use of IGNS.
---
paper_title: Improved Demons Technique with Orthogonal Gradient Information for Medical Image Registration

paper_content:

Accurate registration is crucial for medical image analysis. In this letter, we proposed an improved Demons technique (IDT) for medical image registration. The IDT improves registration quality using orthogonal gradient information. The advantage of the proposed IDT is assessed using 14 medical image pairs. Experimental results show that the proposed technique provides about 8% improvement over existing Demons-based techniques in terms of registration accuracy.
---
paper_title: Oral Health—A Neglected Aspect of Subjective Well-Being in Later Life

paper_content:

Objectives ::: This study examined whether oral health is a neglected aspect of subjective well-being (SWB) among older adults. The key research question was whether deterioration in oral health among dentate older adults living in England was associated with decreases in SWB, using measures of eudemonic, evaluative, and affective dimensions of well-being. ::: ::: ::: Methods ::: This secondary analysis used data from the third (2006-2007) and fifth (2010-2011) waves of respondents aged 50 and older from the English Longitudinal Study of Ageing (ELSA). We fitted multivariable regression models to examine the effects of changes in oral impacts on daily life and edentulism (complete tooth loss) on SWB (quality of life, life satisfaction, and depressive symptomatology). ::: ::: ::: Results ::: A worsening in both oral health measures was associated with an increase in depressive symptoms even after adjusting for time-varying confounders including declining health, activities of daily living, and reduced social support. Becoming edentate was also associated with decreases in quality of life and life satisfaction. ::: ::: ::: Discussion ::: A deterioration in oral health and oral health-related quality of life increases the risk of depressive symptoms among older adults and highlights the importance of oral health as a determinant of subjective well-being in later life.
---
paper_title: Augmented Reality-Based Personalized Virtual Operative Anatomy for Neurosurgical Guidance and Training

paper_content:

This paper presents a novel augmented reality (AR) interactive environment for neurosurgical training. Comparing with traditional virtual reality based neurosurgical simulator, our system provides a more natural and intuitive fashion for surgeons. To achieve holographic visualization of virtual brain on 3D-printed skull (workspace), the first step is to reconstruct the personalized anatomy structure from segmented MR imaging. Then, tailored to the computational power of HoloLens, we employ the mass-spring method to model the mechanical response of brain. After that, a precise registration method is employed to map the virtual-real spatial information, which can overlay the virtual operative brain on workspace. In addition, bimanual haptic interface is also integrated into our simulator, which is more similar with real neurosurgery. In experiments, we conduct accuracy validation on our registration method, as well as the validity test on the developed simulators. The results demonstrate that our simulator can provide high-accuracy augmented visualization effects and deep immersion for novice surgeons.
---
paper_title: Revolutionizing Restorative Dentistry: An Overview

paper_content:

The field of science and research is ever changing and the scientific discipline of prosthodontics is no exception. The practice of prosthodontics and the supporting technology involved has evolved tremendously from the traditional to the contemporary. As a result of continual developments in technology, new methods of production and new treatment concepts may be expected. Clinicians must have certain basic knowledge if they are to benefit from these new procedures. This article reviews the contemporary trends in the field of prosthodontics and provides an insight into what one might expect in the near future.
---
paper_title: Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study

paper_content:

PURPOSE ::: We present a novel augmented reality (AR) surgical navigation system based on ultrasound-assisted registration for pedicle screw placement. This system provides the clinically desired targeting accuracy and reduces radiation exposure. ::: ::: ::: METHODS ::: Ultrasound (US) is used to perform registration between preoperative computed tomography (CT) images and patient, and the registration is performed by least-squares fitting of these two three-dimensional (3D) point sets of anatomical landmarks taken from US and CT images. An integral videography overlay device is calibrated to accurately display naked-eye 3D images for surgical navigation. We use a 3.0-mm Kirschner wire (K-wire) instead of a pedicle screw in this study, and the K-wire is calibrated to obtain its orientation and tip location. Based on the above registration and calibration, naked-eye 3D images of the planning path and the spine are superimposed onto patient in situ using our AR navigation system. Simultaneously, a 3D image of the K-wire is overlaid accurately on the real one to guide the insertion procedure. The targeting accuracy is evaluated postoperatively by performing a CT scan. ::: ::: ::: RESULTS ::: An agar phantom experiment was performed. Eight K-wires were inserted successfully after US-assisted registration, and the mean targeting error and angle error were 3.35 mm and [Formula: see text], respectively. Furthermore, an additional sheep cadaver experiment was performed. Four K-wires were inserted successfully. The mean targeting error was 3.79 mm and the mean angle error was [Formula: see text], and US-assisted registration yielded better targeting results than skin markers-based registration (targeting errors: 2.41 vs. 5.18 mm, angle errors: [Formula: see text] vs. [Formula: see text]. ::: ::: ::: CONCLUSION ::: Experimental outcomes demonstrate that the proposed navigation system has acceptable targeting accuracy. In particular, the proposed navigation method reduces repeated radiation exposure to the patient and surgeons. Therefore, it has promising prospects for clinical use.
---
paper_title: Nutrition and oral health in sport: time for action

paper_content:

Elite athletes have poor oral health, common negative self-reported performance effects and a lifelong burden for disease management.1–3 One central determinant of oral health is nutrition and in particular the consumption of dietary sugars, acidic food and drinks and eating behaviours. However, nutrition is also clearly of major importance in managing and improving athlete performance. Therefore, this call to action is to recommend strategies to prevent or mitigate the risk from nutrition to the short-term health and performance of athletes and to the long-term oral health of all active individuals. The target audience includes athletes, health professionals (nutritionists and dieticians, sport and exercise medicine practitioners, dental professionals), scientists in related fields, sport policy organisations and industry. ::: ::: Given that the mechanisms by which carbohydrate-containing or acidic supplements can affect oral health are well established, it is important to develop mitigation strategies to reduce the risk of possible adverse oral effects while maintaining the performance benefits. Based on best evidence for effectiveness,4 …
---
paper_title: Bibliometric Analysis of the Top 100 Most Cited Articles in the First 50 Years of Heart Transplantation

paper_content:

The world celebrates over 50 years since the first human-to-human heart transplant. Bibliometric analysis is a statistical concept that has recently evolved, enabling scientists to study citation patterns and identify characteristics of highly cited scholarly work in different fields. Although it has been widely utilized, such analyses have not been conducted to date on heart transplant literature. We sought to assess the characteristics of the top 100 most referenced citations in the field of heart transplantation. We searched the Scopus database ( www.scopus.com ) to identify all articles relating to heart transplantation. The articles were arranged in descending order from most cited to least cited and selected articles were scrutinized for data extraction. One hundred articles were included in the final list. Of the total 40,660 citations identified, 3,210 (8.0%) were self-citations, which impacted the final rank order. The articles were published in 25 different journals between 1960 and 2013. The most productive 5-year time period was between 2000 and 2005, when 24 of the 100 most cited publications were produced. There was no correlation between the journals' impact factors and the number of articles produced per journal. The presence and type of funding were not associated with the number of citations. Over 85% of first and senior investigators were men. In conclusion, our study highlights key features of the most highly cited scientific literature on heart transplantation and provides insights into trends of published work in this field. Additionally, this work may serve as a useful guide to researchers and funding bodies by highlighting the most prolific areas of cardiac transplant research to date.
---
paper_title: Emerging trends in regenerative medicine : a scientometric analysis in CiteSpace

paper_content:

INTRODUCTION ::: Regenerative medicine involves research in a number of fields and disciplines such as stem cell research, tissue engineering and biological therapy in general. As research in these areas advances rapidly, it is critical to keep abreast of emerging trends and critical turns of the development of the collective knowledge. ::: ::: ::: AREAS COVERED ::: A progressively synthesized network is derived from 35,963 original research and review articles that cite 3875 articles obtained from an initial topic search on regenerative medicine between 2000 and 2011. CiteSpace is used to facilitate the analysis of the intellectual structure and emerging trends. ::: ::: ::: EXPERT OPINION ::: A major ongoing research trend is concerned with finding alternative reprogramming techniques as well as refining existing ones for induced pluripotent stem cells (iPSCs). A more recent emerging trend focuses on the structural and functional equivalence between iPSCs and human embryonic stem cells and potential clinical and therapeutic implications on regenerative medicine in a long run. The two trends overlap in terms of what they cite, but they are distinct and have different implications on future research. Visual analytics of the literature provides a valuable, timely, repeatable and flexible approach in addition to traditional systematic reviews so as to track the development of new emerging trends and identify critical evidence.
---
paper_title: JOURNAL SELF‐CITATION RATES AND IMPACT FACTORS IN DENTISTRY, ORAL SURGERY, AND MEDICINE: A 3‐YEAR BIBLIOMETRIC ANALYSIS

paper_content:

Abstract Objectives The objective of this study was to evaluate the evolution of journal self-citation rates (SCRs) and impact factors (IFs) over time in the dental journals. Materials and Methods The journals listed under the category “dentistry, oral surgery, and medicine” in the Journal Citation Reports for the years 2014-2016 were screened for the following: citations and self-citations to years used in IF calculation, IF, IF without self-citations (corrected IF), SCR, and quartile of the IF distribution the journal occupied (Q1-Q4). Additional data regarding the number of issues published annually, journal's access options, and country of publication were extracted from the “Journal Profile Page.” Results The median SCR significantly declined between 2014 and 2016 (13.725 [0-57.049], 12.687 [0-52.326], and 10.667 [0-53.208], respectively [ P r P > .05) were found for SCR-IF and SCR-corrected IF for all years. Conclusions There was a statistically significant decrease in SCR during the observation period. SCR was not correlated to IF of dental journals. Subspecialty journals and journals publishing more frequently presented significantly higher SCRs. These findings suggest favorable publishing conditions and citation practices in the dental literature.
---
paper_title: A Visual Analysis of Research on Information Security Risk by Using CiteSpace

paper_content:

The security of information issue has become a global problem and has risen the concerns of both researchers and practitioners. When security incidents occur, there is a risk that national military secrets or confidential information of corporations will leak out and cause serious damages to the collective. This paper is aimed to explore the knowledge structure, development, and the future trend of information security area by providing a comprehensive review of the present information security risk (ISR) literature. The visualization analysis was conducted on journal literatures from the Web of Science, IEEE, ACM, and Scopus database, and the results were mapped into the I-model. According to 2748 articles, evaluation methods, e.g., frequency statistics, clustering coefficient, and centrality calculation, are employed to analyze all of the interrelated matrixes that are supported by CiteSpace. Some useful outcomes of a variety of objectives are shown under a significant level, such as author, country/territory, cluster, institute, as well as reference. A synthetical analysis has demonstrated the future research trend on ISR. As for researchers and practitioners, this paper suggests an analysis of integrated visualization in terms of the knowledge and innovation based on the area of ISR.
---
paper_title: Most cited publications in oral and maxillofacial surgery: a bibliometric analysis

paper_content:

BACKGROUND ::: Citation is one of the most important forms of acknowledgment and recognition received by our peers in academia. This study identifies and characterizes the current topmost highly cited publications in journals specifically dedicated to the specialty. The purpose of this study is to identify, using the citation count, works that have made key contributions in the field and to provide insight into the direction the specialty has taken in the last 30 years. ::: ::: ::: METHODS ::: Four journals that were dedicated to the field of oral and maxillofacial surgery (OMFS) were selected. The SCOPUS database was utilized to perform a citation analysis on the top 200 publications in the month of May, 2017. Each publication was individually reviewed for the number of citations, the source journal of the manuscript, its year of publications, the article type, the country of origin, the study type, and the level of evidence. We categorized every manuscript within a subspecialty. ::: ::: ::: RESULTS ::: The number of citations ranged from 2824 to 118 over the last four decades. The majority of the manuscripts were published in the Journal of Oral and Maxillofacial Surgery (60.5%) followed by the International Journal of Oral and Maxillofacial Surgery (25%). Only 3% of these publications were of level I evidence. The most frequent field of publication was benign pathology (18.5%) followed by dental implantology (19%) and then craniomaxillofacial trauma (12%) and craniomaxillofacial deformities (12%). The majority of the publications were articles, of which 59.5% were case-control, case series, or cohort studies. Nine of the top 20 most cited articles and 12.5% of all the most cited papers were related to osteoradionecrosis, osteochemonecrosis, and bisphosphonates. The majority of these publications originated in the USA (44%), followed by Sweden (9.20%), the Netherlands and UK (7.76%), and Germany (5.82%). ::: ::: ::: CONCLUSION ::: Despite considerable advances in the quality of both clinical and laboratory research, only six level I studies were identified. Osteonecrosis/bisphosphonates and platelet-rich plasma were frequent topics indicating the recent interests in these fields. The USA has continued to play a major role in the research; however, there needs to be more international, multi-institutional cooperative collaborations.
---
paper_title: Science Mapping: A Systematic Review of the Literature

paper_content:

Abstract Purpose We present a systematic review of the literature concerning major aspects of science mapping to serve two primary purposes: First, to demonstrate the use of a science mapping approach to perform the review so that researchers may apply the procedure to the review of a scientific domain of their own interest, and second, to identify major areas of research activities concerning science mapping, intellectual milestones in the development of key specialties, evolutionary stages of major specialties involved, and the dynamics of transitions from one specialty to another. Design/methodology/approach We first introduce a theoretical framework of the evolution of a scientific specialty. Then we demonstrate a generic search strategy that can be used to construct a representative dataset of bibliographic records of a domain of research. Next, progressively synthesized co-citation networks are constructed and visualized to aid visual analytic studies of the domain’s structural and dynamic patterns and trends. Finally, trajectories of citations made by particular types of authors and articles are presented to illustrate the predictive potential of the analytic approach. Findings The evolution of the science mapping research involves the development of a number of interrelated specialties. Four major specialties are discussed in detail in terms of four evolutionary stages: conceptualization, tool construction, application, and codification. Underlying connections between major specialties are also explored. The predictive analysis demonstrates citations trajectories of potentially transformative contributions. Research limitations The systematic review is primarily guided by citation patterns in the dataset retrieved from the literature. The scope of the data is limited by the source of the retrieval, i.e. the Web of Science, and the composite query used. An iterative query refinement is possible if one would like to improve the data quality, although the current approach serves our purpose adequately. More in-depth analyses of each specialty would be more revealing by incorporating additional methods such as citation context analysis and studies of other aspects of scholarly publications. Practical implications The underlying analytic process of science mapping serves many practical needs, notably bibliometric mapping, knowledge domain visualization, and visualization of scientific literature. In order to master such a complex process of science mapping, researchers often need to develop a diverse set of skills and knowledge that may span multiple disciplines. The approach demonstrated in this article provides a generic method for conducting a systematic review. Originality/value Incorporating the evolutionary stages of a specialty into the visual analytic study of a research domain is innovative. It provides a systematic methodology for researchers to achieve a good understanding of how scientific fields evolve, to recognize potentially insightful patterns from visually encoded signs, and to synthesize various information so as to capture the state of the art of the domain.
---
paper_title: The 100 most cited articles on oral cancer

paper_content:

BACKGROUND ::: Citations analysis is one of the most widely used bibliometric tools to evaluate the academic importance of a study in a specific area of knowledge. The objective of this study was to identify the 100 most cited articles on oral cancer and to analyse their principal characteristics. ::: ::: ::: METHODS ::: We performed a literature search in the Web of Science database using the Science Citation Index Expanded tool to determine the number of citations of all articles on oral cancer identified up to 10 August 2017. The 100 most referenced articles were then selected, and the following information was gathered: ranking based on the number of citations; citation density; citations in Scopus; number and names of the authors; language and year of publication; country and institution of origin; financial support; journal of publication, with its impact factor, category and quartile; type of research; evidence level; and area of study. ::: ::: ::: RESULTS ::: The number of citations of the 100 articles varied from 1959 to 165, and the number of authors varied from a single author to 23. The oldest article was from 1948, and the most recent was from 2013. All the studies were published in English, the majority (56%) was from the United States, and 80% were published in journals in the first quartile. ::: ::: ::: CONCLUSIONS ::: The majority of articles were of studies that had received financial support, were published in journals with a high impact factor and were focussed on the aetiology and pathogenesis of oral cancer.
---
paper_title: A non-rigid medical image registration method based on improved linear elastic model

paper_content:

Abstract Non-rigid medical image registration is an important research project of medical image processing; it is the basis of medical image fusion. Relative to rigid image, the deformation of non-rigid image is more serious and more complicated. According to the characteristics of non-rigid medical image deformation, this paper proposes an adaptive non-rigid medical image registration algorithm. Firstly, it is based on global registration; secondly, it is about extracting feature points of global registration image and the reference image, and then generating irregular triangle grid according to extracted feature points. Finally, local accurate image registration is achieved using the minimum potential energy as a similar measure. Experimental results show that relative to the traditional non-rigid registration algorithm, this algorithm not only ensures the registration accuracy but also enhances the robustness and anti-noise of registration algorithm.
---
paper_title: Accuracy validation for medical image registration algorithms: a review.

paper_content:

Accuracy validation is essential to clinical application of medical image registration techniques. Registration validation remains a challenging problem in practice mainly due to lack of 'ground truth'.In this paper, an overview of current validation methods for medical image registration is presented with detailed discussion of their benefits and drawbacks.Special focus is on non-rigid registration validation. Promising solution is also discussed.
---
paper_title: Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation

paper_content:

Augmented reality for monocular laparoscopy from a preoperative volume such as CT is achieved in two steps. The first step is to segment the organ in the preoperative volume and reconstruct its 3D model. The second step is to register the preoperative 3D model to an initial intraoperative laparoscopy image. To date, there does not exist an automatic initial registration method to solve the second step for the liver in the de facto operating room conditions of monocular laparoscopy. Existing methods attempt to solve for both deformation and pose simultaneously, leading to nonconvex problems with no optimal solution algorithms. We propose in contrast to break the problem down into two parts, solving for (i) deformation and (ii) pose. Part (i) simulates biomechanical deformations from the preoperative to the intraoperative state to predict the liver’s unknown intraoperative shape by modeling gravity, the abdominopelvic cavity’s pressure and boundary conditions. Part (ii) rigidly registers the simulated shape to the laparoscopy image using contour cues. Our formulation leads to a well-posed problem, contrary to existing methods. This is because it exploits strong environment priors to complement the weak laparoscopic visual cues. Quantitative results with in silico and phantom experiments and qualitative results with laparosurgery images for two patients show that our method outperforms the state-of-the-art in accuracy and registration time.
---
paper_title: Deformable Medical Image Registration: A Survey

paper_content:

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.
---
paper_title: Multimodal medical image registration using Particle Swarm Optimization: A review

paper_content:

Intensity-based registration techniques have been increasingly used in multimodal image co-registration, which is a fundamental task in medical imaging, because it enables to integrate different images into a single representation such that complementary information can be easily accessed and fused. These schemes usually require the optimization of some similarity metric (e.g., Mutual Information) calculated on the input images. Local optimization methods often do not obtain good results, possibly leading to premature convergence to local optima, especially with non-smooth fitness functions. In these cases, we can adopt global optimization methods, and Swarm Intelligence techniques represent a very effective and efficient solution. This paper focuses on biomedical image registration using Particle Swarm Optimization (PSO). Several literature approaches are critically reviewed, by investigating modifications and hybridizations with Evolutionary Strategies. Since biomedical image registration represents a challenging clinical task, the experimental findings encourage further research studies in the near future.
---
paper_title: Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia.

paper_content:

BACKGROUND ::: Through three-dimensional real time imaging, augmented reality (AR) can provide an overlay of the anatomical structure, or visual cues for specific landmarks. In this study, an AR Toolkit was used for distraction osteogenesis with hemifacial microsomia to define the mandibular osteotomy line and assist with intraoral distractor placement. ::: ::: ::: METHODS ::: 20 patients with hemifacial microsomia were studied and were randomly assigned to experimental and control groups. Pre-operative computed tomography was used in both groups, whereas AR was used in the experimental group. Afterwards, pre- and post-operative computed tomographic scans of both groups were superimposed, and several measurements were made and analysed. ::: ::: ::: RESULTS ::: Both the conventional method and AR technique achieved proper positioning of the osteotomy planes, although the AR was more accurate. The difference in average vertical distance from the coronoid and condyle process to the pre- and post-operative cutting planes was significant (p < 0.01) between the two groups, whereas no significant difference (p > 0.05) was observed in the average angle between the two planes. The difference in deviations between the intersection points of the overlaid mandible across two cutting planes was also significant (p < 0.01). ::: ::: ::: CONCLUSION ::: This study reports on an efficient approach for guiding intraoperative distraction osteogenesis. Augmented reality tools such as the AR Toolkit may be helpful for precise positioning of intraoral distractors in patients with hemifacial microsomia in craniofacial surgery.
---
paper_title: A Novel Dental Implant Guided Surgery Based on Integration of Surgical Template and Augmented Reality

paper_content:

Background ::: Stereoscopic visualization concept combined with head-mounted displays may increase the accuracy of computer-aided implant surgery. ::: ::: Purpose ::: The aim of this study was to develop an augmented reality-based dental implant placement system and evaluate the accuracy of the virtually planned versus the actual prepared implant site created in vitro. ::: ::: Materials and Methods ::: Four fully edentulous mandibular and four partially edentulous maxillary duplicated casts were used. Six implants were planned in the mandibular and four in the maxillary casts. A total of 40 osteotomy sites were prepared in the casts using stereolithographic template integrated with augmented reality-based surgical simulation. During the surgery, the dentist could be guided accurately through a head-mounted display by superimposing the virtual auxiliary line and the drill stop. The deviation between planned and prepared positions of the implants was measured via postoperative computer tomography generated scan images. ::: ::: Results ::: Mean and standard deviation of the discrepancy between planned and prepared sites at the entry point, apex, angle, depth, and lateral locations were 0.50 ± 0.33 mm, 0.96 ± 0.36 mm, 2.70 ± 1.55°, 0.33 ± 0.27 mm, and 0.86 ± 0.34 mm, respectively, for the fully edentulous mandible, and 0.46 ± 0.20 mm, 1.23 ± 0.42 mm, 3.33 ± 1.42°, 0.48 ± 0.37 mm, and 1.1 ± 0.39 mm, respectively, for the partially edentulous maxilla. There was a statistically significant difference in the apical deviation between maxilla and mandible in this surgical simulation (p < .05). ::: ::: Conclusions ::: Deviation of implant placement from planned position was significantly reduced by integrating surgical template and augmented reality technology.
---
paper_title: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

paper_content:

AIM ::: We present a newly designed, localiser-free, head-mounted system featuring augmented reality as an aid to maxillofacial bone surgery, and assess the potential utility of the device by conducting a feasibility study and validation. ::: ::: ::: METHODS ::: Our head-mounted wearable system facilitating augmented surgery was developed as a stand-alone, video-based, see-through device in which the visual features were adapted to facilitate maxillofacial bone surgery. We implement a strategy designed to present augmented reality information to the operating surgeon. LeFort1 osteotomy was chosen as the test procedure. The system is designed to exhibit virtual planning overlaying the details of a real patient. We implemented a method allowing performance of waferless, augmented-reality assisted bone repositioning. In vitro testing was conducted on a physical replica of a human skull, and the augmented reality system was used to perform LeFort1 maxillary repositioning. Surgical accuracy was measured with the aid of an optical navigation system that recorded the coordinates of three reference points (located in anterior, posterior right, and posterior left positions) on the repositioned maxilla. The outcomes were compared with those expected to be achievable in a three-dimensional environment. Data were derived using three levels of surgical planning, of increasing complexity, and for nine different operators with varying levels of surgical skill. ::: ::: ::: RESULTS ::: The mean error was 1.70 ± 0.51 mm. The axial errors were 0.89 ± 0.54 mm on the sagittal axis, 0.60 ± 0.20 mm on the frontal axis, and 1.06 ± 0.40 mm on the craniocaudal axis. The simplest plan was associated with a slightly lower mean error (1.58 ± 0.37 mm) compared with the more complex plans (medium: 1.82 ± 0.71 mm; difficult: 1.70 ± 0.45 mm). The mean error for the anterior reference point was lower (1.33 ± 0.58 mm) than those for both the posterior right (1.72 ± 0.24 mm) and posterior left points (2.05 ± 0.47 mm). No significant difference in terms of error was noticed among operators, despite variations in surgical experience. Feedback from surgeons was acceptable; all tests were completed within 15 min and the tool was considered to be both comfortable and usable in practice. ::: ::: ::: CONCLUSION ::: We used a new localiser-free, head-mounted, wearable, stereoscopic, video see-through display to develop a useful strategy affording surgeons access to augmented reality information. Our device appears to be accurate when used to assist in waferless maxillary repositioning. Our results suggest that the method can potentially be extended for use with many surgical procedures on the facial skeleton. Further, our positive results suggest that it would be appropriate to proceed to in vivo testing to assess surgical accuracy under real clinical conditions.
---
paper_title: Image-Based tracking of the teeth for orthodontic augmented reality

paper_content:

We present image-based methods for tracking teeth in a video image with respect to a CT scan of the jaw, in order to enable a novel light-weight augmented reality (AR) system in orthodontistry. Its purpose is guided bracket placement in orthodontic correction. In this context, our goal is to determine the position of the patient maxilla and mandible in a video image solely based on a CT scan. This is suitable for image guidance through an overlay of the video image with the planned position of brackets in a monocular AR system. Our tracking algorithm addresses the contradicting requirements of robustness, accuracy and performance in two problem-specific formulations. First, we exploit a distance-based modulation of two iso-surfaces from the CT image to approximate the appearance of the gum line. Second, back-projection of previous video frames to an iso-surface is used to account for recently placed brackets. In combination, this novel algorithm allowed us to track several sequences of three patient videos of real procedures, despite difficult lighting conditions. Paired with a systematic evaluation, we were able to show practical feasibility of such a system.
---
paper_title: Registration strategy using occlusal splint based on augmented reality for mandibular angle oblique split osteotomy.

paper_content:

BACKGROUND ::: An augmented reality tool allows for visual tracking of real anatomic structures in superposition with volume-rendered computed tomographic or magnetic resonance imaging scans and thus can be used for navigated translocation of important structures during operation. In this feasibility study, ARToolKit was used in mandibular angle oblique split osteotomy to define the cutting planes according to an operative plan. ::: ::: ::: METHODS ::: We overlay the operative plan on the model of a mandible made by rapid prototyping technology, and the technology was successfully used in 15 patients. Before the operation, all patients underwent computed tomographic scan, and dental casts were prepared by surgeons. Then, surgeons make the occlusal splint according to a dental cast to fix the marker, which can be recognized by the ARToolKit. The occlusal splint and marker were transformed to three-dimensional data using a laser scanner, and a programmer that runs on a personal computer named Rapidform matches the marker and the mandible image to generate the virtual image. By this step, the virtual image describing the marker, occlusal splint, and the mandible image of the patient are integrated. During the operation, the operative plan was overlaid on the rapid prototyping model of the mandible as soon as the ARToolKit recognized the marker. ::: ::: ::: RESULTS ::: The technology was successfully used in 15 patients; the virtual image of the mandible and the cutting-plane both overlaid the real model of the mandible. ::: ::: ::: CONCLUSIONS ::: This study has reported a new and effective way for mandibular angle oblique split osteotomy, and using occlusal splint might be a powerful option for the registration of augmented reality. Augmented reality tools like ARToolKit may be helpful for control of maxillary translocation in orthognathic surgery.
---
paper_title: Development of AR Display System for Dental Surgical Simulator

paper_content:

In this paper, we describe about a display system for dental surgical simulator. The design concept of our system is that the system allows the user to learning the dental surgical methods with the real hand and body posture. Therefore our system set a display, which shows virtual teeth model and real teeth and gums, close to user’s hand position and allows the user to manipulating object directly.
---
paper_title: A non-rigid medical image registration method based on improved linear elastic model

paper_content:

Abstract Non-rigid medical image registration is an important research project of medical image processing; it is the basis of medical image fusion. Relative to rigid image, the deformation of non-rigid image is more serious and more complicated. According to the characteristics of non-rigid medical image deformation, this paper proposes an adaptive non-rigid medical image registration algorithm. Firstly, it is based on global registration; secondly, it is about extracting feature points of global registration image and the reference image, and then generating irregular triangle grid according to extracted feature points. Finally, local accurate image registration is achieved using the minimum potential energy as a similar measure. Experimental results show that relative to the traditional non-rigid registration algorithm, this algorithm not only ensures the registration accuracy but also enhances the robustness and anti-noise of registration algorithm.
---
paper_title: Comparison of denture models by means of micro computed tomography

paper_content:

The production of dental inlays and crowns requires precise information on patients’ teeth morphology. The conventional method is the preparation of impressions using mold materials, e.g. a silicone impression material. The disadvantage of this technique is the human choke impulse and the flavor of the material. These discomforts can be avoided by methods where a three-dimensional scanner is used for recording the teeth morphology. The present study reveals the accuracy of three model types, namely conventional impression, rapid prototyping using an oral scanner C.O.S., 3M (Schweiz) AG and milling from a proprietary resin using the oral scanner iTero, Straumann Holding AG. For each method five models were fabricated from a steel reference (standard). Using a nanotom m (phoenixǀx-ray, GE Sensing and Inspection Technologies GmbH), three-dimensional micro computed tomography data sets of the standard and the 15 models were recorded and landmark distances within the data sets were measured with sub-pixel accuracy. To verify these results a coordinate measuring machine (Leitz PMM 864, Hexagon Metrology GmbH) based on tactile detection was used for the measurement of the landmark distances, and a correction of the distances measured by the nanotom m was arranged. The nanotom data sets of the 15 models were also compared to the standard by means of a non-rigid registration algorithm. The calculated deformation field exhibited mean pixel displacement values of (0.19 ± 0.09) mm for the C.O.S. models, (0.12 ± 0.07) mm for the gypsum models and (0.19 ± 0.12) mm for the i-Tero models.
---
paper_title: Non-rigid contour-to-pixel registration of photographic and quantitative light-induced fluorescence imaging of decalcified teeth

paper_content:

Quantitative light-induced fluorescence (QLF) is widely used to assess the damage of a tooth due to decalcification. In digital photographs, decalcification appears as white spot lesions, i.e. white spots on the tooth surface. We propose a novel multimodal registration approach for the matching of digital photographs and QLF images of decalcified teeth. The registration is based on the idea of contour-to-pixel matching. Here, the curve, which represents the shape of the tooth, is extracted from the QLF image using a contour segmentation by binarization and morphological processing. This curve is aligned to the photo with a non-rigid variational registration approach. Thus, the registration problem is formulated as minimization problem with an objective function that consists of a data term and a regularizer for the deformation. To construct the data term, the photo is pointwise classified into tooth and non-tooth regions. Then, the signed distance function of the tooth region allows to measure the mismatch between curve and photo. As regularizer a higher order, linear elastic prior is used. The resulting minimization problem is solved numerically using bilinear Finite Elements for the spatial discretization and the Gauss-Newton algorithm. The evaluation is based on 150 image pairs, where an average of 5 teeth have been captured from 32 subjects. All registrations have been confirmed correctly by a dental expert. The contour-to-pixel methods can directly be used in 3D for surface-to-voxel tasks.
---
paper_title: Atlas-Based Automatic Generation of Subject-Specific Finite Element Tongue Meshes

paper_content:

Generation of subject-specific 3D finite element (FE) models requires the processing of numerous medical images in order to precisely extract geometrical information about subject-specific anatomy. This processing remains extremely challenging. To overcome this difficulty, we present an automatic atlas-based method that generates subject-specific FE meshes via a 3D registration guided by Magnetic Resonance images. The method extracts a 3D transformation by registering the atlas’ volume image to the subject’s one, and establishes a one-to-one correspondence between the two volumes. The 3D transformation field deforms the atlas’ mesh to generate the subject-specific FE mesh. To preserve the quality of the subject-specific mesh, a diffeomorphic non-rigid registration based on B-spline free-form deformations is used, which guarantees a non-folding and one-to-one transformation. Two evaluations of the method are provided. First, a publicly available CT-database is used to assess the capability to accurately capture the complexity of each subject-specific Lung’s geometry. Second, FE tongue meshes are generated for two healthy volunteers and two patients suffering from tongue cancer using MR images. It is shown that the method generates an appropriate representation of the subject-specific geometry while preserving the quality of the FE meshes for subsequent FE analysis. To demonstrate the importance of our method in a clinical context, a subject-specific mesh is used to simulate tongue’s biomechanical response to the activation of an important tongue muscle, before and after cancer surgery.
---
paper_title: A Phase-Based Image Registration Algorithm for Dental Radiograph Identification

paper_content:

Dental radiographs have been used for the accurate assessment and treatment of dental diseases. For an accurate diagnosis, the complete geometric registration between radiographs is required. The perspective projection between two radiographs may be observed, even if they are taken from the same oral regions of the subject. This paper presents an efficient dental radiograph registration algorithm using Phase-Only Correlation (POC) function. The use of phase components in 2D (two-dimensional) discrete Fourier transforms of dental radiograph images makes possible to achieve highly robust image registration and recognition. Experimental evaluation using a dental radiograph database indicates that the proposed algorithm exhibits efficient recognition performance even for distorted radiographs.
---
paper_title: Augmented reality haptics system for dental surgical skills training

paper_content:

We have developed a virtual reality (VR) and an augmented reality (AR) dental training simulator utilizing a haptic device. The simulators utilize volumetric force feedback computation and real time modification of the volumetric data. They include a virtual mirror to facilitate indirect vision during a simulated operation. The AR environment allows students to practice surgery in correct postures by combining the 3D tooth and tool models with the real-world view and displaying the result through a video see-through head-mounted display (HMD). Preliminary results from an initial evaluation show that the system is a promising tool to supplement dental training and that there are advantages of the AR over the VR approach.
---
paper_title: Application of an augmented reality tool for maxillary positioning in orthognathic surgery - a feasibility study.

paper_content:

BACKGROUND ::: An augmented reality tool for computer assisted surgery named X-Scope allows visual tracking of real anatomical structures in superposition with volume rendered CT or MRI scans and thus can be used for navigated translocation of bony segments. ::: ::: ::: METHODS ::: In a feasibility study X-Scope was used in orthognathic surgery to control the translocation of the maxilla after Le Fort I osteotomy within a bimaxillary procedure. The situation achieved was compared with the pre-operative situation by means of cephalometric analysis on lateral and frontal cephalograms. ::: ::: ::: RESULTS ::: The technique was successfully utilized in 5 patients. Maxillary positioning using X-Scope was accomplished accurately within a range of 1mm. The tool was used in all cases in addition to the usual intra-operative splints. A stand-alone application without conventional control does not yet seem reasonable. ::: ::: ::: CONCLUSION ::: Augmented reality tools like X-Scope may be helpful for controlling maxillary translocation in orthognathic surgery. The application to other interventions in cranio-maxillofacial surgery such as Le Fort III osteotomy, fronto-orbital advancement, and cranial vault reshaping or repair may also be considered.
---
paper_title: Video see‐through augmented reality for oral and maxillofacial surgery

paper_content:

BACKGROUND ::: Oral and maxillofacial surgery has not been benefitting from image guidance techniques owing to the limitations in image registration. ::: ::: ::: METHODS ::: A real-time markerless image registration method is proposed by integrating a shape matching method into a 2D tracking framework. The image registration is performed by matching the patient's teeth model with intraoperative video to obtain its pose. The resulting pose is used to overlay relevant models from the same CT space on the camera video for augmented reality. ::: ::: ::: RESULTS ::: The proposed system was evaluated on mandible/maxilla phantoms, a volunteer and clinical data. Experimental results show that the target overlay error is about 1 mm, and the frame rate of registration update yields 3-5 frames per second with a 4 K camera. ::: ::: ::: CONCLUSIONS ::: The significance of this work lies in its simplicity in clinical setting and the seamless integration into the current medical procedure with satisfactory response time and overlay accuracy. Copyright © 2016 John Wiley & Sons, Ltd.
---
paper_title: Computer-assisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display.

paper_content:

There may well be a shift towards 3-dimensional orthognathic surgery when virtual surgical planning can be applied clinically. We present a computer-assisted protocol that uses surgical navigation supplemented by an interactive image-guided visualisation display (IGVD) to transfer virtual maxillary planning precisely. The aim of this study was to analyse its accuracy and versatility in vivo. The protocol consists of maxillofacial imaging, diagnosis, planning of virtual treatment, and intraoperative surgical transfer using an IGV display. The advantage of the interactive IGV display is that the virtually planned maxilla and its real position can be completely superimposed during operation through a video graphics array (VGA) camera, thereby augmenting the surgeon's 3-dimensional perception. Sixteen adult class III patients were treated with by bimaxillary osteotomy. Seven hard tissue variables were chosen to compare (ΔT1-T0) the virtual maxillary planning (T0) with the postoperative result (T1) using 3-dimensional cephalometry. Clinically acceptable precision for the surgical planning transfer of the maxilla (<0.35 mm) was seen in the anteroposterior and mediolateral angles, and in relation to the skull base (<0.35°), and marginal precision was seen in the orthogonal dimension (<0.64 mm). An interactive IGV display complemented surgical navigation, augmented virtual and real-time reality, and provided a precise technique of waferless stereotactic maxillary positioning, which may offer an alternative approach to the use of arbitrary splints and 2-dimensional orthognathic planning.
---
paper_title: Interactive haptic simulation of tooth extraction by a constraint-based haptic rendering approach

paper_content:

Tooth extraction is a typical process in clinical dental operations. Interactive haptic simulation of tooth extraction may provide a useful tool for dental students to learn the correct force pattern and tool posture to accomplish a safe tooth extraction. In this paper, we extended our previous configuration-based optimization approach to simulate the six Degree-of-Freedom (DoF) haptic interaction process of tooth extraction. A multi-phase model was proposed to simulate progressive changes of the connection strength between the target tooth and its surrounding gingiva. An energy accumulation model was proposed to compute the small-scale rotation and translation of the target tooth under active forces from a dental forceps. The proposed approach could support training of coordinated force and motion control skill required for tooth extraction. Experimental results validated the stability and efficiency of the proposed approach to simulate various force-displacement profiles for extracting diversified target teeth.
---
paper_title: Combining Scale-Space and Similarity-Based Aspect Graphs for Fast 3D Object Recognition

paper_content:

This paper describes an approach for recognizing instances of a 3D object in a single camera image and for determining their 3D poses. A hierarchical model is generated solely based on the geometry information of a 3D CAD model of the object. The approach does not rely on texture or reflectance information of the object's surface, making it useful for a wide range of industrial and robotic applications, e.g., bin-picking. A hierarchical view-based approach that addresses typical problems of previous methods is applied: It handles true perspective, is robust to noise, occlusions, and clutter to an extent that is sufficient for many practical applications, and is invariant to contrast changes. For the generation of this hierarchical model, a new model image generation technique by which scale-space effects can be taken into account is presented. The necessary object views are derived using a similarity-based aspect graph. The high robustness of an exhaustive search is combined with an efficient hierarchical search. The 3D pose is refined by using a least-squares adjustment that minimizes geometric distances in the image, yielding a position accuracy of up to 0.12 percent with respect to the object distance, and an orientation accuracy of up to 0.35 degree in our tests. The recognition time is largely independent of the complexity of the object, but depends mainly on the range of poses within which the object may appear in front of the camera. For efficiency reasons, the approach allows the restriction of the pose range depending on the application. Typical runtimes are in the range of a few hundred ms.
---
paper_title: Training of Tooth Scaling by Simulator -- Development of Simulator and Investigation of its Effectiveness --

paper_content:

Dental calculus, hardened plaque on the teeth, is a cause of periodontitis, or gum diseases in the mouth. For dentists and dental hygienists, it is an important task to remove the calculi from patients. The task is called "tooth scaling" and the calculi just beside the gum and between the tooth and gum are usually removed by using a hand scaler. Since the scaling by a hand scaler requires skill, apprentice dentists and dental hygienists must be trained in order to get the skill. Unfortunately in the training environment, however, there are several problems such as difficulty in collecting teeth on which various types of calculi are stuck, lack of cooperative patients, and so on. Authors have newly developed a simulator for the training by using a PHANToM and a video-see-through HMD. The training effectiveness has been evaluated by artificial calculus removal on model teeth, and subjectively evaluated by veterans. The experimental result shows the developed simulator is more effective than the conventional paint removing method.
---
paper_title: A Real-Time Three-Dimensional Tracking and Registration Method in the AR-HUD System

paper_content:

In this paper, we propose a method of 3-D tracking registration with a virtual image and the surroundings in three steps: initial registration matrix acquisition, feature extraction in a road scene, and registration of virtual information in a real environment. We obtain the initial registration matrix according to the relationship between different parts of the system. During transformation matrix solving, we do not match features. Instead, the relationship is based on the characteristic image information. Then, we interpolate another image’s corresponding coordinates and estimate the transform matrix of the map and the image. The average time of the virtual reality registration algorithm can reach 0.0781 s. The average accuracy of our registration method is less than 2 pixels. The proposed algorithm has some advantages. We use features from accelerated segment test feature extraction and the binary robust independent elementary features descriptor to efficiently extract image feature points. The algorithm is more efficient, because it eliminates the feature-matching phase. We apply the proposed virtual reality registration method to a vehicle’s augmented reality head-up display (AR-HUD). We derive the relationship between human eyes, optical system, tracking camera, and real scene in the AR-HUD system. Our experiment proves that the algorithm has a better registration effect on virtual objects, and it enhances road information. Different from the general head-up display (HUD) system, our method achieves the true integration of virtual and real world. It improves the distraction problem caused by the mismatch between the existing HUD virtual image and the real scene.
---
paper_title: Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation.

paper_content:

Abstract Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560 × 1600 elemental image resolution shows the rendering speeds of 50–60 frames per second (fps) for surface models, and 5–8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability.
---
paper_title: Augmented reality surgical navigation with accurate CBCT-patient registration for dental implant placement

paper_content:

It is challenging to achieve high implant accuracy in dental implant placement, because high risk tissues need to be avoided. In this study, we present an augmented reality (AR) surgical navigation with an accurate cone beam computed tomography (CBCT)-patient registration method to provide clinically desired dental implant accuracy. A registration device is used for registration between preoperative data and patient outside the patient’s mouth. After registration, the registration device is worn on the patient’s teeth for tracking the patient. Naked-eye 3D images of the planning path and the mandibular nerve are superimposed onto the patient in situ to form an AR scene. Simultaneously, a 3D image of the drill is overlaid accurately on the real one to guide the implant procedure. Finally, implant accuracy is evaluated postoperatively. A model experiment was performed by an experienced dentist. Totally, ten parallel pins were inserted into five 3D-printed mandible models guided by our AR navigation method and through the dentist’s experience, respectively. AR-guided dental implant placement showed better results than the dentist’s experience (mean target error = 1.25 mm vs. 1.63 mm; mean angle error = 4.03° vs. 6.10°). Experimental results indicate that the proposed method is expected to be applied in the clinic.
---
paper_title: A 3D GIS-based interactive registration mechanism for outdoor augmented reality system

paper_content:

An interactive method is proposed for registration between virtual and real objects.We propose an outdoor augmented reality (AR) system, named ARGIS.The proposed ARGIS combines 3DGIS, HMD, GPS, IMU and interactive devices.ARGIS overcomes the drawback of traditional vision tracking method for registration.We developed a prototype and tested it on the campus of Peking University, China. Registering virtual objects with reference to their corresponding real-world objects plays a key role in augmented reality (AR) system. Although there have been a lot of work on using vision-based method to perform registration for indoor AR system, it is very difficult to apply such registration method for outdoor AR system due to the inability to modify the objects in outdoor environment and the huge range of working area. 3D Geographic Information System (GIS) is capable of providing an outdoor virtual geographic environment where users are located at, which may provide users with a corresponding virtual object for the one in the physical world. In this study, a 3D GIS-based registration mechanism is proposed for outdoor AR system. Specifically, an easy-use interactive method for precise registration was developed to improve the performance of the registration. To implement the registration mechanism, an outdoor AR system built upon 3D GIS was developed, named Augmented Reality Geographical Information System (ARGIS). ARGIS has the capability of performing precise registration in outdoor environment without using traditional vision tracking method, which thus enables users to arbitrarily manipulate the system. A prototype was developed to conduct experiment on the campus of Peking University, Beijing, China to test the proposed registration mechanism. The experiment shows that the developed registration mechanism is feasible and efficient in the outdoor environment. The ARGIS is expected to enrich the applications of outdoor AR system, including but not limited to underground facility mapping, emergency rescue and urban planning.
---
paper_title: An augmented reality based teeth shade matching system

paper_content:

In this paper, we present an augmented reality (AR) based teeth shade matching system for helping dentists verify the correctness of teeth shade matching and evaluate the acceptance of virtual teeth in cosmetic dentistry. The appearance of teeth is captured by a modified camera based on the measurements from cosmetic dentistry. After generating the shade matching map of teeth, we employ a subsurface scattering rendering model to synthesize the photorealistic virtual tooth. The virtual tooth is projected to the original one to test the matching quality. In experiments, we assess the benefits of our system by user evaluations. The experimental results demonstrate that our system can improve the efficiency and accuracy of dentists' analysis of teeth color, and the system has promising potential to be applied in dentistry clinic and dental industry.
---
paper_title: Augmented Reality Navigation With Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

paper_content:

Computer-assisted oral and maxillofacial surgery (OMS) has been rapidly evolving since the last decade. State-of-the-art surgical navigation in OMS still suffers from bulky tracking sensors, troublesome image registration procedures, patient movement, loss of depth perception in visual guidance, and low navigation accuracy. We present an augmented reality navigation system with automatic marker-free image registration using 3-D image overlay and stereo tracking for dental surgery. A customized stereo camera is designed to track both the patient and instrument. Image registration is performed by patient tracking and real-time 3-D contour matching, without requiring any fiducial and reference markers. Real-time autostereoscopic 3-D imaging is implemented with the help of a consumer-level graphics processing unit. The resulting 3-D image of the patient's anatomy is overlaid on the surgical site by a half-silvered mirror using image registration and IP-camera registration to guide the surgeon by exposing hidden critical structures. The 3-D image of the surgical instrument is also overlaid over the real one for an augmented display. The 3-D images present both stereo and motion parallax from which depth perception can be obtained. Experiments were performed to evaluate various aspects of the system; the overall image overlay error of the proposed system was 0.71 mm.
---
paper_title: Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study

paper_content:

To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject's maxillofacial area. The three-dimensional augmented reality system (integral videography display, computed tomography, a position tracker and a computer) was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument's position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon's ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible (<1 mm); this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient's surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.
---
paper_title: Haptic augmented reality dental trainer with automatic performance assessment

paper_content:

We developed an augmented reality (AR) dental training simulator utilizing a haptic (force feedback) device. A number of dental procedures such as crown preparation and opening access to the pulp can be simulated with various shapes of dental drill. The system allows students to practise surgery in the correct postures as in the actual environment by combining 3D tooth and tool models upon the real-world view and displaying the result through a video see-through head mounted display (HMD). The system monitors the important features such as applied forces and tool movement that characterize the quality of the procedure. Automatic performance assessment is achieved by comparing outcome and process features of a student with the best matching expert. Moreover, we incorporated kinematic feedback and hand guidance by haptic device. The result from an initial evaluation shows that the simulator is promising for supplemental training.
---
paper_title: Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study

paper_content:

PURPOSE ::: We present a novel augmented reality (AR) surgical navigation system based on ultrasound-assisted registration for pedicle screw placement. This system provides the clinically desired targeting accuracy and reduces radiation exposure. ::: ::: ::: METHODS ::: Ultrasound (US) is used to perform registration between preoperative computed tomography (CT) images and patient, and the registration is performed by least-squares fitting of these two three-dimensional (3D) point sets of anatomical landmarks taken from US and CT images. An integral videography overlay device is calibrated to accurately display naked-eye 3D images for surgical navigation. We use a 3.0-mm Kirschner wire (K-wire) instead of a pedicle screw in this study, and the K-wire is calibrated to obtain its orientation and tip location. Based on the above registration and calibration, naked-eye 3D images of the planning path and the spine are superimposed onto patient in situ using our AR navigation system. Simultaneously, a 3D image of the K-wire is overlaid accurately on the real one to guide the insertion procedure. The targeting accuracy is evaluated postoperatively by performing a CT scan. ::: ::: ::: RESULTS ::: An agar phantom experiment was performed. Eight K-wires were inserted successfully after US-assisted registration, and the mean targeting error and angle error were 3.35 mm and [Formula: see text], respectively. Furthermore, an additional sheep cadaver experiment was performed. Four K-wires were inserted successfully. The mean targeting error was 3.79 mm and the mean angle error was [Formula: see text], and US-assisted registration yielded better targeting results than skin markers-based registration (targeting errors: 2.41 vs. 5.18 mm, angle errors: [Formula: see text] vs. [Formula: see text]. ::: ::: ::: CONCLUSION ::: Experimental outcomes demonstrate that the proposed navigation system has acceptable targeting accuracy. In particular, the proposed navigation method reduces repeated radiation exposure to the patient and surgeons. Therefore, it has promising prospects for clinical use.
---
paper_title: A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries

paper_content:

BACKGROUND ::: Augmented reality-based surgeries have not been successfully implemented in oral and maxillofacial areas due to limitations in geometric accuracy and image registration. This paper aims to improve the accuracy and depth perception of the augmented video. ::: ::: ::: METHODOLOGY ::: The proposed system consists of a rotational matrix and translation vector algorithm to reduce the geometric error and improve the depth perception by including 2 stereo cameras and a translucent mirror in the operating room. ::: ::: ::: RESULTS ::: The results on the mandible/maxilla area show that the new algorithm improves the video accuracy by 0.30-0.40 mm (in terms of overlay error) and the processing rate to 10-13 frames/s compared to 7-10 frames/s in existing systems. The depth perception increased by 90-100 mm. ::: ::: ::: CONCLUSION ::: The proposed system concentrates on reducing the geometric error. Thus, this study provides an acceptable range of accuracy with a shorter operating time, which provides surgeons with a smooth surgical flow.
---
paper_title: A Stable and Accurate Marker-Less Augmented Reality Registration Method

paper_content:

Markerless Augmented Reality (AR) registration using the standard Homography matrix is unstable, and for image-based registration it has very low accuracy. In this paper, we present a new method to improve the stability and the accuracy of marker-less registration in AR. Based on the Visual Simultaneous Localization and Mapping (V-SLAM) framework, our method adds a three-dimensional dense cloud processing step to the state-of-the-art ORB-SLAM in order to deal with mainly the point cloud fusion and the object recognition. Our algorithm for the object recognition process acts as a stabilizer to improve the registration accuracy during the model to the scene transformation process. This has been achieved by integrating the Hough voting algorithm with the Iterative Closest Points(ICP) method. Our proposed AR framework also further increases the registration accuracy with the use of integrated camera poses on the registration of virtual objects. Our experiments show that the proposed method not only accelerates the speed of camera tracking with a standard SLAM system, but also effectively identifies objects and improves the stability of markerless augmented reality applications.
---
paper_title: Comparison of denture models by means of micro computed tomography

paper_content:

The production of dental inlays and crowns requires precise information on patients’ teeth morphology. The conventional method is the preparation of impressions using mold materials, e.g. a silicone impression material. The disadvantage of this technique is the human choke impulse and the flavor of the material. These discomforts can be avoided by methods where a three-dimensional scanner is used for recording the teeth morphology. The present study reveals the accuracy of three model types, namely conventional impression, rapid prototyping using an oral scanner C.O.S., 3M (Schweiz) AG and milling from a proprietary resin using the oral scanner iTero, Straumann Holding AG. For each method five models were fabricated from a steel reference (standard). Using a nanotom m (phoenixǀx-ray, GE Sensing and Inspection Technologies GmbH), three-dimensional micro computed tomography data sets of the standard and the 15 models were recorded and landmark distances within the data sets were measured with sub-pixel accuracy. To verify these results a coordinate measuring machine (Leitz PMM 864, Hexagon Metrology GmbH) based on tactile detection was used for the measurement of the landmark distances, and a correction of the distances measured by the nanotom m was arranged. The nanotom data sets of the 15 models were also compared to the standard by means of a non-rigid registration algorithm. The calculated deformation field exhibited mean pixel displacement values of (0.19 ± 0.09) mm for the C.O.S. models, (0.12 ± 0.07) mm for the gypsum models and (0.19 ± 0.12) mm for the i-Tero models.
---
paper_title: Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia.

paper_content:

BACKGROUND ::: Through three-dimensional real time imaging, augmented reality (AR) can provide an overlay of the anatomical structure, or visual cues for specific landmarks. In this study, an AR Toolkit was used for distraction osteogenesis with hemifacial microsomia to define the mandibular osteotomy line and assist with intraoral distractor placement. ::: ::: ::: METHODS ::: 20 patients with hemifacial microsomia were studied and were randomly assigned to experimental and control groups. Pre-operative computed tomography was used in both groups, whereas AR was used in the experimental group. Afterwards, pre- and post-operative computed tomographic scans of both groups were superimposed, and several measurements were made and analysed. ::: ::: ::: RESULTS ::: Both the conventional method and AR technique achieved proper positioning of the osteotomy planes, although the AR was more accurate. The difference in average vertical distance from the coronoid and condyle process to the pre- and post-operative cutting planes was significant (p < 0.01) between the two groups, whereas no significant difference (p > 0.05) was observed in the average angle between the two planes. The difference in deviations between the intersection points of the overlaid mandible across two cutting planes was also significant (p < 0.01). ::: ::: ::: CONCLUSION ::: This study reports on an efficient approach for guiding intraoperative distraction osteogenesis. Augmented reality tools such as the AR Toolkit may be helpful for precise positioning of intraoral distractors in patients with hemifacial microsomia in craniofacial surgery.
---
paper_title: Image-Based tracking of the teeth for orthodontic augmented reality

paper_content:

We present image-based methods for tracking teeth in a video image with respect to a CT scan of the jaw, in order to enable a novel light-weight augmented reality (AR) system in orthodontistry. Its purpose is guided bracket placement in orthodontic correction. In this context, our goal is to determine the position of the patient maxilla and mandible in a video image solely based on a CT scan. This is suitable for image guidance through an overlay of the video image with the planned position of brackets in a monocular AR system. Our tracking algorithm addresses the contradicting requirements of robustness, accuracy and performance in two problem-specific formulations. First, we exploit a distance-based modulation of two iso-surfaces from the CT image to approximate the appearance of the gum line. Second, back-projection of previous video frames to an iso-surface is used to account for recently placed brackets. In combination, this novel algorithm allowed us to track several sequences of three patient videos of real procedures, despite difficult lighting conditions. Paired with a systematic evaluation, we were able to show practical feasibility of such a system.
---
paper_title: Deformable Medical Image Registration: A Survey

paper_content:

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.
---
paper_title: Atlas-Based Automatic Generation of Subject-Specific Finite Element Tongue Meshes

paper_content:

Generation of subject-specific 3D finite element (FE) models requires the processing of numerous medical images in order to precisely extract geometrical information about subject-specific anatomy. This processing remains extremely challenging. To overcome this difficulty, we present an automatic atlas-based method that generates subject-specific FE meshes via a 3D registration guided by Magnetic Resonance images. The method extracts a 3D transformation by registering the atlas’ volume image to the subject’s one, and establishes a one-to-one correspondence between the two volumes. The 3D transformation field deforms the atlas’ mesh to generate the subject-specific FE mesh. To preserve the quality of the subject-specific mesh, a diffeomorphic non-rigid registration based on B-spline free-form deformations is used, which guarantees a non-folding and one-to-one transformation. Two evaluations of the method are provided. First, a publicly available CT-database is used to assess the capability to accurately capture the complexity of each subject-specific Lung’s geometry. Second, FE tongue meshes are generated for two healthy volunteers and two patients suffering from tongue cancer using MR images. It is shown that the method generates an appropriate representation of the subject-specific geometry while preserving the quality of the FE meshes for subsequent FE analysis. To demonstrate the importance of our method in a clinical context, a subject-specific mesh is used to simulate tongue’s biomechanical response to the activation of an important tongue muscle, before and after cancer surgery.
---
paper_title: Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

paper_content:

AIM ::: We present a newly designed, localiser-free, head-mounted system featuring augmented reality as an aid to maxillofacial bone surgery, and assess the potential utility of the device by conducting a feasibility study and validation. ::: ::: ::: METHODS ::: Our head-mounted wearable system facilitating augmented surgery was developed as a stand-alone, video-based, see-through device in which the visual features were adapted to facilitate maxillofacial bone surgery. We implement a strategy designed to present augmented reality information to the operating surgeon. LeFort1 osteotomy was chosen as the test procedure. The system is designed to exhibit virtual planning overlaying the details of a real patient. We implemented a method allowing performance of waferless, augmented-reality assisted bone repositioning. In vitro testing was conducted on a physical replica of a human skull, and the augmented reality system was used to perform LeFort1 maxillary repositioning. Surgical accuracy was measured with the aid of an optical navigation system that recorded the coordinates of three reference points (located in anterior, posterior right, and posterior left positions) on the repositioned maxilla. The outcomes were compared with those expected to be achievable in a three-dimensional environment. Data were derived using three levels of surgical planning, of increasing complexity, and for nine different operators with varying levels of surgical skill. ::: ::: ::: RESULTS ::: The mean error was 1.70 ± 0.51 mm. The axial errors were 0.89 ± 0.54 mm on the sagittal axis, 0.60 ± 0.20 mm on the frontal axis, and 1.06 ± 0.40 mm on the craniocaudal axis. The simplest plan was associated with a slightly lower mean error (1.58 ± 0.37 mm) compared with the more complex plans (medium: 1.82 ± 0.71 mm; difficult: 1.70 ± 0.45 mm). The mean error for the anterior reference point was lower (1.33 ± 0.58 mm) than those for both the posterior right (1.72 ± 0.24 mm) and posterior left points (2.05 ± 0.47 mm). No significant difference in terms of error was noticed among operators, despite variations in surgical experience. Feedback from surgeons was acceptable; all tests were completed within 15 min and the tool was considered to be both comfortable and usable in practice. ::: ::: ::: CONCLUSION ::: We used a new localiser-free, head-mounted, wearable, stereoscopic, video see-through display to develop a useful strategy affording surgeons access to augmented reality information. Our device appears to be accurate when used to assist in waferless maxillary repositioning. Our results suggest that the method can potentially be extended for use with many surgical procedures on the facial skeleton. Further, our positive results suggest that it would be appropriate to proceed to in vivo testing to assess surgical accuracy under real clinical conditions.
---
paper_title: A Scalable ADMM Algorithm for Rigid Registration

paper_content:

A fundamental problem that comes up in computer vision, image processing, manifold learning, and sensor networks is that of registering multiple point sets using rigid transforms. A standard result in this regard is that the least-square formulation of the registration problem admits a closed-form solution for two point sets. However, since the group of rigid transforms is not convex, solving the least-square optimization for multiple point sets is computationally challenging. It was recently demonstrated that the least-square formulation can be relaxed into a tractable semidefinite program, and that the relaxation is provably tight under certain assumptions. The difficulty is that standard solvers for semidefinite programming (e.g., interior-point solvers) cannot be scaled to handle large-sized problems. In this letter, we propose an iterative solver based on variable splitting and the alternating direction method of multipliers. Since each iteration essentially involves an eigendecomposition, the proposed solver can be scaled to problems that are beyond the reach of interior-point solvers. We present results on simulated and real data to demonstrate the potential of the solver.
---
paper_title: A review of computer-aided oral and maxillofacial surgery: planning, simulation and navigation.

paper_content:

ABSTRACTIntroduction: Currently, oral and maxillofacial surgery (OMFS) still poses a significant challenge for surgeons due to the anatomic complexity and limited field of view of the oral cavity. With the great development of computer technologies, he computer-aided surgery has been widely used for minimizing the risks and improving the precision of surgery.Areas covered: The major goal of this paper is to provide a comprehensive reference source of current and future development of computer-aided OMFS including surgical planning, simulation and navigation for relevant researchers.Expert commentary: Compared with the traditional OMFS, computer-aided OMFS overcomes the disadvantage that the treatment on the region of anatomically complex maxillofacial depends almost exclusively on the experience of the surgeon.
---
paper_title: Registration strategy using occlusal splint based on augmented reality for mandibular angle oblique split osteotomy.

paper_content:

BACKGROUND ::: An augmented reality tool allows for visual tracking of real anatomic structures in superposition with volume-rendered computed tomographic or magnetic resonance imaging scans and thus can be used for navigated translocation of important structures during operation. In this feasibility study, ARToolKit was used in mandibular angle oblique split osteotomy to define the cutting planes according to an operative plan. ::: ::: ::: METHODS ::: We overlay the operative plan on the model of a mandible made by rapid prototyping technology, and the technology was successfully used in 15 patients. Before the operation, all patients underwent computed tomographic scan, and dental casts were prepared by surgeons. Then, surgeons make the occlusal splint according to a dental cast to fix the marker, which can be recognized by the ARToolKit. The occlusal splint and marker were transformed to three-dimensional data using a laser scanner, and a programmer that runs on a personal computer named Rapidform matches the marker and the mandible image to generate the virtual image. By this step, the virtual image describing the marker, occlusal splint, and the mandible image of the patient are integrated. During the operation, the operative plan was overlaid on the rapid prototyping model of the mandible as soon as the ARToolKit recognized the marker. ::: ::: ::: RESULTS ::: The technology was successfully used in 15 patients; the virtual image of the mandible and the cutting-plane both overlaid the real model of the mandible. ::: ::: ::: CONCLUSIONS ::: This study has reported a new and effective way for mandibular angle oblique split osteotomy, and using occlusal splint might be a powerful option for the registration of augmented reality. Augmented reality tools like ARToolKit may be helpful for control of maxillary translocation in orthognathic surgery.
---
paper_title: Development of AR Display System for Dental Surgical Simulator

paper_content:

In this paper, we describe about a display system for dental surgical simulator. The design concept of our system is that the system allows the user to learning the dental surgical methods with the real hand and body posture. Therefore our system set a display, which shows virtual teeth model and real teeth and gums, close to user’s hand position and allows the user to manipulating object directly.
---
paper_title: Non-rigid contour-to-pixel registration of photographic and quantitative light-induced fluorescence imaging of decalcified teeth

paper_content:

Quantitative light-induced fluorescence (QLF) is widely used to assess the damage of a tooth due to decalcification. In digital photographs, decalcification appears as white spot lesions, i.e. white spots on the tooth surface. We propose a novel multimodal registration approach for the matching of digital photographs and QLF images of decalcified teeth. The registration is based on the idea of contour-to-pixel matching. Here, the curve, which represents the shape of the tooth, is extracted from the QLF image using a contour segmentation by binarization and morphological processing. This curve is aligned to the photo with a non-rigid variational registration approach. Thus, the registration problem is formulated as minimization problem with an objective function that consists of a data term and a regularizer for the deformation. To construct the data term, the photo is pointwise classified into tooth and non-tooth regions. Then, the signed distance function of the tooth region allows to measure the mismatch between curve and photo. As regularizer a higher order, linear elastic prior is used. The resulting minimization problem is solved numerically using bilinear Finite Elements for the spatial discretization and the Gauss-Newton algorithm. The evaluation is based on 150 image pairs, where an average of 5 teeth have been captured from 32 subjects. All registrations have been confirmed correctly by a dental expert. The contour-to-pixel methods can directly be used in 3D for surface-to-voxel tasks.
---
paper_title: A Real-Time Three-Dimensional Tracking and Registration Method in the AR-HUD System

paper_content:

In this paper, we propose a method of 3-D tracking registration with a virtual image and the surroundings in three steps: initial registration matrix acquisition, feature extraction in a road scene, and registration of virtual information in a real environment. We obtain the initial registration matrix according to the relationship between different parts of the system. During transformation matrix solving, we do not match features. Instead, the relationship is based on the characteristic image information. Then, we interpolate another image’s corresponding coordinates and estimate the transform matrix of the map and the image. The average time of the virtual reality registration algorithm can reach 0.0781 s. The average accuracy of our registration method is less than 2 pixels. The proposed algorithm has some advantages. We use features from accelerated segment test feature extraction and the binary robust independent elementary features descriptor to efficiently extract image feature points. The algorithm is more efficient, because it eliminates the feature-matching phase. We apply the proposed virtual reality registration method to a vehicle’s augmented reality head-up display (AR-HUD). We derive the relationship between human eyes, optical system, tracking camera, and real scene in the AR-HUD system. Our experiment proves that the algorithm has a better registration effect on virtual objects, and it enhances road information. Different from the general head-up display (HUD) system, our method achieves the true integration of virtual and real world. It improves the distraction problem caused by the mismatch between the existing HUD virtual image and the real scene.
---
paper_title: Nonrigid Point Set Registration by Preserving Local Connectivity

paper_content:

This paper is concerned with the nonrigid point set registration problem and a probability-based registration algorithm with local connectivity preservation is proposed. A unified formulation for point set registration problem is introduced and the derived energy function is composed of three parts, distance measurement item, transformation constraint item, and correspondence constraint item. In order to preserve the local structure of point set, the definitions of ${k}$ -connected neighbors and connectivity matrix are given and the local connectivity constraint is constructed as a weighted least square error item. The point set registration problem is formulated in the expectation-maximization algorithm scheme and the optimal spatial transformation and correspondence matrix are estimated simultaneously. The effectiveness of the proposed method is verified by applying the method to synthetic point sets and real scenarios of hand shapes and surface-mount technology components.
---
paper_title: Atlas-Based Automatic Generation of Subject-Specific Finite Element Tongue Meshes

paper_content:

Generation of subject-specific 3D finite element (FE) models requires the processing of numerous medical images in order to precisely extract geometrical information about subject-specific anatomy. This processing remains extremely challenging. To overcome this difficulty, we present an automatic atlas-based method that generates subject-specific FE meshes via a 3D registration guided by Magnetic Resonance images. The method extracts a 3D transformation by registering the atlas’ volume image to the subject’s one, and establishes a one-to-one correspondence between the two volumes. The 3D transformation field deforms the atlas’ mesh to generate the subject-specific FE mesh. To preserve the quality of the subject-specific mesh, a diffeomorphic non-rigid registration based on B-spline free-form deformations is used, which guarantees a non-folding and one-to-one transformation. Two evaluations of the method are provided. First, a publicly available CT-database is used to assess the capability to accurately capture the complexity of each subject-specific Lung’s geometry. Second, FE tongue meshes are generated for two healthy volunteers and two patients suffering from tongue cancer using MR images. It is shown that the method generates an appropriate representation of the subject-specific geometry while preserving the quality of the FE meshes for subsequent FE analysis. To demonstrate the importance of our method in a clinical context, a subject-specific mesh is used to simulate tongue’s biomechanical response to the activation of an important tongue muscle, before and after cancer surgery.
---
paper_title: A Phase-Based Image Registration Algorithm for Dental Radiograph Identification

paper_content:

Dental radiographs have been used for the accurate assessment and treatment of dental diseases. For an accurate diagnosis, the complete geometric registration between radiographs is required. The perspective projection between two radiographs may be observed, even if they are taken from the same oral regions of the subject. This paper presents an efficient dental radiograph registration algorithm using Phase-Only Correlation (POC) function. The use of phase components in 2D (two-dimensional) discrete Fourier transforms of dental radiograph images makes possible to achieve highly robust image registration and recognition. Experimental evaluation using a dental radiograph database indicates that the proposed algorithm exhibits efficient recognition performance even for distorted radiographs.
---
paper_title: Augmented reality: a review.

paper_content:

Augmented reality is a technology that overlays digital information on objects or places in the real world for the purpose of enhancing the user experience. It is not virtual reality, that is, the technology that creates a totally digital or computer created environment. Augmented reality, with its ability to combine reality and digital information, is being studied and implemented in medicine, marketing, museums, fashion, and numerous other areas. This article presents an overview of augmented reality, discussing what it is, how it works, its current implementations, and its potential impact on libraries.
---
paper_title: Augmented reality haptics system for dental surgical skills training

paper_content:

We have developed a virtual reality (VR) and an augmented reality (AR) dental training simulator utilizing a haptic device. The simulators utilize volumetric force feedback computation and real time modification of the volumetric data. They include a virtual mirror to facilitate indirect vision during a simulated operation. The AR environment allows students to practice surgery in correct postures by combining the 3D tooth and tool models with the real-world view and displaying the result through a video see-through head-mounted display (HMD). Preliminary results from an initial evaluation show that the system is a promising tool to supplement dental training and that there are advantages of the AR over the VR approach.
---
paper_title: Joint 3D Reconstruction of a Static Scene and Moving Objects

paper_content:

We present a technique for simultaneous 3D reconstruction of static regions and rigidly moving objects in a scene. An RGB-D frame is represented as a collection of features, which are points and planes. We classify the features into static and dynamic regions and grow separate maps, static and object maps, for each of them. To robustly classify the features in each frame, we fuse multiple RANSAC-based registration results obtained by registering different groups of the features to different maps, including (1) all the features to the static map, (2) all the features to each object map, and (3) subsets of the features, each forming a segment, to each object map. This multi-group registration approach is designed to overcome the following challenges: scenes can be dominated by static regions, making object tracking more difficult; and moving object might have larger pose variation between frames compared to the static regions. We show qualitative results from indoor scenes with objects in various shapes. The technique enables on-the-fly object model generation to be used for robotic manipulation.
---
paper_title: AR in VR: assessing surgical augmented reality visualizations in a steerable virtual reality environment

paper_content:

Augmented reality (AR) has emerged as a promising approach to support surgeries; however, its application in real world scenarios is still very limited. Besides sophisticated registration tasks that need to be solved, surgical AR visualizations have not been studied in a standardized and comparative manner. To foster the development of future AR applications, a steerable framework is urgently needed to rapidly evaluate new visualization techniques, explore their individual parameter spaces and define relevant application scenarios. Inspired by its beneficial usage in the automotive industry, the underlying concept of virtual reality (VR) is capable of transforming complex real environments into controllable virtual ones. We present an interactive VR framework, called Augmented Visualization Box (AVB), in which visualizations for AR can be systematically investigated without explicitly performing an error-prone registration. As use case, a virtual laparoscopic scenario with anatomical surface models was created in a computer game engine. In a study with eleven surgeons, we analyzed this VR setting under different environmental factors and its applicability for a quantitative assessment of different AR overlay concepts. According to the surgeons, the visual impression of the VR scene is mostly influenced by 2D surface details and lighting conditions. The AR evaluation shows that, depending on the visualization used and its capability to encode depth, 37% to 91% of the experts made wrong decisions, but were convinced of their correctness. These results show that surgeons have more confidence in their decisions, although they are wrong, when supported by AR visualizations. With AVB, intraoperative situations are realistically simulated to quantitatively benchmark current AR overlay methods. Successful surgical task execution in an AR system can only be facilitated if visualizations are customized toward the surgical task.
---
paper_title: Training of Tooth Scaling by Simulator -- Development of Simulator and Investigation of its Effectiveness --

paper_content:

Dental calculus, hardened plaque on the teeth, is a cause of periodontitis, or gum diseases in the mouth. For dentists and dental hygienists, it is an important task to remove the calculi from patients. The task is called "tooth scaling" and the calculi just beside the gum and between the tooth and gum are usually removed by using a hand scaler. Since the scaling by a hand scaler requires skill, apprentice dentists and dental hygienists must be trained in order to get the skill. Unfortunately in the training environment, however, there are several problems such as difficulty in collecting teeth on which various types of calculi are stuck, lack of cooperative patients, and so on. Authors have newly developed a simulator for the training by using a PHANToM and a video-see-through HMD. The training effectiveness has been evaluated by artificial calculus removal on model teeth, and subjectively evaluated by veterans. The experimental result shows the developed simulator is more effective than the conventional paint removing method.
---
paper_title: Augmented reality haptics system for dental surgical skills training

paper_content:

We have developed a virtual reality (VR) and an augmented reality (AR) dental training simulator utilizing a haptic device. The simulators utilize volumetric force feedback computation and real time modification of the volumetric data. They include a virtual mirror to facilitate indirect vision during a simulated operation. The AR environment allows students to practice surgery in correct postures by combining the 3D tooth and tool models with the real-world view and displaying the result through a video see-through head-mounted display (HMD). Preliminary results from an initial evaluation show that the system is a promising tool to supplement dental training and that there are advantages of the AR over the VR approach.
---
paper_title: Training of Tooth Scaling by Simulator -- Development of Simulator and Investigation of its Effectiveness --

paper_content:

Dental calculus, hardened plaque on the teeth, is a cause of periodontitis, or gum diseases in the mouth. For dentists and dental hygienists, it is an important task to remove the calculi from patients. The task is called "tooth scaling" and the calculi just beside the gum and between the tooth and gum are usually removed by using a hand scaler. Since the scaling by a hand scaler requires skill, apprentice dentists and dental hygienists must be trained in order to get the skill. Unfortunately in the training environment, however, there are several problems such as difficulty in collecting teeth on which various types of calculi are stuck, lack of cooperative patients, and so on. Authors have newly developed a simulator for the training by using a PHANToM and a video-see-through HMD. The training effectiveness has been evaluated by artificial calculus removal on model teeth, and subjectively evaluated by veterans. The experimental result shows the developed simulator is more effective than the conventional paint removing method.
---
paper_title: Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation.

paper_content:

Abstract Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560 × 1600 elemental image resolution shows the rendering speeds of 50–60 frames per second (fps) for surface models, and 5–8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability.
---
paper_title: Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study

paper_content:

To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject's maxillofacial area. The three-dimensional augmented reality system (integral videography display, computed tomography, a position tracker and a computer) was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument's position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon's ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible (<1 mm); this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient's surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.
---
paper_title: Application of an augmented reality tool for maxillary positioning in orthognathic surgery - a feasibility study.

paper_content:

BACKGROUND ::: An augmented reality tool for computer assisted surgery named X-Scope allows visual tracking of real anatomical structures in superposition with volume rendered CT or MRI scans and thus can be used for navigated translocation of bony segments. ::: ::: ::: METHODS ::: In a feasibility study X-Scope was used in orthognathic surgery to control the translocation of the maxilla after Le Fort I osteotomy within a bimaxillary procedure. The situation achieved was compared with the pre-operative situation by means of cephalometric analysis on lateral and frontal cephalograms. ::: ::: ::: RESULTS ::: The technique was successfully utilized in 5 patients. Maxillary positioning using X-Scope was accomplished accurately within a range of 1mm. The tool was used in all cases in addition to the usual intra-operative splints. A stand-alone application without conventional control does not yet seem reasonable. ::: ::: ::: CONCLUSION ::: Augmented reality tools like X-Scope may be helpful for controlling maxillary translocation in orthognathic surgery. The application to other interventions in cranio-maxillofacial surgery such as Le Fort III osteotomy, fronto-orbital advancement, and cranial vault reshaping or repair may also be considered.
---
paper_title: Video see‐through augmented reality for oral and maxillofacial surgery

paper_content:

BACKGROUND ::: Oral and maxillofacial surgery has not been benefitting from image guidance techniques owing to the limitations in image registration. ::: ::: ::: METHODS ::: A real-time markerless image registration method is proposed by integrating a shape matching method into a 2D tracking framework. The image registration is performed by matching the patient's teeth model with intraoperative video to obtain its pose. The resulting pose is used to overlay relevant models from the same CT space on the camera video for augmented reality. ::: ::: ::: RESULTS ::: The proposed system was evaluated on mandible/maxilla phantoms, a volunteer and clinical data. Experimental results show that the target overlay error is about 1 mm, and the frame rate of registration update yields 3-5 frames per second with a 4 K camera. ::: ::: ::: CONCLUSIONS ::: The significance of this work lies in its simplicity in clinical setting and the seamless integration into the current medical procedure with satisfactory response time and overlay accuracy. Copyright © 2016 John Wiley & Sons, Ltd.
---
paper_title: Computer-assisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display.

paper_content:

There may well be a shift towards 3-dimensional orthognathic surgery when virtual surgical planning can be applied clinically. We present a computer-assisted protocol that uses surgical navigation supplemented by an interactive image-guided visualisation display (IGVD) to transfer virtual maxillary planning precisely. The aim of this study was to analyse its accuracy and versatility in vivo. The protocol consists of maxillofacial imaging, diagnosis, planning of virtual treatment, and intraoperative surgical transfer using an IGV display. The advantage of the interactive IGV display is that the virtually planned maxilla and its real position can be completely superimposed during operation through a video graphics array (VGA) camera, thereby augmenting the surgeon's 3-dimensional perception. Sixteen adult class III patients were treated with by bimaxillary osteotomy. Seven hard tissue variables were chosen to compare (ΔT1-T0) the virtual maxillary planning (T0) with the postoperative result (T1) using 3-dimensional cephalometry. Clinically acceptable precision for the surgical planning transfer of the maxilla (<0.35 mm) was seen in the anteroposterior and mediolateral angles, and in relation to the skull base (<0.35°), and marginal precision was seen in the orthogonal dimension (<0.64 mm). An interactive IGV display complemented surgical navigation, augmented virtual and real-time reality, and provided a precise technique of waferless stereotactic maxillary positioning, which may offer an alternative approach to the use of arbitrary splints and 2-dimensional orthognathic planning.
---
paper_title: Real-Time FEM-Based Registration of 3-D to 2.5-D Transrectal Ultrasound Images

paper_content:

We present a novel technique for real-time deformable registration of 3-D to 2.5-D transrectal ultrasound (TRUS) images for image-guided, robot-assisted laparoscopic radical prostatectomy (RALRP). For RALRP, a pre-operatively acquired 3-D TRUS image is registered to thin-volumes comprised of consecutive intra-operative 2-D TRUS images, where the optimal transformation is found using a gradient descent method based on analytical first and second order derivatives. Our method relies on an efficient algorithm for real-time extraction of arbitrary slices from a 3-D image deformed given a discrete mesh representation. We also propose and demonstrate an evaluation method that generates simulated models and images for RALRP by modeling tissue deformation through patient-specific finite-element models (FEM). We evaluated our method on in-vivo data from 11 patients collected during RALRP and focal therapy interventions. In the presence of an average landmark deformation of 3.89 and 4.62 mm, we achieved accuracies of 1.15 and 0.72 mm, respectively, on the synthetic and in-vivo data sets, with an average registration computation time of 264 ms, using MATLAB on a conventional PC. The results show that the real-time tracking of the prostate motion and deformation is feasible, enabling a real-time augmented reality-based guidance system for RALRP.].
---
paper_title: Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation.

paper_content:

Abstract Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560 × 1600 elemental image resolution shows the rendering speeds of 50–60 frames per second (fps) for surface models, and 5–8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability.
---
paper_title: Haptic augmented reality dental trainer with automatic performance assessment

paper_content:

We developed an augmented reality (AR) dental training simulator utilizing a haptic (force feedback) device. A number of dental procedures such as crown preparation and opening access to the pulp can be simulated with various shapes of dental drill. The system allows students to practise surgery in the correct postures as in the actual environment by combining 3D tooth and tool models upon the real-world view and displaying the result through a video see-through head mounted display (HMD). The system monitors the important features such as applied forces and tool movement that characterize the quality of the procedure. Automatic performance assessment is achieved by comparing outcome and process features of a student with the best matching expert. Moreover, we incorporated kinematic feedback and hand guidance by haptic device. The result from an initial evaluation shows that the simulator is promising for supplemental training.
---
paper_title: A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries

paper_content:

BACKGROUND ::: Augmented reality-based surgeries have not been successfully implemented in oral and maxillofacial areas due to limitations in geometric accuracy and image registration. This paper aims to improve the accuracy and depth perception of the augmented video. ::: ::: ::: METHODOLOGY ::: The proposed system consists of a rotational matrix and translation vector algorithm to reduce the geometric error and improve the depth perception by including 2 stereo cameras and a translucent mirror in the operating room. ::: ::: ::: RESULTS ::: The results on the mandible/maxilla area show that the new algorithm improves the video accuracy by 0.30-0.40 mm (in terms of overlay error) and the processing rate to 10-13 frames/s compared to 7-10 frames/s in existing systems. The depth perception increased by 90-100 mm. ::: ::: ::: CONCLUSION ::: The proposed system concentrates on reducing the geometric error. Thus, this study provides an acceptable range of accuracy with a shorter operating time, which provides surgeons with a smooth surgical flow.
---
paper_title: A GPU-Accelerated Deformable Image Registration Algorithm With Applications to Right Ventricular Segmentation

paper_content:

Delineation of the cardiac right ventricle is essential in generating clinical measurements such as ejection fraction and stroke volume. Given manual segmentation on the first frame, one approach to segment right ventricle from all of the magnetic resonance images is to find point correspondence between the sequence of images. Finding the point correspondence with non-rigid transformation requires a deformable image registration algorithm, which often involves computationally expensive optimization. The central processing unit (CPU)-based implementation of point correspondence algorithm has been shown to be accurate in delineating organs from a sequence of images in recent studies. The purpose of this study is to develop computationally efficient approaches for deformable image registration. We propose a graphics processing unit (GPU) accelerated approach to improve the efficiency. The proposed approach consists of two parallelization components: Parallel compute unified device architecture (CUDA) version of the deformable registration algorithm; and the application of an image concatenation approach to further parallelize the algorithm. Three versions of the algorithm were implemented: 1) CPU; 2) GPU with only intra-image parallelization (sequential image registration); and 3) GPU with inter and intra-image parallelization (concatenated image registration). The proposed methods were evaluated over a data set of 16 subjects. CPU, GPU sequential image, and GPU concatenated image methods took an average of 113.13, 16.50, and 5.96 s to segment a sequence of 20 images, respectively. The proposed parallelization approach offered a computational performance improvement of around $19\times $ in comparison to the CPU implementation while retaining the same level of segmentation accuracy. This paper demonstrated that the GPU computing could be utilized for improving the computational performance of a non-rigid image registration algorithm without compromising the accuracy.
---

Title: Registration Technology of Augmented Reality in Oral Medicine: A Review

Section 1: INTRODUCTION
Description 1: Provide a comprehensive overview of Augmented Reality (AR) technology, its origins, and its introduction into the field of oral medicine, explaining its importance, benefits, and applications within this specific field.

Section 2: LITERATURE SEARCH
Description 2: Outline the methodology used for the literature search, including details on the planning, searching, and assessing phases. 

Section 3: ANALYSIS OF RESEARCH HOTSPOTS BASED ON CITESPACE
Description 3: Analyze research hotspots using the Citespace tool, explaining the bibliometric method and presenting keyword co-occurrence network maps to identify trends and developments.

Section 4: STATIC REGISTRATION METHOD OF AUGMENTED REALITY IN ORAL MEDICINE
Description 4: Describe the static registration methods including rigid and non-rigid registration techniques used for AR in oral medicine, providing examples and discussing their applications.

Section 5: REAL-TIME REGISTRATION METHOD OF AUGMENTED REALITY IN ORAL MEDICINE
Description 5: Discuss real-time registration methods in detail, exploring their importance in practical applications such as preoperative navigation, oral surgery, and dental training.

Section 6: DISCUSSION AND FUTURE DIRECTIONS
Description 6: Summarize the key technologies and current limitations of static and real-time registration methods, and propose potential future research directions to enhance AR applications in oral medicine.

Section 7: CONCLUSION
Description 7: Provide a summary of the review findings, emphasizing the significance and impact of AR registration methods in oral medicine, along with a brief outline of future pathways as suggested in the discussion.