Slice-to-volume medical image registration: a survey
---
paper_title: Deformable Medical Image Registration: A Survey

paper_content:

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.
---
paper_title: Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images.

paper_content:

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.
---
paper_title: A review of 3D/2D registration methods for image-guided interventions

paper_content:

Registration of pre- and intra-interventional data is one of the key technologies for image-guided radiation therapy, radiosurgery, minimally invasive surgery, endoscopy, and interventional radiology. In this paper, we survey those 3D/2D data registration methods that utilize 3D computer tomography or magnetic resonance images as the pre-interventional data and 2D X-ray projection images as the intra-interventional data. The 3D/2D registration methods are reviewed with respect to image modality, image dimensionality, registration basis, geometric transformation, user interaction, optimization procedure, subject, and object of registration.
---
paper_title: Automatic non-linear MRI-ultrasound registration for the correction of intra-operative brain deformations

paper_content:

Objective: Movements of brain tissue during neurosurgical procedures reduce the effectiveness of using pre-operative images for intra-operative surgical guidance. In this paper, we explore the use of acquiring intra-operative ultrasound (US) images for the quantification of and correction for non-linear brain deformations.Materials and Methods: We will present a multi-modal registration strategy that automatically matches pre-operative images (e.g., MRI) to intra-operative US to correct for these deformations. The strategy involves using the predicted appearance of neuroanatomical structures in US images to build “pseudo ultrasound” images based on pre-operative segmented MRI. These images can then be non-linearly registered to intra-operative US using cross-correlation measurements within the ANIMAL package. The feasibility of the theory is demonstrated through its application to clinical patient data acquired during 12 neurosurgical procedures.Results: Results of applying the method to 12 surgical cases...
---
paper_title: Motion Correction in fMRI by Mapping Slice-to-Volume with Concurrent Field-Inhomogeneity Correction

paper_content:

Head motion is the major source of error in measuring intensity changes related to given stimuli in fMRI. The effects of head motion are image shifts and field inhomogeneity variations which cause local changes in geometric distortions. The previously developed motion correction method, mapping slice-to-volume (MSV), retrospectively remaps slices that are shifted by head motion to their spatially correct locations in an anatomical reference. Images exhibiting spatially varying geometric distortions require non-linear mapping solutions. An accurate field map can be used for the correction of such spatial distortions. However, field-map changes with head motion and, in practice, only one field-map is available typically. This work evaluates the improved motion correction capability of MSV with concurrent iterative field-corrected reconstruction using only an initial field-map. The results from simulated motion data show effective convergence and accuracy in image registration for the correction of image artifacts complicated by the motion induced field effects.
---
paper_title: Registration of Whole-Mount Histology and Volumetric Imaging of the Prostate Using Particle Filtering

paper_content:

Registration of histological slices to volumetric imaging of the prostate is an important task that can be used to optimize imaging for cancer detection. Such registration is challenging due to physical changes of the specimen during excision and fixation, and misalignment of the histological slices during preparation and digital scanning. In this work, we consider a multi-slice to volume registration method in which a stack of sparse, unaligned 2-D whole-mount histological slices is registered to a 3-D volumetric imaging of the prostate. We propose a particle filtering framework to contend with the high dimensionality of the search space and multimodal nature of the optimization. Such framework allows modeling of the uncertainty in the pose of the slices and in the imaged information, in order to derive optimal registration parameters in a Bayesian approach. Intensity-, region-, and point-based similarity metrics were incorporated into the registration algorithm to account for different imaging modalities. We demonstrate and evaluate our method on a diverse set of data that includes a synthetic volume, ex vivo and in vivo magnetic resonance imaging, and in vivo ultrasound.
---
paper_title: Improved fMRI time-series registration using joint probability density priors

paper_content:

Functional MRI (fMRI) time-series studies are plagued by varying degrees of subject head motion. Faithful head ::: motion correction is essential to accurately detect brain activation using statistical analyses of these time-series. ::: Mutual information (MI) based slice-to-volume (SV) registration is used for motion estimation when the rate of ::: change of head position is large. SV registration accounts for head motion between slice acquisitions by estimating ::: an independent rigid transformation for each slice in the time-series. Consequently each MI optimization uses ::: intensity counts from a single time-series slice, making the algorithm susceptible to noise for low complexity endslices ::: (i.e., slices near the top of the head scans). This work focuses on improving the accuracy of MI-based SV ::: registration of end-slices by using joint probability density priors derived from registered high complexity centerslices ::: (i.e., slices near the middle of the head scans). Results show that the use of such priors can significantly ::: improve SV registration accuracy.
---
paper_title: A Fast CT and CT-Fluoroscopy Registration Algorithm With Respiratory Motion Compensation for Image-Guided Lung Intervention

paper_content:

CT-fluoroscopy (CTF) is an efficient imaging technique for guiding percutaneous lung intervention such as biopsy and ablation. In CTF-guided procedures, four to ten axial images are captured in a very short time period during breath holding to provide near real-time feedback of patients' anatomy so that physicians can adjust the needle as it is advanced toward a target lesion. Although popularly used in clinics, this procedure requires frequent scans to guide the needle, which may cause increased procedure time, complication rates, and radiation exposure to both clinicians and patients. In addition, CTF only generates a limited number of 2-D axial images and does not provide sufficient 3-D anatomical information. Therefore, how to provide volumetric anatomical information using CTF while reducing intraoperative scan is an important and challenging problem. In this paper, we propose a fast CT-CTF deformable registration algorithm that warps the inhale preprocedural CT onto the intraprocedural CTF for guidance in 3-D. In the algorithm, the deformation in the transverse plane is modeled using 2-D B-Spline, and the deformation along z-direction is regularized by smoothness constraint. A respiratory motion compensation framework is also incorporated for accurate registration. A parallel implementation strategy is adopted to accomplish the registration in several seconds. With electromagnetic tracking, the needle position can be superimposed onto the deformed inhale CT image, thereby providing 3-D image guidance during breath holding. Experiments were conducted using both simulated CTF images with known deformation and real CTF images captured during lung cancer biopsy studies. The experiments demonstrated satisfactory registration results of our proposed fast CT-CTF registration algorithm.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: Rigid Registration of Untracked Freehand 2D Ultrasound Sweeps to 3D CT of Liver Tumours

paper_content:

We present a rigid registration framework for freehand 2D ultrasound sweeps to 3D CT of liver tumours. The method registers the 2D sweeps in a group-wise manner, without the need for prior 3D ultrasound compounding or probe tracking during acquisition. We first introduce a specific acquisition model to keep the dimension of this problem reasonable. Only seven parameters are indeed required to register the images. These are estimated using simulated annealing optimization of a robust modality-independent similarity measure. The framework contrasts the current methods that rely on tracking devices and phantom calibration, which are often difficult to use routinely in clinical practice. Our results on both synthetic and real data show that the method is well-suited for such ultrasound-CT registration of liver tumours.
---
paper_title: Correlating 2D histological slice with 3D MRI image volume using smart phone as an interactive tool for muscle study

paper_content:

In muscle dystrophy studies, registration of histological image with MRI image volume enables cross validation of MRI biomarkers using pathological result. However, correlation of 2D histology slice with 3D MRI volume is technically challenging due to the potentially non-orthogonal slice plane and incomplete or distorted histological slice. This paper presents an efficient method to directly perform the 2D-3D registration. The method is unique in that it uses smart phone as a navigation tool for initial alignment followed by an overlap invariant mutual information-based refinement. Experimental results using animal muscle samples images from a 3T MRI and HE stained histological images show that the proposed method is capable of aligning the histological slice with an oblique slice in MR volume.
---
paper_title: Cascaded slice to volume registration for moving fetal FMRI

paper_content:

Motion correction of MRI sequences is a very active area of research. Several postprocessing techniques for volume correction and more recently slice correction have been proposed. Slice motion correction of fMRI data typically involves iterative registration of slices to a target volume. The target volume is usually reconstructed at each iteration using current slice motion estimates, with all possible views of the subject. However, in the presence of large movements, the quality of the reconstruction can be greatly degraded without a good initialization, and this can in turn greatly affect the accuracy and precision of the slice registration. This paper introduces a cascaded motion correction pipeline that uses subsets of data to generate coherent volumes for initial target volume based slice registration in fMRI. The pipeline incorporates distortion correction, frame motion estimation and two levels of slice to volume registration. The proposed method has been tested on moving adult and fetal fMRI data.
---
paper_title: 2D/3D registration of endoscopic ultrasound to CT volume data.

paper_content:

This paper describes a computer-aided navigation system using image fusion to support endoscopic interventions such as the accurate collection of biopsy specimens. An endoscope provides the physician with real-time ultrasound (US) and a video image. An image slice that corresponds to the corresponding image from the US scan head is derived from a preoperative computed tomography (CT) or magnetic resonance image volume data set using oblique reformatting and displayed side by side with the US image. The position of the image acquired by the US scan head is determined by a miniaturized electromagnetic tracking system (EMTS) after calibrating the endoscope's scan head. The transformation between the patient coordinate system and the preoperative data set is calculated using a 2D/3D registration. This is achieved by calibrating an intraoperative interventional CT slice with an optical tracking system (OTS) using the same algorithm as for the US calibration. The slice is then used for 2D/3D registration with the coordinate system of the preoperative volume. The fiducial registration error (FRE) for the US calibration was 2.0 mm +/- 0.4 mm; the interventional CT FRE was 0.36 +/- 0.12 mm; and the 2D/3D registration target registration error (TRE) was 1.8 +/- 0.3 mm. The point-to-point registration between the OTS and the EMTS had an FRE of 0.9 +/- 0.4 mm. Finally, we found an overall TRE for the complete system to be 3.9 +/- 0.6 mm.
---
paper_title: Dynamic MR-based respiratory motion compensation for hybrid PET/MR system

paper_content:

This paper describes a novel respiratory motion compensation (MC) technique for hybrid PET/MR system. Existing PET/MR respiratory MC techniques mainly rely on respiratory gating to reconstruct 3D MR images for different respiratory phases, and use image registration techniques to estimate the deformation between different respiratory phases. A well known limitation of respiratory gating is its assumption that breathing motion is perfectly periodic, therefore MC techniques based on gated MR are sensitive to irregular breathing patterns. To address this limitation, the proposed technique uses dynamic 2D MRIs for PET/MR respiratory motion estimation. A static 3D MRI and a series of dynamic 2D MRIs are acquired before and during the PET data acquisition, respectively. The dynamic 2D MRIs are registered against the static 3D MRI using a novel deformable 2D+t to 3D image registration method to derive a 3D+t deformation field, which is then used for motion corrected PET reconstruction. The proposed method is validated on synthetic PET/MR data and real MR data, demonstrating its efficacy to compensate for irregular respiratory motion and to deblur the PET image.
---
paper_title: A Unified Approach to Diffusion Direction Sensitive Slice Registration and 3-D DTI Reconstruction From Moving Fetal Brain Anatomy

paper_content:

This paper presents an approach to 3-D diffusion tensor image (DTI) reconstruction from multi-slice diffusion weighted (DW) magnetic resonance imaging acquisitions of the moving fetal brain. Motion scatters the slice measurements in the spatial and spherical diffusion domain with respect to the underlying anatomy. Previous image registration techniques have been described to estimate the between slice fetal head motion, allowing the reconstruction of 3D a diffusion estimate on a regular grid using interpolation. We propose Approach to Unified Diffusion Sensitive Slice Alignment and Reconstruction (AUDiSSAR) that explicitly formulates a process for diffusion direction sensitive DW-slice-to-DTI-volume alignment. This also incorporates image resolution modeling to iteratively deconvolve the effects of the imaging point spread function using the multiple views provided by thick slices acquired in different anatomical planes. The algorithm is implemented using a multi-resolution iterative scheme and multiple real and synthetic data are used to evaluate the performance of the technique. An accuracy experiment using synthetically created motion data of an adult head and an experiment using synthetic motion added to sedated fetal monkey dataset show a significant improvement in motion-trajectory estimation compared to current state-of-the-art approaches. The performance of the method is then evaluated on challenging but clinically typical in utero fetal scans of four different human cases, showing improved rendition of cortical anatomy and extraction of white matter tracts. While the experimental work focuses on DTI reconstruction (second-order tensor model), the proposed reconstruction framework can employ any 5-D diffusion volume model that can be represented by the spatial parameterizations of an orientation distribution function.
---
paper_title: A standardized method for brain-cutting suitable for both stereology and MRI-brain co-registration

paper_content:

Abstract We have developed an agar-embedding method for brain-slicing that minimizes the geometrical distortions which arise from handling and slicing the fixed postmortem brain. To facilitate postmortem brain-magnetic resonance imaging (MRI) co-registration, each hemisphere is processed separately. We embed the fixed brain hemisphere with reference markers in agar. The block containing the brain and markers is sliced at a fixed interval using a rotary slicer. Each slice is photographed with a high-resolution digital camera. The digital images are realigned as a 3-dimensional volume via a control point-based registration method for multi-slice registration. The realigned multiple slices of the reconstructed postmortem hemisphere are then co-registered to corresponding slices of an in vivo reference MRI-volume. We illustrate these postmortem MRI-brain co-registration methods to correlate in vivo T 2 -weighted MRI hyperintensities in gray and white matter with underlying pathology. For design-based stereology, the volume of interest (VOI) is defined using reproducible anatomical boundaries. This method is suitable for stereologic measures of structures ranging from defined nuclei to whole brain.
---
paper_title: Single slice US-MRI registration for neurosurgical MRI-guided US

paper_content:

Image-based ultrasound to magnetic resonance image (US-MRI) registration can be an invaluable tool in image-guided neuronavigation systems. State-of-the-art commercial and research systems utilize image-based registration to assist in functions such as brain-shift correction, image fusion, and probe calibration. Since traditional US-MRI registration techniques use reconstructed US volumes or a series of tracked US slices, the functionality of this approach can be compromised by the limitations of optical or magnetic tracking systems in the neurosurgical operating room. These drawbacks include ergonomic issues, line-of-sight/magnetic interference, and maintenance of the sterile field. For those seeking a US vendor-agnostic system, these issues are compounded with the challenge of instrumenting the probe without permanent modification and calibrating the probe face to the tracking tool. To address these challenges, this paper explores the feasibility of a real-time US-MRI volume registration in a small virtual craniotomy site using a single slice. We employ the Linear Correlation of Linear Combination (LC2) similarity metric in its patch-based form on data from MNI’s Brain Images for Tumour Evaluation (BITE) dataset as a PyCUDA enabled Python module in Slicer. By retaining the original orientation information, we are able to improve on the poses using this approach. To further assist the challenge of US-MRI registration, we also present the BOXLC2 metric which demonstrates a speed improvement to LC2, while retaining a similar accuracy in this context.
---
paper_title: Self-similarity Weighted Mutual Information: A New Nonrigid Image Registration Metric

paper_content:

Extending mutual information (MI), which has been widely used as a similarity measure for rigid registration of multi-modal images, to deformable registration is an active field of research. We propose a self-similarity weighted graph-based implementation of α-mutual information (α-MI) for nonrigid image registration. The new Self Similarity \(\underline \alpha\)-MI (SeSaMI) metric takes local structures into account and is robust against signal non-stationarity and intensity distortions. We have used SeSaMI as the similarity measure in a regularized cost function with B-spline deformation field. Since the gradient of SeSaMI can be derived analytically, the cost function can be efficiently optimized using stochastic gradient descent. We show that SeSaMI produces a robust and smooth cost function and outperforms the state of the art statistical based similarity metrics in simulation and using data from image-guided neurosurgery.
---
paper_title: A comparative study on manual and automatic slice-to-volume registration of CT images

paper_content:

In order to assess the clinical relevance of a slice-to-volume registration algorithm, this technique was compared to manual registration. Reformatted images obtained from a diagnostic CT examination of the lower abdomen were reviewed and manually registered by 41 individuals. The results were refined by the algorithm. Furthermore, a fully automatic registration of the single slices to the whole CT examination, without manual initialization, was also performed. The manual registration error for rotation and translation was found to be 2.7±2.8 ° and 4.0±2.5 mm. The automated registration algorithm significantly reduced the registration error to 1.6±2.6 ° and 1.3±1.6 mm (p = 0.01). In 3 of 41 (7.3%) registration cases, the automated registration algorithm failed completely. On average, the time required for manual registration was 213±197 s; automatic registration took 82±15 s. Registration was also performed without any human interaction. The resulting registration error of the algorithm without manual pre-registration was found to be 2.9±2.9 ° and 1.1±0.2 mm. Here, a registration took 91±6 s, on average. Overall, the automated registration algorithm improved the accuracy of manual registration by 59% in rotation and 325% in translation. The absolute values are well within a clinically relevant range.
---
paper_title: Non-rigid 2D-3D Medical Image Registration using Markov Random Fields

paper_content:

The aim of this paper is to propose a novel mapping algorithm between 2D images and a 3D volume seeking simultaneously a linear plane transformation and an in-plane dense deformation. We adopt a metric free locally over-parametrized graphical model that combines linear and deformable parameters within a coupled formulation on a 5-dimensional space. Image similarity is encoded in singleton terms, while geometric linear consistency of the solution (common/single plane) and in-plane deformations smoothness are modeled in a pair-wise term. The robustness of the method and its promising results with respect to the state of the art demonstrate the extreme potential of this approach.
---
paper_title: Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI

paper_content:

Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acquired by these techniques, inter-slice motion leads to severe motion artifacts that are apparent in out-of-plane views. Slice sequential acquisitions do not enable 3-D volume representation. In this study, we have developed a novel technique based on a slice acquisition model, which enables the reconstruction of a volumetric image from multiple-scan slice acquisitions. The super-resolution volume reconstruction is formulated as an inverse problem of finding the underlying structure generating the acquired slices. We have developed a robust M-estimation solution which minimizes a robust error norm function between the model-generated slices and the acquired slices. The accuracy and robustness of this novel technique has been quantitatively assessed through simulations with digital brain phantom images as well as high-resolution newborn images. We also report here successful application of our new technique for the reconstruction of volumetric fetal brain MRI from clinically acquired data.
---
paper_title: Simulation and Fully Automatic Multimodal Registration of Medical Ultrasound

paper_content:

The fusion of 3D freehand ultrasound with CT and CTA has benefits for a variety of clinical applications, however a lot of manual work is usually required for correct registration. We developed new methods that allow one to simulate medical ultrasound from CT in real-time, reproducing the majority of ultrasonic imaging effects. The second novelty is a robust similarity measure that assesses the correlation of a combination of multiple signals extracted from CT with ultrasound, without knowing the influence of each signal. This serves as the foundation of a fully automatic registration, which aligns a freehand ultrasound sweep with the corresponding 3D modality using a rigid or an affine transformation model, without any manual interaction. We also present the used initialization, global and local parameter optimization schemes, and validation on abdominal CTA and ultrasound imaging of 10 patients.
---
paper_title: An overlap invariant entropy measure of 3D medical image alignment

paper_content:

Abstract This paper is concerned with the development of entropy-based registration criteria for automated 3D multi-modality medical image alignment. In this application where misalignment can be large with respect to the imaged field of view, invariance to overlap statistics is an important consideration. Current entropy measures are reviewed and a normalised measure is proposed which is simply the ratio of the sum of the marginal entropies and the joint entropy. The effect of changing overlap on current entropy measures and this normalised measure are compared using a simple image model and experiments on clinical image data. Results indicate that the normalised entropy measure provides significantly improved behaviour over a range of imaged fields of view.
---
paper_title: Recovery of Slice Rotations with the Stack Alignment Transform in Cardiac MR Series

paper_content:

Generic solutions for misalignment correction in image stacks are based on slice-byslice registration to a reference volume. Slice misalignment in cardiac Magnetic Resonance cine series typically includes translational and rotational components. Recovery of rotation in medial down to apical slices with the basic slice-to-volume registration may provide unreliable results because of rotational symmetry of the short-axis view of the left ventricle. This paper describes a novel method for correction of stack misalignment in cardiac cine series with recovery of the rotational component. The core of the presented method is a custom spatial transform which parametrises a set of independent centred rigid transforms, one per slice in a stack. The stack alignment transform was designed to circumvent the weaknesses associated with slice-to-volume registration and improve the reliability of misalignment correction over the slice-by-slice registration approach. The image similarity metric for every iteration during the stack alignment transform optimisation is calculated on the whole image stack with all slice correction parameters contributing to the result. The presented method was evaluated on 50 clinical datasets and the results demonstrate an improvement of misalignment correction with the proposed transform. The study also provides evidence of the importance of recovery of slice rotations in this application; the presented method is general enough to be applied to similar problems in other image modalities.
---
paper_title: Confocal bi-protocol: a new strategy for isotropic 3D live cell imaging

paper_content:

The conventional approach for microscopic 3D cellular imaging is based on axial through-stack image series which has some significant limitations such as anisotropic resolution and axial aberration. To overcome these drawbacks, we have recently introduced an alternative approach based on micro-rotation image series. Unfortunately, this new technique suffers from a huge burden of computation that makes its use quite difficult for current applications. To address these problems we propose a new imaging strategy called bi-protocol, which consists of coupling micro-rotation acquisition and conventional z-stack acquisition. We experimentally prove bi-protocol 3D reconstruction produces similar quality to that of pure micro-rotation, but offers the advantage of reduced computation burden because it uses the z-stack volume to accelerate the registration of the micro-rotation images.
---
paper_title: Registration and fusion of SPECT, high-resolution MRI, and interventional MRI for thermal ablation of prostate cancer

paper_content:

We are investigating interventional MRI (iMRI) guided radiofrequency thermal ablation for the minimally invasive treatment of the prostate cancer. Nuclear medicine can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of functional images such as nuclear medicine SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high-resolution MRI volume. Then by registering the high-resolution MR image with live-time iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to live-time iMRI images for improved tumor targeting. For the first step, we used a three-dimensional mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm with special features. The concept was tested using image data from three patients and three volunteers. The SV registration accuracy was 0.4 mm /spl plusmn/ 0.2 mm as compared to our volume- to-volume registration that was previously shown to be quite accurate for these image pairs. With our image registration and fusion software, simulation experiments show that it is quite feasible to incorporate SPECT and high-resolution MRI into the iMRI-guided minimally invasive treatment procedures.
---
paper_title: Registration of two-dimensional cardiac images to preprocedural three-dimensional images for interventional applications

paper_content:

PURPOSE ::: To evaluate the accuracy and efficiency of rigid-body registration of two-dimensional fast cine and real-time cardiac images to high-resolution and SNR three-dimensional preprocedural reference volumes for application during MRI-guided interventional procedures. ::: ::: ::: MATERIALS AND METHODS ::: Mutual information (MI) and correlation ratio (CR) similarity measures were evaluated. The dependence of registration accuracy and efficiency on different resolution and SNR parameters, and also on cardiac-phase differences was evaluated in a porcine model. Two-dimensional images were initially misoriented at distances (d) of 2-10 mm, and rotations of +/-5 degrees about all axes. Registration error and computation time were evaluated, and performance was also assessed visually. ::: ::: ::: RESULTS ::: The maximum registration error using MI (<2.7 mm and <3.6 degrees ) occurred for d = 10 mm, misrotation of +/-5 degrees , and relative SNR = 1. The computation time was 15 seconds for MI and 10 seconds for CR. ::: ::: ::: CONCLUSION ::: Registration accuracy was not highly dependent on the relative timing, within the cycle, between the two-dimensional and three-dimensional images. Registration using CR was faster than that using MI, although accuracy was marginally higher with MI. J.
---
paper_title: Atlas-Based Indexing of Brain Sections via 2-D to 3-D Image Registration

paper_content:

A 2-D to 3-D nonlinear intensity-based registration method is proposed in which the alignment of histological brain sections with a volumetric brain atlas is performed. First, sparsely cut brain sections were linearly matched with an oblique slice automatically extracted from the atlas. Second, a planar-to-curved surface alignment was employed in order to match each section with its corresponding image overlaid on a curved-surface within the atlas. For the latter, a PDE-based registration technique was developed that is driven by a local normalized-mutual-information similarity measure. We demonstrate the method and evaluate its performance with simulated and real data experiments. An atlas-guided segmentation of mouse brains' hippocampal complex, retrieved from the Mouse Brain Library (MBL) database, is demonstrated with the proposed algorithm.
---
paper_title: Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images.

paper_content:

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.
---
paper_title: Model-based 3D/2D deformable registration of MR images

paper_content:

A method is proposed for automatic registration of 3D preoperative magnetic resonance images of deformable tissue to a sequence of its 2D intraoperative images. The algorithm employs a dynamic continuum mechanics model of the deformation and similarity (distance) measures such as correlation ratio, mutual information or sum of squared differences for registration. The registration is solely based on information present in the 3D preoperative and 2D intraoperative images and does not require fiducial markers, feature extraction or image segmentation. Results of experiments with a biopsy training breast phantom show that the proposed method can perform well in the presence of large deformations. This is particularly useful for clinical applications such as MR-based breast biopsy where large tissue deformations occur.
---
paper_title: Improved Motion Correction in fMRI by Joint Mapping of Slices into an Anatomical Volume

paper_content:

Motion correction in fMRI time series is essential for accurate statistical analyses. Typically motion correction is applied in a rigid fashion between time series volumes. Such corrections assume no relative motion between slices. An improved motion correction scheme, map-slice-to-volume (MSV), was developed previously using mutual information (MI) to register individual fMRI slices onto an anatomical volume to account for inter-slice motion [6]. As each slice’s orientation represents a statistically independent sampling of the patient’s motion at multiple intervals throughout the acquisition, a smoothed estimate of the patient’s trajectory in time can be computed by jointly estimating each slice’s orientation while minimizing the implied acceleration of the patient’s head subject to applying a priori, slice-related weights based on known registration reliabilities. The results of this joint mapping of slices into a volume (JMSV) show further substantial improvement in slice registration and subsequent motion estimation.
---
paper_title: Sparse Registration for Three-Dimensional Stress Echocardiography

paper_content:

Three-dimensional (3-D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction. It involves evaluating wall motion of the left ventricle, by visually analyzing ultrasound images obtained in rest and in different stages of stress. Since the acquisitions are performed minutes apart, variabilities may exist in the visualized cross-sections. To improve anatomical correspondence between rest and stress, aligning the images is essential. We developed a new intensity-based, sparse registration method to retrieve standard anatomical views from 3-D stress images that were equivalent to the manually selected views in the rest images. Using sparse image planes, the influence of common image artifacts could be reduced. We investigated different similarity measures and different levels of sparsity. The registration was tested using data of 20 patients and quantitatively evaluated based on manually defined anatomical landmarks. Alignment was best using sparse registration with two long-axis and two short-axis views; registration errors were reduced significantly, to the range of interobserver variabilities. In 91% of the cases, the registration result was qualitatively assessed as better than or equal to the manual alignment. In conclusion, sparse registration improves the alignment of rest and stress images, with a performance similar to manual alignment. This is an important step towards objective quantification in 3-D stress echocardiography.
---
paper_title: Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

paper_content:

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.
---
paper_title: Co-registration of Multiple Postmortem Brain Slices to Corresponding MRIs Using Voxel Similarity Measures and Slice-to-Volume Transformation

paper_content:

New methods to register multiple hemispheric slices of the postmortem brain to anatomically corresponding in-vivo MRI slices within a 3D volumetric MRI are presented. Gel-embedding and fiducial markers are used to reduce geometrical distortions in the postmortem brain volume. The registration algorithm relies on a recursive extraction of warped MRI slices from the reference MRI volume using a modified non-linear polynomial transformation until matching slices are found. Eight different voxel similarity measures are tested to get the best co-registration cost and the results show that combination of two different similarity measures shows the best performance. After validating the implementation and approach through simulation studies, the presented methods are applied to real data. The results demonstrate the feasibility and practicability of the presented co­registration methods, thus providing a means of MR signal analysis and histological examination of tissue lesions via co­registered images of postmortem brain slices and their corresponding MRI sections. With this approach, it is possible to investigate the pathology of a disease through both routinely acquired MRls and postmortem brain slices, thus improving the understanding of the pathological substrates and their progression.
---
paper_title: Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

paper_content:

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.
---
paper_title: Active Contours without Edges

paper_content:

We propose a new model for active contours to detect objects in a given image, based on techniques of curve evolution, Mumford-Shah (1989) functional for segmentation and level sets. Our model can detect objects whose boundaries are not necessarily defined by the gradient. We minimize an energy which can be seen as a particular case of the minimal partition problem. In the level set formulation, the problem becomes a "mean-curvature flow"-like evolving the active contour, which will stop on the desired boundary. However, the stopping term does not depend on the gradient of the image, as in the classical active contour models, but is instead related to a particular segmentation of the image. We give a numerical algorithm using finite differences. Finally, we present various experimental results and in particular some examples for which the classical snakes methods based on the gradient are not applicable. Also, the initial curve can be anywhere in the image, and interior contours are automatically detected.
---
paper_title: Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies

paper_content:

Targeted prostate biopsy is challenging because no currently established imaging modality is both accurate for prostate cancer diagnosis and cost-effective for real-time procedure guidance. A system that fuses real-time transrectal ultrasound images with previously acquired endorectal coil MRI images for prostate biopsy guidance is presented here. The system uses electromagnetic tracking and intraoperative image registration to superimpose the MRI data on the ultrasound image. Prostate motion is tracked and compensated for without the need for fiducial markers. The accuracy of the system in phantom studies was shown to be 2.4 ± 1.2 mm. The fusion system has been used in more than 20 patients to guide biopsies with almost no modification of the conventional protocol. Retrospective clinical evaluation suggests that clinically acceptable spatial accuracy can be achieved.
---
paper_title: Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices

paper_content:

Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D/3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fetal Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available.
---
paper_title: Temporal registration of partial data using particle filtering

paper_content:

We propose a particle filtering framework for rigid registration of a model image to a time-series of partially observed images. The method incorporates a model-based segmentation technique in order to track the pose dynamics of an underlying observed object with time. An applicable algorithm is derived by employing the proposed framework for registration of a 3D model of an anatomical structure, which was segmented from preoperative images, to consecutive axial 2D slices of a magnetic resonance imaging (MRI) scan, which are acquired intraoperatively over time. The process is fast and robust with respect to image noise and clutter, variations of illumination, and different imaging modalities.
---
paper_title: User-friendly freehand ultrasound calibration using Lego bricks and automatic registration

paper_content:

PURPOSE ::: As an inexpensive, noninvasive, and portable clinical imaging modality, ultrasound (US) has been widely employed in many interventional procedures for monitoring potential tissue deformation, surgical tool placement, and locating surgical targets. The application requires the spatial mapping between 2D US images and 3D coordinates of the patient. Although positions of the devices (i.e., ultrasound transducer) and the patient can be easily recorded by a motion tracking system, the spatial relationship between the US image and the tracker attached to the US transducer needs to be estimated through an US calibration procedure. Previously, various calibration techniques have been proposed, where a spatial transformation is computed to match the coordinates of corresponding features in a physical phantom and those seen in the US scans. However, most of these methods are difficult to use for novel users. ::: ::: ::: METHODS ::: We proposed an ultrasound calibration method by constructing a phantom from simple Lego bricks and applying an automated multi-slice 2D-3D registration scheme without volumetric reconstruction. The method was validated for its calibration accuracy and reproducibility. ::: ::: ::: RESULTS ::: Our method yields a calibration accuracy of [Formula: see text] mm and a calibration reproducibility of 1.29 mm. ::: ::: ::: CONCLUSION ::: We have proposed a robust, inexpensive, and easy-to-use ultrasound calibration method.
---
paper_title: MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration

paper_content:

Deformable registration of images obtained from different modalities remains a challenging task in medical image analysis. This paper addresses this important problem and proposes a modality independent neighbourhood descriptor (MIND) for both linear and deformable multi-modal registration. Based on the similarity of small image patches within one image, it aims to extract the distinctive structure in a local neighbourhood, which is preserved across modalities. The descriptor is based on the concept of image self-similarity, which has been introduced for non-local means filtering for image denoising. It is able to distinguish between different types of features such as corners, edges and homogeneously textured regions. MIND is robust to the most considerable differences between modalities: non-functional intensity relations, image noise and non-uniform bias fields. The multi-dimensional descriptor can be efficiently computed in a dense fashion across the whole image and provides point-wise local similarity across modalities based on the absolute or squared difference between descriptors, making it applicable for a wide range of transformation models and optimisation algorithms. We use the sum of squared differences of the MIND representations of the images as a similarity metric within a symmetric non-parametric Gauss-Newton registration framework. In principle, MIND would be applicable to the registration of arbitrary modalities. In this work, we apply and validate it for the registration of clinical 3D thoracic CT scans between inhale and exhale as well as the alignment of 3D CT and MRI scans. Experimental results show the advantages of MIND over state-of-the-art techniques such as conditional mutual information and entropy images, with respect to clinically annotated landmark locations.
---
paper_title: The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration

paper_content:

Over the last five years, new “voxel-based” approaches have allowed important progress in multimodal image registration, notably due to the increasing use of information-theoretic similarity measures. Their wide success has led to the progressive abandon of measures using standard image statistics (mean and variance). Until now, such measures have essentially been based on heuristics. In this paper, we address the determination of a new measure based on standard statistics from a theoretical point of view. We show that it naturally leads to a known concept of probability theory, the correlation ratio. In our derivation, we take as the hypothesis the functional dependence between the image intensities. Although such a hypothesis is not as general as possible, it enables us to model the image smoothness prior very easily. We also demonstrate results of multimodal rigid registration involving Magnetic Resonance (MR), Computed Tomography (CT), and Positron Emission Tomography (PET) images. These results suggest that the correlation ratio provides a good trade-off between accuracy and robustness.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: MRI-Guided Prostate Motion Tracking using Multislice-to-Volume Registration

paper_content:

Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2010-07-13 16:41:52.223
---
paper_title: Reconstruction of fetal brain MRI with intensity matching and complete outlier removal

paper_content:

We propose a method for the reconstruction of volumetric fetal MRI from 2D slices, comprising super-resolution reconstruction of the volume interleaved with slice-to-volume registration to correct for the motion. The method incorporates novel intensity matching of acquired 2D slices and robust statistics which completely excludes identified misregistered or corrupted voxels and slices. The reconstruction method is applied to motion-corrupted data simulated from MRI of a preterm neonate, as well as 10 clinically acquired thick-slice fetal MRI scans and three scan-sequence optimized thin-slice fetal datasets. The proposed method produced high quality reconstruction results from all the datasets to which it was applied. Quantitative analysis performed on simulated and clinical data shows that both intensity matching and robust statistics result in statistically significant improvement of super-resolution reconstruction. The proposed novel EM-based robust statistics also improves the reconstruction when compared to previously proposed Huber robust statistics. The best results are obtained when thin-slice data and the correct approximation of the point spread function is used. This paper addresses the need for a comprehensive reconstruction algorithm of 3D fetal MRI, so far lacking in the scientific literature.
---
paper_title: Registration of 2D histological sections with 3D micro-CT datasets from small animal vertebrae and tibiae

paper_content:

The aim of this study was the registration of digitized thin 2D sections of mouse vertebrae and tibiae used for histomorphometry of trabecular bone structure into 3D micro computed tomography (μCT) datasets of the samples from which the sections were prepared. Intensity-based and segmentation-based registrations (SegRegs) of 2D sections and 3D μCT datasets were applied. As the 2D sections were deformed during their preparation, affine registration for the vertebrae was used instead of rigid registration. Tibiae sections were additionally cut on the distal end, which subsequently undergone more deformation so that elastic registration was necessary. The Jaccard distance was used as registration quality measure. The quality of intensity-based registrations and SegRegs was practically equal, although precision errors of the elastic registration of segmentation masks in tibiae were lower, while those in vertebrae were lower for the intensity-based registration. Results of SegReg significantly depended on the ...
---
paper_title: A Fast 3D Volume Reconstruction for Confocal Micro-rotation Cell Imaging

paper_content:

We propose a fast reconstruction algorithm for nonadherent living cell images from micro-rotational confocal microscopy. This procedure is called bi-protocol. Its key point consists in coupling micro-rotation data and conventional z-stack. Doing so, the estimation of the micro-rotation slice positions becomes a slice-to-volume registration allowing to reduce drastically the computing time. The validation of our procedure is performed on real data which gives significant improvement compared with the conventional z-stacking imaging.
---
paper_title: Slice-to-volume registration using mutual information between probabilistic image classifications

paper_content:

Intensity based registration algorithms have proved to be accurate and robust for 3D-3D registration tasks. However, these methods utilise the information content within an image, and therefore their performance is hindered for image data that is sparse. This is the case for the registration of a single image slice to a 3D image volume. There are some important applications that could benefit from improved slice-to-volume registration, for example, the planning of magnetic resonance (MR) scans or cardiac MR imaging, where images are acquired as stacks of single slices. We have developed and validated an information based slice-to-volume registration algorithm that uses vector valued probabilistic images of tissue classification that have been derived from the original intensity images. We believe that using such methods inherently incorporates into the registration framework more information about the images, especially in images containing severe partial volume artifacts. Initial experimental results indicate that the suggested method can achieve a more robust registration compared to standard intensity based methods for the rigid registration of a single thick brain MR slice, containing severe partial volume artifacts in the through-plane direction, to a complete 3D MR brain volume.
---
paper_title: Particle filtering for slice-to-volume motion correction in EPI based functional MRI

paper_content:

Head movement during scanning introduces artificial signal changes and impedes activation detection in fMRI studies. The head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) sequence can be estimated and compensated by aligning the images onto a reference volume through image registration. Registering EPI images volume by volume fails to consider head motion between slices, leading to biased head motion estimates. Slice-to-volume registration is used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, it is prone to image noise and geometric distortion, resulting in high variance estimates. In this work, we propose a Gaussian particle filter based head motion tracking algorithm to reduce the image misregistration errors. The algorithm models head motion by using a dynamic state space model (SSM) to model continuous slice acquisition thereby providing more accurate motion estimates and voxel position estimates. We demonstrate significant performance improvement of the proposed approach as compared to previous registration-only methods of head motion estimation.
---
paper_title: Nonrigid Registration of Ultrasound and MRI Using Contextual Conditioned Mutual Information

paper_content:

Mutual information (MI) quantifies the information that is shared between two random variables and has been widely used as a similarity metric for multi-modal and uni-modal image registration. A drawback of MI is that it only takes into account the intensity values of corresponding pixels and not of neighborhoods. Therefore, it treats images as “bag of words” and the contextual information is lost. In this work, we present Contextual Conditioned Mutual Information (CoCoMI), which conditions MI estimation on similar structures. Our rationale is that it is more likely for similar structures to undergo similar intensity transformations. The contextual analysis is performed on one of the images offline. Therefore, CoCoMI does not significantly change the registration time. We use CoCoMI as the similarity measure in a regularized cost function with a B-spline deformation field and efficiently optimize the cost function using a stochastic gradient descent method. We show that compared to the state of the art local MI based similarity metrics, CoCoMI does not distort images to enforce erroneous identical intensity transformations for different image structures. We further present the results on nonrigid registration of ultrasound (US) and magnetic resonance (MR) patient data from image-guided neurosurgery trials performed in our institute and publicly available in the BITE dataset. We show that CoCoMI performs significantly better than the state of the art similarity metrics in US to MR registration. It reduces the average mTRE over 13 patients from 4.12 mm to 2.35 mm, and the maximum mTRE from 9.38 mm to 3.22 mm.
---
paper_title: Inter-breath-hold registration for the production of high resolution cardiac MR volumes

paper_content:

High resolutionMRI images of the beating heart permit observation of detailed anatomical features and enable quantification of small changes in metrics of cardiac function. To obtain approximately isotropic sampling with an adequate spatial and temporal resolution, these images need to be acquired in multiple breath-holds. They are, therefore, often affected by through-plane discontinuities due to inconsistent breath-hold positions. This paper presents a method to correct for these discontinuities by performing breath-hold-by-breath-hold registration of high resolution 3D data to radial long axis images. The corrected images appear free of discontinuities, and it was found that they could be delineated more reproducibly than uncorrected images. This reduces the sample size required to detect systematic changes in blood pool volume by 57% at end systole and 78% at end diastole.
---
paper_title: Deformable Ultrasound Registration without Reconstruction

paper_content:

Ultrasound (US) imaging is often proposed as an interoperative imaging modality. This use nearly always requires that the collected data be registered to preoperative data of another modality. Existing intensity-based registration approaches all begin by reconstructing a 3D US volume from the collected 2D slices. We propose to directly register the set of 2D slices to the preoperative images. We argue this has a number of advantages, including the omission of the potentially complex reconstruction step, greater adaptability of the similarity measures, and easier parallelization. We describe a system for performing this task and present results on phantom data that show that our slice based method consistently outperforms a reconstruction based method in both speed and accuracy.
---
paper_title: Sub-cortical brain structure segmentation using F-CNN'S

paper_content:

In this paper we propose a deep learning approach for segmenting sub-cortical structures of the human brain in Magnetic Resonance (MR) image data. We draw inspiration from a state-of-the-art Fully-Convolutional Neural Network (F-CNN) architecture for semantic segmentation of objects in natural images, and adapt it to our task. Unlike previous CNN-based methods that operate on image patches, our model is applied on a full blown 2D image, without any alignment or registration steps at testing time. We further improve segmentation results by interpreting the CNN output as potentials of a Markov Random Field (MRF), whose topology corresponds to a volumetric grid. Alpha-expansion is used to perform approximate inference imposing spatial volumetric homogeneity to the CNN priors. We compare the performance of the proposed pipeline with a similar system using Random Forest-based priors, as well as state-of-art segmentation algorithms, and show promising results on two different brain MRI datasets.
---
paper_title: Correction of misaligned slices in multi-slice MR cardiac examinations by using slice-to-volume registration

paper_content:

One of the main challenges with magnetic resonance (MR) cardiac image acquisition is to account for cardiac motion due to respiration. A popular technique to reduce respiratory motion is to perform a multi-slice acquisition in which a patient holds their breath multiple times during the scan. This paper explores the feasibility of using rigid slice-to-volume registration to correct for misalignments of slice stacks in such images due to differing breath-hold positions. The experimental results indicate that slice-to-volume registration is sufficiently accurate and robust to compensate for the typical misalignments expected. We show that correction of misalignments in such data results in anatomically more correct images, as well as improved left ventricular volume measurements. It also shows that by correcting for misalignments in short axis (SA) images, one can improve the interstudy reproducibility and hence reduce the number of samples needed for cardiac MR studies to show the same statistical significance.
---
paper_title: Automatic 3-D Breath-Hold Related Motion Correction of Dynamic Multislice MRI

paper_content:

Magnetic resonance (MR) cine images are often used to clinically assess left ventricular cardiac function. In a typical study, multiple 2-D long axis (LA) and short axis (SA) cine images are acquired, each in a different breath-hold. Differences in lung volume during breath-hold and overall patient motion distort spatial alignment of the images thus complicating spatial integration of all image data in three dimensions. We present a fully automatic postprocessing approach to correct these slice misalignments. The approach is based on the constrained optimization of the intensity similarity of intersecting image lines after the automatic definition of a region of interest. It uses all views and all time frames simultaneously. Our method models both in-plane and out-of-plane translations and full 3-D rotations, can be applied retrospectively and does not require a cardiac wall segmentation. The method was validated on both healthy volunteer and patient data with simulated misalignments, as well as on clinical multibreath-hold patient data. For the simulated data, subpixel accuracy could be obtained using translational correction. The possibilities and limitations of rotational correction were investigated and discussed. For the clinical multibreath-hold patient data sets, the median discrepancy between manual SA and LA contours was reduced from 2.83 to 1.33 mm using the proposed correction method. We have also shown the usefulness of the correction method for functional analysis on clinical image data. The same clinical multibreath-hold data sets were resegmented after positional correction, taking newly available complementary information of intersecting slices into account, further reducing the median discrepancy to 0.43 mm. This is due to the integration of the 2-D slice information into 3-D space.
---
paper_title: Ultrasound-CT registration of vertebrae without reconstruction

paper_content:

Purpose ::: While robust and accurate, our previously developed volume-to-volume ultrasound-CT registration of vertebrae required that the 2D ultrasound slices be reconstructed into a 3D volume, a time-consuming step that increased the total registration time per vertebra. We have modified our registration technique to a slices-to-volume strategy to eliminate the ultrasound reconstruction step in order to make the total registration time more practical intraoperatively.
---
paper_title: Intersection based registration of slice stacks to form 3D images of the human fetal brain

paper_content:

Clinical fetal MR imaging of the brain commonly makes use of fast 2D acquisitions of multiple sets of approximately orthogonal 2D slices. We and others have previously proposed an iterative slice-to-volume registration process to recover a geometrically consistent 3D image. However, these approaches depend on a 3D volume reconstruction step during the slice alignment. This is both computationally expensive and makes the convergence of the registration process poorly defined. In this paper our key contribution is a new approach which considers the collective alignment of all slices directly, via shared structure in their intersections, rather than to an estimated 3D volume. We derive an analytical expression for the gradient of the collective similarity of the slices along their intersections, with respect to the 3D location and orientation of each 2D slice. We include examples of the approach applied to simulated data and clinically acquired fetal images.
---
paper_title: Determining histology-MRI slice correspondences for defining MRI-based disease signatures of prostate cancer

paper_content:

Abstract Mapping the spatial disease extent in a certain anatomical organ/tissue from histology images to radiological images is important in defining the disease signature in the radiological images. One such scenario is in the context of men with prostate cancer who have had pre-operative magnetic resonance imaging (MRI) before radical prostatectomy. For these cases, the prostate cancer extent from ex vivo whole-mount histology is to be mapped to in vivo MRI. The need for determining radiology-image-based disease signatures is important for (a) training radiologist residents and (b) for constructing an MRI-based computer aided diagnosis (CAD) system for disease detection in vivo . However, a prerequisite for this data mapping is the determination of slice correspondences (i.e. indices of each pair of corresponding image slices) between histological and magnetic resonance images. The explicit determination of such slice correspondences is especially indispensable when an accurate 3D reconstruction of the histological volume cannot be achieved because of (a) the limited tissue slices with unknown inter-slice spacing, and (b) obvious histological image artifacts (tissue loss or distortion). In the clinic practice, the histology-MRI slice correspondences are often determined visually by experienced radiologists and pathologists working in unison, but this procedure is laborious and time-consuming. We present an iterative method to automatically determine slice correspondence between images from histology and MRI via a group-wise comparison scheme, followed by 2D and 3D registration. The image slice correspondences obtained using our method were compared with the ground truth correspondences determined via consensus of multiple experts over a total of 23 patient studies. In most instances, the results of our method were very close to the results obtained via visual inspection by these experts.
---
paper_title: Registration-based Approach for Reconstruction of High Resolution In Utero Fetal MR Brain Images

paper_content:

Rationale and Objectives This paper describes a novel approach to forming high-resolution MR images of the human fetal brain. It addresses the key problem of fetal motion by proposing a registration-refined compounding of multiple sets of orthogonal fast two-dimensional MRI slices, which are currently acquired for clinical studies, into a single high-resolution MRI volume. Materials and Methods A robust multiresolution slice alignment is applied iteratively to the data to correct motion of the fetus that occurs between two-dimensional acquisitions. This is combined with an intensity correction step and a super-resolution reconstruction step, to form a single high isotropic resolution volume of the fetal brain. Results Experimental validation on synthetic image data with known motion types and underlying anatomy, together with retrospective application to sets of clinical acquisitions, are included. Conclusion Results indicate that this method promises a unique route to acquiring high-resolution MRI of the fetal brain in vivo allowing comparable quality to that of neonatal MRI. Such data provide a highly valuable window into the process of normal and abnormal brain development, which is directly applicable in a clinical setting.
---
paper_title: Automatic Deformable MR-Ultrasound Registration for Image-Guided Neurosurgery

paper_content:

In this work, we present a novel algorithm for registration of 3-D volumetric ultrasound (US) and MR using Robust PaTch-based cOrrelation Ratio (RaPTOR). RaPTOR computes local correlation ratio (CR) values on small patches and adds the CR values to form a global cost function. It is therefore invariant to large amounts of spatial intensity inhomogeneity. We also propose a novel outlier suppression technique based on the orientations of the RaPTOR gradients. Our deformation is modeled with free-form cubic B-splines. We analytically derive the derivatives of RaPTOR with respect to the transformation, i.e., the displacement of the B-spline nodes, and optimize RaPTOR using a stochastic gradient descent approach. RaPTOR is validated on MR and tracked US images of neurosurgery. Deformable registration of the US and MR images acquired, respectively, preoperation and postresection is of significant clinical significance, but challenging due to, among others, the large amount of missing correspondences between the two images. This work is also novel in that it performs automatic registration of this challenging dataset. To validate the results, we manually locate corresponding anatomical landmarks in the US and MR images of tumor resection in brain surgery. Compared to rigid registration based on the tracking system alone, RaPTOR reduces the mean initial mTRE over 13 patients from 5.9 to 2.9 mm, and the maximum initial TRE from 17.0 to 5.9 mm. Each volumetric registration using RaPTOR takes about 30 sec on a single CPU core. An important challenge in the field of medical image analysis is the shortage of publicly available dataset, which can both facilitate the advancement of new algorithms to clinical settings and provide a benchmark for comparison. To address this problem, we will make our manually located landmarks available online.
---
paper_title: Automatic 3D registration for interventional MRI-guided treatment of prostate cancer.

paper_content:

The goal of this research is to register real-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained high-resolution MRI image volume, which in turn can be registered with functional images such as those Urom SPECT. The immediate application is in iMRI-guided treatment of prostate cancer, where additional images are desired to improve tumor targeting. In this article, simulation experiments are performed to demonstrate the feasibility of slice-to-volume registration for this application. We acquired 3D volume images from a 1.5-T MRI system and simulated low-field iMRI image slices by creating thick slices and adding noise. We created a slice-to-volume mutual information registration algorithm with special features to improve robustness. Features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. To assess the quality of registration, we calculated 3D displacements on a voxel-by-voxel basis over a volume of in...
---
paper_title: Multiscale Registration of Real-Time and Prior MRI Data for Image-Guided Cardiac Interventions

paper_content:

Abstract Recently, there is a growing interest in using magnetic resonance imaging (MRI) to guide interventional procedures due to its excellent soft tissue contrast and lack of ionizing radiation compared to traditional radiographic guidance. One of these applications is the use of MRI to guide radio frequency ablation of anatomic substrates, within the left ventricle, responsible for ventricular tachycardia. However, different MRI acquisition schemes have significant tradeoffs between image quality and acquisition time. Guidance using high-quality preoperative 3-D MR images is limited in the case of cardiac interventions because the heart moves dynamically during the procedure. On the other hand, 2-D real-time MR images acquired during the intervention sacrifice image quality for shorter image acquisition time, leading to real-time positional updates of cardiac anatomy. Ideally, we wish to combine the advantages of live feedback from real-time images and accurate visualization of anatomical structures from preoperative images. Therefore, to improve the MRI guidance capabilities for cardiac interventions, we describe a novel multiscale rigid registration framework to correct for respiratory motion between the prior and real-time datasets. In the proposed approach, we use a weighted total variation flow algorithm to extract coarse-to-fine features from the input images and subsequently register the corresponding scales in a hierarchical manner. Registration experiments were performed with in vivo human imaging data, and the target registration error achieved was 1.51 mm. Thus, the feasibility of motion correction in an interventional setting has been demonstrated, which may lead to significant improvements in the guidance of cardiac interventions.
---
paper_title: Registration of Whole-Mount Histology and Volumetric Imaging of the Prostate Using Particle Filtering

paper_content:

Registration of histological slices to volumetric imaging of the prostate is an important task that can be used to optimize imaging for cancer detection. Such registration is challenging due to physical changes of the specimen during excision and fixation, and misalignment of the histological slices during preparation and digital scanning. In this work, we consider a multi-slice to volume registration method in which a stack of sparse, unaligned 2-D whole-mount histological slices is registered to a 3-D volumetric imaging of the prostate. We propose a particle filtering framework to contend with the high dimensionality of the search space and multimodal nature of the optimization. Such framework allows modeling of the uncertainty in the pose of the slices and in the imaged information, in order to derive optimal registration parameters in a Bayesian approach. Intensity-, region-, and point-based similarity metrics were incorporated into the registration algorithm to account for different imaging modalities. We demonstrate and evaluate our method on a diverse set of data that includes a synthetic volume, ex vivo and in vivo magnetic resonance imaging, and in vivo ultrasound.
---
paper_title: A method for registration of 3-D shapes

paper_content:

The authors describe a general-purpose, representation-independent method for the accurate and computationally efficient registration of 3-D shapes including free-form curves and surfaces. The method handles the full six degrees of freedom and is based on the iterative closest point (ICP) algorithm, which requires only a procedure to find the closest point on a geometric entity to a given point. The ICP algorithm always converges monotonically to the nearest local minimum of a mean-square distance metric, and the rate of convergence is rapid during the first few iterations. Therefore, given an adequate set of initial rotations and translations for a particular class of objects with a certain level of 'shape complexity', one can globally minimize the mean-square distance metric over all six degrees of freedom by testing each initial registration. One important application of this method is to register sensed data from unfixtured rigid objects with an ideal geometric model, prior to shape inspection. Experimental results show the capabilities of the registration algorithm on point sets, curves, and surfaces. >
---
paper_title: Object recognition from local scale-invariant features

paper_content:

An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
---
paper_title: Shape registration in implicit spaces using information theory and free form deformations

paper_content:

We present a novel, variational and statistical approach for shape registration. Shapes of interest are implicitly embedded in a higher-dimensional space of distance transforms. In this implicit embedding space, registration is formulated in a hierarchical manner: the mutual information criterion supports various transformation models and is optimized to perform global registration; then, a B-spline-based incremental free form deformations (IFFD) model is used to minimize a sum-of-squared-differences (SSD) measure and further recover a dense local nonrigid registration field. The key advantage of such framework is twofold: 1) it naturally deals with shapes of arbitrary dimension (2D, 3D, or higher) and arbitrary topology (multiple parts, closed/open) and 2) it preserves shape topology during local deformation and produces local registration fields that are smooth, continuous, and establish one-to-one correspondences. Its invariance to initial conditions is evaluated through empirical validation, and various hard 2D/3D geometric shape registration examples are used to show its robustness to noise, severe occlusion, and missing parts. We demonstrate the power of the proposed framework using two applications: one for statistical modeling of anatomical structures, another for 3D face scan registration and expression tracking. We also compare the performance of our algorithm with that of several other well-known shape registration algorithms
---
paper_title: Non-rigid registration using distance functions

paper_content:

Abstract This paper deals with the registration of geometric shapes. Our primary contribution is the use of a simple and robust shape representation (distance functions) for global-to-local alignment. We propose a rigid-invariant variational framework that can deal as well with local non-rigid transformations. To this end, the registration map consists of a linear motion model and a local deformations field, incrementally recovered. In order to demonstrate the performance of the selected representation a simple criterion is considered, the sum of square differences. Empirical validation and promising results were obtained on examples that exhibit large global motion as well as important local deformations and arbitrary topological changes.
---
paper_title: Mapping and characterizing endometrial implants by registering 2D transvaginal ultrasound to 3D pelvic magnetic resonance images

paper_content:

Mapping an endometrial tissue seen in Transvaginal Ultrasound (TVUS) to a Magnetic Resonance (MR) image is an important approach to improve the accuracy of diagnosis and preoperative surgery planning. To date, no technical methods solve this problem. In a clinical setting where only one 2D TVUS image is available, methods in the state of the art require the selection of a 2D MR slice and use 2D/2D deformable image registration. This is a limitation since the standard TVUS and MR imaging techniques used for diagnosing endometriosis are 2D and 3D, respectively. In this work, we introduce a novel deformable slice-to-volume registration process to overcome this limitation. To this end, we register a 2D TVUS image to a 3D MR data in order to transform the TVUS planar image to a curved 2D surface in MR volume. The proposed method facilitates the transfer of two types of information from a 2D TVUS image to the MR volume: 1) the location and shape of small implants and 2) the implants' depth of infiltration in the host tissue. Our TVUS-MR registration method uses contour to surface correspondences through a novel variational one-step deformable Iterative Closest Point (ICP) method. We demonstrate the accuracy of the proposed method by quantitative and qualitative tests on semi-synthetic and clinical data sets. Experimental error analysis shows high registration accuracy, illustrating the robustness of the method.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Registration with Uncertainties and Statistical Modeling of Shapes with Variable Metric Kernels

paper_content:

Registration and modeling of shapes are two important problems in computer vision and pattern recognition. Despite enormous progress made over the past decade, these problems are still open. In this paper, we advance the state of the art in both directions. First we consider an efficient registration method that aims to recover a one-to-one correspondence between shapes and introduce measures of uncertainties driven from the data which explain the local support of the recovered transformations. To this end, a free form deformation is used to describe the deformation model. The transformation is combined with an objective function defined in the space of implicit functions used to represent shapes. Once the registration parameters have been recovered, we introduce a novel technique for model building and statistical interpretation of the training examples based on a variable bandwidth kernel approach. The support on the kernels varies spatially and is determined according to the uncertainties of the registration process. Such a technique introduces the ability to account for potential registration errors in the model. Hand-written character recognition and knowledge-based object extraction in medical images are examples of applications that demonstrate the potentials of the proposed framework.
---
paper_title: Deformable medical image registration: setting the state of the art with discrete methods.

paper_content:

This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.
---
paper_title: Do Convnets Learn Correspondence?

paper_content:

Convolutional neural nets (convnets) trained from massive labeled datasets [1] have substantially improved the state-of-the-art in image classification [2] and object detection [3]. However, visual understanding requires establishing correspondence on a finer level than object category. Given their large pooling regions and training from whole-image labels, it is not clear that convnets derive their success from an accurate correspondence model which could be used for precise localization. In this paper, we study the effectiveness of convnet activation features for tasks requiring correspondence. We present evidence that convnet features localize at a much finer scale than their receptive field sizes, that they can be used to perform intraclass aligment as well as conventional hand-engineered features, and that they outperform conventional features in keypoint prediction on objects from PASCAL VOC 2011 [4].
---
paper_title: A combined corner and edge detector

paper_content:

The problem we are addressing in Alvey Project MMI149 is that of using computer vision to understand the unconstrained 3D world, in which the viewed scenes will in general contain too wide a diversity of objects for topdown recognition techniques to work. For example, we desire to obtain an understanding of natural scenes, containing roads, buildings, trees, bushes, etc., as typified by the two frames from a sequence illustrated in Figure 1. The solution to this problem that we are pursuing is to use a computer vision system based upon motion analysis of a monocular image sequence from a mobile camera. By extraction and tracking of image features, representations of the 3D analogues of these features can be constructed.
---
paper_title: Ultrasound RF time series for tissue typing: first in vivo clinical results

paper_content:

The low diagnostic value of ultrasound in prostate cancer imaging has resulted in an effort to enhance the tumor contrast using ultrasound-based technologies that go beyond traditional B-mode imaging. Ultrasound RF time series, formed by echo samples originating from the same location over a few seconds of imaging, has been proposed and experimentally used for tissue typing with the goal of cancer detection. In this work, for the first time we report the preliminary results of in vivo clinical use of spectral parameters extracted from RF time series in prostate cancer detection. An image processing pipeline is designed to register the ultrasound data to wholemount histopathology references acquired from prostate specimens that are removed in radical prostatectomy after imaging. Support vector machine classification is used to detect cancer in 524 regions of interest of size 5×5 mm, each forming a feature vector of spectral RF time series parameters. Preliminary ROC curves acquired based on RF time series analysis for individual cases, with leave-one-patient-out cross validation, are presented and compared with B-mode texture analysis.
---
paper_title: Fast feature based multi slice to volume registration using phase congruency

paper_content:

Slice to volume registration is very useful in many medical imaging applications, for example, fusing static high resolution three dimensional (3D) image volumes to dynamic two dimensional (2D) slice data for deriving motion information in 3D. Though information theoretic registration methods such as Mutual Information are usually robust, they are time intensive and typically require a high level of field-of-view correspondence between the source and target images. In single slice to volume registration scenarios, where such correspondence is limited, registration accuracy and robustness often deteriorate. In this paper, we present a novel registration method that maintains robustness and accuracy while significantly increasing registration speed. Our approach employs multiple slice (as opposed to single slice) to volume registration, which increases the amount of potential matching information while maintaining a small number of slices and hence facilitates the often necessary high speed dynamic image acquisition. Our proposed registration approach first extracts phase congruency information from the slices/volume using oriented 2D Gabor wavelets. Using local non maximum suppression, we then automatically obtain a robust and accurate set of feature points that are subsequently matched using an Iterative Closest Point (ICP) approach. Validation on BrainWeb simulated magnetic resonance imaging (MRI) data showed significant gains in speed (∼40-fold increase) when compared to conventional Mutual Information based volumetric registration while maintaining comparable robustness and accuracy levels.
---
paper_title: Registration of freehand 3D ultrasound and magnetic resonance liver images

paper_content:

Abstract We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a “bronze standard” registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the “bronze standard” and this is improved to 3.6 mm after running the intensity-based algorithm.
---
paper_title: Cadaver validation of intensity-based ultrasound to CT registration

paper_content:

A method is presented for the registration of tracked B-mode ultrasound images to a CT volume of a femur or pelvis. This registration can allow tracked surgical instruments to be aligned with the CT image or an associated preoperative plan. Our method requires no manual segmentation of either the ultrasound images or the CT volume. The CT and US images are processed to produce images where the image intensity represents the probability of the presence of a bone edge. These images are then registered together using normalised cross-correlation as a similarity measure. The parameter which represents the speed of sound through tissue has also been included in the registration optimisation process. Experiments have been carried out on six cadaveric femurs and three cadaveric pelves. Registration results were compared with a "gold standard" registration acquired using bone implanted fiducial markers. Results show the registration method to be accurate, on average, to 1.7 mm root-mean-square target registration error.
---
paper_title: Correlating 2D histological slice with 3D MRI image volume using smart phone as an interactive tool for muscle study

paper_content:

In muscle dystrophy studies, registration of histological image with MRI image volume enables cross validation of MRI biomarkers using pathological result. However, correlation of 2D histology slice with 3D MRI volume is technically challenging due to the potentially non-orthogonal slice plane and incomplete or distorted histological slice. This paper presents an efficient method to directly perform the 2D-3D registration. The method is unique in that it uses smart phone as a navigation tool for initial alignment followed by an overlap invariant mutual information-based refinement. Experimental results using animal muscle samples images from a 3T MRI and HE stained histological images show that the proposed method is capable of aligning the histological slice with an oblique slice in MR volume.
---
paper_title: Prospective slice-by-slice motion correction reduces false positive activations in fMRI with task-correlated motion

paper_content:

Abstract Objective We aimed to test the hypothesis that slice-by-slice prospective motion correction at 7 T using an optical tracking system reduces the rate of false positive activations in an fMRI group study with a paradigm that involves task-correlated motion. Materials and methods Brain activation during right leg movement was measured using a block design on 15 volunteers, with and without prospective motion correction. Clearly erroneous activations were compared between both cases, at the individual level. Additionally, conventional group analysis was performed. Results The number of falsely activated voxels with T-values higher than 5 was reduced by 48% using prospective motion correction alone, without additional retrospective realignment. In the group analysis, the statistical power was increased — the peak T-value was 26% greater, and the number of voxels in the cluster representing the right leg was increased by a factor of 9.3. Conclusion Slice-by-slice prospective motion correction in fMRI studies with task-correlated motion can substantially reduce false positive activations and increase statistical power.
---
paper_title: Ultrasound-to-computer-tomography registration for image-guided laparoscopic liver surgery

paper_content:

Background: The application of image-guided surgery (IGS) to laparoscopic liver resection and ablation is currently limited, but it would assist in intraoperative decision making regarding oncologic margins, ablation probe placement, and ablation tracking. Methods: Eight spherical surface targets on a liver phantom were imaged with an optically tracked laparoscopic ultrasound (US) probe. Ten US images of each target were registered to computer tomography (CT) images of the phantoms and then mapped to the CT scans. Accuracy of the registration was assessed by comparing the distance between the predicted target location and the position obtained directly from CT. Results: The average localization error was 5.3 mm. The errors resulted primarily from inaccurate US probe tracking but were otherwise insensitive to the variability that arises from manually identifying targets in US and CT images. Conclusions: The results obtained for US-to-CT registration in a phantom model suggest that further investigations into its clinical use are warranted and that other IGS technologies could be applied to laparoscopic liver surgery as well.
---
paper_title: Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

paper_content:

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.
---
paper_title: Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies

paper_content:

Targeted prostate biopsy is challenging because no currently established imaging modality is both accurate for prostate cancer diagnosis and cost-effective for real-time procedure guidance. A system that fuses real-time transrectal ultrasound images with previously acquired endorectal coil MRI images for prostate biopsy guidance is presented here. The system uses electromagnetic tracking and intraoperative image registration to superimpose the MRI data on the ultrasound image. Prostate motion is tracked and compensated for without the need for fiducial markers. The accuracy of the system in phantom studies was shown to be 2.4 ± 1.2 mm. The fusion system has been used in more than 20 patients to guide biopsies with almost no modification of the conventional protocol. Retrospective clinical evaluation suggests that clinically acceptable spatial accuracy can be achieved.
---
paper_title: Motion-robust MRI through real-time motion tracking and retrospective super-resolution volume reconstruction

paper_content:

Magnetic Resonance Imaging (MRI) is highly sensitive to motion; hence current practice is based on the prevention of motion during scan. In newborns, young children, and patients with limited cooperation, this commonly requires full sedation or general anesthesia, which is time consuming, costly, and is associated with significant risks. Despite progress in prospective motion correction in MRI, the use of motion compensation techniques is limited by the type and amount of motion that can be compensated for, the dependency on the scanner platform, the need for pulse sequence modifications, and/or difficult setup. In this paper we introduce a novel platform-independent motion-robust MRI technique based on prospective real-time motion tracking through a miniature magnetic field sensor and retrospective super-resolution volume reconstruction. The technique is based on fast 2D scans that maintain high-quality of slices in the presence of motion but are degraded in 3D due to inter-slice motion artifacts. The sensor, conveniently attached to the subject forehead, provides real-time estimation of the motion, which in turn gives the relative location of the slice acquisitions. These location parameters are used to compensate the inter-slice motion to reconstruct an isotropic high-resolution volumetric image from slices in a super-resolution reconstruction framework. The quantitative results obtained for phantom and volunteer subject experiments in this study show the efficacy of the developed technique, which is particularly useful for motion-robust high-resolution T2-weighted imaging of newborns and pediatric subjects.
---
paper_title: Registration of freehand 3D ultrasound and magnetic resonance liver images

paper_content:

Abstract We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a “bronze standard” registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the “bronze standard” and this is improved to 3.6 mm after running the intensity-based algorithm.
---
paper_title: Cadaver validation of intensity-based ultrasound to CT registration

paper_content:

A method is presented for the registration of tracked B-mode ultrasound images to a CT volume of a femur or pelvis. This registration can allow tracked surgical instruments to be aligned with the CT image or an associated preoperative plan. Our method requires no manual segmentation of either the ultrasound images or the CT volume. The CT and US images are processed to produce images where the image intensity represents the probability of the presence of a bone edge. These images are then registered together using normalised cross-correlation as a similarity measure. The parameter which represents the speed of sound through tissue has also been included in the registration optimisation process. Experiments have been carried out on six cadaveric femurs and three cadaveric pelves. Registration results were compared with a "gold standard" registration acquired using bone implanted fiducial markers. Results show the registration method to be accurate, on average, to 1.7 mm root-mean-square target registration error.
---
paper_title: Weighted likelihood function of multiple statistical parameters to retrieve 2D TRUS-MR slice correspondence for prostate biopsy

paper_content:

This paper presents a novel method to identify the 2D axial Magnetic Resonance (MR) slice from a pre-acquired MR prostate volume that closely corresponds to the 2D axial Transrectal Ultrasound (TRUS) slice obtained during prostate biopsy. The shape-context representations of the segmented prostate contours in both the imaging modalities are used to establish point correspondences using Bhattacharyya distance. Thereafter, Chi-square distance is used to find the prostate shape similarities between the MR slices and the TRUS slice. Normalized mutual information and correlation coefficient between the TRUS and MR slices are computed to find the information theoretic similarities between the TRUS-MR slices. The maximum of the weighted likelihood function of the afore-mentioned statistical similarity measures finally yields the MR slice that closely resembles the TRUS slice acquired during the biopsy procedure. The method is evaluated for 20 patient datasets and close matches with the ground truth are obtained for 16 cases.
---
paper_title: Shape matching and object recognition using shape contexts

paper_content:

We present a novel approach to measuring similarity between shapes and exploit it for object recognition. In our framework, the measurement of similarity is preceded by: (1) solving for correspondences between points on the two shapes; (2) using the correspondences to estimate an aligning transform. In order to solve the correspondence problem, we attach a descriptor, the shape context, to each point. The shape context at a reference point captures the distribution of the remaining points relative to it, thus offering a globally discriminative characterization. Corresponding points on two similar shapes will have similar shape contexts, enabling us to solve for correspondences as an optimal assignment problem. Given the point correspondences, we estimate the transformation that best aligns the two shapes; regularized thin-plate splines provide a flexible class of transformation maps for this purpose. The dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points, together with a term measuring the magnitude of the aligning transform. We treat recognition in a nearest-neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image. Results are presented for silhouettes, trademarks, handwritten digits, and the COIL data set.
---
paper_title: Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies

paper_content:

Targeted prostate biopsy is challenging because no currently established imaging modality is both accurate for prostate cancer diagnosis and cost-effective for real-time procedure guidance. A system that fuses real-time transrectal ultrasound images with previously acquired endorectal coil MRI images for prostate biopsy guidance is presented here. The system uses electromagnetic tracking and intraoperative image registration to superimpose the MRI data on the ultrasound image. Prostate motion is tracked and compensated for without the need for fiducial markers. The accuracy of the system in phantom studies was shown to be 2.4 ± 1.2 mm. The fusion system has been used in more than 20 patients to guide biopsies with almost no modification of the conventional protocol. Retrospective clinical evaluation suggests that clinically acceptable spatial accuracy can be achieved.
---
paper_title: Phantom validation for ultrasound to statistical shape model registration of human pelvis

paper_content:

Total Hip Replacement (THR) has become a common surgical procedure in recent years, as a result of increasing ::: aging population with osteoarthritis of the hip joint. Localization of the pelvic anatomical coordinate system ::: (PaCS) is a critical step in accurate placement of the femur prosthesis in the acetabulum in THR. Intra-operative ::: ultrasound (US) imaging can provide a radiation-free navigation system for localization of the PaCS. However, ::: US images are noisy and cannot provide any anatomical information beneath the bone surface due to the total ::: reflection of US beam at the bone-soft tissue interface. A solution to this problem is to fuse intra-operative ::: US with pre-operative imaging or a statistical shape model (SSM) of the pelvis. Here, we propose a multi-slice ::: to volume intensity-based registration of the pelvic SSM to a sparse set of 2D US images in order to localize ::: the PaCS in the US. In this registration technique, a set of 2D slices are extracted from a pelvic SSM using ::: the approximate location and orientation of their corresponding 2D US images. During the registration, the ::: comparison between the SSM slices and the US images is made using an ultrasound simulation technique and ::: a correlation-based similarity metric. We demonstrate the feasibility of our proposed approach in localizing ::: the PaCS on five patient-based phantoms. These results indicate the necessity of including pubic symphysis ::: landmarks in the 2D US slices in order to have a precise estimation of the PaCS.
---
paper_title: User-friendly freehand ultrasound calibration using Lego bricks and automatic registration

paper_content:

PURPOSE ::: As an inexpensive, noninvasive, and portable clinical imaging modality, ultrasound (US) has been widely employed in many interventional procedures for monitoring potential tissue deformation, surgical tool placement, and locating surgical targets. The application requires the spatial mapping between 2D US images and 3D coordinates of the patient. Although positions of the devices (i.e., ultrasound transducer) and the patient can be easily recorded by a motion tracking system, the spatial relationship between the US image and the tracker attached to the US transducer needs to be estimated through an US calibration procedure. Previously, various calibration techniques have been proposed, where a spatial transformation is computed to match the coordinates of corresponding features in a physical phantom and those seen in the US scans. However, most of these methods are difficult to use for novel users. ::: ::: ::: METHODS ::: We proposed an ultrasound calibration method by constructing a phantom from simple Lego bricks and applying an automated multi-slice 2D-3D registration scheme without volumetric reconstruction. The method was validated for its calibration accuracy and reproducibility. ::: ::: ::: RESULTS ::: Our method yields a calibration accuracy of [Formula: see text] mm and a calibration reproducibility of 1.29 mm. ::: ::: ::: CONCLUSION ::: We have proposed a robust, inexpensive, and easy-to-use ultrasound calibration method.
---
paper_title: Image Guidance of Intracardiac Ultrasound with Fusion of Pre-operative Images ⋆

paper_content:

This paper presents a method for registering 3D intracardiac echo (ICE) to pre-operative images. A magnetic tracking sensor is integrated on the ICE catheter tip to provide the 3D location and orientation. The user guides the catheter into the patient heart to acquire a series of ultrasound images covering the anatomy of the heart chambers. An automatic intensity-based registration algorithm is applied to align these ultrasound images with pre-operative images. One of the important applications is to help electrophysiology doctors to treat complicated atrial fibrillation cases. After registration, the doctor can see the position and orientation of the ICE catheter and other tracked catheters inside the heart anatomy in real time. The image guidance provided by this technique may increase the ablation accuracy and reduce the amount of time for the electrophysiology procedures. We show successful image registration results from animal experiments.
---
paper_title: Ultrasound-CT registration of vertebrae without reconstruction

paper_content:

Purpose ::: While robust and accurate, our previously developed volume-to-volume ultrasound-CT registration of vertebrae required that the 2D ultrasound slices be reconstructed into a 3D volume, a time-consuming step that increased the total registration time per vertebra. We have modified our registration technique to a slices-to-volume strategy to eliminate the ultrasound reconstruction step in order to make the total registration time more practical intraoperatively.
---
paper_title: Multi-slice to volume registration of ultrasound data to a statistical atlas of human pelvis

paper_content:

Identifying the proper orientation of the pelvis is a critical step in accurate placement of the femur prosthesis in the ::: acetabulum in Total Hip Replacement (THR) surgeries. The general approach to localize the orientation of the pelvis ::: coordinate system is to use X-ray fluoroscopy to guide the procedure. An alternative can be employing intra-operative ::: ultrasound (US) imaging with pre-operative CT scan or fluoroscopy imaging. In this paper, we propose to replace the ::: need of pre-operative imaging by using a statistical shape model of the pelvis, constructed from several CT images. We ::: then propose an automatic deformable intensity-based registration of the anatomical atlas to a sparse set of 2D ::: ultrasound images of the pelvis in order to localize its anatomical coordinate system. In this registration technique, we ::: first extract a set of 2D slices from a single instance of the pelvic atlas. Each individual 2D slice is generated based on ::: the location of a corresponding 2D ultrasound image. Next, we create simulated ultrasound images out of the 2D atlas ::: slices and calculate a similarity metric between the simulated images and the actual ultrasound images. The similarity ::: metric guides an optimizer to generate an instance of the atlas that best matches the ultrasound data. We demonstrated ::: the feasibility of our proposed approach on two male human cadaver data. The registration was able to localize a ::: patient-specific pelvic coordinate system with origin translation error of 2 mm and 3.45 mm, and average axes rotation ::: error of 3.5 degrees and 3.9 degrees for the two cadavers, respectively.
---
paper_title: Motion Correction in fMRI by Mapping Slice-to-Volume with Concurrent Field-Inhomogeneity Correction

paper_content:

Head motion is the major source of error in measuring intensity changes related to given stimuli in fMRI. The effects of head motion are image shifts and field inhomogeneity variations which cause local changes in geometric distortions. The previously developed motion correction method, mapping slice-to-volume (MSV), retrospectively remaps slices that are shifted by head motion to their spatially correct locations in an anatomical reference. Images exhibiting spatially varying geometric distortions require non-linear mapping solutions. An accurate field map can be used for the correction of such spatial distortions. However, field-map changes with head motion and, in practice, only one field-map is available typically. This work evaluates the improved motion correction capability of MSV with concurrent iterative field-corrected reconstruction using only an initial field-map. The results from simulated motion data show effective convergence and accuracy in image registration for the correction of image artifacts complicated by the motion induced field effects.
---
paper_title: Registration of freehand 3D ultrasound and magnetic resonance liver images

paper_content:

Abstract We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a “bronze standard” registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the “bronze standard” and this is improved to 3.6 mm after running the intensity-based algorithm.
---
paper_title: A comparative study on manual and automatic slice-to-volume registration of CT images

paper_content:

In order to assess the clinical relevance of a slice-to-volume registration algorithm, this technique was compared to manual registration. Reformatted images obtained from a diagnostic CT examination of the lower abdomen were reviewed and manually registered by 41 individuals. The results were refined by the algorithm. Furthermore, a fully automatic registration of the single slices to the whole CT examination, without manual initialization, was also performed. The manual registration error for rotation and translation was found to be 2.7±2.8 ° and 4.0±2.5 mm. The automated registration algorithm significantly reduced the registration error to 1.6±2.6 ° and 1.3±1.6 mm (p = 0.01). In 3 of 41 (7.3%) registration cases, the automated registration algorithm failed completely. On average, the time required for manual registration was 213±197 s; automatic registration took 82±15 s. Registration was also performed without any human interaction. The resulting registration error of the algorithm without manual pre-registration was found to be 2.9±2.9 ° and 1.1±0.2 mm. Here, a registration took 91±6 s, on average. Overall, the automated registration algorithm improved the accuracy of manual registration by 59% in rotation and 325% in translation. The absolute values are well within a clinically relevant range.
---
paper_title: Correction of misaligned slices in multi-slice cardiovascular magnetic resonance using slice-to-volume registration

paper_content:

A popular technique to reduce respiratory motion for cardiovascular magnetic resonance is to perform a multi-slice acquisition in which a patient holds their breath multiple times during the scan. The feasibility of rigid slice-to-volume registration to correct for misalignments of slice stacks in such images due to differing breath-hold positions is explored. Experimental results indicate that slice-to-volume registration can compensate for the typical misalignments expected. Correction of slice misalignment results in anatomically more correct images, as well as improved left ventricular volume measurements. The interstudy reproducibility has also been improved reducing the number of samples needed for cardiac MR studies.
---
paper_title: Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI

paper_content:

Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acquired by these techniques, inter-slice motion leads to severe motion artifacts that are apparent in out-of-plane views. Slice sequential acquisitions do not enable 3-D volume representation. In this study, we have developed a novel technique based on a slice acquisition model, which enables the reconstruction of a volumetric image from multiple-scan slice acquisitions. The super-resolution volume reconstruction is formulated as an inverse problem of finding the underlying structure generating the acquired slices. We have developed a robust M-estimation solution which minimizes a robust error norm function between the model-generated slices and the acquired slices. The accuracy and robustness of this novel technique has been quantitatively assessed through simulations with digital brain phantom images as well as high-resolution newborn images. We also report here successful application of our new technique for the reconstruction of volumetric fetal brain MRI from clinically acquired data.
---
paper_title: Recovery of Slice Rotations with the Stack Alignment Transform in Cardiac MR Series

paper_content:

Generic solutions for misalignment correction in image stacks are based on slice-byslice registration to a reference volume. Slice misalignment in cardiac Magnetic Resonance cine series typically includes translational and rotational components. Recovery of rotation in medial down to apical slices with the basic slice-to-volume registration may provide unreliable results because of rotational symmetry of the short-axis view of the left ventricle. This paper describes a novel method for correction of stack misalignment in cardiac cine series with recovery of the rotational component. The core of the presented method is a custom spatial transform which parametrises a set of independent centred rigid transforms, one per slice in a stack. The stack alignment transform was designed to circumvent the weaknesses associated with slice-to-volume registration and improve the reliability of misalignment correction over the slice-by-slice registration approach. The image similarity metric for every iteration during the stack alignment transform optimisation is calculated on the whole image stack with all slice correction parameters contributing to the result. The presented method was evaluated on 50 clinical datasets and the results demonstrate an improvement of misalignment correction with the proposed transform. The study also provides evidence of the importance of recovery of slice rotations in this application; the presented method is general enough to be applied to similar problems in other image modalities.
---
paper_title: Confocal bi-protocol: a new strategy for isotropic 3D live cell imaging

paper_content:

The conventional approach for microscopic 3D cellular imaging is based on axial through-stack image series which has some significant limitations such as anisotropic resolution and axial aberration. To overcome these drawbacks, we have recently introduced an alternative approach based on micro-rotation image series. Unfortunately, this new technique suffers from a huge burden of computation that makes its use quite difficult for current applications. To address these problems we propose a new imaging strategy called bi-protocol, which consists of coupling micro-rotation acquisition and conventional z-stack acquisition. We experimentally prove bi-protocol 3D reconstruction produces similar quality to that of pure micro-rotation, but offers the advantage of reduced computation burden because it uses the z-stack volume to accelerate the registration of the micro-rotation images.
---
paper_title: Registration of two-dimensional cardiac images to preprocedural three-dimensional images for interventional applications

paper_content:

PURPOSE ::: To evaluate the accuracy and efficiency of rigid-body registration of two-dimensional fast cine and real-time cardiac images to high-resolution and SNR three-dimensional preprocedural reference volumes for application during MRI-guided interventional procedures. ::: ::: ::: MATERIALS AND METHODS ::: Mutual information (MI) and correlation ratio (CR) similarity measures were evaluated. The dependence of registration accuracy and efficiency on different resolution and SNR parameters, and also on cardiac-phase differences was evaluated in a porcine model. Two-dimensional images were initially misoriented at distances (d) of 2-10 mm, and rotations of +/-5 degrees about all axes. Registration error and computation time were evaluated, and performance was also assessed visually. ::: ::: ::: RESULTS ::: The maximum registration error using MI (<2.7 mm and <3.6 degrees ) occurred for d = 10 mm, misrotation of +/-5 degrees , and relative SNR = 1. The computation time was 15 seconds for MI and 10 seconds for CR. ::: ::: ::: CONCLUSION ::: Registration accuracy was not highly dependent on the relative timing, within the cycle, between the two-dimensional and three-dimensional images. Registration using CR was faster than that using MI, although accuracy was marginally higher with MI. J.
---
paper_title: Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images.

paper_content:

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.
---
paper_title: Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

paper_content:

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.
---
paper_title: Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

paper_content:

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.
---
paper_title: Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices

paper_content:

Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D/3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fetal Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available.
---
paper_title: Temporal registration of partial data using particle filtering

paper_content:

We propose a particle filtering framework for rigid registration of a model image to a time-series of partially observed images. The method incorporates a model-based segmentation technique in order to track the pose dynamics of an underlying observed object with time. An applicable algorithm is derived by employing the proposed framework for registration of a 3D model of an anatomical structure, which was segmented from preoperative images, to consecutive axial 2D slices of a magnetic resonance imaging (MRI) scan, which are acquired intraoperatively over time. The process is fast and robust with respect to image noise and clutter, variations of illumination, and different imaging modalities.
---
paper_title: Towards real time 2D to 3D registration for ultrasound-guided endoscopic and laparoscopic procedures

paper_content:

Purpose ::: A method to register endoscopic and laparoscopic ultrasound (US) images in real time with pre-operative computed tomography (CT) data sets has been developed with the goal of improving diagnosis, biopsy guidance, and surgical interventions in the abdomen.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: MRI of Moving Subjects Using Multislice Snapshot Images With Volume Reconstruction (SVR): Application to Fetal, Neonatal, and Adult Brain Studies

paper_content:

Motion degrades magnetic resonance (MR) images and prevents acquisition of self-consistent and high-quality volume images. A novel methodology, Snapshot magnetic resonance imaging (MRI) with Volume Reconstruction (SVR) has been developed for imaging moving subjects at high resolution and high signal-to-noise ratio (SNR). The method combines registered 2-D slices from sequential dynamic single-shot scans. The SVR approach requires that the anatomy in question is not changing shape or size and is moving at a rate that allows snapshot images to be acquired. After imaging the target volume repeatedly to guarantee sufficient sampling every where, a robust slice-to-volume registration method has been implemented that achieves alignment of each slice within 0.3 mm in the examples tested. Multilevel scattered interpolation has been used to obtain high-fidelity reconstruction with root-mean-square (rms) error that is less than the noise level in the images. The SVR method has been performed successfully for brain studies on subjects that cannot stay still, and in some cases were moving substantially during scanning. For example, awake neonates, deliberately moved adults and, especially, on fetuses, for which no conventional high-resolution 3-D method is currently available. Fine structure of the in-utero fetal brain is clearly revealed for the first time and substantial SNR improvement is realized by having many individually acquired slices contribute to each voxel in the reconstructed image.
---
paper_title: Particle filtering for slice-to-volume motion correction in EPI based functional MRI

paper_content:

Head movement during scanning introduces artificial signal changes and impedes activation detection in fMRI studies. The head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) sequence can be estimated and compensated by aligning the images onto a reference volume through image registration. Registering EPI images volume by volume fails to consider head motion between slices, leading to biased head motion estimates. Slice-to-volume registration is used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, it is prone to image noise and geometric distortion, resulting in high variance estimates. In this work, we propose a Gaussian particle filter based head motion tracking algorithm to reduce the image misregistration errors. The algorithm models head motion by using a dynamic state space model (SSM) to model continuous slice acquisition thereby providing more accurate motion estimates and voxel position estimates. We demonstrate significant performance improvement of the proposed approach as compared to previous registration-only methods of head motion estimation.
---
paper_title: Distortion correction in fetal EPI using non-rigid registration with Laplacian constraint

paper_content:

Geometric distortion induced by the main B0 field disrupts the consistency of fetal echo planar imaging (EPI) data, on which diffusion and functional magnetic resonance imaging is based. In this paper, we present a novel data-driven method for simultaneous motion and distortion correction of fetal EPI. A motion-corrected and reconstructed T2 weighted single shot fast spin echo (ssFSE) volume is used as a model of undistorted fetal brain anatomy. Our algorithm interleaves two registration steps: estimation of fetal motion parameters by aligning EPI slices to the model; and deformable registration of EPI slices to slices simulated from the undistorted model to estimate the distortion field. The deformable registration is regularized by a physically inspired Laplacian constraint, to model distortion induced by a source-free background B0 field. Our experiments show that distortion correction significantly improves consistency of reconstructed EPI volumes with ssFSE volumes. In addition, the estimated distortion fields are consistent with fields calculated from acquired field maps, and the Laplacian constraint is essential for estimation of plausible distortion fields. The EPI volumes reconstructed from different scans of the same subject were more consistent when the proposed method was used in comparison with EPI volumes reconstructed from data distortion corrected using a separately acquired B0 field map.
---
paper_title: Target Motion Tracking in MRI-guided Transrectal Robotic Prostate Biopsy

paper_content:

Purpose: MRI-guided prostate needle biopsy requires compensation for organ motion between target planning and needle placement. Two questions are studied and answered in this paper: 1) is rigid registration sufficient in tracking the targets with an error smaller than the clinically significant size of prostate cancer and 2) what is the effect of the number of intraoperative slices on registration accuracy and speed? Methods: we propose multislice-to-volume registration algorithms for tracking the biopsy targets within the prostate. Three orthogonal plus additional transverse intraoperative slices are acquired in the approximate center of the prostate and registered with a high-resolution target planning volume. Both rigid and deformable scenarios were implemented. Both simulated and clinical MRI-guided robotic prostate biopsy data were used to assess tracking accuracy. Results: average registration errors in clinical patient data were 2.6 mm for the rigid algorithm and 2.1 mm for the deformable algorithm. Conclusion: rigid tracking appears to be promising. Three tracking slices yield significantly high registration speed with an affordable error.
---
paper_title: Automatic 3-D Breath-Hold Related Motion Correction of Dynamic Multislice MRI

paper_content:

Magnetic resonance (MR) cine images are often used to clinically assess left ventricular cardiac function. In a typical study, multiple 2-D long axis (LA) and short axis (SA) cine images are acquired, each in a different breath-hold. Differences in lung volume during breath-hold and overall patient motion distort spatial alignment of the images thus complicating spatial integration of all image data in three dimensions. We present a fully automatic postprocessing approach to correct these slice misalignments. The approach is based on the constrained optimization of the intensity similarity of intersecting image lines after the automatic definition of a region of interest. It uses all views and all time frames simultaneously. Our method models both in-plane and out-of-plane translations and full 3-D rotations, can be applied retrospectively and does not require a cardiac wall segmentation. The method was validated on both healthy volunteer and patient data with simulated misalignments, as well as on clinical multibreath-hold patient data. For the simulated data, subpixel accuracy could be obtained using translational correction. The possibilities and limitations of rotational correction were investigated and discussed. For the clinical multibreath-hold patient data sets, the median discrepancy between manual SA and LA contours was reduced from 2.83 to 1.33 mm using the proposed correction method. We have also shown the usefulness of the correction method for functional analysis on clinical image data. The same clinical multibreath-hold data sets were resegmented after positional correction, taking newly available complementary information of intersecting slices into account, further reducing the median discrepancy to 0.43 mm. This is due to the integration of the 2-D slice information into 3-D space.
---
paper_title: Intersection based registration of slice stacks to form 3D images of the human fetal brain

paper_content:

Clinical fetal MR imaging of the brain commonly makes use of fast 2D acquisitions of multiple sets of approximately orthogonal 2D slices. We and others have previously proposed an iterative slice-to-volume registration process to recover a geometrically consistent 3D image. However, these approaches depend on a 3D volume reconstruction step during the slice alignment. This is both computationally expensive and makes the convergence of the registration process poorly defined. In this paper our key contribution is a new approach which considers the collective alignment of all slices directly, via shared structure in their intersections, rather than to an estimated 3D volume. We derive an analytical expression for the gradient of the collective similarity of the slices along their intersections, with respect to the 3D location and orientation of each 2D slice. We include examples of the approach applied to simulated data and clinically acquired fetal images.
---
paper_title: Determining histology-MRI slice correspondences for defining MRI-based disease signatures of prostate cancer

paper_content:

Abstract Mapping the spatial disease extent in a certain anatomical organ/tissue from histology images to radiological images is important in defining the disease signature in the radiological images. One such scenario is in the context of men with prostate cancer who have had pre-operative magnetic resonance imaging (MRI) before radical prostatectomy. For these cases, the prostate cancer extent from ex vivo whole-mount histology is to be mapped to in vivo MRI. The need for determining radiology-image-based disease signatures is important for (a) training radiologist residents and (b) for constructing an MRI-based computer aided diagnosis (CAD) system for disease detection in vivo . However, a prerequisite for this data mapping is the determination of slice correspondences (i.e. indices of each pair of corresponding image slices) between histological and magnetic resonance images. The explicit determination of such slice correspondences is especially indispensable when an accurate 3D reconstruction of the histological volume cannot be achieved because of (a) the limited tissue slices with unknown inter-slice spacing, and (b) obvious histological image artifacts (tissue loss or distortion). In the clinic practice, the histology-MRI slice correspondences are often determined visually by experienced radiologists and pathologists working in unison, but this procedure is laborious and time-consuming. We present an iterative method to automatically determine slice correspondence between images from histology and MRI via a group-wise comparison scheme, followed by 2D and 3D registration. The image slice correspondences obtained using our method were compared with the ground truth correspondences determined via consensus of multiple experts over a total of 23 patient studies. In most instances, the results of our method were very close to the results obtained via visual inspection by these experts.
---
paper_title: Registration-based Approach for Reconstruction of High Resolution In Utero Fetal MR Brain Images

paper_content:

Rationale and Objectives This paper describes a novel approach to forming high-resolution MR images of the human fetal brain. It addresses the key problem of fetal motion by proposing a registration-refined compounding of multiple sets of orthogonal fast two-dimensional MRI slices, which are currently acquired for clinical studies, into a single high-resolution MRI volume. Materials and Methods A robust multiresolution slice alignment is applied iteratively to the data to correct motion of the fetus that occurs between two-dimensional acquisitions. This is combined with an intensity correction step and a super-resolution reconstruction step, to form a single high isotropic resolution volume of the fetal brain. Results Experimental validation on synthetic image data with known motion types and underlying anatomy, together with retrospective application to sets of clinical acquisitions, are included. Conclusion Results indicate that this method promises a unique route to acquiring high-resolution MRI of the fetal brain in vivo allowing comparable quality to that of neonatal MRI. Such data provide a highly valuable window into the process of normal and abnormal brain development, which is directly applicable in a clinical setting.
---
paper_title: Automatic 3D registration for interventional MRI-guided treatment of prostate cancer.

paper_content:

The goal of this research is to register real-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained high-resolution MRI image volume, which in turn can be registered with functional images such as those Urom SPECT. The immediate application is in iMRI-guided treatment of prostate cancer, where additional images are desired to improve tumor targeting. In this article, simulation experiments are performed to demonstrate the feasibility of slice-to-volume registration for this application. We acquired 3D volume images from a 1.5-T MRI system and simulated low-field iMRI image slices by creating thick slices and adding noise. We created a slice-to-volume mutual information registration algorithm with special features to improve robustness. Features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. To assess the quality of registration, we calculated 3D displacements on a voxel-by-voxel basis over a volume of in...
---
paper_title: Multiscale Registration of Real-Time and Prior MRI Data for Image-Guided Cardiac Interventions

paper_content:

Abstract Recently, there is a growing interest in using magnetic resonance imaging (MRI) to guide interventional procedures due to its excellent soft tissue contrast and lack of ionizing radiation compared to traditional radiographic guidance. One of these applications is the use of MRI to guide radio frequency ablation of anatomic substrates, within the left ventricle, responsible for ventricular tachycardia. However, different MRI acquisition schemes have significant tradeoffs between image quality and acquisition time. Guidance using high-quality preoperative 3-D MR images is limited in the case of cardiac interventions because the heart moves dynamically during the procedure. On the other hand, 2-D real-time MR images acquired during the intervention sacrifice image quality for shorter image acquisition time, leading to real-time positional updates of cardiac anatomy. Ideally, we wish to combine the advantages of live feedback from real-time images and accurate visualization of anatomical structures from preoperative images. Therefore, to improve the MRI guidance capabilities for cardiac interventions, we describe a novel multiscale rigid registration framework to correct for respiratory motion between the prior and real-time datasets. In the proposed approach, we use a weighted total variation flow algorithm to extract coarse-to-fine features from the input images and subsequently register the corresponding scales in a hierarchical manner. Registration experiments were performed with in vivo human imaging data, and the target registration error achieved was 1.51 mm. Thus, the feasibility of motion correction in an interventional setting has been demonstrated, which may lead to significant improvements in the guidance of cardiac interventions.
---
paper_title: Registration of Whole-Mount Histology and Volumetric Imaging of the Prostate Using Particle Filtering

paper_content:

Registration of histological slices to volumetric imaging of the prostate is an important task that can be used to optimize imaging for cancer detection. Such registration is challenging due to physical changes of the specimen during excision and fixation, and misalignment of the histological slices during preparation and digital scanning. In this work, we consider a multi-slice to volume registration method in which a stack of sparse, unaligned 2-D whole-mount histological slices is registered to a 3-D volumetric imaging of the prostate. We propose a particle filtering framework to contend with the high dimensionality of the search space and multimodal nature of the optimization. Such framework allows modeling of the uncertainty in the pose of the slices and in the imaged information, in order to derive optimal registration parameters in a Bayesian approach. Intensity-, region-, and point-based similarity metrics were incorporated into the registration algorithm to account for different imaging modalities. We demonstrate and evaluate our method on a diverse set of data that includes a synthetic volume, ex vivo and in vivo magnetic resonance imaging, and in vivo ultrasound.
---
paper_title: Registering 2D and 3D imaging data of bone during healing

paper_content:

AbstractPurpose/Aims of the study: Bone’s hierarchical structure can be visualized using a variety of methods. Many techniques, such as light and electron microscopy generate two-dimensional (2D) images, while micro-computed tomography (µCT) allows a direct representation of the three-dimensional (3D) structure. In addition, different methods provide complementary structural information, such as the arrangement of organic or inorganic compounds. The overall aim of the present study is to answer bone research questions by linking information of different 2D and 3D imaging techniques. A great challenge in combining different methods arises from the fact that they usually reflect different characteristics of the real structure.Materials and methods: We investigated bone during healing by means of µCT and a couple of 2D methods. Backscattered electron images were used to qualitatively evaluate the tissue’s calcium content and served as a position map for other experimental data. Nanoindentation and X-ray scat...
---
paper_title: Atlas-Based Indexing of Brain Sections via 2-D to 3-D Image Registration

paper_content:

A 2-D to 3-D nonlinear intensity-based registration method is proposed in which the alignment of histological brain sections with a volumetric brain atlas is performed. First, sparsely cut brain sections were linearly matched with an oblique slice automatically extracted from the atlas. Second, a planar-to-curved surface alignment was employed in order to match each section with its corresponding image overlaid on a curved-surface within the atlas. For the latter, a PDE-based registration technique was developed that is driven by a local normalized-mutual-information similarity measure. We demonstrate the method and evaluate its performance with simulated and real data experiments. An atlas-guided segmentation of mouse brains' hippocampal complex, retrieved from the Mouse Brain Library (MBL) database, is demonstrated with the proposed algorithm.
---
paper_title: User-friendly freehand ultrasound calibration using Lego bricks and automatic registration

paper_content:

PURPOSE ::: As an inexpensive, noninvasive, and portable clinical imaging modality, ultrasound (US) has been widely employed in many interventional procedures for monitoring potential tissue deformation, surgical tool placement, and locating surgical targets. The application requires the spatial mapping between 2D US images and 3D coordinates of the patient. Although positions of the devices (i.e., ultrasound transducer) and the patient can be easily recorded by a motion tracking system, the spatial relationship between the US image and the tracker attached to the US transducer needs to be estimated through an US calibration procedure. Previously, various calibration techniques have been proposed, where a spatial transformation is computed to match the coordinates of corresponding features in a physical phantom and those seen in the US scans. However, most of these methods are difficult to use for novel users. ::: ::: ::: METHODS ::: We proposed an ultrasound calibration method by constructing a phantom from simple Lego bricks and applying an automated multi-slice 2D-3D registration scheme without volumetric reconstruction. The method was validated for its calibration accuracy and reproducibility. ::: ::: ::: RESULTS ::: Our method yields a calibration accuracy of [Formula: see text] mm and a calibration reproducibility of 1.29 mm. ::: ::: ::: CONCLUSION ::: We have proposed a robust, inexpensive, and easy-to-use ultrasound calibration method.
---
paper_title: Registration of 2D histological sections with 3D micro-CT datasets from small animal vertebrae and tibiae

paper_content:

The aim of this study was the registration of digitized thin 2D sections of mouse vertebrae and tibiae used for histomorphometry of trabecular bone structure into 3D micro computed tomography (μCT) datasets of the samples from which the sections were prepared. Intensity-based and segmentation-based registrations (SegRegs) of 2D sections and 3D μCT datasets were applied. As the 2D sections were deformed during their preparation, affine registration for the vertebrae was used instead of rigid registration. Tibiae sections were additionally cut on the distal end, which subsequently undergone more deformation so that elastic registration was necessary. The Jaccard distance was used as registration quality measure. The quality of intensity-based registrations and SegRegs was practically equal, although precision errors of the elastic registration of segmentation masks in tibiae were lower, while those in vertebrae were lower for the intensity-based registration. Results of SegReg significantly depended on the ...
---
paper_title: A new registration/visualization paradigm for CT-Fluoroscopy guided RF liver ablation

paper_content:

2D-3D slice-to-volume registration for abdominal organs like liver is difficult due to the breathing motion and tissue deformation. The purpose of our approach is to ease CT-fluoroscopy (CT-fluoro) based needle insertion for the Radiofrequency Liver Ablation procedure using high resolution contrasted preoperative data. In this case, low signal-to-noise ratio, absence of contrast and additional presence of needle in CT-fluoro makes it difficult to guarantee the solution of any deformable slice-to-volume registration algorithm. In this paper, we first propose a method for creating a set of ground truth (GT) simulation data based on a non-linear deformation of the CT-fluoro volume obtained from real patients. Second, we split the CT-fluoro image and apply intensity based rigid and affine registration to each section. We then propose a novel solution, which consists of intuitive visualization sequences of optimal sub-volumes of preinterventional data based on the registration results. Experiments on synthetic and real patient data and direct feedback of two interventionalists validate our alternative approach.
---
paper_title: Determining histology-MRI slice correspondences for defining MRI-based disease signatures of prostate cancer

paper_content:

Abstract Mapping the spatial disease extent in a certain anatomical organ/tissue from histology images to radiological images is important in defining the disease signature in the radiological images. One such scenario is in the context of men with prostate cancer who have had pre-operative magnetic resonance imaging (MRI) before radical prostatectomy. For these cases, the prostate cancer extent from ex vivo whole-mount histology is to be mapped to in vivo MRI. The need for determining radiology-image-based disease signatures is important for (a) training radiologist residents and (b) for constructing an MRI-based computer aided diagnosis (CAD) system for disease detection in vivo . However, a prerequisite for this data mapping is the determination of slice correspondences (i.e. indices of each pair of corresponding image slices) between histological and magnetic resonance images. The explicit determination of such slice correspondences is especially indispensable when an accurate 3D reconstruction of the histological volume cannot be achieved because of (a) the limited tissue slices with unknown inter-slice spacing, and (b) obvious histological image artifacts (tissue loss or distortion). In the clinic practice, the histology-MRI slice correspondences are often determined visually by experienced radiologists and pathologists working in unison, but this procedure is laborious and time-consuming. We present an iterative method to automatically determine slice correspondence between images from histology and MRI via a group-wise comparison scheme, followed by 2D and 3D registration. The image slice correspondences obtained using our method were compared with the ground truth correspondences determined via consensus of multiple experts over a total of 23 patient studies. In most instances, the results of our method were very close to the results obtained via visual inspection by these experts.
---
paper_title: Nonrigid registration using free-form deformations: application to breast MR images

paper_content:

In this paper the authors present a new approach for the nonrigid registration of contrast-enhanced breast MRI. A hierarchical transformation model of the motion of the breast has been developed. The global motion of the breast is modeled by an affine transformation while the local breast motion is described by a free-form deformation (FFD) based on B-splines. Normalized mutual information is used as a voxel-based similarity measure which is insensitive to intensity changes as a result of the contrast enhancement. Registration is achieved by minimizing a cost function, which represents a combination of the cost associated with the smoothness of the transformation and the cost associated with the image similarity. The algorithm has been applied to the fully automated registration of three-dimensional (3-D) breast MRI in volunteers and patients. In particular, the authors have compared the results of the proposed nonrigid registration algorithm to those obtained using rigid and affine registration techniques. The results clearly indicate that the nonrigid registration algorithm is much better able to recover the motion and deformation of the breast than rigid or affine registration algorithms.
---
paper_title: A Fast CT and CT-Fluoroscopy Registration Algorithm With Respiratory Motion Compensation for Image-Guided Lung Intervention

paper_content:

CT-fluoroscopy (CTF) is an efficient imaging technique for guiding percutaneous lung intervention such as biopsy and ablation. In CTF-guided procedures, four to ten axial images are captured in a very short time period during breath holding to provide near real-time feedback of patients' anatomy so that physicians can adjust the needle as it is advanced toward a target lesion. Although popularly used in clinics, this procedure requires frequent scans to guide the needle, which may cause increased procedure time, complication rates, and radiation exposure to both clinicians and patients. In addition, CTF only generates a limited number of 2-D axial images and does not provide sufficient 3-D anatomical information. Therefore, how to provide volumetric anatomical information using CTF while reducing intraoperative scan is an important and challenging problem. In this paper, we propose a fast CT-CTF deformable registration algorithm that warps the inhale preprocedural CT onto the intraprocedural CTF for guidance in 3-D. In the algorithm, the deformation in the transverse plane is modeled using 2-D B-Spline, and the deformation along z-direction is regularized by smoothness constraint. A respiratory motion compensation framework is also incorporated for accurate registration. A parallel implementation strategy is adopted to accomplish the registration in several seconds. With electromagnetic tracking, the needle position can be superimposed onto the deformed inhale CT image, thereby providing 3-D image guidance during breath holding. Experiments were conducted using both simulated CTF images with known deformation and real CTF images captured during lung cancer biopsy studies. The experiments demonstrated satisfactory registration results of our proposed fast CT-CTF registration algorithm.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: Dynamic MR-based respiratory motion compensation for hybrid PET/MR system

paper_content:

This paper describes a novel respiratory motion compensation (MC) technique for hybrid PET/MR system. Existing PET/MR respiratory MC techniques mainly rely on respiratory gating to reconstruct 3D MR images for different respiratory phases, and use image registration techniques to estimate the deformation between different respiratory phases. A well known limitation of respiratory gating is its assumption that breathing motion is perfectly periodic, therefore MC techniques based on gated MR are sensitive to irregular breathing patterns. To address this limitation, the proposed technique uses dynamic 2D MRIs for PET/MR respiratory motion estimation. A static 3D MRI and a series of dynamic 2D MRIs are acquired before and during the PET data acquisition, respectively. The dynamic 2D MRIs are registered against the static 3D MRI using a novel deformable 2D+t to 3D image registration method to derive a 3D+t deformation field, which is then used for motion corrected PET reconstruction. The proposed method is validated on synthetic PET/MR data and real MR data, demonstrating its efficacy to compensate for irregular respiratory motion and to deblur the PET image.
---
paper_title: Non-rigid 2D-3D Medical Image Registration using Markov Random Fields

paper_content:

The aim of this paper is to propose a novel mapping algorithm between 2D images and a 3D volume seeking simultaneously a linear plane transformation and an in-plane dense deformation. We adopt a metric free locally over-parametrized graphical model that combines linear and deformable parameters within a coupled formulation on a 5-dimensional space. Image similarity is encoded in singleton terms, while geometric linear consistency of the solution (common/single plane) and in-plane deformations smoothness are modeled in a pair-wise term. The robustness of the method and its promising results with respect to the state of the art demonstrate the extreme potential of this approach.
---
paper_title: Deformable Medical Image Registration: A Survey

paper_content:

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.
---
paper_title: Model-based 3D/2D deformable registration of MR images

paper_content:

A method is proposed for automatic registration of 3D preoperative magnetic resonance images of deformable tissue to a sequence of its 2D intraoperative images. The algorithm employs a dynamic continuum mechanics model of the deformation and similarity (distance) measures such as correlation ratio, mutual information or sum of squared differences for registration. The registration is solely based on information present in the 3D preoperative and 2D intraoperative images and does not require fiducial markers, feature extraction or image segmentation. Results of experiments with a biopsy training breast phantom show that the proposed method can perform well in the presence of large deformations. This is particularly useful for clinical applications such as MR-based breast biopsy where large tissue deformations occur.
---
paper_title: Deformable medical image registration: setting the state of the art with discrete methods.

paper_content:

This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: Registration of 2D histological sections with 3D micro-CT datasets from small animal vertebrae and tibiae

paper_content:

The aim of this study was the registration of digitized thin 2D sections of mouse vertebrae and tibiae used for histomorphometry of trabecular bone structure into 3D micro computed tomography (μCT) datasets of the samples from which the sections were prepared. Intensity-based and segmentation-based registrations (SegRegs) of 2D sections and 3D μCT datasets were applied. As the 2D sections were deformed during their preparation, affine registration for the vertebrae was used instead of rigid registration. Tibiae sections were additionally cut on the distal end, which subsequently undergone more deformation so that elastic registration was necessary. The Jaccard distance was used as registration quality measure. The quality of intensity-based registrations and SegRegs was practically equal, although precision errors of the elastic registration of segmentation masks in tibiae were lower, while those in vertebrae were lower for the intensity-based registration. Results of SegReg significantly depended on the ...
---
paper_title: Deformable Ultrasound Registration without Reconstruction

paper_content:

Ultrasound (US) imaging is often proposed as an interoperative imaging modality. This use nearly always requires that the collected data be registered to preoperative data of another modality. Existing intensity-based registration approaches all begin by reconstructing a 3D US volume from the collected 2D slices. We propose to directly register the set of 2D slices to the preoperative images. We argue this has a number of advantages, including the omission of the potentially complex reconstruction step, greater adaptability of the similarity measures, and easier parallelization. We describe a system for performing this task and present results on phantom data that show that our slice based method consistently outperforms a reconstruction based method in both speed and accuracy.
---
paper_title: Target Motion Tracking in MRI-guided Transrectal Robotic Prostate Biopsy

paper_content:

Purpose: MRI-guided prostate needle biopsy requires compensation for organ motion between target planning and needle placement. Two questions are studied and answered in this paper: 1) is rigid registration sufficient in tracking the targets with an error smaller than the clinically significant size of prostate cancer and 2) what is the effect of the number of intraoperative slices on registration accuracy and speed? Methods: we propose multislice-to-volume registration algorithms for tracking the biopsy targets within the prostate. Three orthogonal plus additional transverse intraoperative slices are acquired in the approximate center of the prostate and registered with a high-resolution target planning volume. Both rigid and deformable scenarios were implemented. Both simulated and clinical MRI-guided robotic prostate biopsy data were used to assess tracking accuracy. Results: average registration errors in clinical patient data were 2.6 mm for the rigid algorithm and 2.1 mm for the deformable algorithm. Conclusion: rigid tracking appears to be promising. Three tracking slices yield significantly high registration speed with an affordable error.
---
paper_title: Correction of local deformations in fMRI by 3D non-linear warping in map-slice-to-volume approach

paper_content:

Zntroduction Recently, we have demonstrilted the capability of retrospectively mapping a slice image into rin anatomical reference volume (i.e. map-slice-to-volume, MSV) from the same subject using a simple rigid body transformation of $x degrees of freedom (DOF) [l]. In practice, deformation in a sliQe image is often presented as a nonlinear warping problem that his not limited to the imaging plane. Out-of-plane warp is typically encountered in multislice fMRI data due to the local field induced deformations or localized outof-plane motion artifacts. This study presents correction of the 3D warping artifacts localized in the regions that undergo local changes that result in spatial displacement of voxels typically observed in single shot ac@isitions [2]. The local warping solution is implemented follolwing the rigid body MSV transform that determines the patient’s~ rigid head motion causing in-plane and out-of-plane displacemend, Methods A slice image from time series EPI data was mapped into a high resolution reference yolume space, T1-weighted MRI volume data using a registration algorithm implementing either affine or a control pbint based thin-plate-spline (TPS) function. The automated registration is driven by the mutual information (Z) computed from the gray values of the image pair. By allowing the optimizer to move the control points in the homologok 3D space to positions that minimize the cost metric @&=-I), the iterative process determines the transformation coefficients for a coordinate mapping. For each iteration the affine or TPS warp transformation performs trilinear interpolation in which the original voxel grey values are used. This approach ensures that there is no undesired intensity change from interpolation effects such as round-off or smoothing across iterations. Results Retrospective image registration was performed to map a GREEPI slice image into a volumetric SPGR MRI data acquired from a 3T MRI system. The tota?ly automatic registration routine for each fMRI slice can be initiaipd using an initial three control point set, i.e., rigid body MSV, atid then can automatically increment DOF to multiple control point TPS warping transforms. The six DOF rotate/translate transfoF determines the spatial slice profile relative to the volumetric refqrence data at the time of acquisition. This process accommodates’ the patient’s head movement [l]. The following warping solution is initiated by using the optimized rigid body model to map the control points (> 4 pts) in the reference space into the homologous space. The TPS warping is subsequently computed, agati, by optimizing MI through iterative movement of the control points in homologous space. Improvement in registration using TPS warping solution is demonstrated in Fig. 1 as, compared to rigid-body in l(a). Deformatiotis in the center and the posterior edge are not removed by rigid body and the improved mapping with TPS solution is also indicated by lower MI index. Note that the final mapping of an acquired slice image into the-anatomical reference space is achieved through ?D warping, that corrects out-of-pime as well as in-plane local deformations besides the patient head motion.
---
paper_title: Non-rigid 2D-3D Medical Image Registration using Markov Random Fields

paper_content:

The aim of this paper is to propose a novel mapping algorithm between 2D images and a 3D volume seeking simultaneously a linear plane transformation and an in-plane dense deformation. We adopt a metric free locally over-parametrized graphical model that combines linear and deformable parameters within a coupled formulation on a 5-dimensional space. Image similarity is encoded in singleton terms, while geometric linear consistency of the solution (common/single plane) and in-plane deformations smoothness are modeled in a pair-wise term. The robustness of the method and its promising results with respect to the state of the art demonstrate the extreme potential of this approach.
---
paper_title: Dense Image Registration through MRFs and Efficient Linear Programming

paper_content:

In this paper, we introduce a novel and efficient approach to dense image registration, which does not require a derivative of the employed cost function. In such a context, the registration problem is formulated using a discrete Markov random field objective function. First, towards dimensionality reduction on the variables we assume that the dense deformation field can be expressed using a small number of control points (registration grid) and an interpolation strategy. Then, the registration cost is expressed using a discrete sum over image costs (using an arbitrary similarity measure) projected on the control points, and a smoothness term that penalizes local deviations on the deformation field according to a neighborhood system on the grid. Towards a discrete approach, the search space is quantized resulting in a fully discrete model. In order to account for large deformations and produce results on a high resolution level, a multi-scale incremental approach is considered where the optimal solution is iteratively updated. This is done through successive morphings of the source towards the target image. Efficient linear programming using the primal dual principles is considered to recover the lowest potential of the cost function. Very promising results using synthetic data with known deformations and real data demonstrate the potentials of our approach.
---
paper_title: Deformable medical image registration: setting the state of the art with discrete methods.

paper_content:

This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: Improved fMRI time-series registration using joint probability density priors

paper_content:

Functional MRI (fMRI) time-series studies are plagued by varying degrees of subject head motion. Faithful head ::: motion correction is essential to accurately detect brain activation using statistical analyses of these time-series. ::: Mutual information (MI) based slice-to-volume (SV) registration is used for motion estimation when the rate of ::: change of head position is large. SV registration accounts for head motion between slice acquisitions by estimating ::: an independent rigid transformation for each slice in the time-series. Consequently each MI optimization uses ::: intensity counts from a single time-series slice, making the algorithm susceptible to noise for low complexity endslices ::: (i.e., slices near the top of the head scans). This work focuses on improving the accuracy of MI-based SV ::: registration of end-slices by using joint probability density priors derived from registered high complexity centerslices ::: (i.e., slices near the middle of the head scans). Results show that the use of such priors can significantly ::: improve SV registration accuracy.
---
paper_title: Reconstruction of a geometrically correct diffusion tensor image of a moving human fetal brain

paper_content:

Recent studies reported the development of methods for rigid registration of 2D fetal brain imaging data to correct for unconstrained fetal and maternal motion, and allow the formation of a true 3D image of conventional fetal brain anatomy from conventional MRI. Diffusion tensor imaging provides additional valuable insight into the developing brain anatomy, however the correction of motion artifacts in clinical fetal diffusion imaging is still a challenging problem. This is due to the challenging problem of matching lower signal-to-noise ratio diffusion weighted EPI slice data to recover between-slice motion, compounded by the presence of possible geometric distortions in the EPI data. In addition, the problem of estimating a diffusion model (such as a tensor) on a regular grid that takes into account the inconsistent spatial and orientation sampling of the diffusion measurements needs to be solved in a robust way. Previous methods have used slice to volume registration within the diffusion dataset. In this work, we describe an alternative approach that makes use of an alignment of diffusion weighted EPI slices to a conventional structural MRI scan which provides a geometrically correct reference image. After spatial realignment of each diffusion slice, a tensor field representing the diffusion profile is estimated by weighted least squared fitting. By qualitative and quantitative evaluation of the results, we confirm the proposed algorithm successfully corrects the motion and reconstructs the diffusion tensor field.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: Dynamic MR-based respiratory motion compensation for hybrid PET/MR system

paper_content:

This paper describes a novel respiratory motion compensation (MC) technique for hybrid PET/MR system. Existing PET/MR respiratory MC techniques mainly rely on respiratory gating to reconstruct 3D MR images for different respiratory phases, and use image registration techniques to estimate the deformation between different respiratory phases. A well known limitation of respiratory gating is its assumption that breathing motion is perfectly periodic, therefore MC techniques based on gated MR are sensitive to irregular breathing patterns. To address this limitation, the proposed technique uses dynamic 2D MRIs for PET/MR respiratory motion estimation. A static 3D MRI and a series of dynamic 2D MRIs are acquired before and during the PET data acquisition, respectively. The dynamic 2D MRIs are registered against the static 3D MRI using a novel deformable 2D+t to 3D image registration method to derive a 3D+t deformation field, which is then used for motion corrected PET reconstruction. The proposed method is validated on synthetic PET/MR data and real MR data, demonstrating its efficacy to compensate for irregular respiratory motion and to deblur the PET image.
---
paper_title: A Unified Approach to Diffusion Direction Sensitive Slice Registration and 3-D DTI Reconstruction From Moving Fetal Brain Anatomy

paper_content:

This paper presents an approach to 3-D diffusion tensor image (DTI) reconstruction from multi-slice diffusion weighted (DW) magnetic resonance imaging acquisitions of the moving fetal brain. Motion scatters the slice measurements in the spatial and spherical diffusion domain with respect to the underlying anatomy. Previous image registration techniques have been described to estimate the between slice fetal head motion, allowing the reconstruction of 3D a diffusion estimate on a regular grid using interpolation. We propose Approach to Unified Diffusion Sensitive Slice Alignment and Reconstruction (AUDiSSAR) that explicitly formulates a process for diffusion direction sensitive DW-slice-to-DTI-volume alignment. This also incorporates image resolution modeling to iteratively deconvolve the effects of the imaging point spread function using the multiple views provided by thick slices acquired in different anatomical planes. The algorithm is implemented using a multi-resolution iterative scheme and multiple real and synthetic data are used to evaluate the performance of the technique. An accuracy experiment using synthetically created motion data of an adult head and an experiment using synthetic motion added to sedated fetal monkey dataset show a significant improvement in motion-trajectory estimation compared to current state-of-the-art approaches. The performance of the method is then evaluated on challenging but clinically typical in utero fetal scans of four different human cases, showing improved rendition of cortical anatomy and extraction of white matter tracts. While the experimental work focuses on DTI reconstruction (second-order tensor model), the proposed reconstruction framework can employ any 5-D diffusion volume model that can be represented by the spatial parameterizations of an orientation distribution function.
---
paper_title: Recovery of Slice Rotations with the Stack Alignment Transform in Cardiac MR Series

paper_content:

Generic solutions for misalignment correction in image stacks are based on slice-byslice registration to a reference volume. Slice misalignment in cardiac Magnetic Resonance cine series typically includes translational and rotational components. Recovery of rotation in medial down to apical slices with the basic slice-to-volume registration may provide unreliable results because of rotational symmetry of the short-axis view of the left ventricle. This paper describes a novel method for correction of stack misalignment in cardiac cine series with recovery of the rotational component. The core of the presented method is a custom spatial transform which parametrises a set of independent centred rigid transforms, one per slice in a stack. The stack alignment transform was designed to circumvent the weaknesses associated with slice-to-volume registration and improve the reliability of misalignment correction over the slice-by-slice registration approach. The image similarity metric for every iteration during the stack alignment transform optimisation is calculated on the whole image stack with all slice correction parameters contributing to the result. The presented method was evaluated on 50 clinical datasets and the results demonstrate an improvement of misalignment correction with the proposed transform. The study also provides evidence of the importance of recovery of slice rotations in this application; the presented method is general enough to be applied to similar problems in other image modalities.
---
paper_title: Confocal bi-protocol: a new strategy for isotropic 3D live cell imaging

paper_content:

The conventional approach for microscopic 3D cellular imaging is based on axial through-stack image series which has some significant limitations such as anisotropic resolution and axial aberration. To overcome these drawbacks, we have recently introduced an alternative approach based on micro-rotation image series. Unfortunately, this new technique suffers from a huge burden of computation that makes its use quite difficult for current applications. To address these problems we propose a new imaging strategy called bi-protocol, which consists of coupling micro-rotation acquisition and conventional z-stack acquisition. We experimentally prove bi-protocol 3D reconstruction produces similar quality to that of pure micro-rotation, but offers the advantage of reduced computation burden because it uses the z-stack volume to accelerate the registration of the micro-rotation images.
---
paper_title: Model-based 3D/2D deformable registration of MR images

paper_content:

A method is proposed for automatic registration of 3D preoperative magnetic resonance images of deformable tissue to a sequence of its 2D intraoperative images. The algorithm employs a dynamic continuum mechanics model of the deformation and similarity (distance) measures such as correlation ratio, mutual information or sum of squared differences for registration. The registration is solely based on information present in the 3D preoperative and 2D intraoperative images and does not require fiducial markers, feature extraction or image segmentation. Results of experiments with a biopsy training breast phantom show that the proposed method can perform well in the presence of large deformations. This is particularly useful for clinical applications such as MR-based breast biopsy where large tissue deformations occur.
---
paper_title: Co-registration of Multiple Postmortem Brain Slices to Corresponding MRIs Using Voxel Similarity Measures and Slice-to-Volume Transformation

paper_content:

New methods to register multiple hemispheric slices of the postmortem brain to anatomically corresponding in-vivo MRI slices within a 3D volumetric MRI are presented. Gel-embedding and fiducial markers are used to reduce geometrical distortions in the postmortem brain volume. The registration algorithm relies on a recursive extraction of warped MRI slices from the reference MRI volume using a modified non-linear polynomial transformation until matching slices are found. Eight different voxel similarity measures are tested to get the best co-registration cost and the results show that combination of two different similarity measures shows the best performance. After validating the implementation and approach through simulation studies, the presented methods are applied to real data. The results demonstrate the feasibility and practicability of the presented co­registration methods, thus providing a means of MR signal analysis and histological examination of tissue lesions via co­registered images of postmortem brain slices and their corresponding MRI sections. With this approach, it is possible to investigate the pathology of a disease through both routinely acquired MRls and postmortem brain slices, thus improving the understanding of the pathological substrates and their progression.
---
paper_title: Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

paper_content:

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: Adaptive visualization for needle guidance in RF liver ablation: taking organ deformation into account

paper_content:

Interventional procedures on deformable organs pose difficulties for the radiologists when inserting the probe ::: towards a lesion. The deformation due to the breathing makes a reliable and automated alignment of the ::: interventional 2D CT-Fluoro to the pre-interventional 3D CT-Volume very challenging. Such alignment is highly ::: desirable since, during the intervention, the CT-Volume brings more information as it is enhanced with contrast ::: agent and has a higher resolution than the CT-Fluoro slice. A reasonable solution for the alignment is obtained ::: by employing a robust optimization technique. However, since we would like to help the needle guidance towards ::: the lesion, due to the involved deformation, a single slice of the 3D CT-Volume is not satisfactory. ::: The main contribution of this paper consists in visualizing slices of the 3D CT-Volume that are resulting ::: from the out-of-plane motion parameters along weighted isosurfaces in the convergence basin of the similarity ::: function used during the alignment. This visualization copes with the uncertainty in estimating the deformation ::: and brings much more information than a single registered slice. Three experienced interventional radiologists ::: were consulted and their evaluation clearly highlighted that such visualization unfolding the neighborhood with ::: the belonging structures, like vessels and lesion spread, will help the needle guidance.
---
paper_title: Deformable Ultrasound Registration without Reconstruction

paper_content:

Ultrasound (US) imaging is often proposed as an interoperative imaging modality. This use nearly always requires that the collected data be registered to preoperative data of another modality. Existing intensity-based registration approaches all begin by reconstructing a 3D US volume from the collected 2D slices. We propose to directly register the set of 2D slices to the preoperative images. We argue this has a number of advantages, including the omission of the potentially complex reconstruction step, greater adaptability of the similarity measures, and easier parallelization. We describe a system for performing this task and present results on phantom data that show that our slice based method consistently outperforms a reconstruction based method in both speed and accuracy.
---
paper_title: Target Motion Tracking in MRI-guided Transrectal Robotic Prostate Biopsy

paper_content:

Purpose: MRI-guided prostate needle biopsy requires compensation for organ motion between target planning and needle placement. Two questions are studied and answered in this paper: 1) is rigid registration sufficient in tracking the targets with an error smaller than the clinically significant size of prostate cancer and 2) what is the effect of the number of intraoperative slices on registration accuracy and speed? Methods: we propose multislice-to-volume registration algorithms for tracking the biopsy targets within the prostate. Three orthogonal plus additional transverse intraoperative slices are acquired in the approximate center of the prostate and registered with a high-resolution target planning volume. Both rigid and deformable scenarios were implemented. Both simulated and clinical MRI-guided robotic prostate biopsy data were used to assess tracking accuracy. Results: average registration errors in clinical patient data were 2.6 mm for the rigid algorithm and 2.1 mm for the deformable algorithm. Conclusion: rigid tracking appears to be promising. Three tracking slices yield significantly high registration speed with an affordable error.
---
paper_title: Automatic 3-D Breath-Hold Related Motion Correction of Dynamic Multislice MRI

paper_content:

Magnetic resonance (MR) cine images are often used to clinically assess left ventricular cardiac function. In a typical study, multiple 2-D long axis (LA) and short axis (SA) cine images are acquired, each in a different breath-hold. Differences in lung volume during breath-hold and overall patient motion distort spatial alignment of the images thus complicating spatial integration of all image data in three dimensions. We present a fully automatic postprocessing approach to correct these slice misalignments. The approach is based on the constrained optimization of the intensity similarity of intersecting image lines after the automatic definition of a region of interest. It uses all views and all time frames simultaneously. Our method models both in-plane and out-of-plane translations and full 3-D rotations, can be applied retrospectively and does not require a cardiac wall segmentation. The method was validated on both healthy volunteer and patient data with simulated misalignments, as well as on clinical multibreath-hold patient data. For the simulated data, subpixel accuracy could be obtained using translational correction. The possibilities and limitations of rotational correction were investigated and discussed. For the clinical multibreath-hold patient data sets, the median discrepancy between manual SA and LA contours was reduced from 2.83 to 1.33 mm using the proposed correction method. We have also shown the usefulness of the correction method for functional analysis on clinical image data. The same clinical multibreath-hold data sets were resegmented after positional correction, taking newly available complementary information of intersecting slices into account, further reducing the median discrepancy to 0.43 mm. This is due to the integration of the 2-D slice information into 3-D space.
---
paper_title: Intersection based registration of slice stacks to form 3D images of the human fetal brain

paper_content:

Clinical fetal MR imaging of the brain commonly makes use of fast 2D acquisitions of multiple sets of approximately orthogonal 2D slices. We and others have previously proposed an iterative slice-to-volume registration process to recover a geometrically consistent 3D image. However, these approaches depend on a 3D volume reconstruction step during the slice alignment. This is both computationally expensive and makes the convergence of the registration process poorly defined. In this paper our key contribution is a new approach which considers the collective alignment of all slices directly, via shared structure in their intersections, rather than to an estimated 3D volume. We derive an analytical expression for the gradient of the collective similarity of the slices along their intersections, with respect to the 3D location and orientation of each 2D slice. We include examples of the approach applied to simulated data and clinically acquired fetal images.
---
paper_title: Registration-based Approach for Reconstruction of High Resolution In Utero Fetal MR Brain Images

paper_content:

Rationale and Objectives This paper describes a novel approach to forming high-resolution MR images of the human fetal brain. It addresses the key problem of fetal motion by proposing a registration-refined compounding of multiple sets of orthogonal fast two-dimensional MRI slices, which are currently acquired for clinical studies, into a single high-resolution MRI volume. Materials and Methods A robust multiresolution slice alignment is applied iteratively to the data to correct motion of the fetus that occurs between two-dimensional acquisitions. This is combined with an intensity correction step and a super-resolution reconstruction step, to form a single high isotropic resolution volume of the fetal brain. Results Experimental validation on synthetic image data with known motion types and underlying anatomy, together with retrospective application to sets of clinical acquisitions, are included. Conclusion Results indicate that this method promises a unique route to acquiring high-resolution MRI of the fetal brain in vivo allowing comparable quality to that of neonatal MRI. Such data provide a highly valuable window into the process of normal and abnormal brain development, which is directly applicable in a clinical setting.
---
paper_title: Motion Correction in fMRI by Mapping Slice-to-Volume with Concurrent Field-Inhomogeneity Correction

paper_content:

Head motion is the major source of error in measuring intensity changes related to given stimuli in fMRI. The effects of head motion are image shifts and field inhomogeneity variations which cause local changes in geometric distortions. The previously developed motion correction method, mapping slice-to-volume (MSV), retrospectively remaps slices that are shifted by head motion to their spatially correct locations in an anatomical reference. Images exhibiting spatially varying geometric distortions require non-linear mapping solutions. An accurate field map can be used for the correction of such spatial distortions. However, field-map changes with head motion and, in practice, only one field-map is available typically. This work evaluates the improved motion correction capability of MSV with concurrent iterative field-corrected reconstruction using only an initial field-map. The results from simulated motion data show effective convergence and accuracy in image registration for the correction of image artifacts complicated by the motion induced field effects.
---
paper_title: Registration of freehand 3D ultrasound and magnetic resonance liver images

paper_content:

Abstract We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a “bronze standard” registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the “bronze standard” and this is improved to 3.6 mm after running the intensity-based algorithm.
---
paper_title: Cadaver validation of intensity-based ultrasound to CT registration

paper_content:

A method is presented for the registration of tracked B-mode ultrasound images to a CT volume of a femur or pelvis. This registration can allow tracked surgical instruments to be aligned with the CT image or an associated preoperative plan. Our method requires no manual segmentation of either the ultrasound images or the CT volume. The CT and US images are processed to produce images where the image intensity represents the probability of the presence of a bone edge. These images are then registered together using normalised cross-correlation as a similarity measure. The parameter which represents the speed of sound through tissue has also been included in the registration optimisation process. Experiments have been carried out on six cadaveric femurs and three cadaveric pelves. Registration results were compared with a "gold standard" registration acquired using bone implanted fiducial markers. Results show the registration method to be accurate, on average, to 1.7 mm root-mean-square target registration error.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: 2D/3D registration of endoscopic ultrasound to CT volume data.

paper_content:

This paper describes a computer-aided navigation system using image fusion to support endoscopic interventions such as the accurate collection of biopsy specimens. An endoscope provides the physician with real-time ultrasound (US) and a video image. An image slice that corresponds to the corresponding image from the US scan head is derived from a preoperative computed tomography (CT) or magnetic resonance image volume data set using oblique reformatting and displayed side by side with the US image. The position of the image acquired by the US scan head is determined by a miniaturized electromagnetic tracking system (EMTS) after calibrating the endoscope's scan head. The transformation between the patient coordinate system and the preoperative data set is calculated using a 2D/3D registration. This is achieved by calibrating an intraoperative interventional CT slice with an optical tracking system (OTS) using the same algorithm as for the US calibration. The slice is then used for 2D/3D registration with the coordinate system of the preoperative volume. The fiducial registration error (FRE) for the US calibration was 2.0 mm +/- 0.4 mm; the interventional CT FRE was 0.36 +/- 0.12 mm; and the 2D/3D registration target registration error (TRE) was 1.8 +/- 0.3 mm. The point-to-point registration between the OTS and the EMTS had an FRE of 0.9 +/- 0.4 mm. Finally, we found an overall TRE for the complete system to be 3.9 +/- 0.6 mm.
---
paper_title: A comparative study on manual and automatic slice-to-volume registration of CT images

paper_content:

In order to assess the clinical relevance of a slice-to-volume registration algorithm, this technique was compared to manual registration. Reformatted images obtained from a diagnostic CT examination of the lower abdomen were reviewed and manually registered by 41 individuals. The results were refined by the algorithm. Furthermore, a fully automatic registration of the single slices to the whole CT examination, without manual initialization, was also performed. The manual registration error for rotation and translation was found to be 2.7±2.8 ° and 4.0±2.5 mm. The automated registration algorithm significantly reduced the registration error to 1.6±2.6 ° and 1.3±1.6 mm (p = 0.01). In 3 of 41 (7.3%) registration cases, the automated registration algorithm failed completely. On average, the time required for manual registration was 213±197 s; automatic registration took 82±15 s. Registration was also performed without any human interaction. The resulting registration error of the algorithm without manual pre-registration was found to be 2.9±2.9 ° and 1.1±0.2 mm. Here, a registration took 91±6 s, on average. Overall, the automated registration algorithm improved the accuracy of manual registration by 59% in rotation and 325% in translation. The absolute values are well within a clinically relevant range.
---
paper_title: A simplex method for function minimization

paper_content:

A method is described for the minimization of a function of n variables, which depends on the comparison of function values at the (n 41) vertices of a general simplex, followed by the replacement of the vertex with the highest value by another point. The simplex adapts itself to the local landscape, and contracts on to the final minimum. The method is shown to be effective and computationally compact. A procedure is given for the estimation of the Hessian matrix in the neighbourhood of the minimum, needed in statistical estimation problems.
---
paper_title: Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI

paper_content:

Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acquired by these techniques, inter-slice motion leads to severe motion artifacts that are apparent in out-of-plane views. Slice sequential acquisitions do not enable 3-D volume representation. In this study, we have developed a novel technique based on a slice acquisition model, which enables the reconstruction of a volumetric image from multiple-scan slice acquisitions. The super-resolution volume reconstruction is formulated as an inverse problem of finding the underlying structure generating the acquired slices. We have developed a robust M-estimation solution which minimizes a robust error norm function between the model-generated slices and the acquired slices. The accuracy and robustness of this novel technique has been quantitatively assessed through simulations with digital brain phantom images as well as high-resolution newborn images. We also report here successful application of our new technique for the reconstruction of volumetric fetal brain MRI from clinically acquired data.
---
paper_title: Registration and fusion of SPECT, high-resolution MRI, and interventional MRI for thermal ablation of prostate cancer

paper_content:

We are investigating interventional MRI (iMRI) guided radiofrequency thermal ablation for the minimally invasive treatment of the prostate cancer. Nuclear medicine can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of functional images such as nuclear medicine SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high-resolution MRI volume. Then by registering the high-resolution MR image with live-time iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to live-time iMRI images for improved tumor targeting. For the first step, we used a three-dimensional mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm with special features. The concept was tested using image data from three patients and three volunteers. The SV registration accuracy was 0.4 mm /spl plusmn/ 0.2 mm as compared to our volume- to-volume registration that was previously shown to be quite accurate for these image pairs. With our image registration and fusion software, simulation experiments show that it is quite feasible to incorporate SPECT and high-resolution MRI into the iMRI-guided minimally invasive treatment procedures.
---
paper_title: Improved Motion Correction in fMRI by Joint Mapping of Slices into an Anatomical Volume

paper_content:

Motion correction in fMRI time series is essential for accurate statistical analyses. Typically motion correction is applied in a rigid fashion between time series volumes. Such corrections assume no relative motion between slices. An improved motion correction scheme, map-slice-to-volume (MSV), was developed previously using mutual information (MI) to register individual fMRI slices onto an anatomical volume to account for inter-slice motion [6]. As each slice’s orientation represents a statistically independent sampling of the patient’s motion at multiple intervals throughout the acquisition, a smoothed estimate of the patient’s trajectory in time can be computed by jointly estimating each slice’s orientation while minimizing the implied acceleration of the patient’s head subject to applying a priori, slice-related weights based on known registration reliabilities. The results of this joint mapping of slices into a volume (JMSV) show further substantial improvement in slice registration and subsequent motion estimation.
---
paper_title: Sparse Registration for Three-Dimensional Stress Echocardiography

paper_content:

Three-dimensional (3-D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction. It involves evaluating wall motion of the left ventricle, by visually analyzing ultrasound images obtained in rest and in different stages of stress. Since the acquisitions are performed minutes apart, variabilities may exist in the visualized cross-sections. To improve anatomical correspondence between rest and stress, aligning the images is essential. We developed a new intensity-based, sparse registration method to retrieve standard anatomical views from 3-D stress images that were equivalent to the manually selected views in the rest images. Using sparse image planes, the influence of common image artifacts could be reduced. We investigated different similarity measures and different levels of sparsity. The registration was tested using data of 20 patients and quantitatively evaluated based on manually defined anatomical landmarks. Alignment was best using sparse registration with two long-axis and two short-axis views; registration errors were reduced significantly, to the range of interobserver variabilities. In 91% of the cases, the registration result was qualitatively assessed as better than or equal to the manual alignment. In conclusion, sparse registration improves the alignment of rest and stress images, with a performance similar to manual alignment. This is an important step towards objective quantification in 3-D stress echocardiography.
---
paper_title: Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

paper_content:

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.
---
paper_title: User-friendly freehand ultrasound calibration using Lego bricks and automatic registration

paper_content:

PURPOSE ::: As an inexpensive, noninvasive, and portable clinical imaging modality, ultrasound (US) has been widely employed in many interventional procedures for monitoring potential tissue deformation, surgical tool placement, and locating surgical targets. The application requires the spatial mapping between 2D US images and 3D coordinates of the patient. Although positions of the devices (i.e., ultrasound transducer) and the patient can be easily recorded by a motion tracking system, the spatial relationship between the US image and the tracker attached to the US transducer needs to be estimated through an US calibration procedure. Previously, various calibration techniques have been proposed, where a spatial transformation is computed to match the coordinates of corresponding features in a physical phantom and those seen in the US scans. However, most of these methods are difficult to use for novel users. ::: ::: ::: METHODS ::: We proposed an ultrasound calibration method by constructing a phantom from simple Lego bricks and applying an automated multi-slice 2D-3D registration scheme without volumetric reconstruction. The method was validated for its calibration accuracy and reproducibility. ::: ::: ::: RESULTS ::: Our method yields a calibration accuracy of [Formula: see text] mm and a calibration reproducibility of 1.29 mm. ::: ::: ::: CONCLUSION ::: We have proposed a robust, inexpensive, and easy-to-use ultrasound calibration method.
---
paper_title: Improved map-slice-to-volume motion correction with b0 inhomogeneity correction: validation of activation detection algorithms using ROC curve analyses

paper_content:

Head motion is a significant source of error in fMRI activation detection and a common approach is to apply 3D volumetric rigid body motion correction techniques. However, in 2D multislice fMRI, each slice may have a distinct set of motion parameters due to inter-slice motion. Here, we apply an automated mutual information based slice-to-volume rigid body registration technique on time series data synthesized from a T2 MRI brain dataset with simulated motion, functional activation, noise and geometric distortion. The map-slice-to-volume (MSV) technique was previously applied to patient data without ground truths for motion and activation regions. In this study, the activation images and area under the receiver operating characteristic curves for various time series datasets indicate that the MSV registration improves the activation detection capability when compared to results obtained from Statistical Parametric Mapping (SPM). The effect of temporal median filtering of motion parameters on activation detection performance was also investigated.
---
paper_title: A new registration/visualization paradigm for CT-Fluoroscopy guided RF liver ablation

paper_content:

2D-3D slice-to-volume registration for abdominal organs like liver is difficult due to the breathing motion and tissue deformation. The purpose of our approach is to ease CT-fluoroscopy (CT-fluoro) based needle insertion for the Radiofrequency Liver Ablation procedure using high resolution contrasted preoperative data. In this case, low signal-to-noise ratio, absence of contrast and additional presence of needle in CT-fluoro makes it difficult to guarantee the solution of any deformable slice-to-volume registration algorithm. In this paper, we first propose a method for creating a set of ground truth (GT) simulation data based on a non-linear deformation of the CT-fluoro volume obtained from real patients. Second, we split the CT-fluoro image and apply intensity based rigid and affine registration to each section. We then propose a novel solution, which consists of intuitive visualization sequences of optimal sub-volumes of preinterventional data based on the registration results. Experiments on synthetic and real patient data and direct feedback of two interventionalists validate our alternative approach.
---
paper_title: Particle filtering for slice-to-volume motion correction in EPI based functional MRI

paper_content:

Head movement during scanning introduces artificial signal changes and impedes activation detection in fMRI studies. The head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) sequence can be estimated and compensated by aligning the images onto a reference volume through image registration. Registering EPI images volume by volume fails to consider head motion between slices, leading to biased head motion estimates. Slice-to-volume registration is used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, it is prone to image noise and geometric distortion, resulting in high variance estimates. In this work, we propose a Gaussian particle filter based head motion tracking algorithm to reduce the image misregistration errors. The algorithm models head motion by using a dynamic state space model (SSM) to model continuous slice acquisition thereby providing more accurate motion estimates and voxel position estimates. We demonstrate significant performance improvement of the proposed approach as compared to previous registration-only methods of head motion estimation.
---
paper_title: Inter-breath-hold registration for the production of high resolution cardiac MR volumes

paper_content:

High resolutionMRI images of the beating heart permit observation of detailed anatomical features and enable quantification of small changes in metrics of cardiac function. To obtain approximately isotropic sampling with an adequate spatial and temporal resolution, these images need to be acquired in multiple breath-holds. They are, therefore, often affected by through-plane discontinuities due to inconsistent breath-hold positions. This paper presents a method to correct for these discontinuities by performing breath-hold-by-breath-hold registration of high resolution 3D data to radial long axis images. The corrected images appear free of discontinuities, and it was found that they could be delineated more reproducibly than uncorrected images. This reduces the sample size required to detect systematic changes in blood pool volume by 57% at end systole and 78% at end diastole.
---
paper_title: Image Guidance of Intracardiac Ultrasound with Fusion of Pre-operative Images ⋆

paper_content:

This paper presents a method for registering 3D intracardiac echo (ICE) to pre-operative images. A magnetic tracking sensor is integrated on the ICE catheter tip to provide the 3D location and orientation. The user guides the catheter into the patient heart to acquire a series of ultrasound images covering the anatomy of the heart chambers. An automatic intensity-based registration algorithm is applied to align these ultrasound images with pre-operative images. One of the important applications is to help electrophysiology doctors to treat complicated atrial fibrillation cases. After registration, the doctor can see the position and orientation of the ICE catheter and other tracked catheters inside the heart anatomy in real time. The image guidance provided by this technique may increase the ablation accuracy and reduce the amount of time for the electrophysiology procedures. We show successful image registration results from animal experiments.
---
paper_title: Determining histology-MRI slice correspondences for defining MRI-based disease signatures of prostate cancer

paper_content:

Abstract Mapping the spatial disease extent in a certain anatomical organ/tissue from histology images to radiological images is important in defining the disease signature in the radiological images. One such scenario is in the context of men with prostate cancer who have had pre-operative magnetic resonance imaging (MRI) before radical prostatectomy. For these cases, the prostate cancer extent from ex vivo whole-mount histology is to be mapped to in vivo MRI. The need for determining radiology-image-based disease signatures is important for (a) training radiologist residents and (b) for constructing an MRI-based computer aided diagnosis (CAD) system for disease detection in vivo . However, a prerequisite for this data mapping is the determination of slice correspondences (i.e. indices of each pair of corresponding image slices) between histological and magnetic resonance images. The explicit determination of such slice correspondences is especially indispensable when an accurate 3D reconstruction of the histological volume cannot be achieved because of (a) the limited tissue slices with unknown inter-slice spacing, and (b) obvious histological image artifacts (tissue loss or distortion). In the clinic practice, the histology-MRI slice correspondences are often determined visually by experienced radiologists and pathologists working in unison, but this procedure is laborious and time-consuming. We present an iterative method to automatically determine slice correspondence between images from histology and MRI via a group-wise comparison scheme, followed by 2D and 3D registration. The image slice correspondences obtained using our method were compared with the ground truth correspondences determined via consensus of multiple experts over a total of 23 patient studies. In most instances, the results of our method were very close to the results obtained via visual inspection by these experts.
---
paper_title: Correction of local deformations in fMRI by 3D non-linear warping in map-slice-to-volume approach

paper_content:

Zntroduction Recently, we have demonstrilted the capability of retrospectively mapping a slice image into rin anatomical reference volume (i.e. map-slice-to-volume, MSV) from the same subject using a simple rigid body transformation of $x degrees of freedom (DOF) [l]. In practice, deformation in a sliQe image is often presented as a nonlinear warping problem that his not limited to the imaging plane. Out-of-plane warp is typically encountered in multislice fMRI data due to the local field induced deformations or localized outof-plane motion artifacts. This study presents correction of the 3D warping artifacts localized in the regions that undergo local changes that result in spatial displacement of voxels typically observed in single shot ac@isitions [2]. The local warping solution is implemented follolwing the rigid body MSV transform that determines the patient’s~ rigid head motion causing in-plane and out-of-plane displacemend, Methods A slice image from time series EPI data was mapped into a high resolution reference yolume space, T1-weighted MRI volume data using a registration algorithm implementing either affine or a control pbint based thin-plate-spline (TPS) function. The automated registration is driven by the mutual information (Z) computed from the gray values of the image pair. By allowing the optimizer to move the control points in the homologok 3D space to positions that minimize the cost metric @&=-I), the iterative process determines the transformation coefficients for a coordinate mapping. For each iteration the affine or TPS warp transformation performs trilinear interpolation in which the original voxel grey values are used. This approach ensures that there is no undesired intensity change from interpolation effects such as round-off or smoothing across iterations. Results Retrospective image registration was performed to map a GREEPI slice image into a volumetric SPGR MRI data acquired from a 3T MRI system. The tota?ly automatic registration routine for each fMRI slice can be initiaipd using an initial three control point set, i.e., rigid body MSV, atid then can automatically increment DOF to multiple control point TPS warping transforms. The six DOF rotate/translate transfoF determines the spatial slice profile relative to the volumetric refqrence data at the time of acquisition. This process accommodates’ the patient’s head movement [l]. The following warping solution is initiated by using the optimized rigid body model to map the control points (> 4 pts) in the reference space into the homologous space. The TPS warping is subsequently computed, agati, by optimizing MI through iterative movement of the control points in homologous space. Improvement in registration using TPS warping solution is demonstrated in Fig. 1 as, compared to rigid-body in l(a). Deformatiotis in the center and the posterior edge are not removed by rigid body and the improved mapping with TPS solution is also indicated by lower MI index. Note that the final mapping of an acquired slice image into the-anatomical reference space is achieved through ?D warping, that corrects out-of-pime as well as in-plane local deformations besides the patient head motion.
---
paper_title: Automatic 3D registration for interventional MRI-guided treatment of prostate cancer.

paper_content:

The goal of this research is to register real-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained high-resolution MRI image volume, which in turn can be registered with functional images such as those Urom SPECT. The immediate application is in iMRI-guided treatment of prostate cancer, where additional images are desired to improve tumor targeting. In this article, simulation experiments are performed to demonstrate the feasibility of slice-to-volume registration for this application. We acquired 3D volume images from a 1.5-T MRI system and simulated low-field iMRI image slices by creating thick slices and adding noise. We created a slice-to-volume mutual information registration algorithm with special features to improve robustness. Features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. To assess the quality of registration, we calculated 3D displacements on a voxel-by-voxel basis over a volume of in...
---
paper_title: Multiscale Registration of Real-Time and Prior MRI Data for Image-Guided Cardiac Interventions

paper_content:

Abstract Recently, there is a growing interest in using magnetic resonance imaging (MRI) to guide interventional procedures due to its excellent soft tissue contrast and lack of ionizing radiation compared to traditional radiographic guidance. One of these applications is the use of MRI to guide radio frequency ablation of anatomic substrates, within the left ventricle, responsible for ventricular tachycardia. However, different MRI acquisition schemes have significant tradeoffs between image quality and acquisition time. Guidance using high-quality preoperative 3-D MR images is limited in the case of cardiac interventions because the heart moves dynamically during the procedure. On the other hand, 2-D real-time MR images acquired during the intervention sacrifice image quality for shorter image acquisition time, leading to real-time positional updates of cardiac anatomy. Ideally, we wish to combine the advantages of live feedback from real-time images and accurate visualization of anatomical structures from preoperative images. Therefore, to improve the MRI guidance capabilities for cardiac interventions, we describe a novel multiscale rigid registration framework to correct for respiratory motion between the prior and real-time datasets. In the proposed approach, we use a weighted total variation flow algorithm to extract coarse-to-fine features from the input images and subsequently register the corresponding scales in a hierarchical manner. Registration experiments were performed with in vivo human imaging data, and the target registration error achieved was 1.51 mm. Thus, the feasibility of motion correction in an interventional setting has been demonstrated, which may lead to significant improvements in the guidance of cardiac interventions.
---
paper_title: Linear Intensity-based Image Registration by Markov Random Fields and Discrete Optimization

paper_content:

We propose a framework for intensity-based registration of images by linear transformations, based on a discrete Markov random field (MRF) formulation. Here, the challenge arises from the fact that optimizing the energy associated with this problem requires a high-order MRF model. Currently, methods for optimizing such high-order models are less general, easy to use, and efficient, than methods for the popular second-order models. Therefore, we propose an approximation to the original energy by an MRF with tractable second-order terms. The approximation at a certain point p in the parameter space is the normalized sum of evaluations of the original energy at projections of p to two-dimensional subspaces. We demonstrate the quality of the proposed approximation by computing the correlation with the original energy, and show that registration can be performed by discrete optimization of the approximated energy in an iteration loop. A search space refinement strategy is employed over iterations to achieve sub-pixel accuracy, while keeping the number of labels small for efficiency. The proposed framework can encode any similarity measure is robust to the settings of the internal parameters, and allows an intuitive control of the parameter ranges. We demonstrate the applicability of the framework by intensity-based registration, and 2D–3D registration of medical images. The evaluation is performed by random studies and real registration tasks. The tests indicate increased robustness and precision compared to corresponding standard optimization of the original energy, and demonstrate robustness to noise. Finally, the proposed framework allows the transfer of advances in MRF optimization to linear registration problems.
---
paper_title: Rigid Slice-To-Volume Medical Image Registration through Markov Random Fields

paper_content:

Rigid slice-to-volume registration is a challenging task, which finds application in medical imaging problems like image fusion for image guided surgeries and motion correction for volume reconstruction. It is usually formulated as an optimization problem and solved using standard continuous methods. In this paper, we discuss how this task be formulated as a discrete labeling problem on a graph. Inspired by previous works on discrete estimation of linear transformations using Markov Random Fields (MRFs), we model it using a pairwise MRF, where the nodes are associated to the rigid parameters, and the edges encode the relation between the variables. We compare the performance of the proposed method to a continuous formulation optimized using simplex, and we discuss how it can be used to further improve the accuracy of our approach. Promising results are obtained using a monomodal dataset composed of magnetic resonance images (MRI) of a beating heart.
---
paper_title: Non-rigid 2D-3D Medical Image Registration using Markov Random Fields

paper_content:

The aim of this paper is to propose a novel mapping algorithm between 2D images and a 3D volume seeking simultaneously a linear plane transformation and an in-plane dense deformation. We adopt a metric free locally over-parametrized graphical model that combines linear and deformable parameters within a coupled formulation on a 5-dimensional space. Image similarity is encoded in singleton terms, while geometric linear consistency of the solution (common/single plane) and in-plane deformations smoothness are modeled in a pair-wise term. The robustness of the method and its promising results with respect to the state of the art demonstrate the extreme potential of this approach.
---
paper_title: Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey

paper_content:

In this paper, we present a comprehensive survey of Markov Random Fields (MRFs) in computer vision and image understanding, with respect to the modeling, the inference and the learning. While MRFs were introduced into the computer vision field about two decades ago, they started to become a ubiquitous tool for solving visual perception problems around the turn of the millennium following the emergence of efficient inference methods. During the past decade, a variety of MRF models as well as inference and learning methods have been developed for addressing numerous low, mid and high-level vision problems. While most of the literature concerns pairwise MRFs, in recent years we have also witnessed significant progress in higher-order MRFs, which substantially enhances the expressiveness of graph-based models and expands the domain of solvable problems. This survey provides a compact and informative summary of the major literature in this research topic.
---
paper_title: Approximate Labeling via Graph Cuts Based on Linear Programming

paper_content:

A new framework is presented for both understanding and developing graph-cut-based combinatorial algorithms suitable for the approximate optimization of a very wide class of Markov random fields (MRFs) that are frequently encountered in computer vision. The proposed framework utilizes tools from the duality theory of linear programming in order to provide an alternative and more general view of state-of-the-art techniques like the alpha-expansion algorithm, which is included merely as a special case. Moreover, contrary to alpha-expansion, the derived algorithms generate solutions with guaranteed optimality properties for a much wider class of problems, for example, even for MRFs with nonmetric potentials. In addition, they are capable of providing per-instance suboptimality bounds in all occasions, including discrete MRFs with an arbitrary potential function. These bounds prove to be very tight in practice (that is, very close to 1), which means that the resulting solutions are almost optimal. Our algorithms' effectiveness is demonstrated by presenting experimental results on a variety of low-level vision tasks, such as stereo matching, image restoration, image completion, and optical flow estimation, as well as on synthetic problems.
---
paper_title: An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision

paper_content:

Minimum cut/maximum flow algorithms on graphs have emerged as an increasingly useful tool for exactor approximate energy minimization in low-level vision. The combinatorial optimization literature provides many min-cut/max-flow algorithms with different polynomial time complexity. Their practical efficiency, however, has to date been studied mainly outside the scope of computer vision. The goal of this paper is to provide an experimental comparison of the efficiency of min-cut/max flow algorithms for applications in vision. We compare the running times of several standard algorithms, as well as a new algorithm that we have recently developed. The algorithms we study include both Goldberg-Tarjan style "push -relabel" methods and algorithms based on Ford-Fulkerson style "augmenting paths." We benchmark these algorithms on a number of typical graphs in the contexts of image restoration, stereo, and segmentation. In many cases, our new algorithm works several times faster than any of the other methods, making near real-time performance possible. An implementation of our max-flow/min-cut algorithm is available upon request for research purposes.
---
paper_title: MRF Energy Minimization and Beyond via Dual Decomposition

paper_content:

This paper introduces a new rigorous theoretical framework to address discrete MRF-based optimization in computer vision. Such a framework exploits the powerful technique of Dual Decomposition. It is based on a projected subgradient scheme that attempts to solve an MRF optimization problem by first decomposing it into a set of appropriately chosen subproblems, and then combining their solutions in a principled way. In order to determine the limits of this method, we analyze the conditions that these subproblems have to satisfy and demonstrate the extreme generality and flexibility of such an approach. We thus show that by appropriately choosing what subproblems to use, one can design novel and very powerful MRF optimization algorithms. For instance, in this manner we are able to derive algorithms that: 1) generalize and extend state-of-the-art message-passing methods, 2) optimize very tight LP-relaxations to MRF optimization, and 3) take full advantage of the special structure that may exist in particular MRFs, allowing the use of efficient inference techniques such as, e.g., graph-cut-based methods. Theoretical analysis on the bounds related with the different algorithms derived from our framework and experimental results/comparisons using synthetic and real data for a variety of tasks in computer vision demonstrate the extreme potentials of our approach.
---
paper_title: Efficient MRF Deformation Model for Non-Rigid Image Matching

paper_content:

We propose a novel MRF-based model for deformable image matching. Given two images, the task is to estimate a mapping from one image to the other maximizing the quality of the match. We consider mappings defined by a discrete deformation field constrained to preserve 2D continuity. We pose the task as finding MAP configurations of a pairwise MRF. We propose a more compact MRF representation of the problem which leads to a weaker, though computationally more tractable, linear programming relaxation -the approximation technique we choose to apply. The number of dual LP variables grows linearly with the search window side, rather than quadratically as in previous approaches. To solve the relaxed problem (suboptimally), we apply TRW-S (Sequential Tree-Reweighted Message passing) algorithm [13, 5]. Using our representation and the chosen optimization scheme, we are able to match much wider deformations than was considered previously in global optimization framework. We further elaborate on continuity and data terms to achieve more appropriate description of smooth deformations. The performance of our technique is demonstrated on both synthetic and real-world experiments.
---
paper_title: MAP estimation via agreement on trees: message-passing and linear programming

paper_content:

We develop and analyze methods for computing provably optimal maximum a posteriori probability (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of tree-structured distributions, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original distribution. Next we develop two approaches to attempting to obtain tight upper bounds: a) a tree-relaxed linear program (LP), which is derived from the Lagrangian dual of the upper bounds; and b) a tree-reweighted max-product message-passing algorithm that is related to but distinct from the max-product algorithm. In this way, we establish a connection between a certain LP relaxation of the mode-finding problem and a reweighted form of the max-product (min-sum) message-passing algorithm.
---
paper_title: Deformable medical image registration: setting the state of the art with discrete methods.

paper_content:

This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.
---
paper_title: Loopy Belief Propagation for Approximate Inference: An Empirical Study

paper_content:

Recently, researchers have demonstrated that "loopy belief propagation" -- the use of Pearl's polytree algorithm in a Bayesian network with loops -- can perform well in the context of error-correcting codes. The most dramatic instance of this is the near Shannon-limit performance of "Turbo Codes" -- codes whose decoding algorithm is equivalent to loopy belief propagation in a chain-structured Bayesian network. ::: ::: In this paper we ask: is there something special about the error-correcting code context, or does loopy propagation work as an approximate inference scheme in a more general setting? We compare the marginals computed using loopy propagation to the exact ones in four Bayesian network architectures, including two real-world networks: ALARM and QMR. We find that the loopy beliefs often converge and when they do, they give a good approximation to the correct marginals. However, on the QMR network, the loopy beliefs oscillated and had no obvious relationship to the correct posteriors. We present some initial investigations into the cause of these oscillations, and show that some simple methods of preventing them lead to the wrong results.
---
paper_title: The lazy flipper: efficient depth-limited exhaustive search in discrete graphical models

paper_content:

We propose a new exhaustive search algorithm for optimization in discrete graphical models. When pursued to the full search depth (typically intractable), it is guaranteed to converge to a global optimum, passing through a series of monotonously improving local optima that are guaranteed to be optimal within a given and increasing Hamming distance. For a search depth of 1, it specializes to ICM. Between these extremes, a tradeoff between approximation quality and runtime is established. We show this experimentally by improving approximations for the non-submodular models in the MRF benchmark [1] and Decision Tree Fields [2].
---
paper_title: Implicit planar and in-plane deformable mapping in medical images through high order graphs

paper_content:

In this paper we propose a novel method based on discrete optimization of high order graphs, to perform deformable slice-to-volume registration of 2D images and 3D volumes. To this end, a 2D grid superimposed to the image is considered with control points deforming in 3D and their deformations corresponding to the label space. Geometrical consistency (unique plane selection) and deformation smoothness (in-plane deformations) as well as image similarity (visual matching) are encoded in different third order cliques. The proposed formulation is optimized through its mapping to a factor graph using conventional graph optimization methods. A dataset composed of 2D slices and 3D MRI volumes of the heart was used to evaluate its accuracy leading to very promising results.
---
paper_title: Optimizing Binary MRFs via Extended Roof Duality

paper_content:

Many computer vision applications rely on the efficient optimization of challenging, so-called non-submodular, binary pairwise MRFs. A promising graph cut based approach for optimizing such MRFs known as "roof duality" was recently introduced into computer vision. We study two methods which extend this approach. First, we discuss an efficient implementation of the "probing" technique introduced recently by Bows et al. (2006). It simplifies the MRF while preserving the global optimum. Our code is 400-700 faster on some graphs than the implementation of the work of Bows et al. (2006). Second, we present a new technique which takes an arbitrary input labeling and tries to improve its energy. We give theoretical characterizations of local minima of this procedure. We applied both techniques to many applications, including image segmentation, new view synthesis, super-resolution, diagram recognition, parameter learning, texture restoration, and image deconvolution. For several applications we see that we are able to find the global minimum very efficiently, and considerably outperform the original roof duality approach. In comparison to existing techniques, such as graph cut, TRW, BP, ICM, and simulated annealing, we nearly always find a lower energy.
---
paper_title: Finding MAPs for belief networks is NP-hard

paper_content:

Given a probabilistic world model, an important problem is to find the maximum a-posteriori probability (MAP) instantiation of all the random variables given the evidence. Numerous researchers using such models employ some graph representation for the distributions, such as a Bayesian belief network. This representation simplifies the complexity of specifying the distributions from exponential in n, the number of variables in the model, to linear in n, in many interesting cases. We show, however, that finding the MAP is NP-hard in the general case when these representations are used, even if the size of the representation happens to be linear in n. Furthermore, minor modifications to the proof show that the problem remains NP-hard for various restrictions of the topology of the graphs. The same technique can be applied to the results of a related paper (by Cooper), to further restrict belief network topology in the proof that probabilistic inference is NP-hard.
---
paper_title: Statistical analysis of dirty pictures

paper_content:

A continuous two-dimensional region is partitioned into a fine rectangular array of sites, or ‘pixels', each pixel having a particular '‘colour’ belonging to a prescribed finite set. The true colouring of the region is unknown but, associated with each pixel, there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model. The aim is to reconstruct the true scene, with the additional knowledge that pixels close together tend to have the same or similar colours. In this paper, it is assumed that the local characteristics of the true scene can be represented by a non-degenerate Markov random field. Such information can be combined with the records by Bayes' theorem and the true scene can be estimated according to standard criteria. However, the computational burden is enormous and the reconstruction may reflect undesirable large-scale properties of the random field. Thus, a simple, iterative method of reconstruction is proposed, which does ...
---
paper_title: Fast approximate energy minimization via graph cuts

paper_content:

In this paper we address the problem of minimizing a large class of energy functions that occur in early vision. The major restriction is that the energy function's smoothness term must only involve pairs of pixels. We propose two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed. The first move we consider is an /spl alpha/-/spl beta/-swap: for a pair of labels /spl alpha/,/spl beta/, this move exchanges the labels between an arbitrary set of pixels labeled a and another arbitrary set labeled /spl beta/. Our first algorithm generates a labeling such that there is no swap move that decreases the energy. The second move we consider is an /spl alpha/-expansion: for a label a, this move assigns an arbitrary set of pixels the label /spl alpha/. Our second algorithm, which requires the smoothness term to be a metric, generates a labeling such that there is no expansion move that decreases the energy. Moreover, this solution is within a known factor of the global minimum. We experimentally demonstrate the effectiveness of our approach on image restoration, stereo and motion.
---
paper_title: Rigid Registration of Untracked Freehand 2D Ultrasound Sweeps to 3D CT of Liver Tumours

paper_content:

We present a rigid registration framework for freehand 2D ultrasound sweeps to 3D CT of liver tumours. The method registers the 2D sweeps in a group-wise manner, without the need for prior 3D ultrasound compounding or probe tracking during acquisition. We first introduce a specific acquisition model to keep the dimension of this problem reasonable. Only seven parameters are indeed required to register the images. These are estimated using simulated annealing optimization of a robust modality-independent similarity measure. The framework contrasts the current methods that rely on tracking devices and phantom calibration, which are often difficult to use routinely in clinical practice. Our results on both synthetic and real data show that the method is well-suited for such ultrasound-CT registration of liver tumours.
---
paper_title: Atlas-Based Indexing of Brain Sections via 2-D to 3-D Image Registration

paper_content:

A 2-D to 3-D nonlinear intensity-based registration method is proposed in which the alignment of histological brain sections with a volumetric brain atlas is performed. First, sparsely cut brain sections were linearly matched with an oblique slice automatically extracted from the atlas. Second, a planar-to-curved surface alignment was employed in order to match each section with its corresponding image overlaid on a curved-surface within the atlas. For the latter, a PDE-based registration technique was developed that is driven by a local normalized-mutual-information similarity measure. We demonstrate the method and evaluate its performance with simulated and real data experiments. An atlas-guided segmentation of mouse brains' hippocampal complex, retrieved from the Mouse Brain Library (MBL) database, is demonstrated with the proposed algorithm.
---
paper_title: Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images.

paper_content:

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.
---
paper_title: Localization of sections within the brain via 2D to 3D image registration

paper_content:

The mouse brain library (MBL) is a database of brain images, each consisting of sparse coronal or horizontal sections. To facilitate morphometric research, it is necessary to index each of these sections by its location within a canonical 3D atlas (NeuroTerrain). This is done with a 2D to 3D matching technique which was developed in this study. The registration method is imaged-based and uses a genetic algorithm to find the local-affine transformation that maximizes a mutual information metric. The average distance between the registration results achieved with the proposed method as compared with manual matching by an expert was 250 microns. This compares well with repeated manual trials where inter-trial matching distance was on average 200 microns.
---
paper_title: Target Motion Tracking in MRI-guided Transrectal Robotic Prostate Biopsy

paper_content:

Purpose: MRI-guided prostate needle biopsy requires compensation for organ motion between target planning and needle placement. Two questions are studied and answered in this paper: 1) is rigid registration sufficient in tracking the targets with an error smaller than the clinically significant size of prostate cancer and 2) what is the effect of the number of intraoperative slices on registration accuracy and speed? Methods: we propose multislice-to-volume registration algorithms for tracking the biopsy targets within the prostate. Three orthogonal plus additional transverse intraoperative slices are acquired in the approximate center of the prostate and registered with a high-resolution target planning volume. Both rigid and deformable scenarios were implemented. Both simulated and clinical MRI-guided robotic prostate biopsy data were used to assess tracking accuracy. Results: average registration errors in clinical patient data were 2.6 mm for the rigid algorithm and 2.1 mm for the deformable algorithm. Conclusion: rigid tracking appears to be promising. Three tracking slices yield significantly high registration speed with an affordable error.
---
paper_title: Multi-slice to volume registration of ultrasound data to a statistical atlas of human pelvis

paper_content:

Identifying the proper orientation of the pelvis is a critical step in accurate placement of the femur prosthesis in the ::: acetabulum in Total Hip Replacement (THR) surgeries. The general approach to localize the orientation of the pelvis ::: coordinate system is to use X-ray fluoroscopy to guide the procedure. An alternative can be employing intra-operative ::: ultrasound (US) imaging with pre-operative CT scan or fluoroscopy imaging. In this paper, we propose to replace the ::: need of pre-operative imaging by using a statistical shape model of the pelvis, constructed from several CT images. We ::: then propose an automatic deformable intensity-based registration of the anatomical atlas to a sparse set of 2D ::: ultrasound images of the pelvis in order to localize its anatomical coordinate system. In this registration technique, we ::: first extract a set of 2D slices from a single instance of the pelvic atlas. Each individual 2D slice is generated based on ::: the location of a corresponding 2D ultrasound image. Next, we create simulated ultrasound images out of the 2D atlas ::: slices and calculate a similarity metric between the simulated images and the actual ultrasound images. The similarity ::: metric guides an optimizer to generate an instance of the atlas that best matches the ultrasound data. We demonstrated ::: the feasibility of our proposed approach on two male human cadaver data. The registration was able to localize a ::: patient-specific pelvic coordinate system with origin translation error of 2 mm and 3.45 mm, and average axes rotation ::: error of 3.5 degrees and 3.9 degrees for the two cadavers, respectively.
---
paper_title: Registration of Whole-Mount Histology and Volumetric Imaging of the Prostate Using Particle Filtering

paper_content:

Registration of histological slices to volumetric imaging of the prostate is an important task that can be used to optimize imaging for cancer detection. Such registration is challenging due to physical changes of the specimen during excision and fixation, and misalignment of the histological slices during preparation and digital scanning. In this work, we consider a multi-slice to volume registration method in which a stack of sparse, unaligned 2-D whole-mount histological slices is registered to a 3-D volumetric imaging of the prostate. We propose a particle filtering framework to contend with the high dimensionality of the search space and multimodal nature of the optimization. Such framework allows modeling of the uncertainty in the pose of the slices and in the imaged information, in order to derive optimal registration parameters in a Bayesian approach. Intensity-, region-, and point-based similarity metrics were incorporated into the registration algorithm to account for different imaging modalities. We demonstrate and evaluate our method on a diverse set of data that includes a synthetic volume, ex vivo and in vivo magnetic resonance imaging, and in vivo ultrasound.
---
paper_title: Reconstruction of a geometrically correct diffusion tensor image of a moving human fetal brain

paper_content:

Recent studies reported the development of methods for rigid registration of 2D fetal brain imaging data to correct for unconstrained fetal and maternal motion, and allow the formation of a true 3D image of conventional fetal brain anatomy from conventional MRI. Diffusion tensor imaging provides additional valuable insight into the developing brain anatomy, however the correction of motion artifacts in clinical fetal diffusion imaging is still a challenging problem. This is due to the challenging problem of matching lower signal-to-noise ratio diffusion weighted EPI slice data to recover between-slice motion, compounded by the presence of possible geometric distortions in the EPI data. In addition, the problem of estimating a diffusion model (such as a tensor) on a regular grid that takes into account the inconsistent spatial and orientation sampling of the diffusion measurements needs to be solved in a robust way. Previous methods have used slice to volume registration within the diffusion dataset. In this work, we describe an alternative approach that makes use of an alignment of diffusion weighted EPI slices to a conventional structural MRI scan which provides a geometrically correct reference image. After spatial realignment of each diffusion slice, a tensor field representing the diffusion profile is estimated by weighted least squared fitting. By qualitative and quantitative evaluation of the results, we confirm the proposed algorithm successfully corrects the motion and reconstructs the diffusion tensor field.
---
paper_title: Slice-to-Volume Nonrigid Registration of Histological Sections to MR Images of the Human Brain

paper_content:

Registration of histological images to three-dimensional imaging modalities is an important step in quantitative analysis of brain structure, in architectonic mapping of the brain, and in investigation of the pathology of a brain disease. Reconstruction of histology volume from serial sections is a well-established procedure, but it does not address registration of individual slices from sparse sections, which is the aim of the slice-to-volume approach. This study presents a flexible framework for intensity-based slice-to-volume nonrigid registration algorithms with a geometric transformation deformation field parametrized by various classes of spline functions: thin-plate splines (TPS), Gaussian elastic body splines (GEBS), or cubic B-splines. Algorithms are applied to cross-modality registration of histological and magnetic resonance images of the human brain. Registration performance is evaluated across a range of optimization algorithms and intensity-based cost functions. For a particular case of histological data, best results are obtained with a TPS three-dimensional (3D) warp, a new unconstrained optimization algorithm (NEWUOA), and a correlation-coefficient-based cost function.
---
paper_title: Rigid Registration of Untracked Freehand 2D Ultrasound Sweeps to 3D CT of Liver Tumours

paper_content:

We present a rigid registration framework for freehand 2D ultrasound sweeps to 3D CT of liver tumours. The method registers the 2D sweeps in a group-wise manner, without the need for prior 3D ultrasound compounding or probe tracking during acquisition. We first introduce a specific acquisition model to keep the dimension of this problem reasonable. Only seven parameters are indeed required to register the images. These are estimated using simulated annealing optimization of a robust modality-independent similarity measure. The framework contrasts the current methods that rely on tracking devices and phantom calibration, which are often difficult to use routinely in clinical practice. Our results on both synthetic and real data show that the method is well-suited for such ultrasound-CT registration of liver tumours.
---
paper_title: Diffusion tensor imaging (DTI) of the brain in moving subjects: application to in-utero fetal and ex-utero studies.

paper_content:

We present a methodology to achieve 3D high-resolution diffusion tensor image reconstruction of the brain in moving subjects. The source data is diffusion-sensitized single-shot echo-planar images. After continuous scanning to acquire a repeated series of parallel slices with 15 diffusion directions, image registration is used to realign the images to correct for subject motion. Once aligned, the diffusion images are treated as irregularly-sampled data where each voxel is associated with an appropriately rotated diffusion direction. This data is used to estimate the diffusion tensor on a regular grid. The method has been tested on data acquired at 1.5T from adults who deliberately moved and from eight fetuses imaged in utero. Maps of apparent diffusion coefficient (ADC) were reliably produced in all cases and promising performance was achieved for fractional anisotropy maps. Results from normal fetal brains were found to be consistent with published data from premature infants of similar gestational age.
---
paper_title: Dynamic MR-based respiratory motion compensation for hybrid PET/MR system

paper_content:

This paper describes a novel respiratory motion compensation (MC) technique for hybrid PET/MR system. Existing PET/MR respiratory MC techniques mainly rely on respiratory gating to reconstruct 3D MR images for different respiratory phases, and use image registration techniques to estimate the deformation between different respiratory phases. A well known limitation of respiratory gating is its assumption that breathing motion is perfectly periodic, therefore MC techniques based on gated MR are sensitive to irregular breathing patterns. To address this limitation, the proposed technique uses dynamic 2D MRIs for PET/MR respiratory motion estimation. A static 3D MRI and a series of dynamic 2D MRIs are acquired before and during the PET data acquisition, respectively. The dynamic 2D MRIs are registered against the static 3D MRI using a novel deformable 2D+t to 3D image registration method to derive a 3D+t deformation field, which is then used for motion corrected PET reconstruction. The proposed method is validated on synthetic PET/MR data and real MR data, demonstrating its efficacy to compensate for irregular respiratory motion and to deblur the PET image.
---
paper_title: Correction of misaligned slices in multi-slice cardiovascular magnetic resonance using slice-to-volume registration

paper_content:

A popular technique to reduce respiratory motion for cardiovascular magnetic resonance is to perform a multi-slice acquisition in which a patient holds their breath multiple times during the scan. The feasibility of rigid slice-to-volume registration to correct for misalignments of slice stacks in such images due to differing breath-hold positions is explored. Experimental results indicate that slice-to-volume registration can compensate for the typical misalignments expected. Correction of slice misalignment results in anatomically more correct images, as well as improved left ventricular volume measurements. The interstudy reproducibility has also been improved reducing the number of samples needed for cardiac MR studies.
---
paper_title: Confocal bi-protocol: a new strategy for isotropic 3D live cell imaging

paper_content:

The conventional approach for microscopic 3D cellular imaging is based on axial through-stack image series which has some significant limitations such as anisotropic resolution and axial aberration. To overcome these drawbacks, we have recently introduced an alternative approach based on micro-rotation image series. Unfortunately, this new technique suffers from a huge burden of computation that makes its use quite difficult for current applications. To address these problems we propose a new imaging strategy called bi-protocol, which consists of coupling micro-rotation acquisition and conventional z-stack acquisition. We experimentally prove bi-protocol 3D reconstruction produces similar quality to that of pure micro-rotation, but offers the advantage of reduced computation burden because it uses the z-stack volume to accelerate the registration of the micro-rotation images.
---
paper_title: Mapping and characterizing endometrial implants by registering 2D transvaginal ultrasound to 3D pelvic magnetic resonance images

paper_content:

Mapping an endometrial tissue seen in Transvaginal Ultrasound (TVUS) to a Magnetic Resonance (MR) image is an important approach to improve the accuracy of diagnosis and preoperative surgery planning. To date, no technical methods solve this problem. In a clinical setting where only one 2D TVUS image is available, methods in the state of the art require the selection of a 2D MR slice and use 2D/2D deformable image registration. This is a limitation since the standard TVUS and MR imaging techniques used for diagnosing endometriosis are 2D and 3D, respectively. In this work, we introduce a novel deformable slice-to-volume registration process to overcome this limitation. To this end, we register a 2D TVUS image to a 3D MR data in order to transform the TVUS planar image to a curved 2D surface in MR volume. The proposed method facilitates the transfer of two types of information from a 2D TVUS image to the MR volume: 1) the location and shape of small implants and 2) the implants' depth of infiltration in the host tissue. Our TVUS-MR registration method uses contour to surface correspondences through a novel variational one-step deformable Iterative Closest Point (ICP) method. We demonstrate the accuracy of the proposed method by quantitative and qualitative tests on semi-synthetic and clinical data sets. Experimental error analysis shows high registration accuracy, illustrating the robustness of the method.
---
paper_title: Sparse Registration for Three-Dimensional Stress Echocardiography

paper_content:

Three-dimensional (3-D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction. It involves evaluating wall motion of the left ventricle, by visually analyzing ultrasound images obtained in rest and in different stages of stress. Since the acquisitions are performed minutes apart, variabilities may exist in the visualized cross-sections. To improve anatomical correspondence between rest and stress, aligning the images is essential. We developed a new intensity-based, sparse registration method to retrieve standard anatomical views from 3-D stress images that were equivalent to the manually selected views in the rest images. Using sparse image planes, the influence of common image artifacts could be reduced. We investigated different similarity measures and different levels of sparsity. The registration was tested using data of 20 patients and quantitatively evaluated based on manually defined anatomical landmarks. Alignment was best using sparse registration with two long-axis and two short-axis views; registration errors were reduced significantly, to the range of interobserver variabilities. In 91% of the cases, the registration result was qualitatively assessed as better than or equal to the manual alignment. In conclusion, sparse registration improves the alignment of rest and stress images, with a performance similar to manual alignment. This is an important step towards objective quantification in 3-D stress echocardiography.
---
paper_title: Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

paper_content:

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.
---
paper_title: Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices

paper_content:

Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D/3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fetal Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available.
---
paper_title: Temporal registration of partial data using particle filtering

paper_content:

We propose a particle filtering framework for rigid registration of a model image to a time-series of partially observed images. The method incorporates a model-based segmentation technique in order to track the pose dynamics of an underlying observed object with time. An applicable algorithm is derived by employing the proposed framework for registration of a 3D model of an anatomical structure, which was segmented from preoperative images, to consecutive axial 2D slices of a magnetic resonance imaging (MRI) scan, which are acquired intraoperatively over time. The process is fast and robust with respect to image noise and clutter, variations of illumination, and different imaging modalities.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: Implicit planar and in-plane deformable mapping in medical images through high order graphs

paper_content:

In this paper we propose a novel method based on discrete optimization of high order graphs, to perform deformable slice-to-volume registration of 2D images and 3D volumes. To this end, a 2D grid superimposed to the image is considered with control points deforming in 3D and their deformations corresponding to the label space. Geometrical consistency (unique plane selection) and deformation smoothness (in-plane deformations) as well as image similarity (visual matching) are encoded in different third order cliques. The proposed formulation is optimized through its mapping to a factor graph using conventional graph optimization methods. A dataset composed of 2D slices and 3D MRI volumes of the heart was used to evaluate its accuracy leading to very promising results.
---
paper_title: A Fast 3D Volume Reconstruction for Confocal Micro-rotation Cell Imaging

paper_content:

We propose a fast reconstruction algorithm for nonadherent living cell images from micro-rotational confocal microscopy. This procedure is called bi-protocol. Its key point consists in coupling micro-rotation data and conventional z-stack. Doing so, the estimation of the micro-rotation slice positions becomes a slice-to-volume registration allowing to reduce drastically the computing time. The validation of our procedure is performed on real data which gives significant improvement compared with the conventional z-stacking imaging.
---
paper_title: MRI of Moving Subjects Using Multislice Snapshot Images With Volume Reconstruction (SVR): Application to Fetal, Neonatal, and Adult Brain Studies

paper_content:

Motion degrades magnetic resonance (MR) images and prevents acquisition of self-consistent and high-quality volume images. A novel methodology, Snapshot magnetic resonance imaging (MRI) with Volume Reconstruction (SVR) has been developed for imaging moving subjects at high resolution and high signal-to-noise ratio (SNR). The method combines registered 2-D slices from sequential dynamic single-shot scans. The SVR approach requires that the anatomy in question is not changing shape or size and is moving at a rate that allows snapshot images to be acquired. After imaging the target volume repeatedly to guarantee sufficient sampling every where, a robust slice-to-volume registration method has been implemented that achieves alignment of each slice within 0.3 mm in the examples tested. Multilevel scattered interpolation has been used to obtain high-fidelity reconstruction with root-mean-square (rms) error that is less than the noise level in the images. The SVR method has been performed successfully for brain studies on subjects that cannot stay still, and in some cases were moving substantially during scanning. For example, awake neonates, deliberately moved adults and, especially, on fetuses, for which no conventional high-resolution 3-D method is currently available. Fine structure of the in-utero fetal brain is clearly revealed for the first time and substantial SNR improvement is realized by having many individually acquired slices contribute to each voxel in the reconstructed image.
---
paper_title: Particle filtering for slice-to-volume motion correction in EPI based functional MRI

paper_content:

Head movement during scanning introduces artificial signal changes and impedes activation detection in fMRI studies. The head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) sequence can be estimated and compensated by aligning the images onto a reference volume through image registration. Registering EPI images volume by volume fails to consider head motion between slices, leading to biased head motion estimates. Slice-to-volume registration is used to estimate motion parameters for each slice by more accurately representing the image acquisition sequence. However, it is prone to image noise and geometric distortion, resulting in high variance estimates. In this work, we propose a Gaussian particle filter based head motion tracking algorithm to reduce the image misregistration errors. The algorithm models head motion by using a dynamic state space model (SSM) to model continuous slice acquisition thereby providing more accurate motion estimates and voxel position estimates. We demonstrate significant performance improvement of the proposed approach as compared to previous registration-only methods of head motion estimation.
---
paper_title: Deformable Ultrasound Registration without Reconstruction

paper_content:

Ultrasound (US) imaging is often proposed as an interoperative imaging modality. This use nearly always requires that the collected data be registered to preoperative data of another modality. Existing intensity-based registration approaches all begin by reconstructing a 3D US volume from the collected 2D slices. We propose to directly register the set of 2D slices to the preoperative images. We argue this has a number of advantages, including the omission of the potentially complex reconstruction step, greater adaptability of the similarity measures, and easier parallelization. We describe a system for performing this task and present results on phantom data that show that our slice based method consistently outperforms a reconstruction based method in both speed and accuracy.
---
paper_title: Correction of misaligned slices in multi-slice MR cardiac examinations by using slice-to-volume registration

paper_content:

One of the main challenges with magnetic resonance (MR) cardiac image acquisition is to account for cardiac motion due to respiration. A popular technique to reduce respiratory motion is to perform a multi-slice acquisition in which a patient holds their breath multiple times during the scan. This paper explores the feasibility of using rigid slice-to-volume registration to correct for misalignments of slice stacks in such images due to differing breath-hold positions. The experimental results indicate that slice-to-volume registration is sufficiently accurate and robust to compensate for the typical misalignments expected. We show that correction of misalignments in such data results in anatomically more correct images, as well as improved left ventricular volume measurements. It also shows that by correcting for misalignments in short axis (SA) images, one can improve the interstudy reproducibility and hence reduce the number of samples needed for cardiac MR studies to show the same statistical significance.
---
paper_title: Target Motion Tracking in MRI-guided Transrectal Robotic Prostate Biopsy

paper_content:

Purpose: MRI-guided prostate needle biopsy requires compensation for organ motion between target planning and needle placement. Two questions are studied and answered in this paper: 1) is rigid registration sufficient in tracking the targets with an error smaller than the clinically significant size of prostate cancer and 2) what is the effect of the number of intraoperative slices on registration accuracy and speed? Methods: we propose multislice-to-volume registration algorithms for tracking the biopsy targets within the prostate. Three orthogonal plus additional transverse intraoperative slices are acquired in the approximate center of the prostate and registered with a high-resolution target planning volume. Both rigid and deformable scenarios were implemented. Both simulated and clinical MRI-guided robotic prostate biopsy data were used to assess tracking accuracy. Results: average registration errors in clinical patient data were 2.6 mm for the rigid algorithm and 2.1 mm for the deformable algorithm. Conclusion: rigid tracking appears to be promising. Three tracking slices yield significantly high registration speed with an affordable error.
---
paper_title: Ultrasound-CT registration of vertebrae without reconstruction

paper_content:

Purpose ::: While robust and accurate, our previously developed volume-to-volume ultrasound-CT registration of vertebrae required that the 2D ultrasound slices be reconstructed into a 3D volume, a time-consuming step that increased the total registration time per vertebra. We have modified our registration technique to a slices-to-volume strategy to eliminate the ultrasound reconstruction step in order to make the total registration time more practical intraoperatively.
---
paper_title: Intersection based registration of slice stacks to form 3D images of the human fetal brain

paper_content:

Clinical fetal MR imaging of the brain commonly makes use of fast 2D acquisitions of multiple sets of approximately orthogonal 2D slices. We and others have previously proposed an iterative slice-to-volume registration process to recover a geometrically consistent 3D image. However, these approaches depend on a 3D volume reconstruction step during the slice alignment. This is both computationally expensive and makes the convergence of the registration process poorly defined. In this paper our key contribution is a new approach which considers the collective alignment of all slices directly, via shared structure in their intersections, rather than to an estimated 3D volume. We derive an analytical expression for the gradient of the collective similarity of the slices along their intersections, with respect to the 3D location and orientation of each 2D slice. We include examples of the approach applied to simulated data and clinically acquired fetal images.
---
paper_title: Bias Field Inconsistency Correction of Motion-Scattered Multislice MRI for Improved 3D Image Reconstruction

paper_content:

A common solution to clinical MR imaging in the presence of large anatomical motion is to use fast multislice 2D studies to reduce slice acquisition time and provide clinically usable slice data. Recently, techniques have been developed which retrospectively correct large scale 3D motion between individual slices allowing the formation of a geometrically correct 3D volume from the multiple slice stacks. One challenge, however, in the final reconstruction process is the possibility of varying intensity bias in the slice data, typically due to the motion of the anatomy relative to imaging coils. As a result, slices which cover the same region of anatomy at different times may exhibit different sensitivity. This bias field inconsistency can induce artifacts in the final 3D reconstruction that can impact both clinical interpretation of key tissue boundaries and the automated analysis of the data. Here we describe a framework to estimate and correct the bias field inconsistency in each slice collectively across all motion corrupted image slices. Experiments using synthetic and clinical data show that the proposed method reduces intensity variability in tissues and improves the distinction between key tissue types.
---
paper_title: Non-iterative relative bias correction for 3D reconstruction of in utero fetal brain MR imaging

paper_content:

The slice intersection motion correction (SIMC) method is a powerful tool to compensate for motion that occurs during in utero acquisition of the multislice magnetic resonance (MR) images of the human fetal brain. The SIMC method makes use of the slice intersection intensity profiles of orthogonally planned slice pairs to simultaneously correct for the relative motion occurring between all the acquired slices. This approach is based on the assumption that the bias field is consistent between slices. However, for some clinical studies where there is a strong bias field combined with significant fetal motion relative to the coils, this assumption is broken and the resulting motion estimate and the reconstruction to a 3D volume can both contain errors. In this work, we propose a method to correct for the relative differences in bias field between all slice pairs. For this, we define the energy function as the mean square difference of the intersection profiles, that is then minimized with respect to the bias field parameters of the slices. A non iterative method which considers the relative bias between each slice simultaneously is used to efficiently remove inconsistencies. The method, when tested on synthetic simulations and actual clinical imaging studies where bias was an issue, brought a significant improvement to the final reconstructed image.
---
paper_title: Registration-based Approach for Reconstruction of High Resolution In Utero Fetal MR Brain Images

paper_content:

Rationale and Objectives This paper describes a novel approach to forming high-resolution MR images of the human fetal brain. It addresses the key problem of fetal motion by proposing a registration-refined compounding of multiple sets of orthogonal fast two-dimensional MRI slices, which are currently acquired for clinical studies, into a single high-resolution MRI volume. Materials and Methods A robust multiresolution slice alignment is applied iteratively to the data to correct motion of the fetus that occurs between two-dimensional acquisitions. This is combined with an intensity correction step and a super-resolution reconstruction step, to form a single high isotropic resolution volume of the fetal brain. Results Experimental validation on synthetic image data with known motion types and underlying anatomy, together with retrospective application to sets of clinical acquisitions, are included. Conclusion Results indicate that this method promises a unique route to acquiring high-resolution MRI of the fetal brain in vivo allowing comparable quality to that of neonatal MRI. Such data provide a highly valuable window into the process of normal and abnormal brain development, which is directly applicable in a clinical setting.
---
paper_title: A comparative study on manual and automatic slice-to-volume registration of CT images

paper_content:

In order to assess the clinical relevance of a slice-to-volume registration algorithm, this technique was compared to manual registration. Reformatted images obtained from a diagnostic CT examination of the lower abdomen were reviewed and manually registered by 41 individuals. The results were refined by the algorithm. Furthermore, a fully automatic registration of the single slices to the whole CT examination, without manual initialization, was also performed. The manual registration error for rotation and translation was found to be 2.7±2.8 ° and 4.0±2.5 mm. The automated registration algorithm significantly reduced the registration error to 1.6±2.6 ° and 1.3±1.6 mm (p = 0.01). In 3 of 41 (7.3%) registration cases, the automated registration algorithm failed completely. On average, the time required for manual registration was 213±197 s; automatic registration took 82±15 s. Registration was also performed without any human interaction. The resulting registration error of the algorithm without manual pre-registration was found to be 2.9±2.9 ° and 1.1±0.2 mm. Here, a registration took 91±6 s, on average. Overall, the automated registration algorithm improved the accuracy of manual registration by 59% in rotation and 325% in translation. The absolute values are well within a clinically relevant range.
---
paper_title: Ultrasound-to-computer-tomography registration for image-guided laparoscopic liver surgery

paper_content:

Background: The application of image-guided surgery (IGS) to laparoscopic liver resection and ablation is currently limited, but it would assist in intraoperative decision making regarding oncologic margins, ablation probe placement, and ablation tracking. Methods: Eight spherical surface targets on a liver phantom were imaged with an optically tracked laparoscopic ultrasound (US) probe. Ten US images of each target were registered to computer tomography (CT) images of the phantoms and then mapped to the CT scans. Accuracy of the registration was assessed by comparing the distance between the predicted target location and the position obtained directly from CT. Results: The average localization error was 5.3 mm. The errors resulted primarily from inaccurate US probe tracking but were otherwise insensitive to the variability that arises from manually identifying targets in US and CT images. Conclusions: The results obtained for US-to-CT registration in a phantom model suggest that further investigations into its clinical use are warranted and that other IGS technologies could be applied to laparoscopic liver surgery as well.
---
paper_title: Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images.

paper_content:

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.
---
paper_title: Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

paper_content:

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.
---
paper_title: Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

paper_content:

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.
---
paper_title: Towards real time 2D to 3D registration for ultrasound-guided endoscopic and laparoscopic procedures

paper_content:

Purpose ::: A method to register endoscopic and laparoscopic ultrasound (US) images in real time with pre-operative computed tomography (CT) data sets has been developed with the goal of improving diagnosis, biopsy guidance, and surgical interventions in the abdomen.
---
paper_title: Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

paper_content:

PURPOSE ::: Prostate needle biopsy is a commonly performed procedure since it is the most definitive form of cancer diagnosis. Magnetic resonance imaging (MRI) allows target-specific biopsies to be performed. However, needle placements are often inaccurate due to intra-operative prostate motion and the lack of motion compensation techniques. This paper detects and determines the extent of tissue displacement during an MRI-guided biopsy so that the needle insertion plan can be adjusted accordingly. ::: ::: ::: METHODS ::: A multi-slice-to-volume registration algorithm was developed to align the pre-operative planning image volume with three intra-operative orthogonal image slices of the prostate acquired immediately before needle insertion. The algorithm consists of an initial rigid transformation followed by a deformable step. ::: ::: ::: RESULTS ::: A total of 14 image sets from 10 patients were studied. Based on prostate contour alignment, the registrations were accurate to within 2 mm. ::: ::: ::: CONCLUSION ::: This algorithm can be used to increase the needle targeting accuracy by alerting the clinician if the biopsy target has moved significantly prior to needle insertion. The proposed method demonstrated feasibility of intra-operative target localization and motion compensation for MRI-guided prostate biopsy.
---
paper_title: A new registration/visualization paradigm for CT-Fluoroscopy guided RF liver ablation

paper_content:

2D-3D slice-to-volume registration for abdominal organs like liver is difficult due to the breathing motion and tissue deformation. The purpose of our approach is to ease CT-fluoroscopy (CT-fluoro) based needle insertion for the Radiofrequency Liver Ablation procedure using high resolution contrasted preoperative data. In this case, low signal-to-noise ratio, absence of contrast and additional presence of needle in CT-fluoro makes it difficult to guarantee the solution of any deformable slice-to-volume registration algorithm. In this paper, we first propose a method for creating a set of ground truth (GT) simulation data based on a non-linear deformation of the CT-fluoro volume obtained from real patients. Second, we split the CT-fluoro image and apply intensity based rigid and affine registration to each section. We then propose a novel solution, which consists of intuitive visualization sequences of optimal sub-volumes of preinterventional data based on the registration results. Experiments on synthetic and real patient data and direct feedback of two interventionalists validate our alternative approach.
---
paper_title: Adaptive visualization for needle guidance in RF liver ablation: taking organ deformation into account

paper_content:

Interventional procedures on deformable organs pose difficulties for the radiologists when inserting the probe ::: towards a lesion. The deformation due to the breathing makes a reliable and automated alignment of the ::: interventional 2D CT-Fluoro to the pre-interventional 3D CT-Volume very challenging. Such alignment is highly ::: desirable since, during the intervention, the CT-Volume brings more information as it is enhanced with contrast ::: agent and has a higher resolution than the CT-Fluoro slice. A reasonable solution for the alignment is obtained ::: by employing a robust optimization technique. However, since we would like to help the needle guidance towards ::: the lesion, due to the involved deformation, a single slice of the 3D CT-Volume is not satisfactory. ::: The main contribution of this paper consists in visualizing slices of the 3D CT-Volume that are resulting ::: from the out-of-plane motion parameters along weighted isosurfaces in the convergence basin of the similarity ::: function used during the alignment. This visualization copes with the uncertainty in estimating the deformation ::: and brings much more information than a single registered slice. Three experienced interventional radiologists ::: were consulted and their evaluation clearly highlighted that such visualization unfolding the neighborhood with ::: the belonging structures, like vessels and lesion spread, will help the needle guidance.
---
paper_title: A Review of Recent Advances in Registration Techniques Applied to Minimally Invasive Therapy

paper_content:

Minimally invasive and less invasive procedure is becoming more and more common in medical therapy. Image guidance is an indispensable component in minimally invasive procedures by providing critical information about the position of the target sites and the optimal manipulation of the devices, while the field of view is limited to naked eyes due to the small incision. Registration is one of the enabling technologies for computer-aided image guidance, which brings high-resolution pre-operative data into the operating room to provide more realistic information about the patient's anatomy. In this paper, we survey the recent advances in registration techniques applied to minimally and/or less invasive therapy, including a wide variety of therapies in surgery, endoscopy, interventional cardiology, interventional radiology, and hybrid procedures. The registration approaches are categorized into several groups, including projection-to-volume, slice-to-volume, video-to-volume, and volume-to-volume registration. The focus is on recent advances in registration techniques that are specifically developed for minimally and/or less invasive procedures in the following medical specialties: neuroradiology and neurosurgery, cardiac applications, and thoracic-abdominal interventions.
---
paper_title: Robust registration method for interventional MRI-guided thermal ablation of prostate cancer

paper_content:

We are investigating methods to register live-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained, high resolution MRI image volume. The immediate application is for iMRI-guided treatments of prostate cancer. We created and evaluated a slice-to-volume mutual information registration algorithm for MR images with special features to improve robustness. Features included a multi-resolution approach and automatic restarting to avoid local minima. We acquired 3D volume images from a 1.5 T MRI system and simulated iMRI images. To assess the quality of registration, we calculated 3D displacement on a voxel-by-voxel basis over a volume of interest between slice-to-volume registration and volume-to-volume registrations that were previously shown to be quite accurate. More than 500 registration experiments were performed on MR images of volunteers. The slice-to-volume registration algorithm was very robust for transverse slice images covering the prostate. A 100% success rate was achieved with an acceptance criterion of < 1.0 mm displacement error over the prostate. Our automatic slice-to-volume mutual information registration algorithm is robust and probably sufficiently accurate to aid in the application of iMRI-guided thermal ablation of prostate cancer.
---
paper_title: Automatic 3D registration for interventional MRI-guided treatment of prostate cancer.

paper_content:

The goal of this research is to register real-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained high-resolution MRI image volume, which in turn can be registered with functional images such as those Urom SPECT. The immediate application is in iMRI-guided treatment of prostate cancer, where additional images are desired to improve tumor targeting. In this article, simulation experiments are performed to demonstrate the feasibility of slice-to-volume registration for this application. We acquired 3D volume images from a 1.5-T MRI system and simulated low-field iMRI image slices by creating thick slices and adding noise. We created a slice-to-volume mutual information registration algorithm with special features to improve robustness. Features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. To assess the quality of registration, we calculated 3D displacements on a voxel-by-voxel basis over a volume of in...
---
paper_title: Motion Correction in fMRI by Mapping Slice-to-Volume with Concurrent Field-Inhomogeneity Correction

paper_content:

Head motion is the major source of error in measuring intensity changes related to given stimuli in fMRI. The effects of head motion are image shifts and field inhomogeneity variations which cause local changes in geometric distortions. The previously developed motion correction method, mapping slice-to-volume (MSV), retrospectively remaps slices that are shifted by head motion to their spatially correct locations in an anatomical reference. Images exhibiting spatially varying geometric distortions require non-linear mapping solutions. An accurate field map can be used for the correction of such spatial distortions. However, field-map changes with head motion and, in practice, only one field-map is available typically. This work evaluates the improved motion correction capability of MSV with concurrent iterative field-corrected reconstruction using only an initial field-map. The results from simulated motion data show effective convergence and accuracy in image registration for the correction of image artifacts complicated by the motion induced field effects.
---
paper_title: Diffusion tensor imaging (DTI) of the brain in moving subjects: application to in-utero fetal and ex-utero studies.

paper_content:

We present a methodology to achieve 3D high-resolution diffusion tensor image reconstruction of the brain in moving subjects. The source data is diffusion-sensitized single-shot echo-planar images. After continuous scanning to acquire a repeated series of parallel slices with 15 diffusion directions, image registration is used to realign the images to correct for subject motion. Once aligned, the diffusion images are treated as irregularly-sampled data where each voxel is associated with an appropriately rotated diffusion direction. This data is used to estimate the diffusion tensor on a regular grid. The method has been tested on data acquired at 1.5T from adults who deliberately moved and from eight fetuses imaged in utero. Maps of apparent diffusion coefficient (ADC) were reliably produced in all cases and promising performance was achieved for fractional anisotropy maps. Results from normal fetal brains were found to be consistent with published data from premature infants of similar gestational age.
---
paper_title: Cascaded slice to volume registration for moving fetal FMRI

paper_content:

Motion correction of MRI sequences is a very active area of research. Several postprocessing techniques for volume correction and more recently slice correction have been proposed. Slice motion correction of fMRI data typically involves iterative registration of slices to a target volume. The target volume is usually reconstructed at each iteration using current slice motion estimates, with all possible views of the subject. However, in the presence of large movements, the quality of the reconstruction can be greatly degraded without a good initialization, and this can in turn greatly affect the accuracy and precision of the slice registration. This paper introduces a cascaded motion correction pipeline that uses subsets of data to generate coherent volumes for initial target volume based slice registration in fMRI. The pipeline incorporates distortion correction, frame motion estimation and two levels of slice to volume registration. The proposed method has been tested on moving adult and fetal fMRI data.
---
paper_title: Correction of misaligned slices in multi-slice cardiovascular magnetic resonance using slice-to-volume registration

paper_content:

A popular technique to reduce respiratory motion for cardiovascular magnetic resonance is to perform a multi-slice acquisition in which a patient holds their breath multiple times during the scan. The feasibility of rigid slice-to-volume registration to correct for misalignments of slice stacks in such images due to differing breath-hold positions is explored. Experimental results indicate that slice-to-volume registration can compensate for the typical misalignments expected. Correction of slice misalignment results in anatomically more correct images, as well as improved left ventricular volume measurements. The interstudy reproducibility has also been improved reducing the number of samples needed for cardiac MR studies.
---
paper_title: Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI

paper_content:

Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acquired by these techniques, inter-slice motion leads to severe motion artifacts that are apparent in out-of-plane views. Slice sequential acquisitions do not enable 3-D volume representation. In this study, we have developed a novel technique based on a slice acquisition model, which enables the reconstruction of a volumetric image from multiple-scan slice acquisitions. The super-resolution volume reconstruction is formulated as an inverse problem of finding the underlying structure generating the acquired slices. We have developed a robust M-estimation solution which minimizes a robust error norm function between the model-generated slices and the acquired slices. The accuracy and robustness of this novel technique has been quantitatively assessed through simulations with digital brain phantom images as well as high-resolution newborn images. We also report here successful application of our new technique for the reconstruction of volumetric fetal brain MRI from clinically acquired data.
---
paper_title: Concurrent correction of geometric distortion and motion using the map-slice-to-volume method in echo-planar imaging.

paper_content:

The accuracy of measuring voxel intensity changes between stimulus and rest images in fMRI echo-planar imaging (EPI) data is severely degraded in the presence of head motion. In addition, EPI is sensitive to susceptibility-induced geometric distortions. Head motion causes image shifts and associated field map changes that induce different geometric distortion at different time points. Conventionally, geometric distortion is "corrected" with a static field map independently of image registration. That approach ignores all field map changes induced by head motion. This work evaluates the improved motion correction capability of mapping slice to volume with concurrent iterative field corrected reconstruction using updated field maps derived from an initial static field map that has been spatially transformed and resampled. It accounts for motion-induced field map changes for translational and in-plane rotation motion. The results from simulated EPI time series data, in which motion, image intensity and activation ground truths are available, show improved accuracy in image registration, field corrected image reconstruction and activation detection.
---
paper_title: Improved Motion Correction in fMRI by Joint Mapping of Slices into an Anatomical Volume

paper_content:

Motion correction in fMRI time series is essential for accurate statistical analyses. Typically motion correction is applied in a rigid fashion between time series volumes. Such corrections assume no relative motion between slices. An improved motion correction scheme, map-slice-to-volume (MSV), was developed previously using mutual information (MI) to register individual fMRI slices onto an anatomical volume to account for inter-slice motion [6]. As each slice’s orientation represents a statistically independent sampling of the patient’s motion at multiple intervals throughout the acquisition, a smoothed estimate of the patient’s trajectory in time can be computed by jointly estimating each slice’s orientation while minimizing the implied acceleration of the patient’s head subject to applying a priori, slice-related weights based on known registration reliabilities. The results of this joint mapping of slices into a volume (JMSV) show further substantial improvement in slice registration and subsequent motion estimation.
---
paper_title: Improved map-slice-to-volume motion correction with b0 inhomogeneity correction: validation of activation detection algorithms using ROC curve analyses

paper_content:

Head motion is a significant source of error in fMRI activation detection and a common approach is to apply 3D volumetric rigid body motion correction techniques. However, in 2D multislice fMRI, each slice may have a distinct set of motion parameters due to inter-slice motion. Here, we apply an automated mutual information based slice-to-volume rigid body registration technique on time series data synthesized from a T2 MRI brain dataset with simulated motion, functional activation, noise and geometric distortion. The map-slice-to-volume (MSV) technique was previously applied to patient data without ground truths for motion and activation regions. In this study, the activation images and area under the receiver operating characteristic curves for various time series datasets indicate that the MSV registration improves the activation detection capability when compared to results obtained from Statistical Parametric Mapping (SPM). The effect of temporal median filtering of motion parameters on activation detection performance was also investigated.
---
paper_title: Comprehensive mathematical simulation of functional magnetic resonance imaging time series including motion-related image distortion and spin saturation effect

paper_content:

There has been vast interest in determining the feasibility of functional magnetic resonance imaging (fMRI) as an accurate method of imaging brain function for patient evaluations. The assessment of fMRI as an accurate tool for activation localization largely depends on the software used to process the time series data. The performance evaluation of different analysis tools is not reliable unless truths in motion and activation are known. Lack of valid truths has been the limiting factor for comparisons of different algorithms. Until now, currently available phantom data do not include comprehensive accounts of head motion. While most fMRI studies assume no interslice motion during the time series acquisition in fMRI data acquired using a multislice and single-shot echo-planar imaging sequence, each slice is subject to a different set of motion parameters. In this study, in addition to known three-dimensional motion parameters applied to each slice, included in the time series computation are geometric distortion from field inhomogeneity and spin saturation effect as a result of out-of-plane head motion. We investigated the effect of these head motion-related artifacts and present a validation of the mapping slice-to-volume (MSV) algorithm for motion correction and activation detection against the known truths. MSV was evaluated, and showed better performance in comparison with other widely used fMRI data processing software, which corrects for head motion with a volume-to-volume realignment method. Furthermore, improvement in signal detection was observed with the implementation of the geometric distortion correction and spin saturation effect compensation features in MSV.
---
paper_title: Automatic 3-D Breath-Hold Related Motion Correction of Dynamic Multislice MRI

paper_content:

Magnetic resonance (MR) cine images are often used to clinically assess left ventricular cardiac function. In a typical study, multiple 2-D long axis (LA) and short axis (SA) cine images are acquired, each in a different breath-hold. Differences in lung volume during breath-hold and overall patient motion distort spatial alignment of the images thus complicating spatial integration of all image data in three dimensions. We present a fully automatic postprocessing approach to correct these slice misalignments. The approach is based on the constrained optimization of the intensity similarity of intersecting image lines after the automatic definition of a region of interest. It uses all views and all time frames simultaneously. Our method models both in-plane and out-of-plane translations and full 3-D rotations, can be applied retrospectively and does not require a cardiac wall segmentation. The method was validated on both healthy volunteer and patient data with simulated misalignments, as well as on clinical multibreath-hold patient data. For the simulated data, subpixel accuracy could be obtained using translational correction. The possibilities and limitations of rotational correction were investigated and discussed. For the clinical multibreath-hold patient data sets, the median discrepancy between manual SA and LA contours was reduced from 2.83 to 1.33 mm using the proposed correction method. We have also shown the usefulness of the correction method for functional analysis on clinical image data. The same clinical multibreath-hold data sets were resegmented after positional correction, taking newly available complementary information of intersecting slices into account, further reducing the median discrepancy to 0.43 mm. This is due to the integration of the 2-D slice information into 3-D space.
---
paper_title: Motion-robust MRI through real-time motion tracking and retrospective super-resolution volume reconstruction

paper_content:

Magnetic Resonance Imaging (MRI) is highly sensitive to motion; hence current practice is based on the prevention of motion during scan. In newborns, young children, and patients with limited cooperation, this commonly requires full sedation or general anesthesia, which is time consuming, costly, and is associated with significant risks. Despite progress in prospective motion correction in MRI, the use of motion compensation techniques is limited by the type and amount of motion that can be compensated for, the dependency on the scanner platform, the need for pulse sequence modifications, and/or difficult setup. In this paper we introduce a novel platform-independent motion-robust MRI technique based on prospective real-time motion tracking through a miniature magnetic field sensor and retrospective super-resolution volume reconstruction. The technique is based on fast 2D scans that maintain high-quality of slices in the presence of motion but are degraded in 3D due to inter-slice motion artifacts. The sensor, conveniently attached to the subject forehead, provides real-time estimation of the motion, which in turn gives the relative location of the slice acquisitions. These location parameters are used to compensate the inter-slice motion to reconstruct an isotropic high-resolution volumetric image from slices in a super-resolution reconstruction framework. The quantitative results obtained for phantom and volunteer subject experiments in this study show the efficacy of the developed technique, which is particularly useful for motion-robust high-resolution T2-weighted imaging of newborns and pediatric subjects.
---
paper_title: Registration-based Approach for Reconstruction of High Resolution In Utero Fetal MR Brain Images

paper_content:

Rationale and Objectives This paper describes a novel approach to forming high-resolution MR images of the human fetal brain. It addresses the key problem of fetal motion by proposing a registration-refined compounding of multiple sets of orthogonal fast two-dimensional MRI slices, which are currently acquired for clinical studies, into a single high-resolution MRI volume. Materials and Methods A robust multiresolution slice alignment is applied iteratively to the data to correct motion of the fetus that occurs between two-dimensional acquisitions. This is combined with an intensity correction step and a super-resolution reconstruction step, to form a single high isotropic resolution volume of the fetal brain. Results Experimental validation on synthetic image data with known motion types and underlying anatomy, together with retrospective application to sets of clinical acquisitions, are included. Conclusion Results indicate that this method promises a unique route to acquiring high-resolution MRI of the fetal brain in vivo allowing comparable quality to that of neonatal MRI. Such data provide a highly valuable window into the process of normal and abnormal brain development, which is directly applicable in a clinical setting.
---
paper_title: Correction of local deformations in fMRI by 3D non-linear warping in map-slice-to-volume approach

paper_content:

Zntroduction Recently, we have demonstrilted the capability of retrospectively mapping a slice image into rin anatomical reference volume (i.e. map-slice-to-volume, MSV) from the same subject using a simple rigid body transformation of $x degrees of freedom (DOF) [l]. In practice, deformation in a sliQe image is often presented as a nonlinear warping problem that his not limited to the imaging plane. Out-of-plane warp is typically encountered in multislice fMRI data due to the local field induced deformations or localized outof-plane motion artifacts. This study presents correction of the 3D warping artifacts localized in the regions that undergo local changes that result in spatial displacement of voxels typically observed in single shot ac@isitions [2]. The local warping solution is implemented follolwing the rigid body MSV transform that determines the patient’s~ rigid head motion causing in-plane and out-of-plane displacemend, Methods A slice image from time series EPI data was mapped into a high resolution reference yolume space, T1-weighted MRI volume data using a registration algorithm implementing either affine or a control pbint based thin-plate-spline (TPS) function. The automated registration is driven by the mutual information (Z) computed from the gray values of the image pair. By allowing the optimizer to move the control points in the homologok 3D space to positions that minimize the cost metric @&=-I), the iterative process determines the transformation coefficients for a coordinate mapping. For each iteration the affine or TPS warp transformation performs trilinear interpolation in which the original voxel grey values are used. This approach ensures that there is no undesired intensity change from interpolation effects such as round-off or smoothing across iterations. Results Retrospective image registration was performed to map a GREEPI slice image into a volumetric SPGR MRI data acquired from a 3T MRI system. The tota?ly automatic registration routine for each fMRI slice can be initiaipd using an initial three control point set, i.e., rigid body MSV, atid then can automatically increment DOF to multiple control point TPS warping transforms. The six DOF rotate/translate transfoF determines the spatial slice profile relative to the volumetric refqrence data at the time of acquisition. This process accommodates’ the patient’s head movement [l]. The following warping solution is initiated by using the optimized rigid body model to map the control points (> 4 pts) in the reference space into the homologous space. The TPS warping is subsequently computed, agati, by optimizing MI through iterative movement of the control points in homologous space. Improvement in registration using TPS warping solution is demonstrated in Fig. 1 as, compared to rigid-body in l(a). Deformatiotis in the center and the posterior edge are not removed by rigid body and the improved mapping with TPS solution is also indicated by lower MI index. Note that the final mapping of an acquired slice image into the-anatomical reference space is achieved through ?D warping, that corrects out-of-pime as well as in-plane local deformations besides the patient head motion.
---
paper_title: Fast Predictive Image Registration

paper_content:

We present a method to predict image deformations based on patch-wise image appearance. Specifically, we design a patch-based deep encoder-decoder network which learns the pixel/voxel-wise mapping between image appearance and registration parameters. Our approach can predict general deformation parameterizations, however, we focus on the large deformation diffeomorphic metric mapping (LDDMM) registration model. By predicting the LDDMM momentum-parameterization we retain the desirable theoretical properties of LDDMM, while reducing computation time by orders of magnitude: combined with patch pruning, we achieve a \(1500\mathsf {x}\)/\(66\mathsf {x}\) speed-up compared to GPU-based optimization for 2D/3D image registration. Our approach has better prediction accuracy than predicting deformation or velocity fields and results in diffeomorphic transformations. Additionally, we create a Bayesian probabilistic version of our network, which allows evaluation of deformation field uncertainty through Monte Carlo sampling using dropout at test time. We show that deformation uncertainty highlights areas of ambiguous deformations. We test our method on the OASIS brain image dataset in 2D and 3D.
---
paper_title: Fully convolutional networks for semantic segmentation

paper_content:

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet [20], the VGG net [31], and GoogLeNet [32]) into fully convolutional networks and transfer their learned representations by fine-tuning [3] to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes less than one fifth of a second for a typical image.
---
paper_title: Single slice US-MRI registration for neurosurgical MRI-guided US

paper_content:

Image-based ultrasound to magnetic resonance image (US-MRI) registration can be an invaluable tool in image-guided neuronavigation systems. State-of-the-art commercial and research systems utilize image-based registration to assist in functions such as brain-shift correction, image fusion, and probe calibration. Since traditional US-MRI registration techniques use reconstructed US volumes or a series of tracked US slices, the functionality of this approach can be compromised by the limitations of optical or magnetic tracking systems in the neurosurgical operating room. These drawbacks include ergonomic issues, line-of-sight/magnetic interference, and maintenance of the sterile field. For those seeking a US vendor-agnostic system, these issues are compounded with the challenge of instrumenting the probe without permanent modification and calibrating the probe face to the tracking tool. To address these challenges, this paper explores the feasibility of a real-time US-MRI volume registration in a small virtual craniotomy site using a single slice. We employ the Linear Correlation of Linear Combination (LC2) similarity metric in its patch-based form on data from MNI’s Brain Images for Tumour Evaluation (BITE) dataset as a PyCUDA enabled Python module in Slicer. By retaining the original orientation information, we are able to improve on the poses using this approach. To further assist the challenge of US-MRI registration, we also present the BOXLC2 metric which demonstrates a speed improvement to LC2, while retaining a similar accuracy in this context.
---
paper_title: Non-rigid 2D-3D Medical Image Registration using Markov Random Fields

paper_content:

The aim of this paper is to propose a novel mapping algorithm between 2D images and a 3D volume seeking simultaneously a linear plane transformation and an in-plane dense deformation. We adopt a metric free locally over-parametrized graphical model that combines linear and deformable parameters within a coupled formulation on a 5-dimensional space. Image similarity is encoded in singleton terms, while geometric linear consistency of the solution (common/single plane) and in-plane deformations smoothness are modeled in a pair-wise term. The robustness of the method and its promising results with respect to the state of the art demonstrate the extreme potential of this approach.
---
paper_title: A CNN Regression Approach for Real-Time 2D/3D Registration

paper_content:

In this paper, we present a Convolutional Neural Network (CNN) regression approach to address the two major limitations of existing intensity-based 2-D/3-D registration technology: 1) slow computation and 2) small capture range. Different from optimization-based methods, which iteratively optimize the transformation parameters over a scalar-valued metric function representing the quality of the registration, the proposed method exploits the information embedded in the appearances of the digitally reconstructed radiograph and X-ray images, and employs CNN regressors to directly estimate the transformation parameters. An automatic feature extraction step is introduced to calculate 3-D pose-indexed features that are sensitive to the variables to be regressed while robust to other factors. The CNN regressors are then trained for local zones and applied in a hierarchical manner to break down the complex regression task into multiple simpler sub-tasks that can be learned separately. Weight sharing is furthermore employed in the CNN regression model to reduce the memory footprint. The proposed approach has been quantitatively evaluated on 3 potential clinical applications, demonstrating its significant advantage in providing highly accurate real-time 2-D/3-D registration with a significantly enlarged capture range when compared to intensity-based methods.
---
paper_title: A Deep Metric for Multimodal Registration

paper_content:

Multimodal registration is a challenging problem in medical imaging due the high variability of tissue appearance under different imaging modalities. The crucial component here is the choice of the right similarity measure. We make a step towards a general learning-based solution that can be adapted to specific situations and present a metric based on a convolutional neural network. Our network can be trained from scratch even from a few aligned image pairs. The metric is validated on intersubject deformable registration on a dataset different from the one used for training, demonstrating good generalization. In this task, we outperform mutual information by a significant margin.
---
paper_title: The Pascal Visual Object Classes (VOC) Challenge

paper_content:

The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection.This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history of the challenge, and proposes directions for future improvement and extension.
---
paper_title: FlowNet: Learning Optical Flow with Convolutional Networks

paper_content:

Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks CNNs succeeded at. In this paper we construct CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations. Since existing ground truth data sets are not sufficiently large to train a CNN, we generate a large synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.
---
paper_title: Deep Neural Networks for Object Detection

paper_content:

Deep Neural Networks (DNNs) have recently shown outstanding performance on image classification tasks [14]. In this paper we go one step further and address the problem of object detection using DNNs, that is not only classifying but also precisely localizing objects of various classes. We present a simple and yet powerful formulation of object detection as a regression problem to object bounding box masks. We define a multi-scale inference procedure which is able to produce high-resolution object detections at a low cost by a few network applications. State-of-the-art performance of the approach is shown on Pascal VOC.
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Online database of clinical MR and ultrasound images of brain tumors.

paper_content:

PURPOSE ::: One of the important challenges in the field of medical imaging is finding real clinical images with which to validate new image processing algorithms. This is particularly true for tracked 3D ultrasound images of the brain. ::: ::: ::: METHODS ::: In 2010, pre- and postoperative magnetic resonance and intraoperative ultrasound images were acquired from brain tumor patients involved in the authors' imaging study at the Montreal Neurological Institute. ::: ::: ::: RESULTS ::: These data are available online at the Montreal Neurological Institute's Brain Images of Tumors for Evaluation database, termed here the MNI BITE database. It contains ultrasound and magnetic resonance images from 14 patients. Each patient underwent a preoperative and a postoperative T1-weighted magnetic resonance scan with gadolinium enhancement, and multiple intraoperative B-mode images were acquired before and after resection. Corresponding features were manually selected in some image pairs for validation. All images are in MINC format, the file format used at the authors' institute for image processing. The MINC tools are available for free download at packages.bic.mni.mcgill.ca. ::: ::: ::: CONCLUSIONS ::: This is the first online database of its kind. These images can be used by image processing scientists as well as clinicians wishing to compare findings from magnetic resonance and ultrasound imaging.
---

Title: Slice-to-Volume Medical Image Registration: A Survey

Section 1: Introduction
Description 1: Provide an overview of image registration, its importance in medical imaging, and the specific challenges and applications of slice-to-volume registration.

Section 2: Definition of Slice-to-Volume Registration
Description 2: Formally define slice-to-volume registration, including the objective function and distinctions between rigid and non-rigid registration.

Section 3: Matching Criterion
Description 3: Describe the different matching criteria used for evaluating the alignment between a 2D slice and a 3D volume, including iconic, geometric, sensor-based, and hybrid methods.

Section 4: Transformation Model
Description 4: Discuss the various transformation models used in slice-to-volume registration, including rigid, linear (affine), and deformable models.

Section 5: Optimization Method
Description 5: Detail the optimization methods used to infer the best transformation model, categorized into continuous, discrete, and miscellaneous approaches.

Section 6: Number of Slices 
Description 6: Explore the approaches for single slice-to-volume and multi slice-to-volume registration, and discuss the advantages and challenges associated with each.

Section 7: Applications
Description 7: Identify and describe the main clinical applications of slice-to-volume registration, categorized under image fusion and image-guided interventions, and motion correction and volume reconstruction.

Section 8: Discussion and Future Directions
Description 8: Summarize key findings from the survey, speculate on future research directions, including potential benefits of deep learning, and address the need for public benchmarks for validation.