Medical Image Fusion: A survey of the state of the art
---
paper_title: Information Fusion in the Immune System

paper_content:

Biologically-inspired methods such as evolutionary algorithms and neural networks are proving useful in the field of information fusion. Artificial Immune Systems (AISs) are a biologically-inspired approach which take inspiration from the biological immune system. Interestingly, recent research has show how AISs which use multi-level information sources as input data can be used to build effective algorithms for real time computer intrusion detection. This research is based on biological information fusion mechanisms used by the human immune system and as such might be of interest to the information fusion community. The aim of this paper is to present a summary of some of the biological information fusion mechanisms seen in the human immune system, and of how these mechanisms have been implemented as AISs
---
paper_title: Task-based scanpath assessment of multi-sensor video fusion in complex scenarios

paper_content:

The combining of visible light and infrared visual representations occurs naturally in some creatures, including the rattlesnake. This process, and the wide-spread use of multi-spectral multi-sensor systems, has influenced research into image fusion methods. Recent advances in image fusion techniques have necessitated the creation of novel ways of assessing fused images, which have previously focused on the use of subjective quality ratings combined with computational metric assessment. Previous work has shown the need to apply a task to the assessment process; the current work continues this approach by extending the novel use of scanpath analysis. In our experiments, participants were shown two video sequences, one in high luminance (HL) and one in low luminance (LL), both featuring a group of people walking around a clearing of trees. Each participant was shown visible and infrared (IR) inputs alone; and side-by-side (SBS); in an average (AVE) fused; a discrete wavelet transform (DWT) fused; and a dual-tree complex wavelet transform (DT-CWT) fused displays. Participants were asked to track one individual in each video sequence, as well as responding by key press when other individuals carried out secondary actions. Results showed the SBS display to lead to much poorer accuracy than the other displays, while reaction times in carrying out the secondary task favoured AVE in the HL sequence and DWT in the LL sequence. Results are discussed in relation to previous findings regarding item saliency and task demands, and the potential for comparative experiments evaluating human performance when viewing fused sequences against naturally occurring fusion processes such as the rattlesnake is highlighted.
---
paper_title: Motion extrapolation of auditory-visual targets

paper_content:

Many tasks involve the precise estimation of speed and position of moving objects, for instance to catch or avoid objects that cohabit in our environment. Many of these objects are characterised by signal representations in more than one modality, such as hearing and vision. The aim of this study was to investigate the extent to which the simultaneous presentation of auditory and visual signals enhances the estimation of motion speed and instantaneous position. Observers are asked to estimate the instant when a moving object arrives at a target spatial position by pressing a response button. This task requires observers to estimate the speed of the moving object and to calibrate the timing of their manual response such that it coincides with the true arrival time of the moving object. When both visual and auditory motion signals are available, the variability in estimating the arrival time of the moving object is significantly reduced compared to the variability in the unimodal conditions. This reduction in variability is consistent with optimal integration of the auditory and visual speed signals. The average bias in the estimated arrival times depends on the motion speed: for medium speeds (17deg/s) observers' subjective arrival times are earlier than the true arrival times; for high speeds (47deg/s) observers exhibit a (much smaller) bias in the other direction. This speed-dependency suggests that the bias is due to an error in estimating the motion speeds rather than an error in calibrating the timing of the motor response. Finally, in this temporal localization task, the bias and variability show similar patterns for motion defined by vision, audition or both.
---
paper_title: Assessing the role of attention in the audiovisual integration of speech

paper_content:

Currently, one of the most controversial topics in the study of multisensory integration in humans (and in its implementation in the development of new technologies for human communication systems) concerns the question of whether or not attention is needed during (or can modulate) the integration of sensory signals that are presented in different sensory modalities. Here, we review the evidence on this question, focusing specifically on the integration of auditory and visual information during the perception of speech. Contrary to the mainstream view that has been prevalent for the last 30years or so, recent studies have now started to reveal that attentional resources are, in fact, recruited during audiovisual multisensory integration, at least under certain conditions. Finally, considering all of the available evidence, we discuss the extent to which audiovisual speech perception should be considered to represent a 'special' case of audiovisual, and more generally, of multisensory integration.
---
paper_title: Applying capacity analyses to psychophysical evaluation of multisensory interactions

paper_content:

Determining when, if, and how information from separate sensory channels has been combined is a fundamental goal of research on multisensory processing in the brain. This can be a particular challenge in psychophysical data, as there is no direct recording of neural output. The most common way to characterize multisensory interactions in behavioral data is to compare responses to multisensory stimulation with the race model, a model of parallel, independent processing constructed from the probability of responses to the two unisensory stimuli which make up the multisensory stimulus. If observed multisensory reaction times are faster than those predicted by the model, it is inferred that information from the two channels is being combined rather than processed independently. Recently, behavioral research has been published employing capacity analyses where comparisons between two conditions are carried out at the level of the integrated hazard function. Capacity analyses seem to be particularly appealing technique for evaluating multisensory functioning, as they describe relationships between conditions across the entire distribution curve, are relatively easy and intuitive to interpret. The current paper presents capacity analysis of a behavioral data set previously analyzed using the race model. While applications of capacity analyses are still somewhat limited due to their novelty, it is hoped that this exploration of capacity and race model analyses will encourage the use of this promising new technique both in multisensory research and other applicable fields.
---
paper_title: A Special Issue on biologically inspired information fusion

paper_content:

An IGFET device isolation structure fabrication scheme includes the formation of electrically insulating isolation structures that extend into the substrate and extend above the surface of the substrate. The isolation structures are formed by providing a first mask to form trenches in the substrate. A layer of silicon dioxide is then deposited, filling the trenches and extending above the surface of the substrate. A second mask layer is formed. The second mask layer shadows the trench regions that were formed in the substrate. The silicon dioxide not shadowed by the second mask layer is removed, leaving isolation structures that extend both into the substrate and which rise above the substrate. A gate structure is formed in the region between two isolation structures, and, in the preferred embodiment, the gate structure extends above the substrate to the same height as the isolation structures. The isolation structures and the gate structure can be used to provide self-aligned doped source/drain regions. Spacers can be added to the isolation structure walls and the gate structure walls to provide heavily-doped self-aligned regions.
---
paper_title: Information Fusion for Anomaly Detection with the Dendritic Cell Algorithm

paper_content:

Dendritic cells are antigen presenting cells that provide a vital link between the innate and adaptive immune system, providing the initial detection of pathogenic invaders. Research into this family of cells has revealed that they perform information fusion which directs immune responses. We have derived a Dendritic Cell Algorithm based on the functionality of these cells, by modelling the biological signals and differentiation pathways to build a control mechanism for an artificial immune system. We present algorithmic details in addition to experimental results, when the algorithm was applied to anomaly detection for the detection of port scans. The results show the Dendritic Cell Algorithm is sucessful at detecting port scans.
---
paper_title: Fusion of Spatial Relationships for Guiding Recognition, Example of Brain Structure Recognition in 3D MRI

paper_content:

Spatial relations play an important role in recognition of structures embedded in a complex environment and for reasoning under imprecision. Several types of relationships can be modeled in a unified way using fuzzy mathematical morphology. Their combination benefits from the powerful framework of fuzzy set theory for fusion tasks and decision making. This paper presents several methods of fusion of information about spatial relationships and illustrates them on the example of model-based recognition of brain structures in 3D magnetic resonance imaging.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Evolutionary algorithm based automated medical image fusion technique: Comparative study with fuzzy fusion approach

paper_content:

Medical image fusion has been used to derive the useful information from multi modal medical images. The proposed methodology introduces evolutionary approaches for robust and automatic extraction of information from different modality images. This evolutionary fusion strategy implements multiresolution decomposition of the input images using wavelet transform. It is because, the analysis of input images at multiple resolutions able to extracts more fine details and improves the quality of the composite fused image. The proposed approach is also independent of any manual marking or knowledge of fiducial points and starts the fusion procedure automatically. The performance of the genetic based evolutionary algorithm is compared with fuzzy based fusion technique using mutual information as the similarity measuring metric. Experimental results show that genetic searching based fusion technique improves the quality of the fused images significantly over the fuzzy approaches.
---
paper_title: Medical Image Fusion Algorithm Based on Clustering Neural Network

paper_content:

This paper proposes a new image fusion algorithm based on clustering analysis for clinical image processing. According to the present image fusion algorithm, pixels of origin images are classified into clustering feature pixels and secondary pixels base on clustering analysis. Feature pixels have more useful medical information we need; secondary pixels have background information of the image. In the new algorithm, build different fusion rules on two types of pixels, rules of feature pixels base on partial gradient and rules of secondary pixels base on average gray. A great deal of experiments have done to testify feasibility of new algorithm, the result show fusion image has more information than origin images and improves the quality of the origin image, fusion image also protects characters of the image and heightens the visual impact, new algorithm is effective.
---
paper_title: PET/CT medical image fusion algorithm based on multiwavelet transform

paper_content:

PET/CT medical image fusion has important clinical significance. As the multiwavelet transform has several particular advantages in comparison with scalar wavelets on image processing, this paper proposes a medical image fusion algorithm based on multiwavelet transform after in-depth study of wavelet theory. The algorithm achieves PET/CT fusion with wavelet coefficients fusion method. Experimental results show that fusion image combines information of the source images, adds more details and texture information, and achieves a good fusion result. Based on the proposed algorithm, we can obtain the best result when using gradient fusion in the low-frequency part and classification fusion in the high-frequency part.
---
paper_title: Medical Image of PET/CT Weighted Fusion Based on Wavelet Transform

paper_content:

Image fusion as a kind of information-integrated technology has been used in many fields recently. For medical images, the medical machine of PET/CT makes the within-machine fusion come true. Although the fusion image not only has the character of CT that can make a precise location of pathological changes but also has the character of PET that can detect the pathological as early as possible. As the within-machine fusion equipments are very expensive, it leads a high fee. Only some large hospitals are equipped with the within-machine fusion equipment. So a large space has been left to the development of fusion differences. Based on the wavelet-transform, the fusion of the CT image and PET image comes true. In the end, we gain the fusion CT/PET image. We can detect and locate the pathological changes exactly and immediately in the fusion image. And actually, this is the predominance of the fusion image.
---
paper_title: A Medical Image Fusion Algorithm Based on Lifting Wavelet Transform

paper_content:

Image fusion is a technigue that integrates the information of multiple images into one image, which makes the fused image more accurate, more comprehensive and reliable. In order to improve the traditional medical image fusion algorithms which often cause details loss, the source medical images are decomposed by way of lifting wavelet, and the high-low frequency weight are utilized according to different fusion rules. Finally the target image is obtained through inverse wavelet transform. The simulation result shows that the algorithm is feasible and effective.
---
paper_title: Redundancy Discrete Wavelet Transform and Contourlet Transform for Multimodality Medical Image Fusion with Quantitative Analysis

paper_content:

Image fusion is the process of combining relevant information from two or more images into a single fused image. The resulting image will be more informative than any of the input images. The fusion in medical images is necessary for efficient diseases diagnosis from multimodality, multidimensional and multiparameter type of images. This paper describes a multimodality medical image fusion system using different fusion techniques and the resultant is analysed with quantitative measures. Initially, the registered images from two different modalities such as CT (anatomical information) and MRI - T2, FLAIR (pathological information) are considered as input, since the diagnosis requires anatomical and pathological information. Then the fusion techniques namely Redundancy Discrete Wavelet Transform (RDWT) and Contour let Transform are applied. Further the fused image is analyzed with five types of quantitative metrics such as Standard Deviation (SD), Entropy (EN), Overall Cross Entropy (OCE), Ratio of Spatial Frequency Error (RSFE), and Power Signal to Noise Ratio (PSNR) for performance evaluation. From the experimental results we observed that RDWT method provides better information (quality) using EN metric and the Contour let Transform gives the difference in source to the fused image using OCE metric and also the fused image obtained from the proposed fusion techniques has more information than the source images are proved through all metrics.
---
paper_title: A Novel Approach to Fast Medical Image Fusion Based on Lifting Wavelet Transform

paper_content:

Medical image fusion as a new and promising research area is very useful in clinical diagnosis and treatment planning. In this paper, a new algorithm based on Lifting Wavelet Transform (LWT) to fuse multi-modality medical images is presented. The lifting scheme is a new idea of constructing wavelets and has several unique advantages in comparison with conventional convolution-based Wavelet Transform (WT). It allows for an in-place implementation of WT and reduces computation time and memory requirement greatly. So in this fusion algorithm, LWT is employed to decompose and reconstruct images to realize fast image fusion. Meanwhile, a new local feature-based fusion rule is put forward to improve fusion quality and extract all significant features from multi-source images. Extensive experiments including the fusion of registered medical CT/MRI, CT/SPECT, MRI/PET images are performed and mutual information is taken as a means of objective assessing fusion images. The experimental results show that the new approach is very effective and has a good performance to fuse medical images.
---
paper_title: Medical image fusion by wavelet transform modulus maxima

paper_content:

Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.
---
paper_title: Medical image fusion method based on wavelet multi-resolution and entropy

paper_content:

After studying the characteristics of DWT and reflecting the statistical features by information entropy, we propose a medical image fusion method on the basis of wavelet entropy decomposition transform. Firstly, we use wavelet transform to decompose the multi-resolution signal from each image. Then we adapt different rules to fuse sub-images with different bandwidths at each resolution level. Finally we could gain fused image through the inverse transform. The method by calculating wavelet entropy can discover slight changes of signal, keep details of tiny regions in the procedure effectively. It also provides favorable evidence for doctors to detect pathological changes in early stage. The experimental results have proved that it is an efficient way to make fused image more adaptable to human vision.
---
paper_title: Multimodal Medical Image Fusion Using Redundant Discrete Wavelet Transform

paper_content:

Medical image fusion has revolutionized medical analysis by improving the precision and performance of computer assisted diagnosis. In this research, a fusion algorithm is proposed to combine pairs of multispectral magnetic resonance imaging such as T1, T2 and Proton Density brain images. The proposed algorithm utilizes different features of Redundant Discrete Wavelet Transform, mutual information based non-linear registration and entropy information to improve performance. Experiments on the BrainWeb database show that the proposed fusion algorithm preserves both edge and component information, and provides improved performance compared to existing Discrete Wavelet Transform based fusion algorithms.
---
paper_title: Multimodal medical image fusion based on Integer Wavelet Transform and Neuro-Fuzzy

paper_content:

Medical image fusion is used to derive useful information from multimodality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide precise information to the doctor and clinical treatment planning system. This paper proposes image fusion based on Integer Wavelet Transform (IWT) and Neuro-Fuzzy. The anatomical and functional images are decomposed using Integer Wavelet Transform. The wavelet coefficients are then fused using neuro-fuzzy algorithm. Then Inverse Integer Wavelet Transform (IIWT) is applied to the fused coefficients to get the fused Image. The performance of this algorithm is compared with image fusion based on Discrete Wavelet Transform (DWT) and neuro-fuzzy using entropy metric. Fusion Symmetry (FS) which quantifies the relative distance in terms of mutual information of the fused image with respect to input images is measured. Fusion Factor (FF) the criterion of maximizing the joint mutual information is also quantified.
---
paper_title: Wavelet Based Medical Image Fusion Using Filter Masks

paper_content:

This paper deals with convolution based image fusion using filter masks and reviews the performance of each with respect to qualitative and quantitative strategies. Fusion is performed using discrete wavelet transformation at two levels. The low and high frequency coefficients obtained are subjected to separate fusion rules. The low frequency approximation coefficients are selected based on a pixel selection rule while high frequency details are selected by convolution using averaging, gaussian, unsharp, prewitt and sobel filter masks of varying sizes. The performance evaluation in each case is conducted using objective strategies like RMSE and PSNR and results are graphically interpreted. Thus a comprehensive analysis is conducted to ensure the best fit mask for medical diagnosis and treatment applications.
---
paper_title: Multimodal Medical Image Fusion through a New DWT Based Technique

paper_content:

Medical image fusion can help physicians to extract the features that may not be normally visible in images by different modalities. In this paper, a novel discrete wavelet transform (DWT) based technique for medical image fusion is presented. Firstly, the medical images to be fused are decomposed by the DWT. Secondly, by considering the characteristics of human visual system (HVS) and the physical meaning of the wavelet coefficients, new different fusion schemes are performed on low frequency and high frequency bands separately followed by a window-based consistency verification. Finally, the fused image is constructed by the inverse DWT (IDWT) with all the combined coefficients. Experimental results on both simulated and real medical images can indicate that the proposed method is effective and provides a significant improvement over several conventional fusion methods.
---
paper_title: A multimodality medical image fusion algorithm based on wavelet transform

paper_content:

According to the characteristics of a medical image, this paper presents a multimodality medical image fusion algorithm based on wavelet transform. For the low-frequency coefficients of the medical image, the fusion algorithm adopts the fusion rule of pixel absolute value maximization; for the high-frequency coefficients, the fusion algorithm uses the fusion rule that combines the regional information entropy contrast degree selection with the weighted averaging method. Then the fusion algorithm obtains the fused medical image with inverse wavelet transform. We select two groups of CT/MRI images and PET/ MRI images to simulate our fusion algorithm and compare its simulation results with the commonly-used wavelet transform fusion algorithm. The simulation results show that our fusion algorithm cannot only preserve more information on a source medical image but also greatly enhance the characteristic and brightness information of a fused medical image, thus being an effective and feasible medical image fusion algorithm.
---
paper_title: The Application of Wavelet Transform to Multi-modality Medical Image Fusion

paper_content:

Medical image fusion has been used to derive useful information from multi-modality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide more information to the doctor and clinical treatment planning system. This paper aims to demonstrate the application of wavelet transformation to multi-modality medical image fusion. This work covers the selection of wavelet function, the use of wavelet based fusion algorithms on medical image fusion of CT and MRI, and the fusion image quality evaluation. We introduce the peak-to-peak signal-to-noise ratio (PSNR) method for measuring fusion effect. The performances of other two methods of image fusion based on pyramid-decomposition and simple image fusion attempts are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme based on wavelet transform
---
paper_title: Medical Image Fusion Based on Wavelet Packet Transform and Self-Adaptive Operator

paper_content:

Combining wavelet packet transform with self- adaptive operator together, a medical image fusion algorithm is put forward. In this algorithm, the medical imagines that have been matched are firstly decomposed using wavelet packet transform, and the wavelet coefficients as well as the sub-images decomposed are processed with self-adaptive operators. Then the fusion imagines are gained by the wavelet packet reconstruction for the decomposed imagines. The simulation results show that this algorithm is feasible and effective for the clinical applications of medical fusion imagines.
---
paper_title: Medical Image Fusion Based on Wavelet Transform and Independent Component Analysis

paper_content:

By analyzing the characters of CT medical image, this paper proposes a novel method for this particular image fusion, which is using discrete wavelet transform and independent component analysis. Firstly, each of CT images were decomposed by 2-D discrete wavelet transform. Then independent component analysis were used to analyse the wavelet coefficients in different level for acquiring independent component. At last, we use wavelet reconstruction to synthesize one CT medical image which could contain more integrated accurate detail information of different soft tissue such as muscles and blood vessels. By contrast, the efficiency of our method is better than weighted average, laplacian pyramid method in medical image fusion field.
---
paper_title: Rapid Multimodal Medical Image Registration and Fusion in 3D Conformal Radiotherapy Treatment Planning

paper_content:

In order to realize effectively and efficiently the automatic registration and fusion of multimodal medical images data in 3D conformal radiotherapy treatment planning (3D CRTP), a rapid image registration and fusion method is proposed in this paper. This proposed registration method is based on hierarchical adaptive free-form deformation(FFD) algorithm, which can be described as follows: First the ROI(region of interest) is extracted by using C-V level sets algorithm, and feature points are matched automatically which is based on parallel computing method. Then, the global rough registration is carried out by employing principal axes algorithm. Next, the automatic fine registration of the multimodal medical images is realized by a FFD method based on hierarchical B-splines. Moreover, in order to speed up the calculation of the FFD coefficients, stochastic gradient descent method-Simultaneous Perturbation(SP) and the criteria of maximum mutual information entropy are adopted. After the registration of multimodal images, their sequence images are fused by applying an image fusion method based on parallel computing and wavelet transform with the fusion rule of combining the local standard deviation and energy. This study demonstrates the superiority of the proposed method.
---
paper_title: Medical image fusion using the convolution of Meridian distributions

paper_content:

The aim of this paper is to introduce a novel non-Gaussian statistical model-based approach for medical image fusion based on the Meridian distribution. The paper also includes a new approach to estimate the parameters of generalized Cauchy distribution. The input images are first decomposed using the Dual-Tree Complex Wavelet Transform (DT-CWT) with the subband coefficients modelled as Meridian random variables. Then, the convolution of Meridian distributions is applied as a probabilistic prior to model the fused coefficients, and the weights used to combine the source images are optimised via Maximum Likelihood (ML) estimation. The superior performance of the proposed method is demonstrated using medical images.
---
paper_title: Medical image fusion based on discrete wavelet transform using Java technology

paper_content:

The importance of information offered by the medical images for diagnosis support can be increased by combining images from different compatible medical devices. The fusion process allows combination of salient feature of these images. In this paper we present different techniques of image fusion, our work for medical image fusion based on discrete wavelet transform and how we understand to integrate this process into a distributed application. The dedicated application considers Java technology for using its facilities as a future development, regarding a remote access mechanism.
---
paper_title: Medical image fusion with a shift-invariant morphological wavelet

paper_content:

This paper presents a novel algorithm for medical image fusion, using a shift-invariant multiscale decomposition scheme. The decomposition scheme is obtained by discarding the downsampling operators of a morphological wavelet. An experiment based on real medical images shows that the proposed method improves the quality of the fused image significantly.
---
paper_title: Medical Image Fusion by Multi-resolution Analysis of Wavelets Transform

paper_content:

A novel algorithm for the multimodalities medical images fusion based on wavelet transform has been proposed and implemented. The autoadaptive weighted coefficients have been calculated recursively to maximize the mutual information between the source image and the result image. Adopting multi-resolution analysis of wavelet transform, we achieved the MRI and CT image fusion. In addition, the new algorithm has been extended to MRI and color image fusion. The experiment results demonstrate that the new algorithm with wavelet transform have better fusion results compared with other mutual information fusion schemes without wavelet transform.
---
paper_title: Medical Image Fusion Based on an Improved Wavelet Coefficient Contrast

paper_content:

Most of previous image fusion methods aim at obtaining as many as information from the different modality images. The fusion criterion is to minimize different error between the fused image and the input images. With respect to the medical diagnosis, the edges and outlines of the interested objects is more important than other information. Therefore, how to preserve the edge-like features is worthy of investigating for medical image fusion. As we know, the image with higher contrast contain more edge-like features. In term of this view, we proposed a new medical image fusion scheme based on an improved wavelet coefficient contrast, which is defined as the ratio of the maximum of detail components to the local mean of the corresponding approximate component. The visual experiments and quantitative assessments demonstrate the effectiveness of this method compared to present image fusion schemes, especially for medical diagnosis.
---
paper_title: A Novel Wavelet Medical Image Fusion Method

paper_content:

This paper proposes a novel global energy merging scheme that is a region-based analysis approach. At first, multi-resolution wavelet decomposition on each source image is performed, and then the energy of the each 3*3 matrix region is calculated. The match measure can produce using wavelet decomposition coefficient and the energy. The three high frequency coefficients of the merging image produce through comparing match measure of several source images. And the low frequency coefficient lies on the choice of the three high frequency coefficient. Finally, by applying the inverse wavelet transform the final fused image is obtained. The composite images can preserve more relevant information about the edges. Experiments showed that the proposed wavelet image fusion method can have better performance than existing medical image fusion methods.
---
paper_title: Medical image pseudo coloring by wavelet fusion

paper_content:

Wavelet image fusion is proposed for medical image pseudo coloring. Wavelet efficiently separates an image into different subspaces that preserve the features of the original images after pseudo coloring. Feature classification information can be directly applied to the coarse approximation of the wavelet decomposed image without segmenting the original image. The proposed method improves the processing speed of the pseudo coloring algorithm and is robust in processing noisy images.
---
paper_title: Research on Medical Image Fusion Based on Orthogonal Wavelet Packets Transformation Combined with 2v-SVM

paper_content:

This paper presents a novel medical image fusion algorithm which imports 2v-SVM -an adaptive SVM learning algorithm to medical image fusion. We combine it with orthogonal wavelet packets to generate a new image fusion rule, which intelligently constructs the "good and bad features-classifier" for improving image fusion. Then we construct a new sort of linear weighted fusion arithmetic operator, which retains the remarkable features and eliminates the redundant information of the two images. We respectively fuse two pairs of images: CT> MRI images and MRI, PET images effectively and obtain good visual effects and quality of fused image. The experiments prove that the proposed algorithm resolve the deficiencies of existing algorithms in characteristics representation and acceptance or rejection of information and provide an advanced method to medical image fusion.
---
paper_title: Medical imaging fusion applications: An overview

paper_content:

Computer aided fusion of multi-modality medical images provides a very promising diagnostic tool with numerous clinical applications. The objective of this paper is to present an overview of medical imaging fusion techniques with an emphasis on the use of neural network algorithms. Case studies derived from oncology (data level fusion), microscopy and ultrasound imaging (feature level and decision level fusion), and lesion placement in pallidotomy (data level fusion) are presented. It is anticipated that these tools will help the physician towards a more realistic and quantitative, assessment of disease.
---
paper_title: Medical Image Fusion Algorithm Based on Clustering Neural Network

paper_content:

This paper proposes a new image fusion algorithm based on clustering analysis for clinical image processing. According to the present image fusion algorithm, pixels of origin images are classified into clustering feature pixels and secondary pixels base on clustering analysis. Feature pixels have more useful medical information we need; secondary pixels have background information of the image. In the new algorithm, build different fusion rules on two types of pixels, rules of feature pixels base on partial gradient and rules of secondary pixels base on average gray. A great deal of experiments have done to testify feasibility of new algorithm, the result show fusion image has more information than origin images and improves the quality of the origin image, fusion image also protects characters of the image and heightens the visual impact, new algorithm is effective.
---
paper_title: Neuro-fuzzy logic based fusion algorithm of medical images

paper_content:

CT, single photon emission computed tomography (SPECT) and nuclear magnetic resonance imaging (MRI) are complementary on reflecting human information. In order to provide more useful information for clinical diagnosis, we have a need to fuse the effective information. In the pixel-level fusion between the medical images, we presented a fusion algorithm based on neuro-fuzzy logic in this paper, and utilized hybrid algorithm which mixes BP algorithm with least mean square (LMS) algorithm to train the parameters of membership function. Employ the data of medical image CT, SPECT and MRI to achieve the fusion simulation, and compare with the simulation results of BP neural network on the basis of the evaluation standards which are the standard deviation and the information entropy. By the contrast and analysis, we got the following conclusions: the fused images based on neuro-fuzzy logic not only reserve more texture features, but also enhance the information characteristics of two original images.
---
paper_title: Multimodal medical image fusion using fuzzy radial basis function neural networks

paper_content:

Most medical images are blurry for two reasons: one is that noise signal blurs the high frequency signal of image edge, the other one is the border of tumor with normal tissues cannot be very well defined on the images, therefore it is difficult for radiology experts to delineate image. It is necessary that an approach based on fuzzy inference fusion is used for coping with these difficulties. The paper puts forward a new method for multimodal medical image fusion, that fuzzy radial basis function neural networks (fuzzy-RBFNN) is used to perform auto-adaptive image fusion. Global genetic algorithm (GA) is employed to train the networks. In the experiments, samples sets include artificial blurred medical image. Experimental results show, comparing with other image fusion method, the proposed approach is more excellent for multimodal medical images both in visual effect of fused images and in objective evaluation criteria, especially for blurry source images.
---
paper_title: Natural Computing Methods in Bioinformatics: A Survey

paper_content:

Often data analysis problems in Bioinformatics concern the fusion of multisensor outputs or the fusion of multisource information, where one must integrate different kinds of biological data. Natural computing provides several possibilities in Bioinformatics, especially by presenting interesting nature-inspired methodologies for handling such complex problems. In this article we survey the role of natural computing in the domains of protein structure prediction, microarray data analysis and gene regulatory network generation. We utilize the learning ability of neural networks for adapting, uncertainty handling capacity of fuzzy sets and rough sets for modeling ambiguity, and the search potential of genetic algorithms for efficiently traversing large search spaces.
---
paper_title: Improving Breast Cancer Risk Stratification Using Resonance-Frequency Electrical Impedance Spectroscopy Through Fusion of Multiple Classifiers

paper_content:

This study aims to improve breast cancer risk stratification. A seven-probe resonance-frequency-based electrical impedance spectroscopy (REIS) system was designed, assembled, and utilized to establish a data set of examinations from 174 women. Three classifiers, including artificial neural network (ANN), support vector machine (SVM), and Gaussian mixture model (GMM), were independently developed to predict the likelihood of each woman to be recommended for biopsy. The performances of these classifiers were compared, and seven fusion methods for integrating these classifiers were investigated. The results showed that among the three classifiers, the ANN yielded the highest performance with an area under the curve (AUC) of 0.81 for the receiver operating characteristic (ROC), while SVM and GMM achieved AUCs of 0.80 and 0.78, respectively. Improvements of up to 3% were obtained using fusion of the three classifiers, with the largest improvement obtained using either a "minimum score" rule or a "weighted sum" rule. Comparing different combinations of two out of the three classifiers, the weighted sum rule provided the most robust and consistent results, with AUCs of 0.81, 0.83, and 0.82 for the different combinations of ANN and SVM, ANN and GMM, and SVM and GMM, respectively. Furthermore, at 90% specificity, the ANN, the weighted sum- and min rule-based classifiers, all detected 67% of the verified cancer cases as compared with 50, 50, and 60% detection of the high risk cases, respectively. The study demonstrated that REIS examinations provide relevant information for developing breast cancer risk stratification tools and that using fusion of several not-fully-correlated classifiers can improve classification performance.
---
paper_title: Medical image fusion using m-PCNN

paper_content:

Medical image fusion plays an important role in clinical applications such as image-guided surgery, image-guided radiotherapy, non-invasive diagnosis, and treatment planning. Pulse coupled neural network (PCNN) is derived from the synchronous neuronal burst phenomena in the cat visual cortex. However, it is very difficult to directly apply original PCNN into the field of image fusion, because its model has some shortcomings. Although a significant amount of research work has been done in developing various medical image algorithms, one disadvantage with the approaches is that they cannot deal with different kinds of medical images. In this instance, we propose a novel multi-channel model -m-PCNN for the first time and apply it to medical image fusion. In the paper, firstly the mathematical model of m-PCNN is described, and then dual-channel model as a special case of m-PCNN is introduced in detail. In order to show that the m-PCNN can deal with multimodal medical images, we used four pairs of medical images with different modalities as our experimental subjects. At the same time, in comparison with other methods (Contrast pyramid, FSD pyramid, Gradient pyramid, Laplacian pyramid, etc.), the performance and relative importance of various methods is investigated using the Mutual Information criteria. Experimental results show our method outperforms other methods, in both visual effect and objective evaluation criteria.
---
paper_title: Adaptive medical image visualization based on hierarchical neural networks and intelligent decision fusion

paper_content:

An adaptive medical image visualization system based on a hierarchical neural network structure and intelligent decision fusion is presented. It consists of a feature generator using both histogram and spatial information computed from a medical image, a wavelet transform for compressing the feature vector, a competitive layer neural net for clustering images into different subclasses, a bi-modal linear estimator and a RBF network based nonlinear estimator for each subclass, as well as an intelligent decision fusion process to integrate estimates from both estimators. Both estimators can adapt to new types of medical images simply by training them with those images. The large training image set is hierarchically organized for efficient user interaction and effective re-mapping of the width/center settings in the training data. Adaptation capabilities are achieved by modifying the width/center values through a mapping function, which is estimated from the width/center settings of some representative images. While the RBF network based estimator performs well for images similar to those in the training set, the bi-modal linear estimator provides reasonable estimation for a wide range of images. The decision fusion step makes the final estimation of the display parameters accurate for trained images and robust for the unknown images. The algorithm has been tested on a wide range of MR images and shown satisfactory results. Although the current algorithm is very comprehensive, its execution time is kept within reasonable range.
---
paper_title: Breast cancer diagnosis using neural-based linear fusion strategies

paper_content:

Breast cancer is one of the leading causes of mortality among women, and the early diagnosis is of significant clinical importance. In this paper, we describe several linear fusion strategies, in particular the Majority Vote, Simple Average, Weighted Average, and Perceptron Average, which are used to combine a group of component multilayer perceptrons with optimal architecture for the classification of breast lesions. In our experiments, we utilize the criteria of mean squared error, absolute classification error, relative error ratio, and Receiver Operating Characteristic (ROC) curve to concretely evaluate and compare the performances of the four fusion strategies. The experimental results demonstrate that the Weighted Average and Perceptron Average strategies can achieve better diagnostic performance compared to the Majority Vote and Simple Average methods.
---
paper_title: Early tumor detection by multiple infrared unsupervised neural nets fusion

paper_content:

The unsupervised classification algorithm called Lagrange constraint neural network (LCNN) has been successfully applied to the sub-pixel multispectral remote sensing, [25]. Here, we apply the LCNN to the early breast cancer detection using two-color mid and long infrared images of the breast. This could be a new paradigm shift that enabled smart neural network algorithm to sort out the underlying malignant heat sources for physician diagnoses. The nonintrusive 2-color passive infrared imaging that could be repeated for record track with no radiation hazard seems to be alternative paradigm shift for the first-line screening against breast cancer. The sub-pixel super-resolution capability of the remote sensing is equivalent to the sub-milimeter scaling of the close-up breast imaging for the vascular and the angiogenesis effects. We demonstrate the potential benefit of the multicolor mid & long infrared imaging capable for detecting the abnormal under-skin thermal textures as well as stage-zero detection of the ductal carcinoma in situ.
---
paper_title: Support vector machine and generalized regression neural network based classification fusion models for cancer diagnosis

paper_content:

This paper presents decision-based fusion models to classify BRCA1, BRCA2 and Sporadic genetic mutations for breast and ovarian cancer. Different ensembles of base classifiers using the stacked generalization technique have been proposed including support vector machines (SVM) with linear, polynomial and radial base function kernels. A generalized regression neural network (GRNN) is then applied to predict the mutation type based on the outputs of base classifiers, and experimental results show that the new proposed fusion methodology for selecting the best and removing weak classifiers outperforms single classification models.
---
paper_title: A new algorithm of multi-modality medical image fusion based on pulse-coupled neural networks

paper_content:

In this paper, a new multi-modality medical image fusion algorithm based on pulse-coupled neural networks (PCNN) is presented. Firstly a multi-scale decomposition on each source image is performed, and then the PCNN is used to combine these decomposition coefficients. Finally an inverse multi-scale transform is taken upon the new fused coefficients to reconstruct fusion image. The new algorithm utilizes the global feature of source images because the PCNN has the global couple and pulse synchronization characteristics. Series of experiments are performed about multi-modality medical images fusion such as CT/MRI, CT/SPECT, MRI/PET, etc. The experimental results show that the new algorithm is very effective and provides a good performance in fusing multi-modality medical images.
---
paper_title: Evolutionary algorithm based automated medical image fusion technique: Comparative study with fuzzy fusion approach

paper_content:

Medical image fusion has been used to derive the useful information from multi modal medical images. The proposed methodology introduces evolutionary approaches for robust and automatic extraction of information from different modality images. This evolutionary fusion strategy implements multiresolution decomposition of the input images using wavelet transform. It is because, the analysis of input images at multiple resolutions able to extracts more fine details and improves the quality of the composite fused image. The proposed approach is also independent of any manual marking or knowledge of fiducial points and starts the fusion procedure automatically. The performance of the genetic based evolutionary algorithm is compared with fuzzy based fusion technique using mutual information as the similarity measuring metric. Experimental results show that genetic searching based fusion technique improves the quality of the fused images significantly over the fuzzy approaches.
---
paper_title: Fuzzy Information Fusion Scheme Used to Segment Brain Tumor from MR Images

paper_content:

A fuzzy information fusion scheme is proposed in this paper to automatically segment tumor areas of human brain from multispectral magnetic resonance images such as T1-weighted, T2-weighted and Proton Density (PD) feature images. The proposed scheme consists of four stages: data-level fusion, space creation of fuzzy features, fusion of fuzzy features and fuzzy decision. Several fuzzy operators are proposed to create the feature-level fusion. The fuzzy information models describing the characteristics of tumor areas in human brain are also established. A fuzzy region growing based on fuzzy connecting is presented to obtain the final segmentation result. The comparison between the result of our method and the hand-labeled segmentation of a radiology expert shows that this scheme is efficient. The experimental results (based on 4 patients studied) show an average probability of correct detection equal to 96% and an average probability of false detection equal to 5%.
---
paper_title: Neuro-fuzzy logic based fusion algorithm of medical images

paper_content:

CT, single photon emission computed tomography (SPECT) and nuclear magnetic resonance imaging (MRI) are complementary on reflecting human information. In order to provide more useful information for clinical diagnosis, we have a need to fuse the effective information. In the pixel-level fusion between the medical images, we presented a fusion algorithm based on neuro-fuzzy logic in this paper, and utilized hybrid algorithm which mixes BP algorithm with least mean square (LMS) algorithm to train the parameters of membership function. Employ the data of medical image CT, SPECT and MRI to achieve the fusion simulation, and compare with the simulation results of BP neural network on the basis of the evaluation standards which are the standard deviation and the information entropy. By the contrast and analysis, we got the following conclusions: the fused images based on neuro-fuzzy logic not only reserve more texture features, but also enhance the information characteristics of two original images.
---
paper_title: Automatic segmentation of subcortical brain structures in MR images using information fusion

paper_content:

Reports a new automated method for the segmentation of internal cerebral structures using an information fusion technique. The information is provided both by images and expert knowledge, and consists in morphological, topological, and tissue constitution data. All this ambiguous, complementary and redundant information is managed using a three-step fusion scheme based on fuzzy logic. The information is first modeled into a common theoretical frame managing its imprecision and incertitude. The models are then fused and a decision is taken in order to reduce the imprecision and to increase the certainty in the location of the structures. The whole process is illustrated on the segmentation of thalamus, putamen, and head of the caudate nucleus from expert knowledge and magnetic resonance images, in a protocol involving 14 healthy volunteers. The quantitative validation is achieved by comparing computed, manually segmented structures and published data by means of indexes assessing the accuracy of volume estimation and spatial location. Results suggest a consistent volume estimation with respect to the expert quantification and published data, and a high spatial similarity of the segmented and computed structures. This method is generic and applicable to any structure that can be defined by expert knowledge and morphological images.
---
paper_title: Multimodal medical image fusion using fuzzy radial basis function neural networks

paper_content:

Most medical images are blurry for two reasons: one is that noise signal blurs the high frequency signal of image edge, the other one is the border of tumor with normal tissues cannot be very well defined on the images, therefore it is difficult for radiology experts to delineate image. It is necessary that an approach based on fuzzy inference fusion is used for coping with these difficulties. The paper puts forward a new method for multimodal medical image fusion, that fuzzy radial basis function neural networks (fuzzy-RBFNN) is used to perform auto-adaptive image fusion. Global genetic algorithm (GA) is employed to train the networks. In the experiments, samples sets include artificial blurred medical image. Experimental results show, comparing with other image fusion method, the proposed approach is more excellent for multimodal medical images both in visual effect of fused images and in objective evaluation criteria, especially for blurry source images.
---
paper_title: IMPROVING MMI WITH ENHANCED-FCM FOR THE FUSION OF BRAIN MR AND SPECT IMAGES

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To overcome these drawbacks of using traditional MMI, a novel scheme, named improved MMI, which is based on fuzzy c-means (FCM) and MMI, is proposed. The experimental results, using MR and SPECT images, to confirm the superior performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: Linguistic recognition system for identification of some possible genes mediating the development of lung adenocarcinoma

paper_content:

In the present article, we develop a linguistic recognition system for identification of some possible genes mediating the development of human lung adenocarcinoma. The methodology involves dimensionality reduction, classifying the genes through incorporation of the notion of linguistic fuzzy sets low, medium and high, and finally selection of some possible genes obtained by a rule generation/grouping technique. The system has been successfully applied on two microarray gene expression data sets. The results are appropriately validated by some earlier investigations, gene expression profiles and t-test. The proposed methodology has been able to find more true positives than an existing one in identifying responsible genes. Moreover, we have found some new genes that may have role in mediating the development of lung adenocarcinoma.
---
paper_title: Fuzzy logic and related methods as a screening tool for detecting gene regulatory networks

paper_content:

The analysis of gene expression microarrays plays an important role in elucidating the functionality of genes, including the discovery of genetic interactions that regulate gene expression. Several methods for modeling such gene regulatory networks exist, including a variety of continuous and discrete models. Methods based on fuzzy logic provide an interesting alternative. However, the guidelines for modeling gene expression with fuzzy logic are fairly open, and the need arises to investigate how adjustments in the modeling scheme will affect the results. In this work, we modify an existing fuzzy logic algorithm to involve an arbitrary number of classification states, and investigate the limiting behavior as the number of states tends to infinity. We also propose a probabilistic model as an alternative to the fuzzy logic model. We investigate the behavior of both models using yeast cell-cycle data and the simulated data of Werhli et al. [A.V. Werhli, M. Grzegorczyk, D. Husmeier, Comparative evaluation of reverse engineering gene regulatory networks with relevance networks, graphical Gaussian models, and Bayesian networks, Bioinformatics 22 (2006) 2523-2531]. We found that altering the number of classification states in both the fuzzy logic and probability models can influence which networks are predicted by both models. As the number of states tends to infinity, the predictions made by both models converge to those of a regression model. Models with a small to moderate number of classification states produced better results from a biological standpoint, compared to models with higher numbers of states. In simulated data, models with differing numbers of classification states produced similar overall results. Thus, increasing the complexity of the models has no apparent benefit, and models with smaller numbers of classification states are therefore preferred based on their ease of linguistic interpretation. The software used in this paper is freely available for non-commercial use at http://louisville.edu/~g0broc01.
---
paper_title: Analytic Implementation for Medical Image Retrieval Based On FCM Using Feature Fusion With Relevance Feedback

paper_content:

A system for endoscopic image which supports image retrieval by content is described here. A new algorithm with multi-feature fusion, which referring to low-level features, is proposed based on the study of single feature including color clustering, texture information. Then, relevance feedback technique is combined with the feature fusion algorithm in order to improve retrieval results. A prototype system which supports query by example is implemented using VC# and SQL server 2000. Finally, the performance of the method is illustrated respectively using examples from an image database composed of 1361 endoscopic images, and the comparison of retrieval results shows that the approach proposed here is effective.
---
paper_title: An approach to sensor fusion in medical robots

paper_content:

This paper proposes a multi-sensor data fusion technique to determine the complex interactions between the sensory, muscular and mechanical components of the human locomotor systems (neuromechanics). We investigated the use of an array of accelerometers, rate gyroscopes, force plates and electromyogram for the assessment of x-y-z components of the hip, thigh, shank and foot acceleration and velocity in the sagittal plane. The objective here is to determine the impact factor each body segment's acceleration has in terms of producing a functional gait. Six able-bodied subjects were employed in this experiment; and subject walked at three different speeds and his/her dynamic/kinematic behaviors of the lower limb recorded with the sensors afore mentioned. Eight distinct gait variability for each subject recorded was detected by our sensing system. Fuzzy fusion technique was employed to evaluate the empirical results and the output matrix shows the relation between the body segments in question in terms of their x-y-z acceleration components. The implementation of this fusion matrix will enhance modeling and building medical robots intended for paraplegic rehabilitation as well as intelligent mobile robots for effective industrial manufacturing.
---
paper_title: Fuzzy rule base classifier fusion for protein mass spectra based ovarian cancer diagnosis

paper_content:

Fuzzy rule base classification systems have been the focus of increased attention in recent years, due to their unique capability of providing human experts with outcomes by means of linguistic rules. In the same time period classifier fusion approaches have been shown to enhance the performance of pattern recognition systems. In the present study we applied a hybrid random subspace fusion scheme that constructs a set of different fuzzy classifiers utilizing different subsets of both the feature space and the sample domain, combining the results of these classifiers using appropriate decision functions. Experimental results using two protein mass spectra datasets of ovarian cancer demonstrate the usefulness of this approach in comparison to other classifier fusion approaches.
---
paper_title: A data fusion approach to tumor delineation

paper_content:

An improved methodology is presented for the localization of tumors for radiation therapy treatment planning applications. The proposed approach focuses on the integration of MRI, CT, and PET imaging data towards the generation of a mathematically optimal tumor boundary. Fuzzy edges derived from CT and MRI are fused to form an integrated edge map which is employed in conjunction with a region based analysis of the PET scan to guide the evolution of a tumor segmenting model. The presented algorithm yields tumor contours which may be employed directly in the radiation therapy treatment planning process.
---
paper_title: Multimodal medical image fusion based on Integer Wavelet Transform and Neuro-Fuzzy

paper_content:

Medical image fusion is used to derive useful information from multimodality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide precise information to the doctor and clinical treatment planning system. This paper proposes image fusion based on Integer Wavelet Transform (IWT) and Neuro-Fuzzy. The anatomical and functional images are decomposed using Integer Wavelet Transform. The wavelet coefficients are then fused using neuro-fuzzy algorithm. Then Inverse Integer Wavelet Transform (IIWT) is applied to the fused coefficients to get the fused Image. The performance of this algorithm is compared with image fusion based on Discrete Wavelet Transform (DWT) and neuro-fuzzy using entropy metric. Fusion Symmetry (FS) which quantifies the relative distance in terms of mutual information of the fused image with respect to input images is measured. Fusion Factor (FF) the criterion of maximizing the joint mutual information is also quantified.
---
paper_title: Natural Computing Methods in Bioinformatics: A Survey

paper_content:

Often data analysis problems in Bioinformatics concern the fusion of multisensor outputs or the fusion of multisource information, where one must integrate different kinds of biological data. Natural computing provides several possibilities in Bioinformatics, especially by presenting interesting nature-inspired methodologies for handling such complex problems. In this article we survey the role of natural computing in the domains of protein structure prediction, microarray data analysis and gene regulatory network generation. We utilize the learning ability of neural networks for adapting, uncertainty handling capacity of fuzzy sets and rough sets for modeling ambiguity, and the search potential of genetic algorithms for efficiently traversing large search spaces.
---
paper_title: Data fusion and fuzzy spatial relationships for locating deep brain stimulation targets in magnetic resonance images

paper_content:

Symptoms of Parkinson's disease can be relieved through Deep Brain Stimulation. This neurosurgical technique relies on high precision positioning of electrodes in specific areas of the basal ganglia and the thalamus. In order to identify these anatomical targets, which are located deep within the brain, we developed a semi-automated method of image analysis, based on data fusion. Information provided by both anatomical magnetic resonance images and expert knowledge is managed in a common possibilistic frame, using a fuzzy logic approach. More specifically, a graph-based virtual atlas modeling theoretical anatomical knowledge is matched to the image data from each patient, through a research algorithm (or strategy) which simultaneously computes an estimation of the location of every structures, thus assisting the neurosurgeon in defining the optimal target. The method was tested on 10 images, with promising results. Location and segmentation results were statistically assessed, opening perspectives for enhancements.
---
paper_title: Fusion of Spatial Relationships for Guiding Recognition, Example of Brain Structure Recognition in 3D MRI

paper_content:

Spatial relations play an important role in recognition of structures embedded in a complex environment and for reasoning under imprecision. Several types of relationships can be modeled in a unified way using fuzzy mathematical morphology. Their combination benefits from the powerful framework of fuzzy set theory for fusion tasks and decision making. This paper presents several methods of fusion of information about spatial relationships and illustrates them on the example of model-based recognition of brain structures in 3D magnetic resonance imaging.
---
paper_title: Multimodality medical image registration and fusion techniques using mutual information and genetic algorithm-based approaches.

paper_content:

Medical image fusion has been used to derive the useful complimentary information from multimodal images. The prior step of fusion is registration or proper alignment of test images for accurate extraction of detail information. For this purpose, the images to be fused are geometrically aligned using mutual information (MI) as similarity measuring metric followed by genetic algorithm to maximize MI. The proposed fusion strategy incorporating multi-resolution approach extracts more fine details from the test images and improves the quality of composite fused image. The proposed fusion approach is independent of any manual marking or knowledge of fiducial points and starts the procedure automatically. The performance of proposed genetic-based fusion methodology is compared with fuzzy clustering algorithm-based fusion approach, and the experimental results show that genetic-based fusion technique improves the quality of the fused image significantly over the fuzzy approaches.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: A framework of fuzzy information fusion for the segmentation of brain tumor tissues on MR images

paper_content:

A framework of fuzzy information fusion is proposed in this paper to automatically segment tumor areas of human brain from multispectral magnetic resonance imaging (MRI) such as T1-weighted, T2-weighted and proton density (PD) images. A priori knowledge about tumors described by radiology experts for different types of MRI are very helpful to guide a automatic and a precise segmentation. However, the terminology used by radiology experts are variable in term of image signal. In order to benefit of these descriptions, we propose to modellize them by fuzzy models. One fuzzy model is built for one type of MRI sequence. The segmentation is finally based on a fusion of different fuzzy information obtained from different types of MRI images. Our algorithm consists of four stages: the registration of multispectral MR images, the creation of fuzzy models describing the characteristics of tumor, the fusion based on fuzzy fusion operators and the adjustment by fuzzy region growing based on fuzzy connecting. The comparison between the obtained results and the hand-tracings of a radiology expert shows that the proposed algorithm is efficient. An average probability of correct detection 96% and an average probability of false detection 5% are obtained through studies of four patients.
---
paper_title: Content Analysis Based Medical Images Fusion with Fuzzy Inference

paper_content:

A content analysis based medical images fusion with fuzzy inference is presented. For well design fuzzy inference rules, the contents of two source images are analyzed with histograms. In the calculation of histograms, section is determined by two source images. Inference rules are designed based on fusion target and histograms of CT image and MRI image. Experiment results comparison from subject and object point of view show that content analysis based medical images fusion with fuzzy inference method is better than ordinary 3 gray-level (dark, medium, bright) fuzzy inference fusion method, the fused image can provide more information.
---
paper_title: Fusion algorithm of medical images based on fuzzy logic

paper_content:

Computed tomography (CT) and nuclear magnetic resonance imaging (MRI) are complementary on reflecting human body information. In order to provide more useful information for clinical diagnosis, we have a need to fuse the effective information. In the pixel-level fusion between the medical images, we presented a Mamdani-type minimum-sum-mean of maximum (MIN-SUM-MOM) algorithm in this paper. In MIN-SUM-MOM algorithm, fuzzy implication operation utilized MIN algorithm; calculating the membership functions of total output fuzzy sets employed SUM algorithm; defuzzification operation used MOM algorithm. Employ the data of medical image CT and MRI to achieve the fusion simulation, and compare with the simulation results of minimum-maximum-Centroid (MIN-MAX-Centroid) algorithm on the basis of the evaluation standards which are the standard deviation and the information entropy. By the contrast and analysis, we got the following conclusions: the fused images based on MIN-SUM-MOM algorithm not only reserve more texture features, but also enhance the information characteristics of two original images.
---
paper_title: An Improved Approach Based on FCM Using Feature Fusion for Medical Image Retrieval

paper_content:

Herein we describe a system for Gastroscopic image which supports image retrieval by content. Attention is focused on content-based approaches for the efficient representation and retrieval of medical images. A new improved image retrieval method with more robustness based on image entropy is proposed, which can be defined as spatial weighted entropy. Then a feature fusion algorithm of medical image retrieval based on FCM using spatial weighted entropy combining with color correlogram is presented. A prototype system which supports query by example is designed and implemented on windows 2003 operating system, using VC# and SQL server 2000.The performance of the method is illustrated using examples from an image database composed of 1361 Gastroscopic images, and the comparison of the retrieval results shows that the approach proposed in this paper is effective.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Implementing Contourlet Transform for Medical Image Fusion on a Heterogenous Platform

paper_content:

The classic wavelet transform has been applied in registration and fusion of medical images for decades. An extension namely, contourlet transform recently indicated its advantages in image fusion with better efficiency in multi-resolution and multi-direction representation and calculation. However, the even higher computational complexity it requires turns out to be a disastrous concern in embedded applications. In this paper, we implement an acceleration system on a heterogeneous TI Da Vinci dual core processor consisting of an ARM processor core and a DSP processor core. The ARM core controls the fusion procedure by extracting the luminance bits and invoking the DSP core to carry out the time-consuming part of the contourlet transform. The partitions of tasks are determined after the program is analyzed and profiled at functional level to make full use of the computational capability of the heterogeneous platform. Initial measured improved performance results are obtained and analyzed with projected further improvements.
---
paper_title: An efficient medical image fusion method using contourlet transform based on PCM

paper_content:

An efficient medical image fusion method has been proposed based on contourlet transform and multi fusion rules. The multimodal medical images were first decomposed using the contourlet transform then fusion rules were applied to low frequency components and high frequency components of contourlet coefficients. For low frequency components principle component analysis (PCA) method was adopted. While for high frequency components region based contourlet contrast was adopted. The final fusion image is obtained by directly applying inverse contourlet transform to the fused low and high frequency components. Using four image quality indicators experimental results showed that the proposed method give extensive fused image on multimodality CT/MRI.
---
paper_title: PET/CT medical image fusion algorithm based on multiwavelet transform

paper_content:

PET/CT medical image fusion has important clinical significance. As the multiwavelet transform has several particular advantages in comparison with scalar wavelets on image processing, this paper proposes a medical image fusion algorithm based on multiwavelet transform after in-depth study of wavelet theory. The algorithm achieves PET/CT fusion with wavelet coefficients fusion method. Experimental results show that fusion image combines information of the source images, adds more details and texture information, and achieves a good fusion result. Based on the proposed algorithm, we can obtain the best result when using gradient fusion in the low-frequency part and classification fusion in the high-frequency part.
---
paper_title: Quantification of brain tissue volumes using MR/MR fusion

paper_content:

We propose in this article to compute White Matter (WM), Gray Matter (GM) and Cerebrospinal Fluid (CF) volumes from MR images of a human brain using possibilistic logic and a fusion scheme. The fusion process is divided into three steps: fuzzy tissue maps are first computed on all images using a possibilistic clustering algorithm. Fusion is then achieved for all tissues with a context based fusion operator. The final segmentation allows for the computation of brain tissue volumes. Applications on a brain model show a great agreement between the results and the true segmentation, thus affording a relevant tissue classification on clinical data.
---
paper_title: The Hermite transform as an efficient model for local image analysis: An application to medical image fusion

paper_content:

The Hermite transform is introduced as an image representation model that can be used to tackle the problem of fusion in multimodal medical imagery. This model includes some important properties of human visual perception, such as local orientation analysis and the Guassian derivative model of early vision. Local analysis is achieved by windowing the image with a Gaussian function, then a local expansion into orthogonal polynomials takes place at every window position. Expansion coefficients are called Hermite coefficients and it is shown that they can be directly obtained by convolving the image with Gaussian derivative filters, in agreement with psychophysical insights of human visual perception. A compact representation can be obtained by locally steering the Hermite coefficients towards the direction of local maximum energy. Image fusion is achieved by combining the steered Hermite coefficients of both source images with the method of verification of consistency. Fusion results are compared with a competitive wavelet-based technique, proving that the Hermite transform provides better reconstruction of relevant image structures.
---
paper_title: Automatic segmentation of subcortical brain structures in MR images using information fusion

paper_content:

Reports a new automated method for the segmentation of internal cerebral structures using an information fusion technique. The information is provided both by images and expert knowledge, and consists in morphological, topological, and tissue constitution data. All this ambiguous, complementary and redundant information is managed using a three-step fusion scheme based on fuzzy logic. The information is first modeled into a common theoretical frame managing its imprecision and incertitude. The models are then fused and a decision is taken in order to reduce the imprecision and to increase the certainty in the location of the structures. The whole process is illustrated on the segmentation of thalamus, putamen, and head of the caudate nucleus from expert knowledge and magnetic resonance images, in a protocol involving 14 healthy volunteers. The quantitative validation is achieved by comparing computed, manually segmented structures and published data by means of indexes assessing the accuracy of volume estimation and spatial location. Results suggest a consistent volume estimation with respect to the expert quantification and published data, and a high spatial similarity of the segmented and computed structures. This method is generic and applicable to any structure that can be defined by expert knowledge and morphological images.
---
paper_title: "Automatic" multimodal medical image fusion

paper_content:

This paper addresses the multimodal medical image fusion problem. To deliver the expected fusion accuracy, most of the state-of-the-art fusion algorithms have the typical requirement - a set of fiducial points between the two modalities of the images to "guide" the fusion. This paper aims at removing this requirement, resulting in an "automatic" multimodal medical image fusion methodology based on an innovative model using ElectroStatic Equilibrium theory called ESE that can be used in clinical diagnoses and evaluations with accepted fusion accuracy. By "automatic", it is meant that the fusion algorithm per se does not require fiducial points; it does require a certain form of human interactions in terms of providing users a list of parameter settings at the beginning of the fusion, that are case-based, anatomy-based, and image-modality-based, and it does even allow users to have an option to change the specific values of the parameter settings to accommodate specific clinical needs. This "automatic" approach allows radiologists to save their time/effort in identifying and marking the fiducial points in the images, allows physicians and radiologists to apply their domain expertise more intelligently in "playing with" different parameter settings in a higher level when running this algorithm in diagnoses, and also allows patients to avoid the inconvenience to be placed under the fiducial markings. Preliminary evaluations against one of the existing fusion methods have shown that ESE holds a great promise in future medical applications.
---
paper_title: Redundancy Discrete Wavelet Transform and Contourlet Transform for Multimodality Medical Image Fusion with Quantitative Analysis

paper_content:

Image fusion is the process of combining relevant information from two or more images into a single fused image. The resulting image will be more informative than any of the input images. The fusion in medical images is necessary for efficient diseases diagnosis from multimodality, multidimensional and multiparameter type of images. This paper describes a multimodality medical image fusion system using different fusion techniques and the resultant is analysed with quantitative measures. Initially, the registered images from two different modalities such as CT (anatomical information) and MRI - T2, FLAIR (pathological information) are considered as input, since the diagnosis requires anatomical and pathological information. Then the fusion techniques namely Redundancy Discrete Wavelet Transform (RDWT) and Contour let Transform are applied. Further the fused image is analyzed with five types of quantitative metrics such as Standard Deviation (SD), Entropy (EN), Overall Cross Entropy (OCE), Ratio of Spatial Frequency Error (RSFE), and Power Signal to Noise Ratio (PSNR) for performance evaluation. From the experimental results we observed that RDWT method provides better information (quality) using EN metric and the Contour let Transform gives the difference in source to the fused image using OCE metric and also the fused image obtained from the proposed fusion techniques has more information than the source images are proved through all metrics.
---
paper_title: Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform

paper_content:

As a novel multiscale geometric analysis tool, contourlet has shown many advantages over the conventional image representation methods. In this paper, a new fusion algorithm for multimodal medical images based on contourlet transform is proposed. All fusion operations are performed in contourlet domain. A novel contourlet contrast measurement is developed, which is proved to be more suitable for human vision system. Other fusion rules like local energy, weighted average and selection are combined with ''region'' idea for coefficient selection in the lowpass and highpass subbands, which can preserve more details in source images and further improve the quality of fused image. The final fusion image is obtained by directly applying inverse contourlet transform to the fused lowpass and highpass subbands. Extensive fusion experiments have been made on three groups of multimodality CT/MR dataset, both visual and quantitative analysis show that comparing with conventional image fusion algorithms, the proposed approach can provide a more satisfactory fusion outcome.
---
paper_title: Risk-factor fusion for predicting multifactorial diseases

paper_content:

A generalized classification methodology is developed to predict the presence or absence of a multifactorial disease from a set of risk factors thought to be correlated with the disease. The methodology includes fusion to combine risk factors into a single feature vector, normalization to overcome the problems associated with fusing features which have different formats and ranges, discrete Karhunen-Loeve transform (DKLT)-based transformation to facilitate parametric classifier development, the selection of features with high interclass separations, and the design of parametric classifiers. The validity of the method is demonstrated by applying it to predict the occurrence of gout from 14 risk factors. Cross-validation evaluations on 96 patients, 48 clinically diagnosed to have gout and 48 diagnosed to not have gout, showed that an average classification accuracy of 75.7% can be obtained. Even more promising is that higher classification accuracies can be achieved through the careful selection of the DKLT transformation matrix which in turn involves selecting design sets that are good representatives of the gout and nongout classes. It is concluded that the generalized methodology developed in this paper is quite effective in predicting multifactorial diseases and can, therefore, assist/support a physician in diagnosing a multifactorial disease.
---
paper_title: Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain

paper_content:

Multimodal medical image fusion, as a powerful tool for the clinical applications, has developed with the advent of various imaging modalities in medical imaging. The main motivation is to capture most relevant information from sources into a single output, which plays an important role in medical diagnosis. In this paper, a novel fusion framework is proposed for multimodal medical images based on non-subsampled contourlet transform (NSCT). The source medical images are first transformed by NSCT followed by combining low- and high-frequency components. Two different fusion rules based on phase congruency and directive contrast are proposed and used to fuse low- and high-frequency coefficients. Finally, the fused image is constructed by the inverse NSCT with all composite coefficients. Experimental results and comparative study show that the proposed fusion framework provides an effective way to enable more accurate analysis of multimodality images. Further, the applicability of the proposed framework is carried out by the three clinical examples of persons affected with Alzheimer, subacute stroke and recurrent tumor.
---
paper_title: Use of quaternionic signal’s representation for analysis and fusion of multi-components 2D medical images

paper_content:

Instead of not widely being used, quaternion numbers have allowed the implementation of several image processing tools. One of these tools, the quaternionic Fourier transform, is utilized in this paper to achieve image fusion. Multi-component images are represented as pure quaternions, and a simple way to implement the quaternionic Fourier transform is also presented. The results are well satisfactory and open a way to an extension of the analytic image concept to quaternionic ones.
---
paper_title: Intermodality registration and fusion of liver images for medical diagnosis

paper_content:

SPECT is valuable for the evaluation of hepatic function and CT can offer better anatomical images for liver and its adjacent structures. Image registration from the two modalities, CT and SPECT, acquires both structural and functional information, and therefore is very useful for clinical diagnoses. The registration and fusion of CT and SPECT images are studied. We propose a two stage registration method based on principal axes transformation and surface fitting to register and fuse CT and SPECT images: First, the centroids and principal axes of these landmarks are calculated and aligned. Then the transformation matrix between CT and SPECT images is modified based on the least square error on the distance image. After registration, uniform volume data is reconstructed for image fusion.
---
paper_title: Multimodal medical image fusion based on Integer Wavelet Transform and Neuro-Fuzzy

paper_content:

Medical image fusion is used to derive useful information from multimodality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide precise information to the doctor and clinical treatment planning system. This paper proposes image fusion based on Integer Wavelet Transform (IWT) and Neuro-Fuzzy. The anatomical and functional images are decomposed using Integer Wavelet Transform. The wavelet coefficients are then fused using neuro-fuzzy algorithm. Then Inverse Integer Wavelet Transform (IIWT) is applied to the fused coefficients to get the fused Image. The performance of this algorithm is compared with image fusion based on Discrete Wavelet Transform (DWT) and neuro-fuzzy using entropy metric. Fusion Symmetry (FS) which quantifies the relative distance in terms of mutual information of the fused image with respect to input images is measured. Fusion Factor (FF) the criterion of maximizing the joint mutual information is also quantified.
---
paper_title: Medical image fusion scheme using complex contourlet transform based on PCA

paper_content:

We present an efficient method for the fusion of medical captured images using different modalities that enhances the original images and combines the complementary information of the various modalities. The contourlet transform has mainly been employed as a fusion technique for images obtained from equal or different modalities. The limitation of directional information of dual-tree complex wavelet (DT-CWT) is rectified in dual-tree complex contourlet transform (DT-CCT) by incorporating directional filter banks (DFB) into the DT-CWT. The DT-CCT produces images with improved contours and textures, while the property of shift invariance is retained. To improve the fused image quality, we propose a new method for fusion rules based on principle component analysis (PCA) which depend on frequency component of DT-CCT coefficients (contourlet domain). For low frequency components, PCA method is adopted and for high frequency components, the salient features are picked up based on local energy. The final fusion image is obtained by directly applying inverse dual tree complex contourlet transform (IDT-CCT) to the fused low and high frequency components. The experimental results showed that the proposed method produces fixed image with extensive features on multimodality.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: Fusion of Multiple Expert Annotations and Overall Score Selection for Medical Image Diagnosis

paper_content:

Two problems especially important for supervised learning and classification in medical image processing are addressed in this study: i) how to fuse medical annotations collected from several medical experts and ii) how to form an image-wise overall score for accurate and reliable automatic diagnosis. Both of the problems are addressed by applying the same receiver operating characteristic (ROC) framework which is made to correspond to the medical practise. The first problem arises from the typical need to collect the medical ground truth from several experts to understand the underlying phenomenon and to increase robustness. However, it is currently unclear how these expert opinions (annotations) should be combined for classification methods. The second problem is due to the ultimate goal of any automatic diagnosis, a patient-based (image-wise) diagnosis, which consequently must be the ultimate evaluation criterion before transferring any methods into practise. Various image processing methods provide several, e.g., spatially distinct, results, which should be combined into a single image-wise score value. We discuss and investigate these two problems in detail, propose good strategies and report experimental results on a diabetic retinopathy database verifying our findings.
---
paper_title: Improving Breast Cancer Risk Stratification Using Resonance-Frequency Electrical Impedance Spectroscopy Through Fusion of Multiple Classifiers

paper_content:

This study aims to improve breast cancer risk stratification. A seven-probe resonance-frequency-based electrical impedance spectroscopy (REIS) system was designed, assembled, and utilized to establish a data set of examinations from 174 women. Three classifiers, including artificial neural network (ANN), support vector machine (SVM), and Gaussian mixture model (GMM), were independently developed to predict the likelihood of each woman to be recommended for biopsy. The performances of these classifiers were compared, and seven fusion methods for integrating these classifiers were investigated. The results showed that among the three classifiers, the ANN yielded the highest performance with an area under the curve (AUC) of 0.81 for the receiver operating characteristic (ROC), while SVM and GMM achieved AUCs of 0.80 and 0.78, respectively. Improvements of up to 3% were obtained using fusion of the three classifiers, with the largest improvement obtained using either a "minimum score" rule or a "weighted sum" rule. Comparing different combinations of two out of the three classifiers, the weighted sum rule provided the most robust and consistent results, with AUCs of 0.81, 0.83, and 0.82 for the different combinations of ANN and SVM, ANN and GMM, and SVM and GMM, respectively. Furthermore, at 90% specificity, the ANN, the weighted sum- and min rule-based classifiers, all detected 67% of the verified cancer cases as compared with 50, 50, and 60% detection of the high risk cases, respectively. The study demonstrated that REIS examinations provide relevant information for developing breast cancer risk stratification tools and that using fusion of several not-fully-correlated classifiers can improve classification performance.
---
paper_title: Medical Image Fusion Based on Wavelet Transform and Independent Component Analysis

paper_content:

By analyzing the characters of CT medical image, this paper proposes a novel method for this particular image fusion, which is using discrete wavelet transform and independent component analysis. Firstly, each of CT images were decomposed by 2-D discrete wavelet transform. Then independent component analysis were used to analyse the wavelet coefficients in different level for acquiring independent component. At last, we use wavelet reconstruction to synthesize one CT medical image which could contain more integrated accurate detail information of different soft tissue such as muscles and blood vessels. By contrast, the efficiency of our method is better than weighted average, laplacian pyramid method in medical image fusion field.
---
paper_title: Classification of co-expressed genes from DNA regulatory regions

paper_content:

The analysis of non-coding DNA regulatory regions is one of the most challenging open problems in computational biology. In this paper we investigate whether we can predict functional information about genes by using information extracted from their sequences together with expression data. We formalize this problem as a classification problem, and we apply Support Vector Machines (SVMs) with non-linear kernels to predict classes of co-expressed genes obtained from clustering procedures. SVMs are trained using information about selected motifs extracted from DNA regulatory regions through combinatorial and statistical methods. In our experiments, we show that functional classes of genes can be predicted from biological sequence data in Saccharomices cerevisiae, achieving results competitive with those recently presented in the literature.
---
paper_title: Text Fusion Watermarking in Medical Image with Semi-reversible for Secure Transfer and Authentication

paper_content:

Nowadays, the transmission of digitized medical information has become very convenient due to the generality of Internet. Internet has created the biggest benefit to achieve the transmission of patient information efficiently. However, it is easier that the hackers can grab or duplicate the digitized information on the Internet. This will cause the following problems of medical security and copyright protection. In order to fulfil the security and convenience issues of the patients following goals like the prevention of medical fault, the real-time detection of abnormal event, the support of clinical decision and developing of medical service based on patient has to be achieved. For this purpose this paper proposes the technique called binary embedding technique. It uses the binary information of the text embedded with semi reversible properties in the image are called binary embedding. This technique prevents the distortion of embedded information in the original image due to addition of noise. The concept of semi reversible property is used retains the original information, image quality before and after the process of watermarking is presented as a statistical analysis
---
paper_title: Classifier fusion approaches for diagnostic cancer models.

paper_content:

Classifier ensembles have produced promising results, improving accuracy, confidence and most importantly feature space coverage in many practical applications. The recent trend is to move from heuristic combinations of classifiers to more statistically sound integrated schemes to produce quantifiable results as far as error bounds and overall generalization capability are concerned. In this study, we are evaluating the use of an ensemble of 8 classifiers based on 15 different fusion strategies on two medical problems. We measure the base classifiers correlation using 11 commonly accepted metrics and provide the grounds for choosing an improved hyper-classifier.
---
paper_title: Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion

paper_content:

Abstract We present a content-based image retrieval framework for diverse collections of medical images of different modalities, anatomical regions, acquisition views, and biological systems. For the image representation, the probabilistic output from multi-class support vector machines (SVMs) with low-level features as inputs are represented as a vector of confidence or membership scores of pre-defined image categories. The outputs are combined for feature-level fusion and retrieval based on the combination rules that are derived by following Bayes’ theorem. We also propose an adaptive similarity fusion approach based on a linear combination of individual feature level similarities. The feature weights are calculated by considering both the precision and the rank order information of top retrieved relevant images as predicted by SVMs. The weights are dynamically updated by the system for each individual search to produce effective results. The experiments and analysis of the results are based on a diverse medical image collection of 11,000 images of 116 categories. The performances of the classification and retrieval algorithms are evaluated both in terms of error rate and precision–recall. Our results demonstrate the effectiveness of the proposed framework as compared to the commonly used approaches based on low-level feature descriptors.
---
paper_title: Fast Fusion of Medical Images Based on Bayesian Risk Minimization and Pixon Map

paper_content:

Fast fusion of multiple registered out-of-focusimages is of great interest in medical imaging; forexample, the thoracic cavity is always too bumpy to befocused on all parts at one shot even when we can omitthe unavoidable hardware vibrations. Previousproposed methods in this field cannot fulfill the realtimerequirement in our multiple camera medicalimaging setting. In this paper, we propose a multiresolutionBayesian risk minimization based method tofuse these chest cavity images. The validity andefficiency of our method are verified by ourexperiments on both out-of-focus medical images andregional motion blurred images. By choosing specialkernel functions for the Pixon map and adoptinguniform distribution as the prior probability, ourmethod can be applied to the real-time medicalimaging situations such as surgical operationmonitoring.
---
paper_title: Multimodal medical image fusion based on IHS and PCA

paper_content:

Abstract The fusion of multimodal brain imaging for a given clinical application is a very important performance. Generally, the PET image indicates the brain function and has a low spatial resolution, the MRI image shows the brain tissue anatomy and contains no functional information. Hence, a perfect fused image should contain both more functional information and more spatial characteristics with no spatial and color distortion. There have been a number of approaches proposed for fusing multitask or multimodal image information. But, every approach has its limited domain for a particular application. Study indicated that intensity-hue-saturation (IHS) transform and principal component analysis (PCA) can preserve more spatial feature and more required functional information with no color distortion. The presented algorithm integrates the advantages of both IHS and PCA fusion methods to improve the fused image quality. Visual and quantitative analysis show that the proposed algorithm significantly improves the fusion quality; compared to fusion methods including PCA, Brovey, discrete wavelet transform (DWT).
---
paper_title: Multichannel fusion models for the parametric classification of multicategory differential brain activity

paper_content:

This work introduces multichannel classification fusion and multichannel data fusion models to fully exploit the different but complementary brain activity information recorded from multiple channels. The goal is to accurately classify differential brain activity into their respective categories. A parametric weighted classification fusion model and three weighted data fusion models (mixture, sum, and concatenation) are introduced. Parametric classifiers are developed for each fusion strategy and the performances of the different strategies are compared by classifying 14-channel evoked potentials (EPs) collected from subjects involved in making explicit match/mismatch comparisons between sequentially presented stimuli. The best performance is obtained using multichannel EP concatenation and the performance improves by incorporating weights in the fusion rules. The fusion strategies introduced are also applicable to other problems involving the classification of multicategory multivariate signals generated from multiple sources.
---
paper_title: Improving protein secondary structure predictions by prediction fusion

paper_content:

Protein secondary structure prediction is still a challenging problem at today. Even if a number of prediction methods have been presented in the literature, the various prediction tools that are available on-line produce results whose quality is not always fully satisfactory. Therefore, a user has to know which predictor to use for a given protein to be analyzed. In this paper, we propose a server implementing a method to improve the accuracy in protein secondary structure prediction. The method is based on integrating the prediction results computed by some available on-line prediction tools to obtain a combined prediction of higher quality. Given an input protein p whose secondary structure has to be predicted, and a group of proteins F, whose secondary structures are known, the server currently works according to a two phase approach: (i) it selects a set of predictors good at predicting the secondary structure of proteins in F (and, therefore, supposedly, that of p as well), and (ii) it integrates the prediction results delivered for p by the selected team of prediction tools. Therefore, by exploiting our system, the user is relieved of the burden of selecting the most appropriate predictor for the given input protein being, at the same time, assumed that a prediction result at least as good as the best available one will be delivered. The correctness of the resulting prediction is measured referring to EVA accuracy parameters used in several editions of CASP.
---
paper_title: Classifier Fusion Using Dempster-Shafer theory of evidence to Predict Breast Cancer Tumors

paper_content:

In classifier fusion models, classifiers outputs are combined to achieve a group decision. The most often used classifiers fusion models are majority vote, probability schemes, weighted averaging and Bayes approach to name few. We propose a model of classifiers fusion by combining the mathematical belief of classifiers. We used Dempster-shafer theory of evidence to determine the mathematical belief of classifiers. Support Vector Machine (SVM) with linear, polynomial and radial kernel has been employed as classifiers. The output of classifiers used as basis for computing beliefs. We combined these beliefs to arrive at one final decision. Our experimental results have shown that the new proposed classifiers fusion methodology have outperforms single classification models.
---
paper_title: Research on Medical Image Fusion Based on Orthogonal Wavelet Packets Transformation Combined with 2v-SVM

paper_content:

This paper presents a novel medical image fusion algorithm which imports 2v-SVM -an adaptive SVM learning algorithm to medical image fusion. We combine it with orthogonal wavelet packets to generate a new image fusion rule, which intelligently constructs the "good and bad features-classifier" for improving image fusion. Then we construct a new sort of linear weighted fusion arithmetic operator, which retains the remarkable features and eliminates the redundant information of the two images. We respectively fuse two pairs of images: CT> MRI images and MRI, PET images effectively and obtain good visual effects and quality of fused image. The experiments prove that the proposed algorithm resolve the deficiencies of existing algorithms in characteristics representation and acceptance or rejection of information and provide an advanced method to medical image fusion.
---
paper_title: Medical Image Retrieval with Query-Dependent Feature Fusion Based on One-Class SVM

paper_content:

Due to the huge growth of the World Wide Web, medical images are now available in large numbers in online repositories, and there exists the need to retrieval the images through automatically extracting visual information of the medical images, which is commonly known as content-based image retrieval (CBIR). Since each feature extracted from images just characterizes certain aspect of image content, multiple features are necessarily employed to improve the retrieval performance. Meanwhile, experiments demonstrate that a special feature is not equally important for different image queries. Most of existed feature fusion methods for image retrieval only utilize query independent feature fusion or rely on explicit user weighting. In this paper, we present a novel query dependent feature fusion method for medical image retrieval based on one class support vector machine. Having considered that a special feature is not equally important for different image queries, the proposed query dependent feature fusion method can learn different feature fusion models for different image queries only based on multiply image samples provided by the user, and the learned feature fusion models can re?ect the different importances of a special feature for different image queries. The experimental results on the IRMA medical image collection demonstrate that the proposed method can improve the retrieval performance effectively and can outperform existed feature fusion methods for image retrieval.
---
paper_title: Support vector machine and generalized regression neural network based classification fusion models for cancer diagnosis

paper_content:

This paper presents decision-based fusion models to classify BRCA1, BRCA2 and Sporadic genetic mutations for breast and ovarian cancer. Different ensembles of base classifiers using the stacked generalization technique have been proposed including support vector machines (SVM) with linear, polynomial and radial base function kernels. A generalized regression neural network (GRNN) is then applied to predict the mutation type based on the outputs of base classifiers, and experimental results show that the new proposed fusion methodology for selecting the best and removing weak classifiers outperforms single classification models.
---
paper_title: Multi-modal medical volumes fusion by surface matching

paper_content:

Presents a fast, six degrees of freedom, registration technique to accurately locate the position and orientation of medical volumes (e.g. CT, MRI) with respect to each other for the same patient. The technique uses surface registration and maximization of mutual information. We have developed a novel technique for surface registration which produces highly accurate results when registering two different volumes of the same individual generated from the same modality, such as preoperative MR and intraoperative MR volumes. In case surface registration is not able to accurately register the different volumes, the result is enhanced by multi-modal volume registration. The gain of this combination is to have an accurate alignment and to reduce time needed for registration. For the multi-modal volume registration, the maximization of mutual information (MI) as a matching criterion is used based on genetic algorithms (GA) as a search engine. Our results demonstrate that our registration technique allows for fast, accurate, robust and completely automatic registration of multimodality medical volumes.
---
paper_title: On information fusion to improve segmentation of MRI sequences

paper_content:

Abstract The paper deals with the problem of segmentation of MRI sequences of vertebrae, in the form of images of their multiple slices, using the Dempster–Shafer theory. This leads to the study of 3-D deformations of the scoliosis. The motivation comes from the inadequacy of the existing techniques based on X-ray image analysis. Such analysis cannot deal with, on the one hand, the complex anatomical structures (“scoliotic rachis”), and the spongy tissue peri-rachidian, and, on the other hand, the choice of slices and the problem of the residue irradiation present in each examination. The main contributions of the paper are: • New architecture for the fusion of MRI data sets. • A novel method to exploit the information contained in MRI sequence. • Model for knowledge representation adapted to specificity of information available (Dempster–Shafer theory). • Choice of the discriminating parameters for the statistical expertise. • Construction of the belief functions. • Choice of the decision criterion. Starting from segmentation by active contour (snake) [Deformable contour: modelling, extraction, detection and classification, Ph.D. Thesis, Wisconsin–Madison University, 1994; Proceedings of the 15th International Conference on Pattern Recognition, vol. 4, Barcelona, 2000, p. 17; Int. J. Comput. Vision 1 (3) (1987) 211], we upgrade it in an attempt to present the doctor with a degree of belief concerning their membership of the contour of the vertebra. We illustrate the proposed fusion architecture by application to actual MRI sequences of the vertebrae, and include perhaps the first example of 3-D reconstruction of the lumbar rachis starting from the results obtained during fusion.
---
paper_title: Importance of the CT/MRI fusion method as a learning tool for CT-based postimplant dosimetry in prostate brachytherapy

paper_content:

Abstract Background and purpose To compare the CT-based and CT/MRI fusion-based postimplant dosimetry after permanent prostate brachytherapy and to evaluate the improvement in CT-based dosimetry by physicians with or without experience in using the CT/MRI fusion method. Patients and methods Thirty-eight consecutive patients agreed to participate in a prospective study. The prostate contours from CT/MRI fusion are the gold standard for determining the prostate volume and dose volume histogram (DVH). CT-based postimplant dosimetries were performed by two physicians. Observer 1 was a radiologist who had never used CT/MRI fusion method for postimplant dosimetric analysis. Observer 2 was a radiation oncologist experienced in postimplant analysis using the CT/MRI fusion method. The prostate dosimetry was evaluated by prostate D90 and V100. Results No significant difference was observed in the mean prostate volumes between the two observers and the CT/MRI fusion data. However, the correlation coefficient value for observer 2 ( R 2 =0.932) was greater than that for observer 1 ( R 2 =0.793). The D90 and V100 values as evaluated by the two observers were significantly underestimated in comparison to those evaluated using the CT/MRI fusion methods. The DVH related parameters were underestimated more frequently by observer 1 than by observer 2: (prostate D90: 99.56% for observer 1, 102.97% for observer 2, 109.37% for CT/MRI fusion. Prostate V100: 88.12% for observer 1, 90.14% for observer 2, 91.91% for CT/MRI fusion). Conclusions The difference in the mean value in D90 and V100 by observer 1 was significantly greater than that for observer 2. These findings suggest that the CT/MRI fusion method provides accurate feedback which thereby improves CT-based postimplant dosimetry for prostate brachytherapy.
---
paper_title: Sequential evaluation of prostate edema after permanent seed prostate brachytherapy using CT-MRI fusion

paper_content:

Purpose: To analyze the extent and time course of prostate edema and its effect on dosimetry after permanent seed prostate brachytherapy. Methods and Materials: Twenty patients scheduled for permanent seed 125 I prostate brachytherapy agreed to a prospective study on postimplant edema. Implants were preplanned using transrectal ultrasonography. Postimplant dosimetry was calculated using computed tomography-magnetic resonance imaging (CT-MRI) fusion on the day of the implant (Day 1) and Days 8 and 30. The prostate was contoured on MRI, and the seeds were located on CT. Factors investigated for an influence on edema were the number of seeds and needles, preimplant prostate volume, transitional zone index (transition zone volume divided by prostate volume), age, and prostate-specific antigen level. Prostate dosimetry was evaluated by the percentage of the prostate volume receiving 100% of the prescribed dose (V 100 ) and percentage of prescribed dose received by 90% of the prostate volume (D 90 ). Results: Prostate edema was maximal on Day 1, with the median prostate volume 31% greater than preimplant transrectal ultrasound volume (range, 0.93–1.72; p p = 0.013) and 5% greater on Day 30 ( p p = 0.016) and the preplan prostate volume ( p = 0.003). The median V 100 on Day 1 was 93.6% (range, 86.0–98.2%) and was 96.3% (range, 85.7–99.5%) on Day 30 ( p = 0.079). Patients with a Day 1 V 100 >93% were less affected by edema resolution, showing a median increase in V 100 of 0.67% on Day 30 compared with 2.77% for patients with a V 100 Conclusion: Despite the extreme range of postimplant edema, the effect on dosimetry was less than expected. Dose coverage of the prostate was good for all patients during Days 1–30. Our data indicate that postimplant dosimetry on the day of implant is sufficient for patients with good dose coverage (Day 1 V 100 >93%).
---
paper_title: 2304 STEREOTACTIC PROSTATE BIOPSY WITH PRE-INTERVENTIONAL MRI AND LIVE US FUSION

paper_content:

INTRODUCTION AND OBJECTIVES: The key challenge for focal therapy of prostate cancer is to identify index lesions. In this context, we describe our initial experience with a new stereotactic prostate biopsy system, which integrates pre-interventional MRI data with periinterventional ultrasound for perineal prostate biopsies. METHODS: 33 patients were stereotactically biopsied (mean age of 64yrs., mean PSA level of 8.2ng/ml and mean prostate vol. of 40ml). 21 of these 33 patients had already had a negative transrectal US-guided biopsy. All men underwent multimodal 3 Tesla MRI without endorectal coil including diffusion weighted imaging and dynamic contrast enhanced sequences as well as MR-spectroscopy. Suspicious lesions were marked by radiologists with over 10 years experience in reading prostate MRI before the obtained data were transferred to the stereotactic biopsy system. Using a custom-made biplane TRUS probe mounted on a stepper device, 3D ultrasound data were generated to set the anatomic landmarks. Then MRI and TRUS imaging data were fused manually. As a result, the suspicious MRI lesions were superimposed onto the TRUS data. Next, 3D biopsy planning was performed including systematic biopsies from the peripheral and transitional zones of the prostate. Perineal biopsies were taken under live US imaging, and the location of each biopsy was documented in 3D. RESULTS: 14 out of 33 patients were diagnosed with prostate cancer. These results showed a positive correlation between MRI findings and histopathology in 22 out of 33 patients. In MRI lesions marked as highly suspicious, the detection rate was 100%. Evaluating the biopsies from lesions marked as highly and as questionable suspicious together, prostate cancer was detected in 27.95%. In comparison, only 8% of the additional systematic biopsies were positive. Target registration error of the first 554 biopsy cores was 1.9 mm. For adverse effects, one patient experienced urinary retention. Postinterventional hemorrhage or urinary tract infection did not occur. CONCLUSIONS: Perineal stereotactic prostate biopsies guided by the combination of MRI and ultrasound enable effective examination of suspicious MRI lesions. Additionally, each biopsy core taken may be documented accurately for its location in 3D. Thus, MRI data may be validated and different treatment options stratified in depth. At the same time the morbidity of the procedure was minimal.
---
paper_title: Image fusion of thallium-201 SPECT and MR imaging for the assessment of recurrent head and neck tumors following flap reconstructive surgery

paper_content:

The aim of this study was to assess the value of fused MR and Tl-201 single photon emission computed tomography (SPECT) images in the diagnosis of recurrent head and neck tumors in patients after flap reconstruction surgery. Twenty-four patients after resection of primary head and neck tumors with flap reconstruction were suspected of having recurrent tumor by follow-up MR examination. Both MR examination and Tl-201 SPECT were prospectively performed to produce fused images. For qualitative analysis, two independent readers separately evaluated the existence of tumor recurrence in the fused images. The Tl-201 uptake of the lesion (Tl index) was also quantitatively compared with that of the normal nuchal muscles. Eighteen patients were histologically proved as having recurrence. The remaining 6 patients, false positive on MRI alone, had non-recurrence. Using the fused images, false positive was found in 1 case for one reader and 2 cases for the other reader. The Tl index of recurrent tumors was significantly higher (p < 0.001) than that of non-recurrent mass lesions. In the assessment of recurrent tumors following flap reconstruction surgery in the head and neck, the use of fused MRI and Tl-201 SPECT images can reduce the number of false positives.
---
paper_title: MRI and PET image fusion by combining IHS and retina-inspired models

paper_content:

Image fusion has become a widely used tool for increasing the interpretation quality of images in medical applications. The acquired data might exhibit either good functional characteristic (such as PET) or high spatial resolution (such as MRI). The MRI image shows the brain tissue anatomy and contains no functional information. The PET image indicates the brain function and has a low spatial resolution. Hence, the image fusion task is carried out to enhance the spatial resolution of the functional images by combining them with a high-resolution anatomic image. A perfect fusion process preserves the original functional characteristics and adds spatial characteristics to the image with no spatial distortion. The intensity-hue-saturation (IHS) algorithm and the retina-inspired model (RIM) fusion technique can preserve more spatial feature and more functional information content, respectively. The presented algorithm integrates the advantages of both IHS and RIM fusion methods to improve the functional and spatial information content. Visual and statistical analyses show that the proposed algorithm significantly improves the fusion quality in terms of: entropy, mutual information, discrepancy, and average gradient; compared to fusion methods including, IHS, Brovey, discrete wavelet transform (DWT), a-trous wavelet and RIM.
---
paper_title: Quantification of brain tissue volumes using MR/MR fusion

paper_content:

We propose in this article to compute White Matter (WM), Gray Matter (GM) and Cerebrospinal Fluid (CF) volumes from MR images of a human brain using possibilistic logic and a fusion scheme. The fusion process is divided into three steps: fuzzy tissue maps are first computed on all images using a possibilistic clustering algorithm. Fusion is then achieved for all tissues with a context based fusion operator. The final segmentation allows for the computation of brain tissue volumes. Applications on a brain model show a great agreement between the results and the true segmentation, thus affording a relevant tissue classification on clinical data.
---
paper_title: An application of multimodal image registration and fusion in a 3D tumor simulation model

paper_content:

In this paper, a three-dimensional (3D) image registration scheme along with fusion tools is developed and incorporated within a model simulating tumor growth and response to radiotherapy. This biosimulation model attempts to predict and visualize in three dimensions the in vivo tumor behavior. The necessary information for the initialization and the adjustment of the model can be provided from different medical examinations. The present study focuses on the development and evaluation of a technique that enables input information provided from different medical imaging modalities to be efficiently exploited by tumor growth simulation algorithms through the registration and fusion of computed tomography (CT) or positron emission tomography (PET) to magnetic resonance imaging (MRI) data.
---
paper_title: Prostate seed implant quality assessment using MR and CT image fusion.

paper_content:

PURPOSE ::: After a seed implant of the prostate, computerized tomography (CT) is ideal for determining seed distribution but soft tissue anatomy is frequently not well visualized. Magnetic resonance (MR) images soft tissue anatomy well but seed visualization is problematic. We describe a method of fusing CT and MR images to exploit the advantages of both of these modalities when assessing the quality of a prostate seed implant. ::: ::: ::: METHODS AND MATERIALS ::: Eleven consecutive prostate seed implant patients were imaged with axial MR and CT scans. MR and CT images were fused in three dimensions using the Pinnacle 3.0 version of the ADAC treatment planning system. The urethra and bladder base were used to "line up" MR and CT image sets during image fusion. Alignment was accomplished using translation and rotation in the three ortho-normal planes. Accuracy of image fusion was evaluated by calculating the maximum deviation in millimeters between the center of the urethra on axial MR versus CT images. Implant quality was determined by comparing dosimetric results to previously set parameters. ::: ::: ::: RESULTS ::: Image fusion was performed with a high degree of accuracy. When lining up the urethra and base of bladder, the maximum difference in axial position of the urethra between MR and CT averaged 2.5 mm (range 1.3-4.0 mm, SD 0.9 mm). By projecting CT-derived dose distributions over MR images of soft tissue structures, qualitative and quantitative evaluation of implant quality is straightforward. ::: ::: ::: CONCLUSIONS ::: The image-fusion process we describe provides a sophisticated way of assessing the quality of a prostate seed implant. Commercial software makes the process time-efficient and available to any clinical practice with a high-quality treatment planning system. While we use MR to image soft tissue structures, the process could be used with any imaging modality that is able to visualize the prostatic urethra (e.g., ultrasound).
---
paper_title: Medical Image Fusion Based on the Structure Similarity Match Measure

paper_content:

In recent years, medical image fusion is extensively used to help doctors improve the accuracy of medical diagnosis by combining multimodality images acquired under different imaging conditions into a single one. Most of the previous methods aim at attaining information as many as possible from source images. However, some of the exacted information is not necessary or useful for medical diagnosis. As Marr’s vision stated, human visual system is being adapted for exacting structure features such as lines, edges, contours from the images. In this paper, we try to develop a new image fusion scheme based on structure similarity match measure (SSIM) to exact structure features from the input images to improve the accuracy of diagnosis. The visual experiments and quantitative assessments demonstrate the effectiveness of this method compared to present image fusion schemes.
---
paper_title: The role of 3D fusion imaging in sentinel lymphadenectomy for vulvar cancer.

paper_content:

OBJECTIVES ::: To evaluate the feasibility of fusion imaging compound tomography (FICT) of CT/MRI and single photon emission tomography (SPECT) versus planar scintigraphy only (plSc) in pre-surgical staging for vulvar cancer. ::: ::: ::: MATERIALS AND METHODS ::: Analysis of consecutive patients with vulvar cancer who preoperatively underwent sentinel scintigraphy (planar and 3D-SPECT imaging) and CT or MRI. Body markers were used for exact anatomical co-registration and fusion datasets were reconstructed using SPECT and CT/MRI. The number and localisation of all intraoperatively identified and resected sentinel lymph nodes (SLN) were compared between planar and 3D fusion imaging. ::: ::: ::: RESULTS ::: Twenty six SLN were localized on planar scintigraphy. Twelve additional SLN were identified after SPECT and CT/MRI reconstruction, all of them were confirmed intraoperatively. In seven cases where single foci were identified at plSc, fusion imaging revealed grouped individual nodes and five additional localisations were discovered at fusion imaging. In seven patients both methods identified SLN contra lateral to the primary tumor site, but only fusion imaging allowed to localise iliac SLN in four patients. All SLN predicted on fusion imaging could be localised and resected during surgery. ::: ::: ::: CONCLUSIONS ::: Fusion imaging using SPECT and CT/MRI can detect SLN in vulvar cancer more precisely than planar imaging regarding number and anatomical localisation. FICT revealed additional information in seven out of ten cases (70%).
---
paper_title: Comparison of MRI-based and CT/MRI fusion-based postimplant dosimetric analysis of prostate brachytherapy.

paper_content:

PURPOSE ::: The aim of this study was to compare the outcomes between magnetic resonance imaging (MRI)-based and computed tomography (CT)/MRI fusion-based postimplant dosimetry methods in permanent prostate brachytherapy. ::: ::: ::: METHODS AND MATERIALS ::: Between October 2004 and March 2006, a total of 52 consecutive patients with prostate cancer were treated by brachytherapy, and postimplant dosimetry was performed using CT/MRI fusion. The accuracy and reproducibility were prospectively compared between MRI-based dosimetry and CT/MRI fusion-based dosimetry based on the dose-volume histogram (DVH) related parameters as recommended by the American Brachytherapy Society. ::: ::: ::: RESULTS ::: The prostate volume was 15.97+/-6.17 cc (mean+/-SD) in MRI-based dosimetry, and 15.97+/-6.02 cc in CT/MRI fusion-based dosimetry without statistical difference. The prostate V100 was 94.5% and 93.0% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.002). The prostate D90 was 119.4% and 114.4% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.004). ::: ::: ::: CONCLUSION ::: Our current results suggested that, as with fusion images, MR images allowed accurate contouring of the organs, but they tended to overestimate the analysis of postimplant dosimetry in comparison to CT/MRI fusion images. Although this MRI-based dosimetric discrepancy was negligible, MRI-based dosimetry was acceptable and reproducible in comparison to CT-based dosimetry, because the difference between MRI-based and CT/MRI fusion-based results was smaller than that between CT-based and CT/MRI fusion-based results as previously reported.
---
paper_title: Effect of edema on postimplant dosimetry in prostate brachytherapy using CT/MRI fusion.

paper_content:

PURPOSE ::: To investigate the time course of prostatic edema and the effect on the dose-volume histograms of the prostate for patients treated with brachytherapy. ::: ::: ::: METHODS AND MATERIALS ::: A total of 74 patients with prostate cancer were enrolled in this prospective study. A transrectal ultrasound-based preplan was performed 4 weeks before implantation and computed tomography/magnetic resonance imaging fusion-based postimplant dosimetry was performed on the day after implantation (Day 1) and 30 days after implantation (Day 30). The prostate volume, prostate volume covered by 100% of the prescription dose (V100), and dose covering 90% of the prostate (D90) were evaluated with prostatic edema over time. ::: ::: ::: RESULTS ::: Prostatic edema was greatest on Day 1, with the mean prostate volume 36% greater than the preplan transrectal ultrasound-based volume; it thereafter decreased over time. It was 9% greater than preplan volume on Day 30. The V(100) increased 5.7% from Day 1 to Day 30, and the D90 increased 13.1% from Day 1 to Day 30. The edema ratio (postplan/preplan) on Day 1 of low-quality implants with a V(100) of <80% was significantly greater than that of intermediate- to high-quality implants (>80% V100; p = 0.0272). The lower V100 on Day 1 showed a greater increase from Day 1 to Day 30. A V100 on Day 1 of >92% is unlikely to increase >0% during the interval studied. ::: ::: ::: CONCLUSION ::: Low-quality implants on Day 1 were highly associated with edema; however, such a low-quality implant on Day 1, with significant edema, tended to improve by Day 30. If a high-quality implant (V100 >92%) can be obtained on Day 1, a re-examination is no longer necessary.
---
paper_title: Image Registration and Fusion for Interventional MRI Guided Thermal Ablation of the Prostate Cancer

paper_content:

We are investigating interventional MRI (iMRI) guided thermal ablation treatment of the prostate cancer. Functional images such as SPECT can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high resolution MRI volume. Then by registering the high-resolution MR image with iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to iMRI images for improved tumor targeting. For the first step, we used a mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm. Image data were acquired from patients and volunteers. Compared to our volume-to-volume registration that was previously evaluated to be quite accurate, the SV registration accuracy is about 0.5mm for transverse images covering the prostate. With our image registration and fusion software, simulation experiments show that it is feasible to incorporate SPECT and high resolution MRI into the iMRI-guided treatment.
---
paper_title: Automatic segmentation of subcortical brain structures in MR images using information fusion

paper_content:

Reports a new automated method for the segmentation of internal cerebral structures using an information fusion technique. The information is provided both by images and expert knowledge, and consists in morphological, topological, and tissue constitution data. All this ambiguous, complementary and redundant information is managed using a three-step fusion scheme based on fuzzy logic. The information is first modeled into a common theoretical frame managing its imprecision and incertitude. The models are then fused and a decision is taken in order to reduce the imprecision and to increase the certainty in the location of the structures. The whole process is illustrated on the segmentation of thalamus, putamen, and head of the caudate nucleus from expert knowledge and magnetic resonance images, in a protocol involving 14 healthy volunteers. The quantitative validation is achieved by comparing computed, manually segmented structures and published data by means of indexes assessing the accuracy of volume estimation and spatial location. Results suggest a consistent volume estimation with respect to the expert quantification and published data, and a high spatial similarity of the segmented and computed structures. This method is generic and applicable to any structure that can be defined by expert knowledge and morphological images.
---
paper_title: Additional value of MR/PET fusion compared with PET/CT in the detection of lymph node metastases in cervical cancer patients.

paper_content:

We evaluated the additional diagnostic value of magnetic resonance/positron emission tomography (MR/PET) fusion in the detection of metastatic lymph nodes in cervical cancer patients. Seventy nine patients with FIGO stage IB-IVA cervical cancer who had undergone both magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) before lymphadenectomy were included in this study. Image analysis was first performed with PET/CT images only. A second analysis was then performed with MR/PET fused images that focused on the additional information obtained from the MR images. Lymphadenectomy involved removing all visible lymph nodes in the surgical field. To enable nodal group-specific comparisons, para-aortic and pelvic lymph nodes were divided into seven nodal groups: para-aortic, both common iliac, both external iliac and both internal iliac/obturator areas. Histopathological evaluation of lymph nodes has been the diagnostic standard. The value of the additional information from the MR images was evaluated by means of receiver operating characteristic (ROC) analysis. Fused MR/PET rendered readers to detect six more metastatic lymph node groups. The sensitivity and specificity of PET/CT and fused MR/PET were 44.1%, 93.9% and 54.2%, 92.7% respectively. The ROC analysis demonstrated a higher diagnostic performance of fused MR/PET compared to PET/CT alone for detecting lymph node metastases (p=0.0259). The findings of this study demonstrate the additional diagnostic value of fused MR/PET images compared with PET/CT in the detection of metastatic lymph nodes in patients with uterine cervical cancer.
---
paper_title: An analysis of intraoperative versus post-operative dosimetry with CT, CT-MRI fusion and XMR for the evaluation of permanent prostate brachytherapy implants.

paper_content:

BACKGROUND AND PURPOSE ::: To assess the agreement between intraoperative and post-operative dosimetry and to identify factors that influence dose calculations of prostate brachytherapy implants. ::: ::: ::: MATERIALS AND METHODS ::: Patients treated with prostate brachytherapy implants underwent post-operative CT and XMR (combined X-ray and MR) imaging. Dose-volume histograms were calculated from CT, XMR and CT-MR fusion data and compared with intraoperative values for two observers. Multiple linear regression models assessed the influences of intraoperative D90, gland oedema, gland volume, source loss and migration, and implanted activity/volume prostate on post-operative D90. ::: ::: ::: RESULTS ::: Forty-nine patients were studied. The mean D90 differences (95% confidence limits) between intraoperative and post-operative CT, XMR and CT-MR fusion assessments were: 11 Gy (-22, 45), 18 Gy (-13, 49) and 20 Gy (-17, 58) for Observer 1; and 15 Gy (-34, 63), 13 Gy (-29, 55) and 14 Gy (-27, 54) for Observer 2. Multiple linear regression modelling showed that the observed oedema and intraoperative D90 were significant independent variables for the prediction of post-operative D90 values for both observers using all modalities. ::: ::: ::: CONCLUSION ::: This is the first study to report Bland-Altman agreement analysis between intraoperative and post-operative dosimetry. Agreement is poor. Post-operative dosimetry is dependent on the intraoperative D90 and the subjectively outlined gland volume.
---
paper_title: Initial clinical assessment of CT-MRI image fusion software in localization of the prostate for 3D conformal radiation therapy☆

paper_content:

Abstract Purpose: To assess the utility of image fusion software and compare MRI prostate localization with CT localization in patients undergoing 3D conformal radiation therapy of prostate cancer. Materials and Methods: After a phantom study was performed to ensure the accuracy of image fusion procedure, 22 prostate cancer patients had CT and MRI studies before the start of radiotherapy. Immobilization casts used during radiation treatment were also used for both imaging studies. After the clinical target volume (CTV) (prostate or prostate + seminal vesicles) was defined on CT, slices from the MRI study were reconstructed to precisely match the CT slices by identifying three common bony landmarks on each study. The CTV was separately defined on the matched MRI slices. Data related to the size and location of the prostate were compared between CT and MRI. The spatial relationship between the tip of urethrogram cone on CT and prostate apex seen on MRI was also estimated. Results: The phantom study showed the registration discrepancies between CT and MRI smaller than 1.0 mm in any pair in comparison. The patient study showed a mean image registration error of 0.9 (± 0.6) mm. The average prostate volume was 63.0 (± 25.8) cm 2 and 50.9 (± 22.9) cm 3 determined by CT and MRI, respectively. The difference in prostate location with the two studies usually differed at the base and at the apex of the prostate. On the transverse MRI, the prostate apex was situated 7.1 (± 4.5) mm dorsal and 15.1 (± 4.0) mm cephalad to the tip of urethrogram cone. Conclusions: CT-MRI image fusion study made it possible to compare the two modalities directly. MRI localization of the prostate is more accurate than CT, and indicates the distance from cone to apex is 15 mm. CT-MRI image fusion technique provides valuable supplements to CT technology for more precise targeting of the prostate cancer.
---
paper_title: Registration, Matching, and Data Fusion in 2D/3D Medical Imaging: Application to DSA and MRA

paper_content:

This paper deals with a new approach of registration in multimodal imaging. Modalities involved are Digital Subtracted Angiography (DSA, 2D) and Magnetic Resonance Angiography (MRA, 3D). Our approach is an hybrid one, mixing feature and intensity based approaches. This approach is based on the extraction of a anatomical referential common to both MRA and DSA. This step appears to be the “geometric-like” aspect. Then, a high level optimization scheme gives the best registration, using an iconic similarity measure. Several ways of matching planar and tomographic imaging are proposed through superimposition, point to point matching or 3D data fusion. The results obtained prove the methods efficiency in a clinical context.
---
paper_title: IMPROVING MMI WITH ENHANCED-FCM FOR THE FUSION OF BRAIN MR AND SPECT IMAGES

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To overcome these drawbacks of using traditional MMI, a novel scheme, named improved MMI, which is based on fuzzy c-means (FCM) and MMI, is proposed. The experimental results, using MR and SPECT images, to confirm the superior performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: Medical image fusion by wavelet transform modulus maxima

paper_content:

Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.
---
paper_title: Registration and fusion of SPECT, high-resolution MRI, and interventional MRI for thermal ablation of prostate cancer

paper_content:

We are investigating interventional MRI (iMRI) guided radiofrequency thermal ablation for the minimally invasive treatment of the prostate cancer. Nuclear medicine can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of functional images such as nuclear medicine SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high-resolution MRI volume. Then by registering the high-resolution MR image with live-time iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to live-time iMRI images for improved tumor targeting. For the first step, we used a three-dimensional mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm with special features. The concept was tested using image data from three patients and three volunteers. The SV registration accuracy was 0.4 mm /spl plusmn/ 0.2 mm as compared to our volume- to-volume registration that was previously shown to be quite accurate for these image pairs. With our image registration and fusion software, simulation experiments show that it is quite feasible to incorporate SPECT and high-resolution MRI into the iMRI-guided minimally invasive treatment procedures.
---
paper_title: Hybrid surface- and voxel-based registration for MR-PET brain fusion

paper_content:

In this paper, we propose a novel technique of registration using hybrid approach for MR-PET brain image fusion. Hybrid approach uses merits of surface- and voxel-based registration. Thus, our method measures similarities using voxel intensities in MR images corresponding to the feature points of the brain in PET images. Proposed method selects the brain threshold using histogram accumulation ratio in PET images. And then, we automatically segment the brain using the inverse region growing with pre-calculated threshold and extract the feature points of the brain using sharpening filter in PET images. In order to find the optimal location for registration, we evaluate the Hybrid-based Cross-Correlation using the voxel intensities in MR images corresponding to the feature points in PET images. In our experiments, we evaluate our method using software phantom and clinical datasets in the aspect of visual inspection, accuracy, robustness, and computation time. Experimental results show that our method is dramatically faster than the voxel-based registration and more accurate than the surface-based registration. In particular, our method can robustly align two datasets with large geometrical displacement and noise at optimal location.
---
paper_title: Reirradiation of recurrent high-grade gliomas using amino acid PET (SPECT)/CT/MRI image fusion to determine gross tumor volume for stereotactic fractionated radiotherapy.

paper_content:

PURPOSE ::: To develop a valid treatment strategy for recurrent high-grade gliomas using stereotactic hypofractionated reirradiation based on biologic imaging and temozolomide. ::: ::: ::: PATIENTS AND METHODS ::: The trial included a total of 44 patients with recurrent high-grade gliomas (1 patient with anaplastic oligodendroglioma, 8 with anaplastic astrocytoma, 33 with glioblastoma multiforme, and 2 with gliosarcoma) after previous surgery and postoperative conventional radiotherapy +/- chemotherapy. For fractionated stereotactic radiotherapy (SFRT) treatment planning, the gross tumor volume was defined by (11)C-methionine positron emission tomography (MET-PET) or (123)I-alpha-methyl-tyrosine (IMT) single-photon computed emission tomography (SPECT)/computed tomography (CT)/magnetic resonance imaging (MRI) fusion in 82% of the patients and by CT/T1+gadolinium-MRI image fusion in 18% of the patients. Six fractions of 5 Gy were administered in 6 days. In 29 of 44 patients (66%), chemotherapy with temozolomide (200 mg/m(2) body surface/day) was given in one to two cycles before and four to five cycles after SFRT. The patients were evaluated in follow-up by clinical investigators and MRI or CT every 3 months after SFRT until death. In cases suspicious for radiation necrosis, a MET-PET or IMT-SPECT investigation was performed. ::: ::: ::: RESULTS ::: The median survival time in the whole group was 8 months. Treatment planning based on PET(SPECT)/CT/MRI imaging was associated with improved survival in comparison to treatment planning using CT/MRI alone: median survival time 9 months vs. 5 months (p = 0.03, log-rank). Median survival time were 11 months for patients who received SFRT based on biologic imaging plus temozolomide and significantly lower, 6 months for patients treated with SFRT without biologic imaging, without temozolomide or without both (p = 0.008, log rank). The most important prognostic factor in univariate analysis was a long interval between initial diagnosis and recurrence (p = 0.0002, log-rank). In the multivariate model, time interval to retreatment (p = 0.006) and temozolomide (p = 0.04) remained statistically significant. No acute neurologic toxicity Grade 3 or higher and no Grade 4 hematologic toxicity was observed. ::: ::: ::: CONCLUSION ::: This is the first study of biologic imaging optimized SFRT plus temozolomide in recurrent high-grade gliomas. It demonstrates the feasibility and safety of this approach. The most striking result of the trial is the statistically significant longer survival time in the univariate analysis for patients reirradiated using MET-PET or IMT-SPECT/CT/MRI image fusion in the treatment planning, in comparison to patients treated based on MRI/CT alone. Multivariate analysis confirmed a significant survival benefit from multimodal treatment (i.e., addition of temozolomide), despite the limited number of patients. Whether treatment planning with SPECT/PET independently influences survival has to be studied in a larger series of patients.
---
paper_title: The Application of Image Fusion in Stereotactic Brachytherapy of Brain Tumours

paper_content:

Summary Background. The visualization of any morphological volume (i.e. CT, MRI) together with an additional second morphological volume (i.e. CT, MRI) or functional data set, which may come from SPECT or PET, is a new method for treatment planning, verification and follow-up of interstitial irradiation. Method. The authors present their experience on interstitial irradiation of brain tumours with stereotactically implanted I-125 seeds supported by image fusion. The image fusion was performed by the BrainLab-Target 1.13 software on Alfa 430 (Digital) workstation before, during, and after interstitial irradiation of brain tumours with Iodine125 seeds. Results and Interpretation. On the basis of 20 brachytherapeutic image fusion of stereotactic CT (slices with fiducials) with additional stereotactic CT, MRI, PET and SPECT images provides more accurate and precise target volume, more exact localization of catheters and isotope seeds (verification fusion), differentiation between the localization and amount of the necrotic and proliferating parts of the tumours and shows the volume changes in consequence of interstitial irradiation. The image fusion should help to improve the accuracy and minimize the perifocal morbidity of interstitial irradiation.
---
paper_title: CT-MR image data fusion for computer-assisted navigated surgery of orbital tumors.

paper_content:

PURPOSE ::: To demonstrate the value of multidetector computed tomography (MDCT) and magnetic resonance imaging (MRI) in the preoperative assessment of orbital tumors, and to present, particularly, CT and MR image data fusion for surgical planning and performance in computer-assisted navigated surgery of orbital tumors. ::: ::: ::: MATERIALS AND METHODS ::: In this retrospective case series, 10 patients with orbital tumors and associated complaints underwent MDCT and MRI of the orbit. MDCT was performed at high resolution, with a bone window level setting in the axial plane. MRI was performed with an axial 3D T1-weighted (w) gradient-echo (GE) contrast-enhanced sequence, in addition to a standard MRI protocol. First, MDCT and MR images were used to diagnose tumorous lesions compared to histology as a standard of reference. Then, the image data sets from CT and 3D T1-w GE sequences were merged on a workstation to create CT-MR fusion images that were used for interventional planning and intraoperative image guidance. The intraoperative accuracy of the navigation unit was measured, defined as the deviation between the same landmark in the navigation image and the patient. Furthermore, the clinical preoperative status was compared to the patients' postoperative outcome. ::: ::: ::: RESULTS ::: Radiological and histological diagnosis, which revealed 7 benign and 3 malignant tumors, were concordant in 7 of 10 cases (70%). The CT-MR fusion images supported the surgeon in the preoperative planning and improved the surgical performance. The mean intraoperative accuracy of the navigation unit was 1.35mm. Postoperatively, orbital complaints showed complete regression in 6 cases, were ameliorated notably in 3 cases, and remained unchanged in 1 case. ::: ::: ::: CONCLUSION ::: CT and MRI are essential for the preoperative assessment of orbital tumors. CT-MR image data fusion is an accurate tool for planning the correct surgical procedure, and can improve surgical results in computer-assisted navigated surgery of orbital tumors.
---
paper_title: Comparison of CT and MR-CT fusion for prostate post-implant dosimetry.

paper_content:

PURPOSE ::: The use of T2 MR for postimplant dosimetry (PID) after prostate brachytherapy allows more anatomically accurate and precise contouring but does not readily permit seed identification. We developed a reproducible technique for performing MR-CT fusion and compared the resulting dosimetry to standard CT-based PID. ::: ::: ::: METHODS AND MATERIALS ::: CT and T1-weighted MR images for 45 patients were fused and aligned based on seed distribution. The T2-weighted MR image was then fused to the aligned T1. Reproducibility of the fusion technique was tested by inter- and intraobserver variability for 13 patients. Dosimetry was computed for the prostate as a whole and for the prostate divided into anterior and posterior sectors of the base, mid-prostate, and apex. ::: ::: ::: RESULTS ::: Inter- and intraobserver variability for the fusion technique showed less than 1% variation in D90. MR-CT fusion D90 and CT D90 were nearly equivalent for the whole prostate, but differed depending on the identification of superior extent of the base (p = 0.007) and on MR/CT prostate volume ratio (p = 0.03). Sector analysis showed a decrease in MR-CT fusion D90 in the anterior base (ratio 0.93 ±0.25, p < 0.05) and an increase in MR-CT fusion D90 in the apex (p < 0.05). The volume of extraprostatic tissue encompassed by the V100 is greater on MR than CT. Factors associated with this difference are the MR/CT volume ratio (p < 0.001) and the difference in identification of the inferior extent of the apex (p = 0.03). ::: ::: ::: CONCLUSIONS ::: We developed a reproducible MR-CT fusion technique that allows MR-based dosimetry. Comparing the resulting postimplant dosimetry with standard CT dosimetry shows several differences, including adequacy of coverage of the base and conformity of the dosimetry around the apex. Given the advantage of MR-based tissue definition, further study of MR-based dosimetry is warranted.
---
paper_title: 11C-Acetate Positron Emission Tomography Imaging and Image Fusion With Computed Tomography and Magnetic Resonance Imaging in Patients With Recurrent Prostate Cancer

paper_content:

Purpose To assess the clinical value of computed tomography (CT) and magnetic resonance imaging (MRI) image fusion with 11 C-acetate (AC) positron emission tomography (PET) imaging for detection and exact location of clinically occult recurrences. Patients and Methods Fifty prostate cancer patients with elevated/increasing serum prostate-specific antigen levels after radical therapy underwent whole-body AC PET. Uptake was initially interpreted as normal, abnormal, or equivocal. In case of abnormal or equivocal uptake, additional conventional imaging techniques, such as CT, MRI, and bone scans, were performed. To precisely define the anatomic location of abnormal uptake and to improve characterization of equivocal lesions, a softwareassisted image fusion (CT-PET, MRI-PET) was performed and evaluated as site-by-site analysis of 51 abnormal (n 37) or equivocal (n 14) sites of all 50 patients. In 17 patients, additional histopathologic evaluation was available. Results In five (10%), 13 (26%), and 32 (64%) of the 50 patients, AC PET studies demonstrated AC uptake judged as normal, equivocal, and abnormal, respectively. Image fusion changed characterization of equivocal lesions as normal in five (10%) of 51 sites and abnormal in nine (18%) of 51 sites. It precisely defined the anatomic location of abnormal uptake in 37 (73%) of 51 sites. AC PET findings did influence patient management in 14 (28%) of 50 patients.
---
paper_title: Photo-realistic representation of anatomical structures for medical education by fusion of volumetric and surface image data

paper_content:

We have produced improved photo-realistic views of anatomical structures for medical education combining data from photographic images of anatomical surfaces with optical, CT and MRI volumetric data such as provided by the NLM Visible Human Project. Volumetric data contains the information needed to construct 3D geometrical models of anatomical structures, but cannot provide a realistic appearance for surfaces. Nieder has captured high quality photographic sequences of anatomy specimens over a range of rotational angles. These have been assembled into QuickTime VR Object movies that can be viewed statically or dynamically. We reuse this surface imagery to produce textures and surface reflectance maps for 3D anatomy models to allow viewing from any orientation and lighting condition. Because the volumetric data comes from different individuals than the surface images, we have to warp these data into alignment. Currently we do not use structured lighting or other direct 3D surface information, so surface shape is recovered from rotational sequences using silhouettes and texture correlations. The results of this work improves the appearance and generality of models, used for anatomy instruction with the PSC Volume Browser.
---
paper_title: A data fusion approach to tumor delineation

paper_content:

An improved methodology is presented for the localization of tumors for radiation therapy treatment planning applications. The proposed approach focuses on the integration of MRI, CT, and PET imaging data towards the generation of a mathematically optimal tumor boundary. Fuzzy edges derived from CT and MRI are fused to form an integrated edge map which is employed in conjunction with a region based analysis of the PET scan to guide the evolution of a tumor segmenting model. The presented algorithm yields tumor contours which may be employed directly in the radiation therapy treatment planning process.
---
paper_title: Advances in the use of neurophysiologycally-based fusion for visualization and pattern recognition of medical imagery

paper_content:

The ever increasing number of image modaliiies available io dociors for diagnosis purposes has established an imporiani, need io develop techniques ihai suppori work-load reduciion and information maximizaiion. To ihis end, we have improved on an image firsion architeciure jirsi developedfor night vision applicaiions. This iechnique, presenied ai Fusion 2002, uiilizes 30 operaiors io combine volumeiric image seis while maximizing informaiion conieni. In our approach, we have combined ihe use of image fusion and user- dejinedpaiiern recognition wiihin a 30 human-compuier inferface. Here, we present our latest advances iowards enhancing informarion visualization and supporting paiiem recognition. We also report on resulis of applying image firsion across a variety of patient cases. Finally, we have also begun ihe assessmeni of paiiem recognition based on 2D vs. 30 fused image feaiures. Initial results indicaie an advaniage io firsing imagery across all ihree dimensions sa as io take advaniage ofihe volumetric information available in medical daia seis. A description of the system and a number of examples will serve io ilhistraie our ongoing results.
---
paper_title: MRI-CT fusion to assess postbrachytherapy prostate volume and the effects of prolonged edema on dosimetry following transperineal interstitial permanent prostate brachytherapy.

paper_content:

PURPOSE ::: Quality assurance through postplan assessment is an integral part of permanent seed prostate implants. The use of MRI-CT fusion for 1-month postimplant dosimetry permits accurate assessment of prostate volume without seed induced artifact and uncertainties of prostate contour inherent to CT assessments. Routine use of MRI-CT fusion reveals significant prostate edema may persist several weeks. This study evaluates the effect of edema, and its subsequent resolution, on dosimetry. ::: ::: ::: METHODS AND MATERIALS ::: From May 2001 to June 2003, 241 men were treated with (125)I seed implants based on a transrectal ultrasound (TRUS) preplan. Quality assessment was performed at 1 month by CT-MRI fusion using VariSeed software. Over this 24-month period, 29 patients (12%) with residual edema at 1 month (12-60% >TRUS plan volume), had repeat CT-MRI fusion at 2-4 months to reassess volume and dosimetry. Eleven of the 29 had received prior androgen ablation to shrink the prostate preimplant. ::: ::: ::: RESULTS ::: For the entire group (n = 241), mean preimplant prostate volume was 33.7 cc and median postplan dosimetric parameters were: V100, 92.2%; D90, 153 Gy; and V150, 53%. For the 29 patients with prolonged edema, mean preimplant volume was 34.8 cc and 1-month volume was 46.1 cc (p <0.001). Mean volume reduction between 1 and 2 months was 13%. The decrease in prostate volume had a significant effect on dosimetry with median increase between 1 month and 2 months in calculated V100 of 9.5%, V150 22.6%, V200 30.1%, and D90 11.5%. ::: ::: ::: CONCLUSIONS ::: Significant residual edema is seen 1-month postimplant in 12% of prostates and may have a profound effect on dosimetry. Further study is underway to characterize the time course of resolution of the edema, and to perform integral dosimetry based on the changing volume.
---
paper_title: Registration and Fusion Techniques for Medical Images: Demonstration and Evaluation

paper_content:

In this work, we present an integrated system for the registration and fusion of medical images, named «dIaGnosis». It is often necessary to align medical images to illustrate the changes between the data retrieved at different times so as to assess the progress of a disease, or to assess the effectiveness of a treatment. The proposed system supports the storage, retrieval, distribution and presentation of medical images from different modalities, such as CT and MRI, in DICOM format. It also supports multiple examinations per patient and uses parallel processing threads to perform the processing of the acquired three-dimensional (3D) images in almost real time. In this paper, the architecture and the working environment of the implemented system are presented in detail, along with a pilot scenario that demonstrates the system in use. Additionally, the registration and fusion algorithms implemented are presented and evaluated, along with the image processing techniques used for the enhancement of medical images. The contribution of the proposed work is multilayered. It provides automatic matching methods based on both segmented surfaces and on different levels of gray, and it improves the alignment process when there is a relative movement and / or distortion of images in the data collected from different imaging systems.
---
paper_title: Fusion of Spatial Relationships for Guiding Recognition, Example of Brain Structure Recognition in 3D MRI

paper_content:

Spatial relations play an important role in recognition of structures embedded in a complex environment and for reasoning under imprecision. Several types of relationships can be modeled in a unified way using fuzzy mathematical morphology. Their combination benefits from the powerful framework of fuzzy set theory for fusion tasks and decision making. This paper presents several methods of fusion of information about spatial relationships and illustrates them on the example of model-based recognition of brain structures in 3D magnetic resonance imaging.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: The Application of Wavelet Transform to Multi-modality Medical Image Fusion

paper_content:

Medical image fusion has been used to derive useful information from multi-modality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide more information to the doctor and clinical treatment planning system. This paper aims to demonstrate the application of wavelet transformation to multi-modality medical image fusion. This work covers the selection of wavelet function, the use of wavelet based fusion algorithms on medical image fusion of CT and MRI, and the fusion image quality evaluation. We introduce the peak-to-peak signal-to-noise ratio (PSNR) method for measuring fusion effect. The performances of other two methods of image fusion based on pyramid-decomposition and simple image fusion attempts are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme based on wavelet transform
---
paper_title: A framework of fuzzy information fusion for the segmentation of brain tumor tissues on MR images

paper_content:

A framework of fuzzy information fusion is proposed in this paper to automatically segment tumor areas of human brain from multispectral magnetic resonance imaging (MRI) such as T1-weighted, T2-weighted and proton density (PD) images. A priori knowledge about tumors described by radiology experts for different types of MRI are very helpful to guide a automatic and a precise segmentation. However, the terminology used by radiology experts are variable in term of image signal. In order to benefit of these descriptions, we propose to modellize them by fuzzy models. One fuzzy model is built for one type of MRI sequence. The segmentation is finally based on a fusion of different fuzzy information obtained from different types of MRI images. Our algorithm consists of four stages: the registration of multispectral MR images, the creation of fuzzy models describing the characteristics of tumor, the fusion based on fuzzy fusion operators and the adjustment by fuzzy region growing based on fuzzy connecting. The comparison between the obtained results and the hand-tracings of a radiology expert shows that the proposed algorithm is efficient. An average probability of correct detection 96% and an average probability of false detection 5% are obtained through studies of four patients.
---
paper_title: Rapid subthalamic nucleus deep brain stimulation lead placement utilising CT/MRI fusion, microelectrode recording and test stimulation.

paper_content:

Subthalamic nucleus (STN) deep brain stimulation (DBS) has become an established treatment strategy for patients with medically refractory Parkinson’s disease (PD). There are however numerous strategies employed for STN lead placement. Variations include method of STN localisation, use of microelectrode recording, number of microelectrode recording passes and time taken for the procedure. We describe a relatively simple and rapid technique of STN lead placement utilising CT / MRI image fusion, microelectrode recording and test stimulation.
---
paper_title: A surface-projection MMI for the fusion of brain MR and SPECT images

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To improve the performance of traditional MMI, a novel scheme named surface-projection MMI algorithm (SP-MMI) is proposed here. SP-MMI is a two-stage registration algorithm included MMI of surface projection and the following MMI of axial plane. The experimental results, using MR and SPECT images, to confirm the good performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: Investigation of PET/MRI image fusion schemes for enhanced breast cancer diagnosis

paper_content:

The benefit of registration and fusion of functional images with anatomical images is well appreciated in the advent of PET/CT. There is an increasing interest in expanding this approach to PET/MRI. The focus of much of the related research has been on registering images from different modalities. However, the importance of appropriately jointly displaying (i.e. fusing) the registered images has often been neglected and underestimated. Our aim is to determine which fusion techniques are most useful for enhanced diagnostic performance, ease of use, efficiency, and accuracy for reading registered MRI and PET breast images. Preliminary results indicate that the radiologists were better able to perform a series of tasks when reading the fused PET/MRI data sets using color tables generated by our new genetic algorithm, as compared to commonly used fire/gray or red/gray schemes.
---
paper_title: FMDIB: a software tool for fusion of MRI and DHC-SPECT images of Brain

paper_content:

Brain disorders are divided into two categories: anatomic and metabolic. Since brain disorders initially appear as metabolic abnormalities and ultimately result in anatomic defects, it is necessary to analyze the metabolic along with the anatomic aspect of the brain tissue in the preliminary stages of diagnosis to greatly increase the chances of a cure. It is possible that metabolic abnormalities exist without an anatomic defect such as Alzheimer disease. The brain anatomic images may achieve from MRI and physiologic and metabolic brain data can achieve from DHC-SPECT or PET system when a radiopharmaceutical such as FDG have been injected to the patient. For fusion of the brain MRI and DHC-SPECT images obtained from individual scanners, we developed the userfriendly FMDIB software in the MATLAB environment. With the help of this software, we resolved the geometrical misalignment of the brain images and subsequently co-registered them with a 1 mm precision. This software enabled us to perform the three dimensional rendering of the image, re-slicing the image with thickness of 1 to 3 mm, slicing the two dimensional DHC-SPECT projections into an arbitrary direction.
---
paper_title: MR and CT image fusion for postimplant analysis in permanent prostate seed implants.

paper_content:

Purpose ::: To compare the outcome of two different image-based postimplant dosimetry methods in permanent seed implantation. ::: Methods and materials ::: Between October 1999 and October 2002, 150 patients with low-risk prostate carcinoma were treated with 125I and 103Pd in our institution. A CT-MRI image fusion protocol was used in 21 consecutive patients treated with exclusive brachytherapy. The accuracy and reproducibility of the method was calculated, and then the CT-based dosimetry was compared with the CT-MRI–based dosimetry using the dose–volume histogram (DVH) related parameters recommended by the American Brachytherapy Society and the American Association of Physicists in Medicine. ::: Results ::: Our method for CT-MRI image fusion was accurate and reproducible (median shift <1 mm). Differences in prostate volume were found, depending on the image modality used. Quality assurance DVH-related parameters strongly depended on the image modality (CT vs. CT-MRI): V100 = 82% vs. 88%, p < 0.05. D90 = 96% vs. 115%, p < 0.05. Those results depend on the institutional implant technique and reflect the importance of lowering inter- and intraobserver discrepancies when outlining prostate and organs at risk for postimplant dosimetry. ::: Conclusions ::: Computed tomography-MRI fused images allow accurate determination of prostate size, significantly improving the dosimetric evaluation based on DVH analysis. This provides a consistent method to judge a prostate seed implant's quality.
---
paper_title: Generalized Local Priority Based Medical Image Fusion Scheme

paper_content:

Computer aided fusion of multi modality medical images provides a very promising diagnostic tool with numerous clinical applications. The objective of this paper is to present a novel scheme of medical imaging fusion. We define “local to be fused” of the process of image fusion as a generalized local, which extends airspace locals to generalized locals, and promotes the airspace local priority to generalized local priority. Based on this priority, the volume-weighted of token coefficient of images is introduced and weighted so that multi-modal medical image fusion is achieved with high quality. This scheme is applied to CT/MRI datasets and results show the effectiveness of the method.
---
paper_title: MRI- Mammography 2D/3D Data Fusion for Breast Pathology Assessment

paper_content:

Increasing use is being made of contrast-enhanced Magnetic Resonance Imaging (Gd-DTPA) for breast cancer assessment since it provides 3D functional information via pharmacokinetic interaction between contrast agent and tumour vascularity, and because it is applicable to women of all ages. Contrast-enhanced MRI (CE-MRI) is complimentary to conventional X-ray mammography since it is a relatively low-resolution functional counterpart of a comparatively high-resolution 2D structural representation. However, despite the additional information provided by MRI, mammography is still an extremely important diagnostic imaging modality, particularly for several common conditions such as ductal carcinoma in-situ (DCIS) where it has been shown that there is a strong correlation between microcalcification clusters and malignancy [1]. Pathological indicators such as calcifications and fine spiculations are not visible in CE-MRI and therefore there is clinical and diagnostic value to fusing the high-resolution structural information available from mammography with the functional data acquired from MRI imaging. This paper presents a novel data fusion technique whereby medio-lateral (ML) and cranio-caudal (CC) mammograms (2D data) are registered to 3D contrast-enhanced MRI volumes. We utilise a combination of pharmacokinetic modelling, projection geometry, wavelet-based landmark detection and thin-plate spline non-rigid registration to transform the coordinates of regions of interest (ROIs) from the 2D mammograms to the spatial reference frame of the contrast-enhanced MRI volume.
---
paper_title: Image-Fusion Operators for 3D Anatomical and Functional Analysis of the Brain

paper_content:

The goal of the present work is to obtain images combining in a useful and precise way the information present in 3D volumes of medical imaging sets. We address two modalities combining anatomy (Magnetic Resonance Imaging or MRI) and functional information (Positron Emission Tomography or PET). Commercial imaging software often offers image fusion tools based on fixed blending or color- channel combination of different modalities, and various popular color Look-Up Tables (LUTs), without considering the anatomical and functional character of the image features. We used a sensible approach for image fusion, taking advantage mainly from the HSL (Hue, Saturation and Luminosity) color space, in order to enhance the fusion results. We further tested operators for gradient and contour extraction to enhance anatomical details, such as feature borders, plus other spatial- domain filters for functional features, which correspond to wide point-spread-function responses in PET images. A set of image-fusion operators was formulated and tested on brain images from PET and MRI acquisitions.
---
paper_title: Adaptive medical image visualization based on hierarchical neural networks and intelligent decision fusion

paper_content:

An adaptive medical image visualization system based on a hierarchical neural network structure and intelligent decision fusion is presented. It consists of a feature generator using both histogram and spatial information computed from a medical image, a wavelet transform for compressing the feature vector, a competitive layer neural net for clustering images into different subclasses, a bi-modal linear estimator and a RBF network based nonlinear estimator for each subclass, as well as an intelligent decision fusion process to integrate estimates from both estimators. Both estimators can adapt to new types of medical images simply by training them with those images. The large training image set is hierarchically organized for efficient user interaction and effective re-mapping of the width/center settings in the training data. Adaptation capabilities are achieved by modifying the width/center values through a mapping function, which is estimated from the width/center settings of some representative images. While the RBF network based estimator performs well for images similar to those in the training set, the bi-modal linear estimator provides reasonable estimation for a wide range of images. The decision fusion step makes the final estimation of the display parameters accurate for trained images and robust for the unknown images. The algorithm has been tested on a wide range of MR images and shown satisfactory results. Although the current algorithm is very comprehensive, its execution time is kept within reasonable range.
---
paper_title: Evaluation of a Semiautomatic 3D Fusion Technique Applied to Molecular Imaging and MRI Brain/Frame Volume Data Sets

paper_content:

A generally applicable 3D fusion method was evaluated using molecular imaging and MRI volumetric data sets from 15 brain tumor patients with stereotactic frames attached to their skull. Point pairs, placed on the frame only, were chosen, polynomial warping coefficients were generated to map voxels from one coordinate space to the other. The MRI frame was considered the reference structure and the standard for “correct” registration. An ANOVA test (p > 0.05) confirmed the point pair choice to be consistent. The 95% confidence interval for the t-test showed the measured distance difference between the registered volumes was within one MRI voxel. A further experiment was conducted to independently evaluate the brain registration based on testing for consistency of randomly selected interior/exterior points. A t-test result (p < 0.05) showed that the consistency (i.e., both interior or both exterior) before and after volume registration were significantly different. This fusion method may be a viable alternative when other methods fail.
---
paper_title: The impact of image fusion in resolving discrepant findings between FDG-PET and MRI/CT in patients with gynaecological cancers

paper_content:

This study was performed to prospectively investigate the impact of image fusion in resolving discrepant findings between fluorine-18 fluorodeoxyglucose positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) or X-ray computed tomography (CT) in patients with gynaecological cancers. Discrepant findings were defined as lesions where the difference between the FDG-PET and MRI/CT images was assigned a value of at least 2 on a 5-point probability scale. The FDG-PET and MRI/CT images were taken within 1 month of each other. Image fusion between FDG-PET and CT was performed by automatic registration between the two images. During an 18-month period, 34 malignant lesions and seven benign lesions from 32 patients who had undergone either surgical excision or a CT-guided histopathological investigation were included for analysis. Among these cases, image fusion was most frequently required to determine the nature and/or the extent of abdominal and pelvic lesions (28/41, 68%), especially as regards peritoneal seeding (8/41, 20%). Image fusion was most useful in providing better localisation for biopsy (16/41, 39%) and in discriminating between lesions with pathological versus physiological FDG uptake (12/41, 29%). Image fusion changed the original diagnosis based on MRI/CT alone in 9/41 lesions (22%), and the original diagnosis based on FDG-PET alone in 5/41 lesions (12%). It led to alteration of treatment planning (surgery or radiotherapy) in seven of the 32 patients (22%). In patients with gynaecological cancers, the technique of image fusion is helpful in discriminating the nature of FDG-avid lesions, in effectively localising lesions for CT-guided biopsy and in providing better surgical or radiotherapy planning.
---
paper_title: Fusion algorithm of medical images based on fuzzy logic

paper_content:

Computed tomography (CT) and nuclear magnetic resonance imaging (MRI) are complementary on reflecting human body information. In order to provide more useful information for clinical diagnosis, we have a need to fuse the effective information. In the pixel-level fusion between the medical images, we presented a Mamdani-type minimum-sum-mean of maximum (MIN-SUM-MOM) algorithm in this paper. In MIN-SUM-MOM algorithm, fuzzy implication operation utilized MIN algorithm; calculating the membership functions of total output fuzzy sets employed SUM algorithm; defuzzification operation used MOM algorithm. Employ the data of medical image CT and MRI to achieve the fusion simulation, and compare with the simulation results of minimum-maximum-Centroid (MIN-MAX-Centroid) algorithm on the basis of the evaluation standards which are the standard deviation and the information entropy. By the contrast and analysis, we got the following conclusions: the fused images based on MIN-SUM-MOM algorithm not only reserve more texture features, but also enhance the information characteristics of two original images.
---
paper_title: Fusion of MRI and DTI to Assist the Treatment Solution of Brain Tumor

paper_content:

With contrast to conventional MRI, diffusion tensor image (DTI) can demonstrate the anatomic structure and pathological process of white matter tracts and it is widely used by clinician. However, at present, DTI and MRI images are diagnosed respectively and clinician gets the image information objectively, and this leads to different diagnosis conclusions. In this article, we develop an integrated approach that aims to maximize the amount of image information for clinician by combining anatomical and functional MR images with nerve fiber bundles derived from diffusion tensor imaging to assist treatment solution for brain tumor. Firstly, segmentation on MRI T2 is operated to decide solid tumor border or edema area et al; and then DTI and MRI images are fused together, which can supplied not only individual information but also fusion information. This new visualization facility which is intuitive for clinician to understand and use can adequately mine this source of image information. Finally, white matter fibers affected by tumor were analyzed quantitatively. The combined visualization and computational methods have the potential to assist in preoperative surgery and radiotherapy planning as well as postoperative treatment evaluation.
---
paper_title: Image fusion of ultrasound and MRI as an aid for assessing anatomical shifts and improving overview and interpretation in ultrasound-guided neurosurgery

paper_content:

Abstract We have developed an image fusion module for combined 3D ultrasound and MRI/CT visualization in image-guided neuronavigation. The module demonstrates alternative image visualization possibilities, which might be integrated in future ultrasound-based neuronavigation systems. The image fusion visualization module reads 3D ultrasound, MRI/CT image data sets and patient registration information which are saved during the operation using the SonoWand ultrasound-based neuronavigation system earlier presented by our group [1] . The images are visualized simultaneously in the same scene by overlay, compositing or splitting visualization techniques which makes perception of information easier as compared to visualizing images in different scenes. Quantitation of brain shift between similar structures recognized in both preoperative MRI and intraoperative ultrasound images was done using both a manual and an automatic image processing algorithm. The results show that alternative image visualizations may be useful in order to get immediate feedback regarding brain shift in the early beginning of the operation. The present paper summarizes the experiences gained using this visualization module on patient image data acquired in our clinic during the last year using the SonoWand ultrasound-based neuronavigation system.
---
paper_title: CT-MR image data fusion for computer assisted navigated neurosurgery of temporal bone tumors.

paper_content:

PURPOSE ::: To demonstrate the value of multi detector computed tomography (MDCT) and magnetic resonance imaging (MRI) in the preoperative work up of temporal bone tumors and to present, especially, CT and MR image fusion for surgical planning and performance in computer assisted navigated neurosurgery of temporal bone tumors. ::: ::: ::: MATERIALS AND METHODS ::: Fifteen patients with temporal bone tumors underwent MDCT and MRI. MDCT was performed in high-resolution bone window level setting in axial plane. The reconstructed MDCT slice thickness was 0.8 mm. MRI was performed in axial and coronal plane with T2-weighted fast spin-echo (FSE) sequences, un-enhanced and contrast-enhanced T1-weighted spin-echo (SE) sequences, and coronal T1-weighted SE sequences with fat suppression and with 3D T1-weighted gradient-echo (GE) contrast-enhanced sequences in axial plane. The 3D T1-weighted GE sequence had a slice thickness of 1mm. Image data sets of CT and 3D T1-weighted GE sequences were merged utilizing a workstation to create CT-MR fusion images. MDCT and MR images were separately used to depict and characterize lesions. The fusion images were utilized for interventional planning and intraoperative image guidance. The intraoperative accuracy of the navigation unit was measured, defined as the deviation between the same landmark in the navigation image and the patient. ::: ::: ::: RESULTS ::: Tumorous lesions of bone and soft tissue were well delineated and characterized by CT and MR images. The images played a crucial role in the differentiation of benign and malignant pathologies, which consisted of 13 benign and 2 malignant tumors. The CT-MR fusion images supported the surgeon in preoperative planning and improved surgical performance. The mean intraoperative accuracy of the navigation system was 1.25 mm. ::: ::: ::: CONCLUSION ::: CT and MRI are essential in the preoperative work up of temporal bone tumors. CT-MR image data fusion presents an accurate tool for planning the correct surgical procedure and is a benefit for the operational results in computer assisted navigated neurosurgery of temporal bone tumors.
---
paper_title: Visualization of the Cortical Potential Field By Medical Imaging Data Fusion

paper_content:

We describe the visualization of the potential field on the scalp and on the cortical surface. The surfaces are derived from magnetic resonance imaging data and the potential fields are reconstructed from electroencephalography data. The visualization tool is validated with clinical and cognitive application studies.
---
paper_title: Multiplanar MRI–CT fusion neuronavigation-guided serial stereotactic biopsy of human brain tumors: proof of a strong correlation between tumor imaging and histopathology by a new technical approach

paper_content:

Purpose ::: Serial stereotactic biopsy is a diagnostic procedure, used when open biopsy or tumor bulk removal seems to be associated with a too high risk of new neurological deficits in tumors of eloquent regions or tumors of deep localizations or in anticipated high surgery related morbidity even in the older patient group. Shortcomings of this method are recognized to be the missed pathohistological information from untargeted areas in heterogeneous tumors. This study shows for the first time a collection of patients with brain tumors with their associated multiplanar MRI–CT fusion imaging during stereotaxis and the histopathological features of serial tumor biopsies along exact trajectorial sites towards the tumor center.
---
paper_title: Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas

paper_content:

A major limitation of the use of endoscopes in minimally invasive surgery is the lack of relative context between the endoscope and its surroundings. The purpose of this work was to fuse images obtained from a tracked endoscope to surfaces derived from three-dimensional (3-D) preoperative magnetic resonance or computed tomography (CT) data, for assistance in surgical planning, training and guidance. We extracted polygonal surfaces from preoperative CT images of a standard brain phantom and digitized endoscopic video images from a tracked neuro-endoscope. The optical properties of the endoscope were characterized using a simple calibration procedure. Registration of the phantom (physical space) and CT images (preoperative image space) was accomplished using fiducial markers that could be identified both on the phantom and within the images. The endoscopic images were corrected for radial lens distortion and then mapped onto the extracted surfaces via a two-dimensional 2-D to 3-D mapping algorithm. The optical tracker has an accuracy of about 0.3 mm at its centroid, which allows the endoscope tip to be localized to within 1.0 mm. The mapping operation allows multiple endoscopic images to be "painted" onto the 3-D brain surfaces, as they are acquired, in the correct anatomical position. This allows panoramic and stereoscopic visualization, as well as navigation of the 3-D surface, painted with multiple endoscopic views, from arbitrary perspectives.
---
paper_title: Efficient Hybrid Registration Method Using a Shell Volume for PET and High Resolution MR Brain Image Fusion

paper_content:

This paper presents an efficient hybrid registration method using a shell volume that consists of high contrast voxels for combining PET and high resolution MR (HR-MR) brain images. This approach automatically selects a brain shell volume from the PET image, and then transforms only the voxels in the brain shell volume into the coordinate space of HR-MR images. Based on the corresponding voxels in HR-MR images, it finally calculates the best-matching voxel positions using normalized mutual information (NMI). The shell volume reduces the computation time by using smaller number of corresponding voxels to be matched, and it even enables a more robust registration. Experimental results on clinical data sets showed that our method successfully aligned all PET and HR-MR image pairs without losing any diagnostic information, while the conventional NMI method failed to align some cases.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Multiclassifier fusion in human brain MR segmentation: modelling convergence

paper_content:

Segmentations of MR images of the human brain can be generated by propagating an existing atlas label volume to the target image. By fusing multiple propagated label volumes, the segmentation can be improved. We developed a model that predicts the improvement of labelling accuracy and precision based on the number of segmentations used as input. Using a cross-validation study on brain image data as well as numerical simulations, we verified the model. Fit parameters of this model are potential indicators of the quality of a given label propagation method or the consistency of the input segmentations used.
---
paper_title: Evidential segmentation scheme of multi-echo MR images for the detection of brain tumors using neighborhood information

paper_content:

In this paper we propose and study an evidential segmentation scheme of multi-echo MR images for the detection of brain tumors. We show that the modeling by means of evidence theory is well suited to the processing of redundant and complementary data as the MR images. Moreover neighborhood relationship between voxels is taken into account via Dempster's combination rule. We show that using this information improves the classification results previously obtained and leads to a real region-based segmentation. Moreover, the combination of spatial information allows to compute a measure of conflict, which reflects the spatial organization of the data: the conflict is higher at the boundaries between different structures. Thus, it provides a new source of evidence that the specialist can aggregate with the segmentation results to soften its own decision.
---
paper_title: Intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy for lung or liver cancers with a fusion of CT and linear accelerator (FOCAL) unit.

paper_content:

PURPOSE ::: To evaluate intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy (SRT) for lung or liver cancers, we checked repeated CT scanning, with a fusion of CT and linear accelerator (FOCAL) unit. ::: ::: ::: METHODS AND MATERIALS ::: The FOCAL unit is a combination of a linear accelerator (Linac), CT scanner, X-ray simulator (X-S), and carbon table, and is designed to achieve CT-guided SRT with daily CT positioning followed by immediate irradiation while patients keep reduced shallow respirations. To evaluate intrafractional tumor position stability, 50 lung or liver lesions in 20 patients were checked by repeated CT scanning just before and after irradiation, and the obtained images were compared. ::: ::: ::: RESULTS ::: There was no case with the intrafractional error judged to be greater than 10 mm. In 68% of cases, the intrafractional positioning errors were negligible (0-5 mm). ::: ::: ::: CONCLUSIONS ::: Using the FOCAL unit, SRT for lung or liver cancers could be performed with intrafractional positioning errors not greater than 10 mm.
---
paper_title: The fusion of anatomic and physiologic imaging in the management of patients with cancer.

paper_content:

Imaging is of major clinical importance in the noninvasive evaluation and management of patients with cancer. Computed tomography (CT) and other anatomic imaging modalities, such as magnetic resonance imaging (MRI) or ultrasound, have a high diagnostic ability by visualizing lesion morphology and by providing the exact localization of malignant sites. Nuclear medicine provides information on the function and metabolism of cancer. Over the last decade, there have been numerous attempts to combine data obtained from different imaging techniques. Fused images of nuclear medicine and CT (or to a lesser extent, MRI) overcome the inherent limitations of both modalities. Valuable physiologic information benefits from a precise topographic localization. Coregistered data have been shown to be useful in the evaluation of patients with cancer at diagnosis and staging, in monitoring the response to treatment, and during follow up, for early detection of recurrence. Time-consuming and difficult realignment and computation for fusion of independent studies have, until now, limited the use of registration techniques to pilot studies performed in a small number of patients. The development of the new technology of single photon emission computed tomography/CT and positron emission tomography/CT that allows for combined functional and anatomic data acquisition has the potential to make fusion an everyday clinical tool.
---
paper_title: Fusion of Magnetic Resonance and Scintimammography Images for Breast Cancer Evaluation: A Pilot Study

paper_content:

BACKGROUND ::: To achieve a more specific method to estimate the real size of breast cancer, we have developed a method to fuse magnetic resonance imaging (MRI) and scintimammography (SM) images. The aim of this study was to assess the feasibility of this method and to evaluate its accuracy to measure the size of breast cancer compared with MRI alone, mammography, and clinical examination, employing pathologic size as the gold standard. ::: ::: ::: METHODS ::: Twenty consecutive breast cancer women at stages IIA-IIIA, scheduled for mastectomies, underwent SM with (99m)Tc-sestamibi and MRI with gadolinium 2-10 days before surgery. All patients had had recent mammographies and were examined clinically. Software was developed in visual language to perform the fusion between MRI and SM images and tumor measurements (MRI/SM). The tumor size, in 3 diameters (anteroposterior, longitudinal, and transverse), for each examination was correlated with pathological measurements using linear regression. ::: ::: ::: RESULTS ::: The MRI/SM technique was successfully performed in all patients, and the principal tumor was measured by this method. The MRI/SM cancer measurements correlated better with pathology than MRI, mammography, and clinical exam in all diameters analyzed (r = 0.88, 0.81, 0.81; SE = 0.11, 0.14, 0.11 in anteroposterior, longitudinal, and transverse diameters, respectively). ::: ::: ::: CONCLUSIONS ::: The MRI/SM is a feasible technique and appears to be more accurate than other examinations (MRI alone, mammography, and clinical exam) to measure breast cancer size.
---
paper_title: Clinical Value of Image Fusion from MR and PET in Patients with Head and Neck Cancer

paper_content:

Purpose ::: This study aimed to evaluate the clinical value of image fusion from magnetic resonance (MR) combined with positron emission tomography (PET) imaging, using 2-deoxy-2-[F-18]fluoro-D-glucose (FDG) in head and neck cancer.
---
paper_title: Information Fusion in the Immune System

paper_content:

Biologically-inspired methods such as evolutionary algorithms and neural networks are proving useful in the field of information fusion. Artificial Immune Systems (AISs) are a biologically-inspired approach which take inspiration from the biological immune system. Interestingly, recent research has show how AISs which use multi-level information sources as input data can be used to build effective algorithms for real time computer intrusion detection. This research is based on biological information fusion mechanisms used by the human immune system and as such might be of interest to the information fusion community. The aim of this paper is to present a summary of some of the biological information fusion mechanisms seen in the human immune system, and of how these mechanisms have been implemented as AISs
---
paper_title: Multi-modal medical volumes fusion by surface matching

paper_content:

Presents a fast, six degrees of freedom, registration technique to accurately locate the position and orientation of medical volumes (e.g. CT, MRI) with respect to each other for the same patient. The technique uses surface registration and maximization of mutual information. We have developed a novel technique for surface registration which produces highly accurate results when registering two different volumes of the same individual generated from the same modality, such as preoperative MR and intraoperative MR volumes. In case surface registration is not able to accurately register the different volumes, the result is enhanced by multi-modal volume registration. The gain of this combination is to have an accurate alignment and to reduce time needed for registration. For the multi-modal volume registration, the maximization of mutual information (MI) as a matching criterion is used based on genetic algorithms (GA) as a search engine. Our results demonstrate that our registration technique allows for fast, accurate, robust and completely automatic registration of multimodality medical volumes.
---
paper_title: Importance of the CT/MRI fusion method as a learning tool for CT-based postimplant dosimetry in prostate brachytherapy

paper_content:

Abstract Background and purpose To compare the CT-based and CT/MRI fusion-based postimplant dosimetry after permanent prostate brachytherapy and to evaluate the improvement in CT-based dosimetry by physicians with or without experience in using the CT/MRI fusion method. Patients and methods Thirty-eight consecutive patients agreed to participate in a prospective study. The prostate contours from CT/MRI fusion are the gold standard for determining the prostate volume and dose volume histogram (DVH). CT-based postimplant dosimetries were performed by two physicians. Observer 1 was a radiologist who had never used CT/MRI fusion method for postimplant dosimetric analysis. Observer 2 was a radiation oncologist experienced in postimplant analysis using the CT/MRI fusion method. The prostate dosimetry was evaluated by prostate D90 and V100. Results No significant difference was observed in the mean prostate volumes between the two observers and the CT/MRI fusion data. However, the correlation coefficient value for observer 2 ( R 2 =0.932) was greater than that for observer 1 ( R 2 =0.793). The D90 and V100 values as evaluated by the two observers were significantly underestimated in comparison to those evaluated using the CT/MRI fusion methods. The DVH related parameters were underestimated more frequently by observer 1 than by observer 2: (prostate D90: 99.56% for observer 1, 102.97% for observer 2, 109.37% for CT/MRI fusion. Prostate V100: 88.12% for observer 1, 90.14% for observer 2, 91.91% for CT/MRI fusion). Conclusions The difference in the mean value in D90 and V100 by observer 1 was significantly greater than that for observer 2. These findings suggest that the CT/MRI fusion method provides accurate feedback which thereby improves CT-based postimplant dosimetry for prostate brachytherapy.
---
paper_title: Sequential evaluation of prostate edema after permanent seed prostate brachytherapy using CT-MRI fusion

paper_content:

Purpose: To analyze the extent and time course of prostate edema and its effect on dosimetry after permanent seed prostate brachytherapy. Methods and Materials: Twenty patients scheduled for permanent seed 125 I prostate brachytherapy agreed to a prospective study on postimplant edema. Implants were preplanned using transrectal ultrasonography. Postimplant dosimetry was calculated using computed tomography-magnetic resonance imaging (CT-MRI) fusion on the day of the implant (Day 1) and Days 8 and 30. The prostate was contoured on MRI, and the seeds were located on CT. Factors investigated for an influence on edema were the number of seeds and needles, preimplant prostate volume, transitional zone index (transition zone volume divided by prostate volume), age, and prostate-specific antigen level. Prostate dosimetry was evaluated by the percentage of the prostate volume receiving 100% of the prescribed dose (V 100 ) and percentage of prescribed dose received by 90% of the prostate volume (D 90 ). Results: Prostate edema was maximal on Day 1, with the median prostate volume 31% greater than preimplant transrectal ultrasound volume (range, 0.93–1.72; p p = 0.013) and 5% greater on Day 30 ( p p = 0.016) and the preplan prostate volume ( p = 0.003). The median V 100 on Day 1 was 93.6% (range, 86.0–98.2%) and was 96.3% (range, 85.7–99.5%) on Day 30 ( p = 0.079). Patients with a Day 1 V 100 >93% were less affected by edema resolution, showing a median increase in V 100 of 0.67% on Day 30 compared with 2.77% for patients with a V 100 Conclusion: Despite the extreme range of postimplant edema, the effect on dosimetry was less than expected. Dose coverage of the prostate was good for all patients during Days 1–30. Our data indicate that postimplant dosimetry on the day of implant is sufficient for patients with good dose coverage (Day 1 V 100 >93%).
---
paper_title: PET/CT medical image fusion algorithm based on multiwavelet transform

paper_content:

PET/CT medical image fusion has important clinical significance. As the multiwavelet transform has several particular advantages in comparison with scalar wavelets on image processing, this paper proposes a medical image fusion algorithm based on multiwavelet transform after in-depth study of wavelet theory. The algorithm achieves PET/CT fusion with wavelet coefficients fusion method. Experimental results show that fusion image combines information of the source images, adds more details and texture information, and achieves a good fusion result. Based on the proposed algorithm, we can obtain the best result when using gradient fusion in the low-frequency part and classification fusion in the high-frequency part.
---
paper_title: An application of multimodal image registration and fusion in a 3D tumor simulation model

paper_content:

In this paper, a three-dimensional (3D) image registration scheme along with fusion tools is developed and incorporated within a model simulating tumor growth and response to radiotherapy. This biosimulation model attempts to predict and visualize in three dimensions the in vivo tumor behavior. The necessary information for the initialization and the adjustment of the model can be provided from different medical examinations. The present study focuses on the development and evaluation of a technique that enables input information provided from different medical imaging modalities to be efficiently exploited by tumor growth simulation algorithms through the registration and fusion of computed tomography (CT) or positron emission tomography (PET) to magnetic resonance imaging (MRI) data.
---
paper_title: Prostate seed implant quality assessment using MR and CT image fusion.

paper_content:

PURPOSE ::: After a seed implant of the prostate, computerized tomography (CT) is ideal for determining seed distribution but soft tissue anatomy is frequently not well visualized. Magnetic resonance (MR) images soft tissue anatomy well but seed visualization is problematic. We describe a method of fusing CT and MR images to exploit the advantages of both of these modalities when assessing the quality of a prostate seed implant. ::: ::: ::: METHODS AND MATERIALS ::: Eleven consecutive prostate seed implant patients were imaged with axial MR and CT scans. MR and CT images were fused in three dimensions using the Pinnacle 3.0 version of the ADAC treatment planning system. The urethra and bladder base were used to "line up" MR and CT image sets during image fusion. Alignment was accomplished using translation and rotation in the three ortho-normal planes. Accuracy of image fusion was evaluated by calculating the maximum deviation in millimeters between the center of the urethra on axial MR versus CT images. Implant quality was determined by comparing dosimetric results to previously set parameters. ::: ::: ::: RESULTS ::: Image fusion was performed with a high degree of accuracy. When lining up the urethra and base of bladder, the maximum difference in axial position of the urethra between MR and CT averaged 2.5 mm (range 1.3-4.0 mm, SD 0.9 mm). By projecting CT-derived dose distributions over MR images of soft tissue structures, qualitative and quantitative evaluation of implant quality is straightforward. ::: ::: ::: CONCLUSIONS ::: The image-fusion process we describe provides a sophisticated way of assessing the quality of a prostate seed implant. Commercial software makes the process time-efficient and available to any clinical practice with a high-quality treatment planning system. While we use MR to image soft tissue structures, the process could be used with any imaging modality that is able to visualize the prostatic urethra (e.g., ultrasound).
---
paper_title: The role of 3D fusion imaging in sentinel lymphadenectomy for vulvar cancer.

paper_content:

OBJECTIVES ::: To evaluate the feasibility of fusion imaging compound tomography (FICT) of CT/MRI and single photon emission tomography (SPECT) versus planar scintigraphy only (plSc) in pre-surgical staging for vulvar cancer. ::: ::: ::: MATERIALS AND METHODS ::: Analysis of consecutive patients with vulvar cancer who preoperatively underwent sentinel scintigraphy (planar and 3D-SPECT imaging) and CT or MRI. Body markers were used for exact anatomical co-registration and fusion datasets were reconstructed using SPECT and CT/MRI. The number and localisation of all intraoperatively identified and resected sentinel lymph nodes (SLN) were compared between planar and 3D fusion imaging. ::: ::: ::: RESULTS ::: Twenty six SLN were localized on planar scintigraphy. Twelve additional SLN were identified after SPECT and CT/MRI reconstruction, all of them were confirmed intraoperatively. In seven cases where single foci were identified at plSc, fusion imaging revealed grouped individual nodes and five additional localisations were discovered at fusion imaging. In seven patients both methods identified SLN contra lateral to the primary tumor site, but only fusion imaging allowed to localise iliac SLN in four patients. All SLN predicted on fusion imaging could be localised and resected during surgery. ::: ::: ::: CONCLUSIONS ::: Fusion imaging using SPECT and CT/MRI can detect SLN in vulvar cancer more precisely than planar imaging regarding number and anatomical localisation. FICT revealed additional information in seven out of ten cases (70%).
---
paper_title: Rectal morbidity after permanent prostate brachytherapy with dose escalation to biologic target volumes identified by SPECT/CT fusion.

paper_content:

PURPOSE ::: To evaluate rectal morbidity after dose escalation to biologic target volumes identified by capromab pendetide (ProstaScint) single-photon emission tomography images coregistered with computed tomography (SPECT/CT). ::: ::: ::: METHODS AND MATERIALS ::: Two hundred thirty-nine consecutive patients diagnosed with T1c-T3b NxM0 adenocarcinoma of the prostate were treated with brachytherapy seed implant (SI) dose escalation to SPECT/CT-identified biologic target volumes, from February 1997 through December 2002. Patients received SI (n=150) or external beam radiation therapy plus SI (n=89). Rectal morbidity was evaluated by clinician scoring using the modified Radiation Therapy Oncology Group criteria. The median followup was 47.2 (range 24.8-96.1) months. ::: ::: ::: RESULTS ::: The rate of acute Grades I and II toxicity was 29.9% and 3.7%, respectively, and chronic Grade I toxicity was 15.4%, 12.4%, 2.3%, and 1.8% at 1, 2, 3, and 4 years postimplant, respectively. Chronic Grade II toxicities were 1.8%, 1.9%, 1.5%, and 0.9% at 1, 2, 3, and 4 years, respectively. No Grade III rectal toxicity was reported. Chronic Grade IV rectal toxicity was 0.5% and 0.6% at 1.5 and 2.5 years, respectively. Ninety-six percent of patients reported freedom from all rectal toxicity after 3 years. ::: ::: ::: CONCLUSIONS ::: Dose intensification to occult tumor targets without increasing rectal toxicity may be achieved using SPECT/CT ProstaScint. Additional research to define the role of molecular imaging in prostate cancer is warranted.
---
paper_title: Comparison of MRI-based and CT/MRI fusion-based postimplant dosimetric analysis of prostate brachytherapy.

paper_content:

PURPOSE ::: The aim of this study was to compare the outcomes between magnetic resonance imaging (MRI)-based and computed tomography (CT)/MRI fusion-based postimplant dosimetry methods in permanent prostate brachytherapy. ::: ::: ::: METHODS AND MATERIALS ::: Between October 2004 and March 2006, a total of 52 consecutive patients with prostate cancer were treated by brachytherapy, and postimplant dosimetry was performed using CT/MRI fusion. The accuracy and reproducibility were prospectively compared between MRI-based dosimetry and CT/MRI fusion-based dosimetry based on the dose-volume histogram (DVH) related parameters as recommended by the American Brachytherapy Society. ::: ::: ::: RESULTS ::: The prostate volume was 15.97+/-6.17 cc (mean+/-SD) in MRI-based dosimetry, and 15.97+/-6.02 cc in CT/MRI fusion-based dosimetry without statistical difference. The prostate V100 was 94.5% and 93.0% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.002). The prostate D90 was 119.4% and 114.4% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.004). ::: ::: ::: CONCLUSION ::: Our current results suggested that, as with fusion images, MR images allowed accurate contouring of the organs, but they tended to overestimate the analysis of postimplant dosimetry in comparison to CT/MRI fusion images. Although this MRI-based dosimetric discrepancy was negligible, MRI-based dosimetry was acceptable and reproducible in comparison to CT-based dosimetry, because the difference between MRI-based and CT/MRI fusion-based results was smaller than that between CT-based and CT/MRI fusion-based results as previously reported.
---
paper_title: Medical Image of PET/CT Weighted Fusion Based on Wavelet Transform

paper_content:

Image fusion as a kind of information-integrated technology has been used in many fields recently. For medical images, the medical machine of PET/CT makes the within-machine fusion come true. Although the fusion image not only has the character of CT that can make a precise location of pathological changes but also has the character of PET that can detect the pathological as early as possible. As the within-machine fusion equipments are very expensive, it leads a high fee. Only some large hospitals are equipped with the within-machine fusion equipment. So a large space has been left to the development of fusion differences. Based on the wavelet-transform, the fusion of the CT image and PET image comes true. In the end, we gain the fusion CT/PET image. We can detect and locate the pathological changes exactly and immediately in the fusion image. And actually, this is the predominance of the fusion image.
---
paper_title: Effect of edema on postimplant dosimetry in prostate brachytherapy using CT/MRI fusion.

paper_content:

PURPOSE ::: To investigate the time course of prostatic edema and the effect on the dose-volume histograms of the prostate for patients treated with brachytherapy. ::: ::: ::: METHODS AND MATERIALS ::: A total of 74 patients with prostate cancer were enrolled in this prospective study. A transrectal ultrasound-based preplan was performed 4 weeks before implantation and computed tomography/magnetic resonance imaging fusion-based postimplant dosimetry was performed on the day after implantation (Day 1) and 30 days after implantation (Day 30). The prostate volume, prostate volume covered by 100% of the prescription dose (V100), and dose covering 90% of the prostate (D90) were evaluated with prostatic edema over time. ::: ::: ::: RESULTS ::: Prostatic edema was greatest on Day 1, with the mean prostate volume 36% greater than the preplan transrectal ultrasound-based volume; it thereafter decreased over time. It was 9% greater than preplan volume on Day 30. The V(100) increased 5.7% from Day 1 to Day 30, and the D90 increased 13.1% from Day 1 to Day 30. The edema ratio (postplan/preplan) on Day 1 of low-quality implants with a V(100) of <80% was significantly greater than that of intermediate- to high-quality implants (>80% V100; p = 0.0272). The lower V100 on Day 1 showed a greater increase from Day 1 to Day 30. A V100 on Day 1 of >92% is unlikely to increase >0% during the interval studied. ::: ::: ::: CONCLUSION ::: Low-quality implants on Day 1 were highly associated with edema; however, such a low-quality implant on Day 1, with significant edema, tended to improve by Day 30. If a high-quality implant (V100 >92%) can be obtained on Day 1, a re-examination is no longer necessary.
---
paper_title: Additional value of MR/PET fusion compared with PET/CT in the detection of lymph node metastases in cervical cancer patients.

paper_content:

We evaluated the additional diagnostic value of magnetic resonance/positron emission tomography (MR/PET) fusion in the detection of metastatic lymph nodes in cervical cancer patients. Seventy nine patients with FIGO stage IB-IVA cervical cancer who had undergone both magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) before lymphadenectomy were included in this study. Image analysis was first performed with PET/CT images only. A second analysis was then performed with MR/PET fused images that focused on the additional information obtained from the MR images. Lymphadenectomy involved removing all visible lymph nodes in the surgical field. To enable nodal group-specific comparisons, para-aortic and pelvic lymph nodes were divided into seven nodal groups: para-aortic, both common iliac, both external iliac and both internal iliac/obturator areas. Histopathological evaluation of lymph nodes has been the diagnostic standard. The value of the additional information from the MR images was evaluated by means of receiver operating characteristic (ROC) analysis. Fused MR/PET rendered readers to detect six more metastatic lymph node groups. The sensitivity and specificity of PET/CT and fused MR/PET were 44.1%, 93.9% and 54.2%, 92.7% respectively. The ROC analysis demonstrated a higher diagnostic performance of fused MR/PET compared to PET/CT alone for detecting lymph node metastases (p=0.0259). The findings of this study demonstrate the additional diagnostic value of fused MR/PET images compared with PET/CT in the detection of metastatic lymph nodes in patients with uterine cervical cancer.
---
paper_title: An analysis of intraoperative versus post-operative dosimetry with CT, CT-MRI fusion and XMR for the evaluation of permanent prostate brachytherapy implants.

paper_content:

BACKGROUND AND PURPOSE ::: To assess the agreement between intraoperative and post-operative dosimetry and to identify factors that influence dose calculations of prostate brachytherapy implants. ::: ::: ::: MATERIALS AND METHODS ::: Patients treated with prostate brachytherapy implants underwent post-operative CT and XMR (combined X-ray and MR) imaging. Dose-volume histograms were calculated from CT, XMR and CT-MR fusion data and compared with intraoperative values for two observers. Multiple linear regression models assessed the influences of intraoperative D90, gland oedema, gland volume, source loss and migration, and implanted activity/volume prostate on post-operative D90. ::: ::: ::: RESULTS ::: Forty-nine patients were studied. The mean D90 differences (95% confidence limits) between intraoperative and post-operative CT, XMR and CT-MR fusion assessments were: 11 Gy (-22, 45), 18 Gy (-13, 49) and 20 Gy (-17, 58) for Observer 1; and 15 Gy (-34, 63), 13 Gy (-29, 55) and 14 Gy (-27, 54) for Observer 2. Multiple linear regression modelling showed that the observed oedema and intraoperative D90 were significant independent variables for the prediction of post-operative D90 values for both observers using all modalities. ::: ::: ::: CONCLUSION ::: This is the first study to report Bland-Altman agreement analysis between intraoperative and post-operative dosimetry. Agreement is poor. Post-operative dosimetry is dependent on the intraoperative D90 and the subjectively outlined gland volume.
---
paper_title: Fusion imaging using a hybrid SPECT-CT camera improves port perfusion scintigraphy for control of hepatic arterial infusion of chemotherapy in colorectal cancer patients

paper_content:

PURPOSE ::: Exclusive and homogeneous perfusion of the liver is considered essential for the efficacy of hepatic arterial infusion of chemotherapy (HAI). The aim of this study was to evaluate port perfusion scintigraphy in colorectal cancer patients using a hybrid SPECT-CT system for control of minimally invasive intra-arterial port systems within the scope of a phase II trial. ::: ::: ::: METHODS ::: In 24 consecutive patients, the perfusion territories of intra-arterial hepatic port systems were assessed by port scintigraphy with( 99m)Tc-labelled macroaggregated albumin employing planar imaging, SPECT and SPECT-CT (acquired with a hybrid SPECT-CT camera). The results of blinded reading of the scintigraphic modalities concerning the intra- and extrahepatic perfusion pattern were compared with combined image analysis (angiography and contrast-enhanced dedicated CT) and patient history for validation. ::: ::: ::: RESULTS ::: Extrahepatic perfusion was correctly seen in three patients, while suspected extrahepatic perfusion could be excluded in one. In 46 liver lobes, perfusion patterns were correctly visualised by SPECT-CT in 100% of cases (planar, 67%; SPECT, 86%). Assessing the perfusion pattern inside the liver on a segmental basis (segments, n=138), SPECT-CT revealed correct segmental assignment of tracer distribution in 100% and was significantly superior to SPECT alone (accuracy, 84%; p<0.001). The scintigraphic findings resulted in changes in therapeutic management in 8/24 patients (33%); in two of these the relevant findings were visualised only by SPECT-CT. ::: ::: ::: CONCLUSION ::: In patients receiving HAI, port perfusion scintigraphy by fusion imaging with a hybrid SPECT-CT system provides important information for therapy optimisation and appears to be superior to SPECT alone.
---
paper_title: Dual-modality 3D brain PET-CT image segmentation based on probabilistic brain atlas and classification fusion

paper_content:

The increasing prevalence of dual medical imaging modalities, such as PET-CT scanners, poses both challenges and opportunities to image segmentation, as they provide distinct but complementary information. In this paper, we propose a novel segmentation algorithm for 3D brain PET-CT images, which classifies each voxel by fusing the voxel's memberships estimated from four points of view using the PET information, CT information, smoothness prior, and probabilistic brain atlas. All memberships having the same dynamic range greatly facilitates weighting the contribution of the four different information sources. The probabilistic brain atlas estimated for each PET-CT image from a set of training samples provides the anatomical information to the segmentation process. We compared the proposed algorithm to three single-classifier based methods, PET-based SPM algorithm, CT-based Otsu thresholding, and PET-CT based MAP-MRF algorithm. The experimental results in 11 clinical brain PET-CT studies demonstrate that the novel algorithm is capable of providing more accurate and reliable segmentation.
---
paper_title: Variability of gross tumor volume delineation in head-and-neck cancer using PET/CT fusion, Part II: the impact of a contouring protocol.

paper_content:

The purpose of this study was to assess the efficacy of a gross tumor volume (GTV) contouring protocol on interobserver variability between 4 physicians in positron emission therapy/computed tomography (PET/CT) treatment planning of head-and-neck cancer. A GTV contouring protocol for PET/CT treatment planning was developed utilizing 4 stages: Preliminary contouring on CT alone, determination of appropriate PET windowing, accurate image registration, and modification of CT contouring with correctly formatted PET/CT display and rules for modality disagreement. Two neuroradiologists and 2 radiation oncologists (designated as A, B, C, and D, respectively) were given a tutorial of PET/CT coregistered imaging individualized to their skill level, which included a step-by-step explanation of the protocol with clinical examples. Opportunities for questions and hands-on practice were given. The physicians were asked to re-contour 16 head-and-neck patients from Part I on PET/CT fusion imaging. Differences in volume magnitude were analyzed for statistical significance by analysis of variance (ANOVA) and paired t-tests (alpha<0.05). Volume overlap was analyzed for statistical significance using Wilcoxon signed-rank tests (alpha<0.05). Volume overlap increased significantly from Part I to Part II (p<0.05). One previously significant difference between physicians disappeared with the protocol in place. The mean fusion volume of Physician C, however, remained significantly larger than that of Physician D (p<0.01). This result is unchanged from Part I. The multidisciplinary contouring protocol significantly improved the coincidence of GTVs contoured by multiple physicians. The magnitudes of the volumes showed marginal improvement in consistency. Developing an institutional contouring protocol for PET/CT treatment planning is highly recommended to reduce interobserver variability.
---
paper_title: Initial clinical assessment of CT-MRI image fusion software in localization of the prostate for 3D conformal radiation therapy☆

paper_content:

Abstract Purpose: To assess the utility of image fusion software and compare MRI prostate localization with CT localization in patients undergoing 3D conformal radiation therapy of prostate cancer. Materials and Methods: After a phantom study was performed to ensure the accuracy of image fusion procedure, 22 prostate cancer patients had CT and MRI studies before the start of radiotherapy. Immobilization casts used during radiation treatment were also used for both imaging studies. After the clinical target volume (CTV) (prostate or prostate + seminal vesicles) was defined on CT, slices from the MRI study were reconstructed to precisely match the CT slices by identifying three common bony landmarks on each study. The CTV was separately defined on the matched MRI slices. Data related to the size and location of the prostate were compared between CT and MRI. The spatial relationship between the tip of urethrogram cone on CT and prostate apex seen on MRI was also estimated. Results: The phantom study showed the registration discrepancies between CT and MRI smaller than 1.0 mm in any pair in comparison. The patient study showed a mean image registration error of 0.9 (± 0.6) mm. The average prostate volume was 63.0 (± 25.8) cm 2 and 50.9 (± 22.9) cm 3 determined by CT and MRI, respectively. The difference in prostate location with the two studies usually differed at the base and at the apex of the prostate. On the transverse MRI, the prostate apex was situated 7.1 (± 4.5) mm dorsal and 15.1 (± 4.0) mm cephalad to the tip of urethrogram cone. Conclusions: CT-MRI image fusion study made it possible to compare the two modalities directly. MRI localization of the prostate is more accurate than CT, and indicates the distance from cone to apex is 15 mm. CT-MRI image fusion technique provides valuable supplements to CT technology for more precise targeting of the prostate cancer.
---
paper_title: Variability of gross tumor volume delineation in head-and-neck cancer using CT and PET/CT fusion.

paper_content:

PURPOSE ::: To assess the need for gross tumor volume (GTV) delineation protocols in head-and-neck cancer (HNC) treatment planning by use of positron emission tomography (PET)/computed tomography (CT) fusion imaging. Assessment will consist of interobserver and intermodality variation analysis. ::: ::: ::: METHODS AND MATERIALS ::: Sixteen HNC patients were accrued for the study. Four physicians (2 neuroradiologists and 2 radiation oncologists) contoured GTV on 16 patients. Physicians were asked to contour GTV on the basis of the CT alone, and then on PET/CT fusion. Statistical analysis included analysis of variance for interobserver variability and Student's paired sample t test for intermodality and interdisciplinary variability. A Boolean pairwise analysis was included to measure degree of overlap. ::: ::: ::: RESULTS ::: Near-significant variation occurred across physicians' CT volumes (p = 0.09) and significant variation occurred across physicians' PET/CT volumes (p = 0.0002). The Boolean comparison correlates with statistical findings. One radiation oncologist's PET/CT fusion volumes were significantly larger than his CT volumes (p < 0.01). Conversely, the other radiation oncologist's CT volumes tended to be larger than his fusion volumes (p = 0.06). No significant interdisciplinary variation was seen. Significant disagreement occurred between radiation oncologists. ::: ::: ::: CONCLUSION ::: Significant differences in GTV delineation were found between multiple observers contouring on PET/CT fusion. The need for delineation protocol has been confirmed.
---
paper_title: CT image fusion in the optimization of replanning during the course of 3-dimensional conformal radiotherapy for non-small-cell lung cancer

paper_content:

We utilized CT image fusion technique for replanning during the course of three-dimensional conformal radiotherapy (3D-CRT), and determined its dosimetric effects on both target volumes and normal tissues. Ten patients with non-small-cell lung cancer (NSCLC) were studied. The primary treatment plan was generated from the pre-treatment planning CT scan. Two treatment plans were established in the replanning of the 3D-CRT following the mid-treatment CT scan. Treatment plan A was generated from the fused images with the reference of the primary treatment plan; treatment plan B independently from the second CT scan images by the physicists who were blinded to the primary treatment plan. For the planning target volume, there were no significant differences between treatment plan A and B in D 99 and D 95 of the whole treatment. However, the doses to normal structures in the whole treatment, mean lung dose (MLD) of the lung, Dmax of the spinal cord and esophagus in the treatment plan A were significantly lower than those in the treatment plan B. CT image fusion is needed in replanning during the course of 3D-CRT for NSCLC, which may ensure safe doses to normal tissues.
---
paper_title: Medical image fusion by wavelet transform modulus maxima

paper_content:

Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.
---
paper_title: Reirradiation of recurrent high-grade gliomas using amino acid PET (SPECT)/CT/MRI image fusion to determine gross tumor volume for stereotactic fractionated radiotherapy.

paper_content:

PURPOSE ::: To develop a valid treatment strategy for recurrent high-grade gliomas using stereotactic hypofractionated reirradiation based on biologic imaging and temozolomide. ::: ::: ::: PATIENTS AND METHODS ::: The trial included a total of 44 patients with recurrent high-grade gliomas (1 patient with anaplastic oligodendroglioma, 8 with anaplastic astrocytoma, 33 with glioblastoma multiforme, and 2 with gliosarcoma) after previous surgery and postoperative conventional radiotherapy +/- chemotherapy. For fractionated stereotactic radiotherapy (SFRT) treatment planning, the gross tumor volume was defined by (11)C-methionine positron emission tomography (MET-PET) or (123)I-alpha-methyl-tyrosine (IMT) single-photon computed emission tomography (SPECT)/computed tomography (CT)/magnetic resonance imaging (MRI) fusion in 82% of the patients and by CT/T1+gadolinium-MRI image fusion in 18% of the patients. Six fractions of 5 Gy were administered in 6 days. In 29 of 44 patients (66%), chemotherapy with temozolomide (200 mg/m(2) body surface/day) was given in one to two cycles before and four to five cycles after SFRT. The patients were evaluated in follow-up by clinical investigators and MRI or CT every 3 months after SFRT until death. In cases suspicious for radiation necrosis, a MET-PET or IMT-SPECT investigation was performed. ::: ::: ::: RESULTS ::: The median survival time in the whole group was 8 months. Treatment planning based on PET(SPECT)/CT/MRI imaging was associated with improved survival in comparison to treatment planning using CT/MRI alone: median survival time 9 months vs. 5 months (p = 0.03, log-rank). Median survival time were 11 months for patients who received SFRT based on biologic imaging plus temozolomide and significantly lower, 6 months for patients treated with SFRT without biologic imaging, without temozolomide or without both (p = 0.008, log rank). The most important prognostic factor in univariate analysis was a long interval between initial diagnosis and recurrence (p = 0.0002, log-rank). In the multivariate model, time interval to retreatment (p = 0.006) and temozolomide (p = 0.04) remained statistically significant. No acute neurologic toxicity Grade 3 or higher and no Grade 4 hematologic toxicity was observed. ::: ::: ::: CONCLUSION ::: This is the first study of biologic imaging optimized SFRT plus temozolomide in recurrent high-grade gliomas. It demonstrates the feasibility and safety of this approach. The most striking result of the trial is the statistically significant longer survival time in the univariate analysis for patients reirradiated using MET-PET or IMT-SPECT/CT/MRI image fusion in the treatment planning, in comparison to patients treated based on MRI/CT alone. Multivariate analysis confirmed a significant survival benefit from multimodal treatment (i.e., addition of temozolomide), despite the limited number of patients. Whether treatment planning with SPECT/PET independently influences survival has to be studied in a larger series of patients.
---
paper_title: The Application of Image Fusion in Stereotactic Brachytherapy of Brain Tumours

paper_content:

Summary Background. The visualization of any morphological volume (i.e. CT, MRI) together with an additional second morphological volume (i.e. CT, MRI) or functional data set, which may come from SPECT or PET, is a new method for treatment planning, verification and follow-up of interstitial irradiation. Method. The authors present their experience on interstitial irradiation of brain tumours with stereotactically implanted I-125 seeds supported by image fusion. The image fusion was performed by the BrainLab-Target 1.13 software on Alfa 430 (Digital) workstation before, during, and after interstitial irradiation of brain tumours with Iodine125 seeds. Results and Interpretation. On the basis of 20 brachytherapeutic image fusion of stereotactic CT (slices with fiducials) with additional stereotactic CT, MRI, PET and SPECT images provides more accurate and precise target volume, more exact localization of catheters and isotope seeds (verification fusion), differentiation between the localization and amount of the necrotic and proliferating parts of the tumours and shows the volume changes in consequence of interstitial irradiation. The image fusion should help to improve the accuracy and minimize the perifocal morbidity of interstitial irradiation.
---
paper_title: CT-MR image data fusion for computer-assisted navigated surgery of orbital tumors.

paper_content:

PURPOSE ::: To demonstrate the value of multidetector computed tomography (MDCT) and magnetic resonance imaging (MRI) in the preoperative assessment of orbital tumors, and to present, particularly, CT and MR image data fusion for surgical planning and performance in computer-assisted navigated surgery of orbital tumors. ::: ::: ::: MATERIALS AND METHODS ::: In this retrospective case series, 10 patients with orbital tumors and associated complaints underwent MDCT and MRI of the orbit. MDCT was performed at high resolution, with a bone window level setting in the axial plane. MRI was performed with an axial 3D T1-weighted (w) gradient-echo (GE) contrast-enhanced sequence, in addition to a standard MRI protocol. First, MDCT and MR images were used to diagnose tumorous lesions compared to histology as a standard of reference. Then, the image data sets from CT and 3D T1-w GE sequences were merged on a workstation to create CT-MR fusion images that were used for interventional planning and intraoperative image guidance. The intraoperative accuracy of the navigation unit was measured, defined as the deviation between the same landmark in the navigation image and the patient. Furthermore, the clinical preoperative status was compared to the patients' postoperative outcome. ::: ::: ::: RESULTS ::: Radiological and histological diagnosis, which revealed 7 benign and 3 malignant tumors, were concordant in 7 of 10 cases (70%). The CT-MR fusion images supported the surgeon in the preoperative planning and improved the surgical performance. The mean intraoperative accuracy of the navigation unit was 1.35mm. Postoperatively, orbital complaints showed complete regression in 6 cases, were ameliorated notably in 3 cases, and remained unchanged in 1 case. ::: ::: ::: CONCLUSION ::: CT and MRI are essential for the preoperative assessment of orbital tumors. CT-MR image data fusion is an accurate tool for planning the correct surgical procedure, and can improve surgical results in computer-assisted navigated surgery of orbital tumors.
---
paper_title: Comparison of CT and MR-CT fusion for prostate post-implant dosimetry.

paper_content:

PURPOSE ::: The use of T2 MR for postimplant dosimetry (PID) after prostate brachytherapy allows more anatomically accurate and precise contouring but does not readily permit seed identification. We developed a reproducible technique for performing MR-CT fusion and compared the resulting dosimetry to standard CT-based PID. ::: ::: ::: METHODS AND MATERIALS ::: CT and T1-weighted MR images for 45 patients were fused and aligned based on seed distribution. The T2-weighted MR image was then fused to the aligned T1. Reproducibility of the fusion technique was tested by inter- and intraobserver variability for 13 patients. Dosimetry was computed for the prostate as a whole and for the prostate divided into anterior and posterior sectors of the base, mid-prostate, and apex. ::: ::: ::: RESULTS ::: Inter- and intraobserver variability for the fusion technique showed less than 1% variation in D90. MR-CT fusion D90 and CT D90 were nearly equivalent for the whole prostate, but differed depending on the identification of superior extent of the base (p = 0.007) and on MR/CT prostate volume ratio (p = 0.03). Sector analysis showed a decrease in MR-CT fusion D90 in the anterior base (ratio 0.93 ±0.25, p < 0.05) and an increase in MR-CT fusion D90 in the apex (p < 0.05). The volume of extraprostatic tissue encompassed by the V100 is greater on MR than CT. Factors associated with this difference are the MR/CT volume ratio (p < 0.001) and the difference in identification of the inferior extent of the apex (p = 0.03). ::: ::: ::: CONCLUSIONS ::: We developed a reproducible MR-CT fusion technique that allows MR-based dosimetry. Comparing the resulting postimplant dosimetry with standard CT dosimetry shows several differences, including adequacy of coverage of the base and conformity of the dosimetry around the apex. Given the advantage of MR-based tissue definition, further study of MR-based dosimetry is warranted.
---
paper_title: 11C-Acetate Positron Emission Tomography Imaging and Image Fusion With Computed Tomography and Magnetic Resonance Imaging in Patients With Recurrent Prostate Cancer

paper_content:

Purpose To assess the clinical value of computed tomography (CT) and magnetic resonance imaging (MRI) image fusion with 11 C-acetate (AC) positron emission tomography (PET) imaging for detection and exact location of clinically occult recurrences. Patients and Methods Fifty prostate cancer patients with elevated/increasing serum prostate-specific antigen levels after radical therapy underwent whole-body AC PET. Uptake was initially interpreted as normal, abnormal, or equivocal. In case of abnormal or equivocal uptake, additional conventional imaging techniques, such as CT, MRI, and bone scans, were performed. To precisely define the anatomic location of abnormal uptake and to improve characterization of equivocal lesions, a softwareassisted image fusion (CT-PET, MRI-PET) was performed and evaluated as site-by-site analysis of 51 abnormal (n 37) or equivocal (n 14) sites of all 50 patients. In 17 patients, additional histopathologic evaluation was available. Results In five (10%), 13 (26%), and 32 (64%) of the 50 patients, AC PET studies demonstrated AC uptake judged as normal, equivocal, and abnormal, respectively. Image fusion changed characterization of equivocal lesions as normal in five (10%) of 51 sites and abnormal in nine (18%) of 51 sites. It precisely defined the anatomic location of abnormal uptake in 37 (73%) of 51 sites. AC PET findings did influence patient management in 14 (28%) of 50 patients.
---
paper_title: Photo-realistic representation of anatomical structures for medical education by fusion of volumetric and surface image data

paper_content:

We have produced improved photo-realistic views of anatomical structures for medical education combining data from photographic images of anatomical surfaces with optical, CT and MRI volumetric data such as provided by the NLM Visible Human Project. Volumetric data contains the information needed to construct 3D geometrical models of anatomical structures, but cannot provide a realistic appearance for surfaces. Nieder has captured high quality photographic sequences of anatomy specimens over a range of rotational angles. These have been assembled into QuickTime VR Object movies that can be viewed statically or dynamically. We reuse this surface imagery to produce textures and surface reflectance maps for 3D anatomy models to allow viewing from any orientation and lighting condition. Because the volumetric data comes from different individuals than the surface images, we have to warp these data into alignment. Currently we do not use structured lighting or other direct 3D surface information, so surface shape is recovered from rotational sequences using silhouettes and texture correlations. The results of this work improves the appearance and generality of models, used for anatomy instruction with the PSC Volume Browser.
---
paper_title: A data fusion approach to tumor delineation

paper_content:

An improved methodology is presented for the localization of tumors for radiation therapy treatment planning applications. The proposed approach focuses on the integration of MRI, CT, and PET imaging data towards the generation of a mathematically optimal tumor boundary. Fuzzy edges derived from CT and MRI are fused to form an integrated edge map which is employed in conjunction with a region based analysis of the PET scan to guide the evolution of a tumor segmenting model. The presented algorithm yields tumor contours which may be employed directly in the radiation therapy treatment planning process.
---
paper_title: Impact of computed tomography and 18F-deoxyglucose coincidence detection emission tomography image fusion for optimization of conformal radiotherapy in non-small-cell lung cancer.

paper_content:

PURPOSE ::: To report a retrospective study concerning the impact of fused 18F-fluoro-deoxy-D-glucose (FDG)-hybrid positron emission tomography (PET) and CT images on three-dimensional conformal radiotherapy planning for patients with non-small-cell lung cancer. ::: ::: ::: METHODS AND MATERIALS ::: A total of 101 patients consecutively treated for Stage I-III non-small-cell lung cancer were studied. Each patient underwent CT and FDG-hybrid PET for simulation treatment in the same treatment position. Images were coregistered using five fiducial markers. Target volume delineation was initially performed on the CT images, and the corresponding FDG-PET data were subsequently used as an overlay to the CT data to define the target volume. ::: ::: ::: RESULTS ::: 18F-fluoro-deoxy-D-glucose-PET identified previously undetected distant metastatic disease in 8 patients, making them ineligible for curative conformal radiotherapy (1 patient presented with some positive uptake corresponding to concomitant pulmonary tuberculosis). Another patient was ineligible for curative treatment because the fused PET-CT images demonstrated excessively extensive intrathoracic disease. The gross tumor volume (GTV) was decreased by CT-PET image fusion in 21 patients (23%) and was increased in 24 patients (26%). The GTV reduction was > or = 25% in 7 patients because CT-PET image fusion reduced the pulmonary GTV in 6 patients (3 patients with atelectasis) and the mediastinal nodal GTV in 1 patient. The GTV increase was > or = 25% in 14 patients owing to an increase in the pulmonary GTV in 11 patients (4 patients with atelectasis) and detection of occult mediastinal lymph node involvement in 3 patients. Of 81 patients receiving a total dose of > or = 60 Gy at the International Commission on Radiation Units and Measurements point, after CT-PET image fusion, the percentage of total lung volume receiving >20 Gy increased in 15 cases and decreased in 22. The percentage of total heart volume receiving >36 Gy increased in 8 patients and decreased in 14. The spinal cord volume receiving at least 45 Gy (2 patients) decreased. Multivariate analysis showed that tumor with atelectasis was the single independent factor that resulted in a significant effect on the modification of the size of the GTV by FDG-PET: tumor with atelectasis (with vs. without atelectasis, p = 0.0001). ::: ::: ::: CONCLUSION ::: The results of our study have confirmed that integrated hybrid PET/CT in the treatment position and coregistered images have an impact on treatment planning and management of non-small-cell lung cancer. However, FDG images using dedicated PET scanners and respiration-gated acquisition protocols could improve the PET-CT image coregistration. Furthermore, the impact on treatment outcome remains to be demonstrated.
---
paper_title: Software-based Fusion of PET and CT Images for Suspected Recurrent Lung Cancer

paper_content:

PURPOSE ::: The purpose of this study was to compare the diagnostic performance of the manual fusion of positron emission tomography (PET) and computed tomography (CT) images with that of CT alone and that of side-by-side PET and CT (PET/CT) in patients with suspected recurrent lung cancer. ::: ::: ::: PROCEDURES ::: Fifty-three patients who had previously had surgery for lung cancer underwent a whole-body 2-deoxy-2-[F-18]fluoro-D-glucose (FDG)-PET scan, followed by a diagnostic CT scan. The PET and CT images were fused on a workstation. CT alone, PET/CT, and fused images were evaluated separately using a five-point grading scale (0=definitely negative, 1=probably negative, 2=equivocal, 3=probably positive, and 4=definitely positive). Lesions of grade 3 or 4 were considered positive, and diagnostic accuracy and certainty were evaluated. ::: ::: ::: RESULTS ::: Overall, 67 lesions in 33 patients were considered true positive pathologically or clinically. Of these 67 lesions, the evaluation of CT, PET/CT, and fused images detected 46, 55, and 66 lesions, respectively, with the number of grade 4 lesions detected being 38, 50, and 63, respectively. The diagnostic accuracy of CT, PET/CT, and fused images according to patients was 75%, 79%, and 87%, respectively. ::: ::: ::: CONCLUSION ::: These results suggest that interpreting fused images increased diagnostic certainty for detecting recurrence and provided more accurate diagnoses.
---
paper_title: Intermodality registration and fusion of liver images for medical diagnosis

paper_content:

SPECT is valuable for the evaluation of hepatic function and CT can offer better anatomical images for liver and its adjacent structures. Image registration from the two modalities, CT and SPECT, acquires both structural and functional information, and therefore is very useful for clinical diagnoses. The registration and fusion of CT and SPECT images are studied. We propose a two stage registration method based on principal axes transformation and surface fitting to register and fuse CT and SPECT images: First, the centroids and principal axes of these landmarks are calculated and aligned. Then the transformation matrix between CT and SPECT images is modified based on the least square error on the distance image. After registration, uniform volume data is reconstructed for image fusion.
---
paper_title: MRI-CT fusion to assess postbrachytherapy prostate volume and the effects of prolonged edema on dosimetry following transperineal interstitial permanent prostate brachytherapy.

paper_content:

PURPOSE ::: Quality assurance through postplan assessment is an integral part of permanent seed prostate implants. The use of MRI-CT fusion for 1-month postimplant dosimetry permits accurate assessment of prostate volume without seed induced artifact and uncertainties of prostate contour inherent to CT assessments. Routine use of MRI-CT fusion reveals significant prostate edema may persist several weeks. This study evaluates the effect of edema, and its subsequent resolution, on dosimetry. ::: ::: ::: METHODS AND MATERIALS ::: From May 2001 to June 2003, 241 men were treated with (125)I seed implants based on a transrectal ultrasound (TRUS) preplan. Quality assessment was performed at 1 month by CT-MRI fusion using VariSeed software. Over this 24-month period, 29 patients (12%) with residual edema at 1 month (12-60% >TRUS plan volume), had repeat CT-MRI fusion at 2-4 months to reassess volume and dosimetry. Eleven of the 29 had received prior androgen ablation to shrink the prostate preimplant. ::: ::: ::: RESULTS ::: For the entire group (n = 241), mean preimplant prostate volume was 33.7 cc and median postplan dosimetric parameters were: V100, 92.2%; D90, 153 Gy; and V150, 53%. For the 29 patients with prolonged edema, mean preimplant volume was 34.8 cc and 1-month volume was 46.1 cc (p <0.001). Mean volume reduction between 1 and 2 months was 13%. The decrease in prostate volume had a significant effect on dosimetry with median increase between 1 month and 2 months in calculated V100 of 9.5%, V150 22.6%, V200 30.1%, and D90 11.5%. ::: ::: ::: CONCLUSIONS ::: Significant residual edema is seen 1-month postimplant in 12% of prostates and may have a profound effect on dosimetry. Further study is underway to characterize the time course of resolution of the edema, and to perform integral dosimetry based on the changing volume.
---
paper_title: CT image fusion in the evaluation of radiation treatment planning for non-small cell lung cancer

paper_content:

Objective ::: We studied the application of CT image fusion in the evaluation of radiation treatment planning for non-small cell lung cancer (NSCLC).
---
paper_title: The Application of Wavelet Transform to Multi-modality Medical Image Fusion

paper_content:

Medical image fusion has been used to derive useful information from multi-modality medical image data. The idea is to improve the image content by fusing images like computer tomography (CT) and magnetic resonance imaging (MRI) images, so as to provide more information to the doctor and clinical treatment planning system. This paper aims to demonstrate the application of wavelet transformation to multi-modality medical image fusion. This work covers the selection of wavelet function, the use of wavelet based fusion algorithms on medical image fusion of CT and MRI, and the fusion image quality evaluation. We introduce the peak-to-peak signal-to-noise ratio (PSNR) method for measuring fusion effect. The performances of other two methods of image fusion based on pyramid-decomposition and simple image fusion attempts are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme based on wavelet transform
---
paper_title: Rapid subthalamic nucleus deep brain stimulation lead placement utilising CT/MRI fusion, microelectrode recording and test stimulation.

paper_content:

Subthalamic nucleus (STN) deep brain stimulation (DBS) has become an established treatment strategy for patients with medically refractory Parkinson’s disease (PD). There are however numerous strategies employed for STN lead placement. Variations include method of STN localisation, use of microelectrode recording, number of microelectrode recording passes and time taken for the procedure. We describe a relatively simple and rapid technique of STN lead placement utilising CT / MRI image fusion, microelectrode recording and test stimulation.
---
paper_title: MR and CT image fusion for postimplant analysis in permanent prostate seed implants.

paper_content:

Purpose ::: To compare the outcome of two different image-based postimplant dosimetry methods in permanent seed implantation. ::: Methods and materials ::: Between October 1999 and October 2002, 150 patients with low-risk prostate carcinoma were treated with 125I and 103Pd in our institution. A CT-MRI image fusion protocol was used in 21 consecutive patients treated with exclusive brachytherapy. The accuracy and reproducibility of the method was calculated, and then the CT-based dosimetry was compared with the CT-MRI–based dosimetry using the dose–volume histogram (DVH) related parameters recommended by the American Brachytherapy Society and the American Association of Physicists in Medicine. ::: Results ::: Our method for CT-MRI image fusion was accurate and reproducible (median shift <1 mm). Differences in prostate volume were found, depending on the image modality used. Quality assurance DVH-related parameters strongly depended on the image modality (CT vs. CT-MRI): V100 = 82% vs. 88%, p < 0.05. D90 = 96% vs. 115%, p < 0.05. Those results depend on the institutional implant technique and reflect the importance of lowering inter- and intraobserver discrepancies when outlining prostate and organs at risk for postimplant dosimetry. ::: Conclusions ::: Computed tomography-MRI fused images allow accurate determination of prostate size, significantly improving the dosimetric evaluation based on DVH analysis. This provides a consistent method to judge a prostate seed implant's quality.
---
paper_title: Fusion Image of Positron Emission Tomography and Computed Tomography for the Diagnosis of Local Recurrence of Rectal Cancer

paper_content:

BACKGROUND ::: The aim of this study was to evaluate the clinical and therapeutic value of digital fusion image (FI) of positron emission tomography (PET) using (18)F-fluorodeoxy glucose and computed tomography (CT) in patients who were suspected of having a local recurrence of rectal cancer. ::: ::: ::: METHODS ::: Forty-two patients (32 men and 10 women; mean age, 61.4 years, range, 40-79 years) with a suspicion of local recurrence after curative resection of rectal cancer were prospectively recruited and underwent (18)F-fluorodeoxy glucose-PET and CT. The FI was reconstructed with a commercially available digital software program, T-B Fusion. Wilcoxon signed rank test was used to compare FI with CT alone or PET alone. ::: ::: ::: RESULTS ::: FI yielded a correct diagnosis in 39 (93%) of 42 patients, whereas CT alone and PET alone did so in 33 (79%) and 37 (88%) patients, respectively. FI had better diagnostic accuracy than CT alone (P = .0138) and PET alone (P = .0156). Overall, FI altered patient management in 11 (26.2%) patients on the basis of additional information, including differentiation of the tumor from the postoperative scar in 6 patients, exact anatomical location in 3 patients, and both in 2 patients. ::: ::: ::: CONCLUSIONS ::: FI has a potential clinical value in the treatment of suspected local recurrence of rectal cancer.
---
paper_title: Potential advantage of studying the lymphatic drainage by sentinel node technique and SPECT-CT image fusion for pelvic irradiation of prostate cancer

paper_content:

Purpose: This study aims to investigate the in vivo drainage of lymphatic spread by using the sentinel node (SN) technique and single-photon emission computed tomography (SPECT)-computed tomography (CT) image fusion, and to analyze the impact of such information on conformal pelvic irradiation. Methods and Materials: Twenty-three prostate cancer patients, candidates for radical prostatectomy already included in a trial studying the SN technique, were enrolled. CT and SPECT images were obtained after intraprostate injection of 115 MBq of 99m Tc-nanocolloid, allowing identification of SN and other pelvic lymph nodes. Target and nontarget structures, including lymph nodes identified by SPECT, were drawn on SPECT-CT fusion images. A three-dimensional conformal treatment plan was performed for each patient. Results: Single-photon emission computed tomography lymph nodal uptake was detected in 20 of 23 cases (87%). The SN was inside the pelvic clinical target volume (CTV 2 ) in 16 of 20 cases (80%) and received no less than the prescribed dose in 17 of 20 cases (85%). The most frequent locations of SN outside the CTV 2 were the common iliac and presacral lymph nodes. Sixteen of the 32 other lymph nodes (50%) identified by SPECT were found outside the CTV 2 . Overall, the SN and other intrapelvic lymph nodes identified by SPECT were not included in the CTV 2 in 5 of 20 (25%) patients. Conclusions: The study of lymphatic drainage can contribute to a better knowledge of the in vivo potential pattern of lymph node metastasis in prostate cancer and can lead to a modification of treatment volume with consequent optimization of pelvic irradiation.
---
paper_title: PET-CT fusion in radiation management of patients with anorectal tumors.

paper_content:

PURPOSE ::: To compare computed tomography (CT) with positron emission tomography-CT (PET-CT) scans with respect to anorectal tumor volumes, correlation in overlap, and influence on radiation treatment fields and patient care. ::: ::: ::: PATIENTS AND METHODS ::: From March to November 2003, 20 patients with rectal cancer and 3 patients with anal cancer were treated with preoperative or definitive chemoradiation, respectively. Computed tomography simulation data generated a CT gross tumor volume (CT-GTV) and CT planning target volume (CT-PTV) and (18)F-fluoro-2-deoxy-glucose PET (FDG-PET) created a PET-GTV and PET-PTV. The PET-CT and CT images were fused using manual coregistration. Patients were treated with three-dimensional conformal therapy to traditional doses. The PET, CT, and overlap volumes (OVs) were measured in cubic centimeters. ::: ::: ::: RESULTS ::: Mean PET-GTV was smaller than the mean CT-GTV (91.7 vs. 99.6 cm(3)). The mean OV was 46.7%. As tumor volume increased, PET and CT OV correlated significantly (p < 0.001). In 17% of patients PET-CT altered the PTV, and in 26% it changed the radiation treatment plan. For 25% of patients with rectal cancer, PET detected distant metastases and changed overall management. Ten rectal cancer patients underwent surgery. When the pretreatment PET standardized uptake value was >10 and the posttreatment PET standardized uptake value was <6, 100% achieved pathologic downstaging (p = 0.047). ::: ::: ::: CONCLUSIONS ::: Variation in volume was significant, with 17% and 26% of patients requiring a change in treatment fields and patient management, respectively. Positron emission tomography can change the management for anorectal tumors by early detection of metastatic disease or disease outside standard radiation fields.
---
paper_title: Design of a simple ultrasound/CT fusion image fusion solution for the evaluation of prostate seed brachytherapy

paper_content:

The purpose of this project is to devise a solution that will enable physicians, medical physicists and dosimetrists to quantitatively evaluate the success of prostate brachytherapy procedures. It is proposed that a simple ultrasound/CT image fusion solution can be developed and implemented to improve the current post-surgical evaluation process.
---
paper_title: The impact of image fusion in resolving discrepant findings between FDG-PET and MRI/CT in patients with gynaecological cancers

paper_content:

This study was performed to prospectively investigate the impact of image fusion in resolving discrepant findings between fluorine-18 fluorodeoxyglucose positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) or X-ray computed tomography (CT) in patients with gynaecological cancers. Discrepant findings were defined as lesions where the difference between the FDG-PET and MRI/CT images was assigned a value of at least 2 on a 5-point probability scale. The FDG-PET and MRI/CT images were taken within 1 month of each other. Image fusion between FDG-PET and CT was performed by automatic registration between the two images. During an 18-month period, 34 malignant lesions and seven benign lesions from 32 patients who had undergone either surgical excision or a CT-guided histopathological investigation were included for analysis. Among these cases, image fusion was most frequently required to determine the nature and/or the extent of abdominal and pelvic lesions (28/41, 68%), especially as regards peritoneal seeding (8/41, 20%). Image fusion was most useful in providing better localisation for biopsy (16/41, 39%) and in discriminating between lesions with pathological versus physiological FDG uptake (12/41, 29%). Image fusion changed the original diagnosis based on MRI/CT alone in 9/41 lesions (22%), and the original diagnosis based on FDG-PET alone in 5/41 lesions (12%). It led to alteration of treatment planning (surgery or radiotherapy) in seven of the 32 patients (22%). In patients with gynaecological cancers, the technique of image fusion is helpful in discriminating the nature of FDG-avid lesions, in effectively localising lesions for CT-guided biopsy and in providing better surgical or radiotherapy planning.
---
paper_title: Integrated PET/CT Fusion Imaging and Endoscopic Ultrasound in the Pre-operative Staging and Evaluation of Esophageal Cancer

paper_content:

PURPOSE ::: Accurate staging of esophageal cancer (ECA) is critical in determining appropriate therapy. Endoscopic ultrasound (EUS), computed tomography (CT) and positron emission tomography (PET) scanning can be used, but limited data exists regarding the use of combined PET/CT fusion imaging and EUS in ECA staging. The objective of this study is to evaluate the role of integrated PET/CT imaging and EUS in the staging of ECA. ::: ::: ::: PROCEDURES ::: Identification of patients diagnosed with ECA from 2004 to 2007 that underwent staging PET/CT and EUS. Data regarding tumor detection, lymph node identification, presence of metastatic disease, and affect on patient management were collected and compared between PET/CT and EUS. ::: ::: ::: RESULTS ::: Eighty-one patients (65 male, 16 female) were identified with mean age of 63.5 years who underwent EUS and PET/CT to stage known ECA. PET/CT identified the primary tumor in 74/81 (91.4%) of cases, compared to 81/81 (100%) with EUS. Locoregional adenopathy was seen by PET/CT in 29/81 (35.8%) of cases, compared to 49/81 (60.5%) by EUS (p = 0.0001). PET/CT identified celiac axis adenopathy in 8/81 (9.9%) of cases, compared to 11/81 (13.6%) with EUS (p = 0.5050). PET/CT identified 17/81 (21.0%) of patients with distant metastases who subsequently did not undergo attempt at curative surgical resection. ::: ::: ::: CONCLUSIONS ::: In ECA, EUS is superior to PET/CT for T staging and in identifying locoregional nodes, while PET/CT provides M staging. EUS and integrated PET/CT appear to independently affect treatment decisions, indicating complimentary and necessary roles in the staging of ECA.
---
paper_title: Fusion algorithm of medical images based on fuzzy logic

paper_content:

Computed tomography (CT) and nuclear magnetic resonance imaging (MRI) are complementary on reflecting human body information. In order to provide more useful information for clinical diagnosis, we have a need to fuse the effective information. In the pixel-level fusion between the medical images, we presented a Mamdani-type minimum-sum-mean of maximum (MIN-SUM-MOM) algorithm in this paper. In MIN-SUM-MOM algorithm, fuzzy implication operation utilized MIN algorithm; calculating the membership functions of total output fuzzy sets employed SUM algorithm; defuzzification operation used MOM algorithm. Employ the data of medical image CT and MRI to achieve the fusion simulation, and compare with the simulation results of minimum-maximum-Centroid (MIN-MAX-Centroid) algorithm on the basis of the evaluation standards which are the standard deviation and the information entropy. By the contrast and analysis, we got the following conclusions: the fused images based on MIN-SUM-MOM algorithm not only reserve more texture features, but also enhance the information characteristics of two original images.
---
paper_title: Incremental Value of 111-In Pentetreotide SPECT/CT Fusion Imaging of Neuroendocrine Tumors

paper_content:

Rationale and Objectives Hybrid single photon-emission computed tomographic (SPECT) and computed tomographic (CT) imaging for the investigation of neuroendocrine tumors allows the fusion of functional and anatomic information in a rapid and efficient method. The aim of this study was to assess the incremental diagnostic value of 111 In pentetreotide SPECT/CT imaging compared with traditional planar and SPECT imaging with respect to lesion localization and characterization and reader confidence. Materials and Methods Forty-nine patients (23 male, 26 female; mean age, 56.9 years; range, 14–88 years) who underwent 111 In pentetreotide planar, SPECT, and SPECT/CT imaging were eligible for this retrospective study, including patients with suspected or confirmed carcinoid tumors ( n = 24), endocrine pancreatic tumors ( n = 18), medullary thyroid cancer ( n = 3), paragangliomas ( n = 2), and multiple endocrine neoplasia type I ( n = 2). Planar and SPECT images were reviewed by two blinded readers, followed by interpretation using additional SPECT/CT images in a subsequent session. A third reader provided consensus in cases with disagreements. Results In 55 of 89 lesions (61.8%), 111 In pentetreotide SPECT/CT imaging improved lesion localization compared to planar and SPECT imaging; in 25 of 89 lesions (28.1%), SPECT/CT imaging changed lesion classification. In 20 of 49 patients (40.8%) for reader 1 and 14 of 49 patients (28.6%) for reader 2, 111 In pentetreotide SPECT/CT imaging provided incremental diagnostic value, which was considered likely to affect patient management in twelve of 20 and seven of 14 patients, respectively. Increased reader confidence was found in 32 of 49 patients (65.3%) for both readers with uniformly high confidence after SPECT/CT interpretation. Conclusions Hybrid 111 In pentetreotide SPECT/CT imaging provides incremental diagnostic value and greater reader confidence over planar and SPECT imaging. This is achieved though superior lesion localization, the identification of physiologic activity, and additional anatomic information derived from the nondiagnostic CT portion of the study.
---
paper_title: CT-MR image data fusion for computer assisted navigated neurosurgery of temporal bone tumors.

paper_content:

PURPOSE ::: To demonstrate the value of multi detector computed tomography (MDCT) and magnetic resonance imaging (MRI) in the preoperative work up of temporal bone tumors and to present, especially, CT and MR image fusion for surgical planning and performance in computer assisted navigated neurosurgery of temporal bone tumors. ::: ::: ::: MATERIALS AND METHODS ::: Fifteen patients with temporal bone tumors underwent MDCT and MRI. MDCT was performed in high-resolution bone window level setting in axial plane. The reconstructed MDCT slice thickness was 0.8 mm. MRI was performed in axial and coronal plane with T2-weighted fast spin-echo (FSE) sequences, un-enhanced and contrast-enhanced T1-weighted spin-echo (SE) sequences, and coronal T1-weighted SE sequences with fat suppression and with 3D T1-weighted gradient-echo (GE) contrast-enhanced sequences in axial plane. The 3D T1-weighted GE sequence had a slice thickness of 1mm. Image data sets of CT and 3D T1-weighted GE sequences were merged utilizing a workstation to create CT-MR fusion images. MDCT and MR images were separately used to depict and characterize lesions. The fusion images were utilized for interventional planning and intraoperative image guidance. The intraoperative accuracy of the navigation unit was measured, defined as the deviation between the same landmark in the navigation image and the patient. ::: ::: ::: RESULTS ::: Tumorous lesions of bone and soft tissue were well delineated and characterized by CT and MR images. The images played a crucial role in the differentiation of benign and malignant pathologies, which consisted of 13 benign and 2 malignant tumors. The CT-MR fusion images supported the surgeon in preoperative planning and improved surgical performance. The mean intraoperative accuracy of the navigation system was 1.25 mm. ::: ::: ::: CONCLUSION ::: CT and MRI are essential in the preoperative work up of temporal bone tumors. CT-MR image data fusion presents an accurate tool for planning the correct surgical procedure and is a benefit for the operational results in computer assisted navigated neurosurgery of temporal bone tumors.
---
paper_title: A SPECT-CT image fusion technique for diagnosis of head-neck cancer

paper_content:

A SPECT-CT image fusion technique is developed to provide a more precise anatomical location and an extension of the basal skull involvement of nasopharyngeal carcinoma (NPC) detected by Tc-99m MDP bone scan, and to achieve a better treatment planning. In this technique, both the SPECT images and the head-and-neck CT images were appropriately separated to two sections, the head (upper) part images and the neck (lower) part images. A multimodality image registration procedure was used to individually align the head/neck section of the SPECT images to the head/neck section of the CT images respectively. The procedure requires: generation of simulated SPECT images using the bone component in the CT images, and then alignment of the measured SPECT images to the simulated ones that are based on the CT images. In order to benefit from the advantages of CT and SPECT images, the two aligned images have been fused together. The fused images allows the user to precisely delineate the regions of interest (ROIs) shown to have involvement of NPC which is difficult to be indicated if one imaging modality is used. Moreover, the fused image is also useful for accurate therapeutic planning in radiation oncology and for consistent evaluation of the results between the pre- and post-treatments.
---
paper_title: Single photon emission computed tomography/spiral computed tomography fusion imaging for the diagnosis of bone metastasis in patients with known cancer

paper_content:

Purpose ::: To evaluate single photon emission computed tomography (SPECT)/spiral computed tomography (CT) fusion imaging for the diagnosis of bone metastasis in patients with known cancer and to compare the diagnostic efficacy of SPECT/CT fusion imaging with that of SPECT alone and with SPECT + CT.
---
paper_title: Fusion of Immunoscintigraphy Single Photon Emission Computed Tomography (SPECT) with CT of the Chest in Patients with Non-Small Cell Lung Cancer

paper_content:

In non-small cell lung cancer (NSCLC), accurate staging is critical in deciding between potentially curative surgery and palliative treatment. Image registration, or fusion, combines the unique functional information provided by SPECT imaging with the excellent anatomic detail offered by computed tomography (CT) or magnetic resonance imaging to better characterize the imformation provided by each separate modality. In this study, we explored the role of fusion of immunoscintigraphy SPECT with CT in the staging of NSCLC. We fused chest CT with 99mTc-labeled IMMU-4 anti-carcinoembryonic antigen Fab′ antibody fragment SPECT in 14 patients with NSCLC using a landmark-based algorithm. The algorithm's accuracy was a measure from the center-to-center distance and the percentage overlap of two regions of interest: one drawn on CT and warped onto SPECT, the other drawn directly on the SPECT. We found that the average center-to-center distance was 1.3 ± 0.8 pixels. Average overlap was 46 ± 20%. CT-SPECT fusion helped differentiate tumor from normal blood pool, necrotic areas within viable tumor, tumor recurrence from scar, and malignant lymphadenopathy from hyperplasia. We conclude that fusion of CT and SPECT augments the information provided by each separate modality. Future clinical applications of fusion in NSCLC staging using immunoscintigraphy appear promising.
---
paper_title: Multiplanar MRI–CT fusion neuronavigation-guided serial stereotactic biopsy of human brain tumors: proof of a strong correlation between tumor imaging and histopathology by a new technical approach

paper_content:

Purpose ::: Serial stereotactic biopsy is a diagnostic procedure, used when open biopsy or tumor bulk removal seems to be associated with a too high risk of new neurological deficits in tumors of eloquent regions or tumors of deep localizations or in anticipated high surgery related morbidity even in the older patient group. Shortcomings of this method are recognized to be the missed pathohistological information from untargeted areas in heterogeneous tumors. This study shows for the first time a collection of patients with brain tumors with their associated multiplanar MRI–CT fusion imaging during stereotaxis and the histopathological features of serial tumor biopsies along exact trajectorial sites towards the tumor center.
---
paper_title: Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas

paper_content:

A major limitation of the use of endoscopes in minimally invasive surgery is the lack of relative context between the endoscope and its surroundings. The purpose of this work was to fuse images obtained from a tracked endoscope to surfaces derived from three-dimensional (3-D) preoperative magnetic resonance or computed tomography (CT) data, for assistance in surgical planning, training and guidance. We extracted polygonal surfaces from preoperative CT images of a standard brain phantom and digitized endoscopic video images from a tracked neuro-endoscope. The optical properties of the endoscope were characterized using a simple calibration procedure. Registration of the phantom (physical space) and CT images (preoperative image space) was accomplished using fiducial markers that could be identified both on the phantom and within the images. The endoscopic images were corrected for radial lens distortion and then mapped onto the extracted surfaces via a two-dimensional 2-D to 3-D mapping algorithm. The optical tracker has an accuracy of about 0.3 mm at its centroid, which allows the endoscope tip to be localized to within 1.0 mm. The mapping operation allows multiple endoscopic images to be "painted" onto the 3-D brain surfaces, as they are acquired, in the correct anatomical position. This allows panoramic and stereoscopic visualization, as well as navigation of the 3-D surface, painted with multiple endoscopic views, from arbitrary perspectives.
---
paper_title: Synergistic value of single-photon emission computed tomography/computed tomography fusion to radioimmunoscintigraphic imaging of prostate cancer.

paper_content:

The rationale on which positron emission tomography/computed tomography (PET/CT) imaging is based, combining the functional features of PET with the anatomic detail of CT, provides many advantages that are easily transferable to single-photon emission computed tomography (SPECT)/CT imaging. Our efforts have focused on applying fused SPECT/CT imaging to identify prostate cancer and its metastasis and recurrence through radioimmunoscintigraphy (RIS). This application of RIS to imaging prostate cancer requires 2 key components: (1) a well-defined target associated with the cancer and (2) a "magic bullet" to seek that target. A well-characterized RIS target for prostate cancer is prostate-specific membrane antigen, or PSMA, and finding the bullet to seek this target with high sensitivity and specificity has been the focus of intensive study for nearly two decades. One of the candidate bullets developed is capromab pendetide, which is a monoclonal antibody that seeks PSMA. This antibody is commercially available as ProstaScint, which can be labeled with indium-111 to localize prostate cancer via SPECT imaging. In the course of applying fused SPECT/CT ProstaScint imaging to more than 800 prostate cancer cases, numerous refinements to our protocol have evolved that are aimed at staging the cancer with utmost accuracy. In addition to optimizing the localization of prostate cancer and its metastasis, these refinements also have been extended toward guiding both the implantation of radioactive seeds in brachytherapy and in other types of radiation therapy which is illustrated through 5 case reports. Progress in the therapeutic targeting of PSMA is also being actively explored, which has more universal ramifications because PSMA is found in the neovasculature of other types of cancers.
---
paper_title: CT-ultrasound fusion prostate brachytherapy: a dynamic dosimetry feedback and improvement method. A report of 54 consecutive cases.

paper_content:

PURPOSE ::: The authors describe a prostate brachytherapy technique with dynamic dosimetry feedback, using coregistered CT and ultrasound (US) images, to map initial dosimetry deficiencies and guide remedial source placement. ::: ::: ::: METHODS AND MATERIALS ::: Fifty-four consecutive patients treated with this method were analyzed for coregistration accuracy and dosimetry outcomes by evaluating the prostate V100, V150, D90, and urethral D50 and D10. Dosimetric improvements created by remedial source placement and preplan/postplan prostate D90 agreement were evaluated. ::: ::: ::: RESULTS ::: Median CT-US coregistration discrepancy with this technique ranged from 0 to 4mm, with the posterior midline prostate and base prostate providing the least consistent and the urethra providing the most consistent coregistration agreement. Final prostate V100 values ranged from 96.1% to 99.8% for all patients. The addition of remedial sources directed by CT-US fusion produced V100 and D90 improvements whose magnitude inversely correlated with the initial result and exceeded the effect of adding quantitatively identical randomly distributed increased millicuries. The final prostate D90 result agreed within (-) 5% to (+) 10% of the preplan result in 98% of all patients. ::: ::: ::: CONCLUSIONS ::: CT-US fusion prostate brachytherapy represents a dynamic dosimetry feedback and remediation method that consistently produced high prostate V100 and D90 values with acceptably low urethra D50 and D10 values in our study. The degree of prostate V100 and D90 dosimetry improvement created by remedial source placement effectively matched the degree of initial dosimetry deficiency. This method produced a high level of correlation between the preplan and final prostate D90 values.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy for lung or liver cancers with a fusion of CT and linear accelerator (FOCAL) unit.

paper_content:

PURPOSE ::: To evaluate intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy (SRT) for lung or liver cancers, we checked repeated CT scanning, with a fusion of CT and linear accelerator (FOCAL) unit. ::: ::: ::: METHODS AND MATERIALS ::: The FOCAL unit is a combination of a linear accelerator (Linac), CT scanner, X-ray simulator (X-S), and carbon table, and is designed to achieve CT-guided SRT with daily CT positioning followed by immediate irradiation while patients keep reduced shallow respirations. To evaluate intrafractional tumor position stability, 50 lung or liver lesions in 20 patients were checked by repeated CT scanning just before and after irradiation, and the obtained images were compared. ::: ::: ::: RESULTS ::: There was no case with the intrafractional error judged to be greater than 10 mm. In 68% of cases, the intrafractional positioning errors were negligible (0-5 mm). ::: ::: ::: CONCLUSIONS ::: Using the FOCAL unit, SRT for lung or liver cancers could be performed with intrafractional positioning errors not greater than 10 mm.
---
paper_title: An application of multimodal image registration and fusion in a 3D tumor simulation model

paper_content:

In this paper, a three-dimensional (3D) image registration scheme along with fusion tools is developed and incorporated within a model simulating tumor growth and response to radiotherapy. This biosimulation model attempts to predict and visualize in three dimensions the in vivo tumor behavior. The necessary information for the initialization and the adjustment of the model can be provided from different medical examinations. The present study focuses on the development and evaluation of a technique that enables input information provided from different medical imaging modalities to be efficiently exploited by tumor growth simulation algorithms through the registration and fusion of computed tomography (CT) or positron emission tomography (PET) to magnetic resonance imaging (MRI) data.
---
paper_title: Additional value of MR/PET fusion compared with PET/CT in the detection of lymph node metastases in cervical cancer patients.

paper_content:

We evaluated the additional diagnostic value of magnetic resonance/positron emission tomography (MR/PET) fusion in the detection of metastatic lymph nodes in cervical cancer patients. Seventy nine patients with FIGO stage IB-IVA cervical cancer who had undergone both magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) before lymphadenectomy were included in this study. Image analysis was first performed with PET/CT images only. A second analysis was then performed with MR/PET fused images that focused on the additional information obtained from the MR images. Lymphadenectomy involved removing all visible lymph nodes in the surgical field. To enable nodal group-specific comparisons, para-aortic and pelvic lymph nodes were divided into seven nodal groups: para-aortic, both common iliac, both external iliac and both internal iliac/obturator areas. Histopathological evaluation of lymph nodes has been the diagnostic standard. The value of the additional information from the MR images was evaluated by means of receiver operating characteristic (ROC) analysis. Fused MR/PET rendered readers to detect six more metastatic lymph node groups. The sensitivity and specificity of PET/CT and fused MR/PET were 44.1%, 93.9% and 54.2%, 92.7% respectively. The ROC analysis demonstrated a higher diagnostic performance of fused MR/PET compared to PET/CT alone for detecting lymph node metastases (p=0.0259). The findings of this study demonstrate the additional diagnostic value of fused MR/PET images compared with PET/CT in the detection of metastatic lymph nodes in patients with uterine cervical cancer.
---
paper_title: Dual-modality 3D brain PET-CT image segmentation based on probabilistic brain atlas and classification fusion

paper_content:

The increasing prevalence of dual medical imaging modalities, such as PET-CT scanners, poses both challenges and opportunities to image segmentation, as they provide distinct but complementary information. In this paper, we propose a novel segmentation algorithm for 3D brain PET-CT images, which classifies each voxel by fusing the voxel's memberships estimated from four points of view using the PET information, CT information, smoothness prior, and probabilistic brain atlas. All memberships having the same dynamic range greatly facilitates weighting the contribution of the four different information sources. The probabilistic brain atlas estimated for each PET-CT image from a set of training samples provides the anatomical information to the segmentation process. We compared the proposed algorithm to three single-classifier based methods, PET-based SPM algorithm, CT-based Otsu thresholding, and PET-CT based MAP-MRF algorithm. The experimental results in 11 clinical brain PET-CT studies demonstrate that the novel algorithm is capable of providing more accurate and reliable segmentation.
---
paper_title: Variability of gross tumor volume delineation in head-and-neck cancer using PET/CT fusion, Part II: the impact of a contouring protocol.

paper_content:

The purpose of this study was to assess the efficacy of a gross tumor volume (GTV) contouring protocol on interobserver variability between 4 physicians in positron emission therapy/computed tomography (PET/CT) treatment planning of head-and-neck cancer. A GTV contouring protocol for PET/CT treatment planning was developed utilizing 4 stages: Preliminary contouring on CT alone, determination of appropriate PET windowing, accurate image registration, and modification of CT contouring with correctly formatted PET/CT display and rules for modality disagreement. Two neuroradiologists and 2 radiation oncologists (designated as A, B, C, and D, respectively) were given a tutorial of PET/CT coregistered imaging individualized to their skill level, which included a step-by-step explanation of the protocol with clinical examples. Opportunities for questions and hands-on practice were given. The physicians were asked to re-contour 16 head-and-neck patients from Part I on PET/CT fusion imaging. Differences in volume magnitude were analyzed for statistical significance by analysis of variance (ANOVA) and paired t-tests (alpha<0.05). Volume overlap was analyzed for statistical significance using Wilcoxon signed-rank tests (alpha<0.05). Volume overlap increased significantly from Part I to Part II (p<0.05). One previously significant difference between physicians disappeared with the protocol in place. The mean fusion volume of Physician C, however, remained significantly larger than that of Physician D (p<0.01). This result is unchanged from Part I. The multidisciplinary contouring protocol significantly improved the coincidence of GTVs contoured by multiple physicians. The magnitudes of the volumes showed marginal improvement in consistency. Developing an institutional contouring protocol for PET/CT treatment planning is highly recommended to reduce interobserver variability.
---
paper_title: Variability of gross tumor volume delineation in head-and-neck cancer using CT and PET/CT fusion.

paper_content:

PURPOSE ::: To assess the need for gross tumor volume (GTV) delineation protocols in head-and-neck cancer (HNC) treatment planning by use of positron emission tomography (PET)/computed tomography (CT) fusion imaging. Assessment will consist of interobserver and intermodality variation analysis. ::: ::: ::: METHODS AND MATERIALS ::: Sixteen HNC patients were accrued for the study. Four physicians (2 neuroradiologists and 2 radiation oncologists) contoured GTV on 16 patients. Physicians were asked to contour GTV on the basis of the CT alone, and then on PET/CT fusion. Statistical analysis included analysis of variance for interobserver variability and Student's paired sample t test for intermodality and interdisciplinary variability. A Boolean pairwise analysis was included to measure degree of overlap. ::: ::: ::: RESULTS ::: Near-significant variation occurred across physicians' CT volumes (p = 0.09) and significant variation occurred across physicians' PET/CT volumes (p = 0.0002). The Boolean comparison correlates with statistical findings. One radiation oncologist's PET/CT fusion volumes were significantly larger than his CT volumes (p < 0.01). Conversely, the other radiation oncologist's CT volumes tended to be larger than his fusion volumes (p = 0.06). No significant interdisciplinary variation was seen. Significant disagreement occurred between radiation oncologists. ::: ::: ::: CONCLUSION ::: Significant differences in GTV delineation were found between multiple observers contouring on PET/CT fusion. The need for delineation protocol has been confirmed.
---
paper_title: Hybrid surface- and voxel-based registration for MR-PET brain fusion

paper_content:

In this paper, we propose a novel technique of registration using hybrid approach for MR-PET brain image fusion. Hybrid approach uses merits of surface- and voxel-based registration. Thus, our method measures similarities using voxel intensities in MR images corresponding to the feature points of the brain in PET images. Proposed method selects the brain threshold using histogram accumulation ratio in PET images. And then, we automatically segment the brain using the inverse region growing with pre-calculated threshold and extract the feature points of the brain using sharpening filter in PET images. In order to find the optimal location for registration, we evaluate the Hybrid-based Cross-Correlation using the voxel intensities in MR images corresponding to the feature points in PET images. In our experiments, we evaluate our method using software phantom and clinical datasets in the aspect of visual inspection, accuracy, robustness, and computation time. Experimental results show that our method is dramatically faster than the voxel-based registration and more accurate than the surface-based registration. In particular, our method can robustly align two datasets with large geometrical displacement and noise at optimal location.
---
paper_title: Reirradiation of recurrent high-grade gliomas using amino acid PET (SPECT)/CT/MRI image fusion to determine gross tumor volume for stereotactic fractionated radiotherapy.

paper_content:

PURPOSE ::: To develop a valid treatment strategy for recurrent high-grade gliomas using stereotactic hypofractionated reirradiation based on biologic imaging and temozolomide. ::: ::: ::: PATIENTS AND METHODS ::: The trial included a total of 44 patients with recurrent high-grade gliomas (1 patient with anaplastic oligodendroglioma, 8 with anaplastic astrocytoma, 33 with glioblastoma multiforme, and 2 with gliosarcoma) after previous surgery and postoperative conventional radiotherapy +/- chemotherapy. For fractionated stereotactic radiotherapy (SFRT) treatment planning, the gross tumor volume was defined by (11)C-methionine positron emission tomography (MET-PET) or (123)I-alpha-methyl-tyrosine (IMT) single-photon computed emission tomography (SPECT)/computed tomography (CT)/magnetic resonance imaging (MRI) fusion in 82% of the patients and by CT/T1+gadolinium-MRI image fusion in 18% of the patients. Six fractions of 5 Gy were administered in 6 days. In 29 of 44 patients (66%), chemotherapy with temozolomide (200 mg/m(2) body surface/day) was given in one to two cycles before and four to five cycles after SFRT. The patients were evaluated in follow-up by clinical investigators and MRI or CT every 3 months after SFRT until death. In cases suspicious for radiation necrosis, a MET-PET or IMT-SPECT investigation was performed. ::: ::: ::: RESULTS ::: The median survival time in the whole group was 8 months. Treatment planning based on PET(SPECT)/CT/MRI imaging was associated with improved survival in comparison to treatment planning using CT/MRI alone: median survival time 9 months vs. 5 months (p = 0.03, log-rank). Median survival time were 11 months for patients who received SFRT based on biologic imaging plus temozolomide and significantly lower, 6 months for patients treated with SFRT without biologic imaging, without temozolomide or without both (p = 0.008, log rank). The most important prognostic factor in univariate analysis was a long interval between initial diagnosis and recurrence (p = 0.0002, log-rank). In the multivariate model, time interval to retreatment (p = 0.006) and temozolomide (p = 0.04) remained statistically significant. No acute neurologic toxicity Grade 3 or higher and no Grade 4 hematologic toxicity was observed. ::: ::: ::: CONCLUSION ::: This is the first study of biologic imaging optimized SFRT plus temozolomide in recurrent high-grade gliomas. It demonstrates the feasibility and safety of this approach. The most striking result of the trial is the statistically significant longer survival time in the univariate analysis for patients reirradiated using MET-PET or IMT-SPECT/CT/MRI image fusion in the treatment planning, in comparison to patients treated based on MRI/CT alone. Multivariate analysis confirmed a significant survival benefit from multimodal treatment (i.e., addition of temozolomide), despite the limited number of patients. Whether treatment planning with SPECT/PET independently influences survival has to be studied in a larger series of patients.
---
paper_title: Ultrasound image fusion for external beam radiotherapy for prostate cancer.

paper_content:

PURPOSE ::: To determine whether real-time ultrasound imaging and targeting system for the treatment of prostate cancer was feasible. The initial phase of this project included a study to develop and determine (a) software for the fusion of ultrasound images to standard x-rays obtained during simulation, and (b) the potential reduction in field size with real-time imaging. ::: ::: ::: METHODS AND MATERIALS ::: During 13 patient simulations a transrectal ultrasound image was obtained. Orthogonal x-ray films were acquired with the rectal probe in place. Both the x-ray and ultrasound images were digitized and a fusion image was created of the prostate position in relation to the probe, bladder, and rectum. The two-dimensional area of the rectum, bladder, and prostate was determined in the lateral projection. Potential conformal blocks were designed for the lateral portals in a four-field treatment technique. ::: ::: ::: RESULTS ::: The transrectal ultrasound probe enabled real-time prostate imaging. The lateral field size can be reduced to 6.08 x 5.68 cm2 +/- 0.62 x 0.48 cm2 from the standard 8 x 8 cm2 field. The posterior rectal wall was physically displaced out of the lateral field. The area of the rectum included in the lateral field is 1.75 cm2 +/- 0.85 cm2. ::: ::: ::: CONCLUSION ::: The prostate position can be determined with certainty on a regular basis with transrectal ultrasonography. The amount of normal tissue in the high dose volume can be reduced. This approach may reduce acute and chronic morbidity and allow further dose escalation.
---
paper_title: The Application of Image Fusion in Stereotactic Brachytherapy of Brain Tumours

paper_content:

Summary Background. The visualization of any morphological volume (i.e. CT, MRI) together with an additional second morphological volume (i.e. CT, MRI) or functional data set, which may come from SPECT or PET, is a new method for treatment planning, verification and follow-up of interstitial irradiation. Method. The authors present their experience on interstitial irradiation of brain tumours with stereotactically implanted I-125 seeds supported by image fusion. The image fusion was performed by the BrainLab-Target 1.13 software on Alfa 430 (Digital) workstation before, during, and after interstitial irradiation of brain tumours with Iodine125 seeds. Results and Interpretation. On the basis of 20 brachytherapeutic image fusion of stereotactic CT (slices with fiducials) with additional stereotactic CT, MRI, PET and SPECT images provides more accurate and precise target volume, more exact localization of catheters and isotope seeds (verification fusion), differentiation between the localization and amount of the necrotic and proliferating parts of the tumours and shows the volume changes in consequence of interstitial irradiation. The image fusion should help to improve the accuracy and minimize the perifocal morbidity of interstitial irradiation.
---
paper_title: A data fusion approach to tumor delineation

paper_content:

An improved methodology is presented for the localization of tumors for radiation therapy treatment planning applications. The proposed approach focuses on the integration of MRI, CT, and PET imaging data towards the generation of a mathematically optimal tumor boundary. Fuzzy edges derived from CT and MRI are fused to form an integrated edge map which is employed in conjunction with a region based analysis of the PET scan to guide the evolution of a tumor segmenting model. The presented algorithm yields tumor contours which may be employed directly in the radiation therapy treatment planning process.
---
paper_title: Impact of computed tomography and 18F-deoxyglucose coincidence detection emission tomography image fusion for optimization of conformal radiotherapy in non-small-cell lung cancer.

paper_content:

PURPOSE ::: To report a retrospective study concerning the impact of fused 18F-fluoro-deoxy-D-glucose (FDG)-hybrid positron emission tomography (PET) and CT images on three-dimensional conformal radiotherapy planning for patients with non-small-cell lung cancer. ::: ::: ::: METHODS AND MATERIALS ::: A total of 101 patients consecutively treated for Stage I-III non-small-cell lung cancer were studied. Each patient underwent CT and FDG-hybrid PET for simulation treatment in the same treatment position. Images were coregistered using five fiducial markers. Target volume delineation was initially performed on the CT images, and the corresponding FDG-PET data were subsequently used as an overlay to the CT data to define the target volume. ::: ::: ::: RESULTS ::: 18F-fluoro-deoxy-D-glucose-PET identified previously undetected distant metastatic disease in 8 patients, making them ineligible for curative conformal radiotherapy (1 patient presented with some positive uptake corresponding to concomitant pulmonary tuberculosis). Another patient was ineligible for curative treatment because the fused PET-CT images demonstrated excessively extensive intrathoracic disease. The gross tumor volume (GTV) was decreased by CT-PET image fusion in 21 patients (23%) and was increased in 24 patients (26%). The GTV reduction was > or = 25% in 7 patients because CT-PET image fusion reduced the pulmonary GTV in 6 patients (3 patients with atelectasis) and the mediastinal nodal GTV in 1 patient. The GTV increase was > or = 25% in 14 patients owing to an increase in the pulmonary GTV in 11 patients (4 patients with atelectasis) and detection of occult mediastinal lymph node involvement in 3 patients. Of 81 patients receiving a total dose of > or = 60 Gy at the International Commission on Radiation Units and Measurements point, after CT-PET image fusion, the percentage of total lung volume receiving >20 Gy increased in 15 cases and decreased in 22. The percentage of total heart volume receiving >36 Gy increased in 8 patients and decreased in 14. The spinal cord volume receiving at least 45 Gy (2 patients) decreased. Multivariate analysis showed that tumor with atelectasis was the single independent factor that resulted in a significant effect on the modification of the size of the GTV by FDG-PET: tumor with atelectasis (with vs. without atelectasis, p = 0.0001). ::: ::: ::: CONCLUSION ::: The results of our study have confirmed that integrated hybrid PET/CT in the treatment position and coregistered images have an impact on treatment planning and management of non-small-cell lung cancer. However, FDG images using dedicated PET scanners and respiration-gated acquisition protocols could improve the PET-CT image coregistration. Furthermore, the impact on treatment outcome remains to be demonstrated.
---
paper_title: Software-based Fusion of PET and CT Images for Suspected Recurrent Lung Cancer

paper_content:

PURPOSE ::: The purpose of this study was to compare the diagnostic performance of the manual fusion of positron emission tomography (PET) and computed tomography (CT) images with that of CT alone and that of side-by-side PET and CT (PET/CT) in patients with suspected recurrent lung cancer. ::: ::: ::: PROCEDURES ::: Fifty-three patients who had previously had surgery for lung cancer underwent a whole-body 2-deoxy-2-[F-18]fluoro-D-glucose (FDG)-PET scan, followed by a diagnostic CT scan. The PET and CT images were fused on a workstation. CT alone, PET/CT, and fused images were evaluated separately using a five-point grading scale (0=definitely negative, 1=probably negative, 2=equivocal, 3=probably positive, and 4=definitely positive). Lesions of grade 3 or 4 were considered positive, and diagnostic accuracy and certainty were evaluated. ::: ::: ::: RESULTS ::: Overall, 67 lesions in 33 patients were considered true positive pathologically or clinically. Of these 67 lesions, the evaluation of CT, PET/CT, and fused images detected 46, 55, and 66 lesions, respectively, with the number of grade 4 lesions detected being 38, 50, and 63, respectively. The diagnostic accuracy of CT, PET/CT, and fused images according to patients was 75%, 79%, and 87%, respectively. ::: ::: ::: CONCLUSION ::: These results suggest that interpreting fused images increased diagnostic certainty for detecting recurrence and provided more accurate diagnoses.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: Investigation of PET/MRI image fusion schemes for enhanced breast cancer diagnosis

paper_content:

The benefit of registration and fusion of functional images with anatomical images is well appreciated in the advent of PET/CT. There is an increasing interest in expanding this approach to PET/MRI. The focus of much of the related research has been on registering images from different modalities. However, the importance of appropriately jointly displaying (i.e. fusing) the registered images has often been neglected and underestimated. Our aim is to determine which fusion techniques are most useful for enhanced diagnostic performance, ease of use, efficiency, and accuracy for reading registered MRI and PET breast images. Preliminary results indicate that the radiologists were better able to perform a series of tasks when reading the fused PET/MRI data sets using color tables generated by our new genetic algorithm, as compared to commonly used fire/gray or red/gray schemes.
---
paper_title: Fusion Image of Positron Emission Tomography and Computed Tomography for the Diagnosis of Local Recurrence of Rectal Cancer

paper_content:

BACKGROUND ::: The aim of this study was to evaluate the clinical and therapeutic value of digital fusion image (FI) of positron emission tomography (PET) using (18)F-fluorodeoxy glucose and computed tomography (CT) in patients who were suspected of having a local recurrence of rectal cancer. ::: ::: ::: METHODS ::: Forty-two patients (32 men and 10 women; mean age, 61.4 years, range, 40-79 years) with a suspicion of local recurrence after curative resection of rectal cancer were prospectively recruited and underwent (18)F-fluorodeoxy glucose-PET and CT. The FI was reconstructed with a commercially available digital software program, T-B Fusion. Wilcoxon signed rank test was used to compare FI with CT alone or PET alone. ::: ::: ::: RESULTS ::: FI yielded a correct diagnosis in 39 (93%) of 42 patients, whereas CT alone and PET alone did so in 33 (79%) and 37 (88%) patients, respectively. FI had better diagnostic accuracy than CT alone (P = .0138) and PET alone (P = .0156). Overall, FI altered patient management in 11 (26.2%) patients on the basis of additional information, including differentiation of the tumor from the postoperative scar in 6 patients, exact anatomical location in 3 patients, and both in 2 patients. ::: ::: ::: CONCLUSIONS ::: FI has a potential clinical value in the treatment of suspected local recurrence of rectal cancer.
---
paper_title: PET-CT fusion in radiation management of patients with anorectal tumors.

paper_content:

PURPOSE ::: To compare computed tomography (CT) with positron emission tomography-CT (PET-CT) scans with respect to anorectal tumor volumes, correlation in overlap, and influence on radiation treatment fields and patient care. ::: ::: ::: PATIENTS AND METHODS ::: From March to November 2003, 20 patients with rectal cancer and 3 patients with anal cancer were treated with preoperative or definitive chemoradiation, respectively. Computed tomography simulation data generated a CT gross tumor volume (CT-GTV) and CT planning target volume (CT-PTV) and (18)F-fluoro-2-deoxy-glucose PET (FDG-PET) created a PET-GTV and PET-PTV. The PET-CT and CT images were fused using manual coregistration. Patients were treated with three-dimensional conformal therapy to traditional doses. The PET, CT, and overlap volumes (OVs) were measured in cubic centimeters. ::: ::: ::: RESULTS ::: Mean PET-GTV was smaller than the mean CT-GTV (91.7 vs. 99.6 cm(3)). The mean OV was 46.7%. As tumor volume increased, PET and CT OV correlated significantly (p < 0.001). In 17% of patients PET-CT altered the PTV, and in 26% it changed the radiation treatment plan. For 25% of patients with rectal cancer, PET detected distant metastases and changed overall management. Ten rectal cancer patients underwent surgery. When the pretreatment PET standardized uptake value was >10 and the posttreatment PET standardized uptake value was <6, 100% achieved pathologic downstaging (p = 0.047). ::: ::: ::: CONCLUSIONS ::: Variation in volume was significant, with 17% and 26% of patients requiring a change in treatment fields and patient management, respectively. Positron emission tomography can change the management for anorectal tumors by early detection of metastatic disease or disease outside standard radiation fields.
---
paper_title: A phase II comparative study of gross tumor volume definition with or without PET/CT fusion in dosimetric planning for non-small-cell lung cancer (NSCLC): primary analysis of Radiation Therapy Oncology Group (RTOG) 0515.

paper_content:

BACKGROUND ::: Radiation Therapy Oncology Group (RTOG) 0515 is a Phase II prospective trial designed to quantify the impact of positron emission tomography (PET)/computed tomography (CT) compared with CT alone on radiation treatment plans (RTPs) and to determine the rate of elective nodal failure for PET/CT-derived volumes. ::: ::: ::: METHODS ::: Each enrolled patient underwent definitive radiation therapy for non-small-cell lung cancer (≥ 60 Gy) and had two RTP datasets generated: gross tumor volume (GTV) derived with CT alone and with PET/CT. Patients received treatment using the PET/CT-derived plan. The primary end point, the impact of PET/CT fusion on treatment plans was measured by differences of the following variables for each patient: GTV, number of involved nodes, nodal station, mean lung dose (MLD), volume of lung exceeding 20 Gy (V20), and mean esophageal dose (MED). Regional failure rate was a secondary end point. The nonparametric Wilcoxon matched-pairs signed-ranks test was used with Bonferroni adjustment for an overall significance level of 0.05. ::: ::: ::: RESULTS ::: RTOG 0515 accrued 52 patients, 47 of whom are evaluable. The follow-up time for all patients is 12.9 months (2.7-22.2). Tumor staging was as follows: II = 6%; IIIA = 40%; and IIIB = 54%. The GTV was statistically significantly smaller for PET/CT-derived volumes (98.7 vs. 86.2 mL; p < 0.0001). MLDs for PET/CT plans were slightly lower (19 vs. 17.8 Gy; p = 0.06). There was no significant difference in the number of involved nodes (2.1 vs. 2.4), V20 (32% vs. 30.8%), or MED (28.7 vs. 27.1 Gy). Nodal contours were altered by PET/CT for 51% of patients. One patient (2%) has developed an elective nodal failure. ::: ::: ::: CONCLUSIONS ::: PET/CT-derived tumor volumes were smaller than those derived by CT alone. PET/CT changed nodal GTV contours in 51% of patients. The elective nodal failure rate for GTVs derived by PET/CT is quite low, supporting the RTOG standard of limiting the target volume to the primary tumor and involved nodes.
---
paper_title: Image-Fusion Operators for 3D Anatomical and Functional Analysis of the Brain

paper_content:

The goal of the present work is to obtain images combining in a useful and precise way the information present in 3D volumes of medical imaging sets. We address two modalities combining anatomy (Magnetic Resonance Imaging or MRI) and functional information (Positron Emission Tomography or PET). Commercial imaging software often offers image fusion tools based on fixed blending or color- channel combination of different modalities, and various popular color Look-Up Tables (LUTs), without considering the anatomical and functional character of the image features. We used a sensible approach for image fusion, taking advantage mainly from the HSL (Hue, Saturation and Luminosity) color space, in order to enhance the fusion results. We further tested operators for gradient and contour extraction to enhance anatomical details, such as feature borders, plus other spatial- domain filters for functional features, which correspond to wide point-spread-function responses in PET images. A set of image-fusion operators was formulated and tested on brain images from PET and MRI acquisitions.
---
paper_title: The impact of image fusion in resolving discrepant findings between FDG-PET and MRI/CT in patients with gynaecological cancers

paper_content:

This study was performed to prospectively investigate the impact of image fusion in resolving discrepant findings between fluorine-18 fluorodeoxyglucose positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) or X-ray computed tomography (CT) in patients with gynaecological cancers. Discrepant findings were defined as lesions where the difference between the FDG-PET and MRI/CT images was assigned a value of at least 2 on a 5-point probability scale. The FDG-PET and MRI/CT images were taken within 1 month of each other. Image fusion between FDG-PET and CT was performed by automatic registration between the two images. During an 18-month period, 34 malignant lesions and seven benign lesions from 32 patients who had undergone either surgical excision or a CT-guided histopathological investigation were included for analysis. Among these cases, image fusion was most frequently required to determine the nature and/or the extent of abdominal and pelvic lesions (28/41, 68%), especially as regards peritoneal seeding (8/41, 20%). Image fusion was most useful in providing better localisation for biopsy (16/41, 39%) and in discriminating between lesions with pathological versus physiological FDG uptake (12/41, 29%). Image fusion changed the original diagnosis based on MRI/CT alone in 9/41 lesions (22%), and the original diagnosis based on FDG-PET alone in 5/41 lesions (12%). It led to alteration of treatment planning (surgery or radiotherapy) in seven of the 32 patients (22%). In patients with gynaecological cancers, the technique of image fusion is helpful in discriminating the nature of FDG-avid lesions, in effectively localising lesions for CT-guided biopsy and in providing better surgical or radiotherapy planning.
---
paper_title: Integrated PET/CT Fusion Imaging and Endoscopic Ultrasound in the Pre-operative Staging and Evaluation of Esophageal Cancer

paper_content:

PURPOSE ::: Accurate staging of esophageal cancer (ECA) is critical in determining appropriate therapy. Endoscopic ultrasound (EUS), computed tomography (CT) and positron emission tomography (PET) scanning can be used, but limited data exists regarding the use of combined PET/CT fusion imaging and EUS in ECA staging. The objective of this study is to evaluate the role of integrated PET/CT imaging and EUS in the staging of ECA. ::: ::: ::: PROCEDURES ::: Identification of patients diagnosed with ECA from 2004 to 2007 that underwent staging PET/CT and EUS. Data regarding tumor detection, lymph node identification, presence of metastatic disease, and affect on patient management were collected and compared between PET/CT and EUS. ::: ::: ::: RESULTS ::: Eighty-one patients (65 male, 16 female) were identified with mean age of 63.5 years who underwent EUS and PET/CT to stage known ECA. PET/CT identified the primary tumor in 74/81 (91.4%) of cases, compared to 81/81 (100%) with EUS. Locoregional adenopathy was seen by PET/CT in 29/81 (35.8%) of cases, compared to 49/81 (60.5%) by EUS (p = 0.0001). PET/CT identified celiac axis adenopathy in 8/81 (9.9%) of cases, compared to 11/81 (13.6%) with EUS (p = 0.5050). PET/CT identified 17/81 (21.0%) of patients with distant metastases who subsequently did not undergo attempt at curative surgical resection. ::: ::: ::: CONCLUSIONS ::: In ECA, EUS is superior to PET/CT for T staging and in identifying locoregional nodes, while PET/CT provides M staging. EUS and integrated PET/CT appear to independently affect treatment decisions, indicating complimentary and necessary roles in the staging of ECA.
---
paper_title: Efficient Hybrid Registration Method Using a Shell Volume for PET and High Resolution MR Brain Image Fusion

paper_content:

This paper presents an efficient hybrid registration method using a shell volume that consists of high contrast voxels for combining PET and high resolution MR (HR-MR) brain images. This approach automatically selects a brain shell volume from the PET image, and then transforms only the voxels in the brain shell volume into the coordinate space of HR-MR images. Based on the corresponding voxels in HR-MR images, it finally calculates the best-matching voxel positions using normalized mutual information (NMI). The shell volume reduces the computation time by using smaller number of corresponding voxels to be matched, and it even enables a more robust registration. Experimental results on clinical data sets showed that our method successfully aligned all PET and HR-MR image pairs without losing any diagnostic information, while the conventional NMI method failed to align some cases.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Fundamental study of proton source based on inertial electrostatic confinement fusion for medical positron emission tomography

paper_content:

Summary form only given. The inertial electrostatic confinement (IEC) fusion has been mainly studied for application to a neutron source. However, it also has a potential applicable to a proton source. The IEC fusion system as a proton source has some advantages of low cost, portableness and easy handling compared with an accelerator such as a cyclotron. In this study, an IEC device was designed and tested for a proton source in order to produce a short-life radioisotope used for a medical positron emission tomography (PET). The device is made of 393-mm diameter, 342-mm high stainless steel cylindrical chamber, in which an open cylindrical grid anode of 200-mm diameter and an open cylindrical grid cathode of 40-mm diameter are set on the axis. In addition, a bucket-type ion source, which consists of sixteen ferrite magnets and sixteen filaments, is integrated with the chamber. A high voltage power supply of 100 kV, 100 mA for dc operation and 10 A for pulsed operation was also designed and used for the IEC device. By using this system, basic discharge characteristics, such as breakdown voltage versus pressure, operating voltage versus discharge current, etc. with changing gas pressure, were measured for hydrogen discharge. Then proton production rate was measured for deuterium discharge with changing operating voltage and discharge current.
---
paper_title: Clinical Value of Image Fusion from MR and PET in Patients with Head and Neck Cancer

paper_content:

Purpose ::: This study aimed to evaluate the clinical value of image fusion from magnetic resonance (MR) combined with positron emission tomography (PET) imaging, using 2-deoxy-2-[F-18]fluoro-D-glucose (FDG) in head and neck cancer.
---
paper_title: Image fusion of thallium-201 SPECT and MR imaging for the assessment of recurrent head and neck tumors following flap reconstructive surgery

paper_content:

The aim of this study was to assess the value of fused MR and Tl-201 single photon emission computed tomography (SPECT) images in the diagnosis of recurrent head and neck tumors in patients after flap reconstruction surgery. Twenty-four patients after resection of primary head and neck tumors with flap reconstruction were suspected of having recurrent tumor by follow-up MR examination. Both MR examination and Tl-201 SPECT were prospectively performed to produce fused images. For qualitative analysis, two independent readers separately evaluated the existence of tumor recurrence in the fused images. The Tl-201 uptake of the lesion (Tl index) was also quantitatively compared with that of the normal nuchal muscles. Eighteen patients were histologically proved as having recurrence. The remaining 6 patients, false positive on MRI alone, had non-recurrence. Using the fused images, false positive was found in 1 case for one reader and 2 cases for the other reader. The Tl index of recurrent tumors was significantly higher (p < 0.001) than that of non-recurrent mass lesions. In the assessment of recurrent tumors following flap reconstruction surgery in the head and neck, the use of fused MRI and Tl-201 SPECT images can reduce the number of false positives.
---
paper_title: The role of 3D fusion imaging in sentinel lymphadenectomy for vulvar cancer.

paper_content:

OBJECTIVES ::: To evaluate the feasibility of fusion imaging compound tomography (FICT) of CT/MRI and single photon emission tomography (SPECT) versus planar scintigraphy only (plSc) in pre-surgical staging for vulvar cancer. ::: ::: ::: MATERIALS AND METHODS ::: Analysis of consecutive patients with vulvar cancer who preoperatively underwent sentinel scintigraphy (planar and 3D-SPECT imaging) and CT or MRI. Body markers were used for exact anatomical co-registration and fusion datasets were reconstructed using SPECT and CT/MRI. The number and localisation of all intraoperatively identified and resected sentinel lymph nodes (SLN) were compared between planar and 3D fusion imaging. ::: ::: ::: RESULTS ::: Twenty six SLN were localized on planar scintigraphy. Twelve additional SLN were identified after SPECT and CT/MRI reconstruction, all of them were confirmed intraoperatively. In seven cases where single foci were identified at plSc, fusion imaging revealed grouped individual nodes and five additional localisations were discovered at fusion imaging. In seven patients both methods identified SLN contra lateral to the primary tumor site, but only fusion imaging allowed to localise iliac SLN in four patients. All SLN predicted on fusion imaging could be localised and resected during surgery. ::: ::: ::: CONCLUSIONS ::: Fusion imaging using SPECT and CT/MRI can detect SLN in vulvar cancer more precisely than planar imaging regarding number and anatomical localisation. FICT revealed additional information in seven out of ten cases (70%).
---
paper_title: Rectal morbidity after permanent prostate brachytherapy with dose escalation to biologic target volumes identified by SPECT/CT fusion.

paper_content:

PURPOSE ::: To evaluate rectal morbidity after dose escalation to biologic target volumes identified by capromab pendetide (ProstaScint) single-photon emission tomography images coregistered with computed tomography (SPECT/CT). ::: ::: ::: METHODS AND MATERIALS ::: Two hundred thirty-nine consecutive patients diagnosed with T1c-T3b NxM0 adenocarcinoma of the prostate were treated with brachytherapy seed implant (SI) dose escalation to SPECT/CT-identified biologic target volumes, from February 1997 through December 2002. Patients received SI (n=150) or external beam radiation therapy plus SI (n=89). Rectal morbidity was evaluated by clinician scoring using the modified Radiation Therapy Oncology Group criteria. The median followup was 47.2 (range 24.8-96.1) months. ::: ::: ::: RESULTS ::: The rate of acute Grades I and II toxicity was 29.9% and 3.7%, respectively, and chronic Grade I toxicity was 15.4%, 12.4%, 2.3%, and 1.8% at 1, 2, 3, and 4 years postimplant, respectively. Chronic Grade II toxicities were 1.8%, 1.9%, 1.5%, and 0.9% at 1, 2, 3, and 4 years, respectively. No Grade III rectal toxicity was reported. Chronic Grade IV rectal toxicity was 0.5% and 0.6% at 1.5 and 2.5 years, respectively. Ninety-six percent of patients reported freedom from all rectal toxicity after 3 years. ::: ::: ::: CONCLUSIONS ::: Dose intensification to occult tumor targets without increasing rectal toxicity may be achieved using SPECT/CT ProstaScint. Additional research to define the role of molecular imaging in prostate cancer is warranted.
---
paper_title: Image Registration and Fusion for Interventional MRI Guided Thermal Ablation of the Prostate Cancer

paper_content:

We are investigating interventional MRI (iMRI) guided thermal ablation treatment of the prostate cancer. Functional images such as SPECT can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high resolution MRI volume. Then by registering the high-resolution MR image with iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to iMRI images for improved tumor targeting. For the first step, we used a mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm. Image data were acquired from patients and volunteers. Compared to our volume-to-volume registration that was previously evaluated to be quite accurate, the SV registration accuracy is about 0.5mm for transverse images covering the prostate. With our image registration and fusion software, simulation experiments show that it is feasible to incorporate SPECT and high resolution MRI into the iMRI-guided treatment.
---
paper_title: Fusion imaging using a hybrid SPECT-CT camera improves port perfusion scintigraphy for control of hepatic arterial infusion of chemotherapy in colorectal cancer patients

paper_content:

PURPOSE ::: Exclusive and homogeneous perfusion of the liver is considered essential for the efficacy of hepatic arterial infusion of chemotherapy (HAI). The aim of this study was to evaluate port perfusion scintigraphy in colorectal cancer patients using a hybrid SPECT-CT system for control of minimally invasive intra-arterial port systems within the scope of a phase II trial. ::: ::: ::: METHODS ::: In 24 consecutive patients, the perfusion territories of intra-arterial hepatic port systems were assessed by port scintigraphy with( 99m)Tc-labelled macroaggregated albumin employing planar imaging, SPECT and SPECT-CT (acquired with a hybrid SPECT-CT camera). The results of blinded reading of the scintigraphic modalities concerning the intra- and extrahepatic perfusion pattern were compared with combined image analysis (angiography and contrast-enhanced dedicated CT) and patient history for validation. ::: ::: ::: RESULTS ::: Extrahepatic perfusion was correctly seen in three patients, while suspected extrahepatic perfusion could be excluded in one. In 46 liver lobes, perfusion patterns were correctly visualised by SPECT-CT in 100% of cases (planar, 67%; SPECT, 86%). Assessing the perfusion pattern inside the liver on a segmental basis (segments, n=138), SPECT-CT revealed correct segmental assignment of tracer distribution in 100% and was significantly superior to SPECT alone (accuracy, 84%; p<0.001). The scintigraphic findings resulted in changes in therapeutic management in 8/24 patients (33%); in two of these the relevant findings were visualised only by SPECT-CT. ::: ::: ::: CONCLUSION ::: In patients receiving HAI, port perfusion scintigraphy by fusion imaging with a hybrid SPECT-CT system provides important information for therapy optimisation and appears to be superior to SPECT alone.
---
paper_title: IMPROVING MMI WITH ENHANCED-FCM FOR THE FUSION OF BRAIN MR AND SPECT IMAGES

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To overcome these drawbacks of using traditional MMI, a novel scheme, named improved MMI, which is based on fuzzy c-means (FCM) and MMI, is proposed. The experimental results, using MR and SPECT images, to confirm the superior performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: Registration and fusion of SPECT, high-resolution MRI, and interventional MRI for thermal ablation of prostate cancer

paper_content:

We are investigating interventional MRI (iMRI) guided radiofrequency thermal ablation for the minimally invasive treatment of the prostate cancer. Nuclear medicine can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of functional images such as nuclear medicine SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high-resolution MRI volume. Then by registering the high-resolution MR image with live-time iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to live-time iMRI images for improved tumor targeting. For the first step, we used a three-dimensional mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm with special features. The concept was tested using image data from three patients and three volunteers. The SV registration accuracy was 0.4 mm /spl plusmn/ 0.2 mm as compared to our volume- to-volume registration that was previously shown to be quite accurate for these image pairs. With our image registration and fusion software, simulation experiments show that it is quite feasible to incorporate SPECT and high-resolution MRI into the iMRI-guided minimally invasive treatment procedures.
---
paper_title: The Application of Image Fusion in Stereotactic Brachytherapy of Brain Tumours

paper_content:

Summary Background. The visualization of any morphological volume (i.e. CT, MRI) together with an additional second morphological volume (i.e. CT, MRI) or functional data set, which may come from SPECT or PET, is a new method for treatment planning, verification and follow-up of interstitial irradiation. Method. The authors present their experience on interstitial irradiation of brain tumours with stereotactically implanted I-125 seeds supported by image fusion. The image fusion was performed by the BrainLab-Target 1.13 software on Alfa 430 (Digital) workstation before, during, and after interstitial irradiation of brain tumours with Iodine125 seeds. Results and Interpretation. On the basis of 20 brachytherapeutic image fusion of stereotactic CT (slices with fiducials) with additional stereotactic CT, MRI, PET and SPECT images provides more accurate and precise target volume, more exact localization of catheters and isotope seeds (verification fusion), differentiation between the localization and amount of the necrotic and proliferating parts of the tumours and shows the volume changes in consequence of interstitial irradiation. The image fusion should help to improve the accuracy and minimize the perifocal morbidity of interstitial irradiation.
---
paper_title: Evaluation of automatic multimodality fusion technique of PET and MRI/CT images for computer assisted brain tumor surgery

paper_content:

Abstract Positron emission tomography (PET) can provide spatial information on metabolic activity in patients with cerebral glioma and functional data in patients with lesions closely related to eloquent brain areas. Because of the limited image resolution of PET, coregistration with anatomic images such as MRI or CT is required to apply PET data for image guided surgery. The accuracy and clinical value of a novel image-fusion technique were evaluated for various PET, MRI and CT imaging modalities using an anatomic brain phantom and clinical patient images. For metabolic studies 2-[ 18 F]-2-desoxy- d -glucose (FDG) and synthetic amino acid PET were obtained in four glioma patients. For mapping cortical functions, such as motor or speech function, specific activation tasks were used during PET acquisition in eight patients with lesions in eloquent brain areas. The PET images were automatically matched and merged with T1-, T2-weighted MRI or CT scans by rigid transformation using an automatic image fusion algorithm (BrainLAB, @Target 1.18). The intensity-based image-fusion technique is robust and capable of registering and fusing various medical image sets regardless of imaging modality and scan orientation. The brain phantom study demonstrated an excellent accuracy for rotate-translate registrations between PET versus T1-, T2-weighted MRI and CT. Excellent spatial correlation between PET and MRI was achieved in all patients during image-guided surgery using a frameless navigation system. By exactly transferring the PET data to the operative site, metabolic data can be correlated with corresponding tumor specimens and functional imaging data with intraoperative brain mapping findings.
---
paper_title: Intermodality registration and fusion of liver images for medical diagnosis

paper_content:

SPECT is valuable for the evaluation of hepatic function and CT can offer better anatomical images for liver and its adjacent structures. Image registration from the two modalities, CT and SPECT, acquires both structural and functional information, and therefore is very useful for clinical diagnoses. The registration and fusion of CT and SPECT images are studied. We propose a two stage registration method based on principal axes transformation and surface fitting to register and fuse CT and SPECT images: First, the centroids and principal axes of these landmarks are calculated and aligned. Then the transformation matrix between CT and SPECT images is modified based on the least square error on the distance image. After registration, uniform volume data is reconstructed for image fusion.
---
paper_title: Advances in the use of neurophysiologycally-based fusion for visualization and pattern recognition of medical imagery

paper_content:

The ever increasing number of image modaliiies available io dociors for diagnosis purposes has established an imporiani, need io develop techniques ihai suppori work-load reduciion and information maximizaiion. To ihis end, we have improved on an image firsion architeciure jirsi developedfor night vision applicaiions. This iechnique, presenied ai Fusion 2002, uiilizes 30 operaiors io combine volumeiric image seis while maximizing informaiion conieni. In our approach, we have combined ihe use of image fusion and user- dejinedpaiiern recognition wiihin a 30 human-compuier inferface. Here, we present our latest advances iowards enhancing informarion visualization and supporting paiiem recognition. We also report on resulis of applying image firsion across a variety of patient cases. Finally, we have also begun ihe assessmeni of paiiem recognition based on 2D vs. 30 fused image feaiures. Initial results indicaie an advaniage io firsing imagery across all ihree dimensions sa as io take advaniage ofihe volumetric information available in medical daia seis. A description of the system and a number of examples will serve io ilhistraie our ongoing results.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: A surface-projection MMI for the fusion of brain MR and SPECT images

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To improve the performance of traditional MMI, a novel scheme named surface-projection MMI algorithm (SP-MMI) is proposed here. SP-MMI is a two-stage registration algorithm included MMI of surface projection and the following MMI of axial plane. The experimental results, using MR and SPECT images, to confirm the good performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: FMDIB: a software tool for fusion of MRI and DHC-SPECT images of Brain

paper_content:

Brain disorders are divided into two categories: anatomic and metabolic. Since brain disorders initially appear as metabolic abnormalities and ultimately result in anatomic defects, it is necessary to analyze the metabolic along with the anatomic aspect of the brain tissue in the preliminary stages of diagnosis to greatly increase the chances of a cure. It is possible that metabolic abnormalities exist without an anatomic defect such as Alzheimer disease. The brain anatomic images may achieve from MRI and physiologic and metabolic brain data can achieve from DHC-SPECT or PET system when a radiopharmaceutical such as FDG have been injected to the patient. For fusion of the brain MRI and DHC-SPECT images obtained from individual scanners, we developed the userfriendly FMDIB software in the MATLAB environment. With the help of this software, we resolved the geometrical misalignment of the brain images and subsequently co-registered them with a 1 mm precision. This software enabled us to perform the three dimensional rendering of the image, re-slicing the image with thickness of 1 to 3 mm, slicing the two dimensional DHC-SPECT projections into an arbitrary direction.
---
paper_title: Potential advantage of studying the lymphatic drainage by sentinel node technique and SPECT-CT image fusion for pelvic irradiation of prostate cancer

paper_content:

Purpose: This study aims to investigate the in vivo drainage of lymphatic spread by using the sentinel node (SN) technique and single-photon emission computed tomography (SPECT)-computed tomography (CT) image fusion, and to analyze the impact of such information on conformal pelvic irradiation. Methods and Materials: Twenty-three prostate cancer patients, candidates for radical prostatectomy already included in a trial studying the SN technique, were enrolled. CT and SPECT images were obtained after intraprostate injection of 115 MBq of 99m Tc-nanocolloid, allowing identification of SN and other pelvic lymph nodes. Target and nontarget structures, including lymph nodes identified by SPECT, were drawn on SPECT-CT fusion images. A three-dimensional conformal treatment plan was performed for each patient. Results: Single-photon emission computed tomography lymph nodal uptake was detected in 20 of 23 cases (87%). The SN was inside the pelvic clinical target volume (CTV 2 ) in 16 of 20 cases (80%) and received no less than the prescribed dose in 17 of 20 cases (85%). The most frequent locations of SN outside the CTV 2 were the common iliac and presacral lymph nodes. Sixteen of the 32 other lymph nodes (50%) identified by SPECT were found outside the CTV 2 . Overall, the SN and other intrapelvic lymph nodes identified by SPECT were not included in the CTV 2 in 5 of 20 (25%) patients. Conclusions: The study of lymphatic drainage can contribute to a better knowledge of the in vivo potential pattern of lymph node metastasis in prostate cancer and can lead to a modification of treatment volume with consequent optimization of pelvic irradiation.
---
paper_title: The impact of image fusion in resolving discrepant findings between FDG-PET and MRI/CT in patients with gynaecological cancers

paper_content:

This study was performed to prospectively investigate the impact of image fusion in resolving discrepant findings between fluorine-18 fluorodeoxyglucose positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) or X-ray computed tomography (CT) in patients with gynaecological cancers. Discrepant findings were defined as lesions where the difference between the FDG-PET and MRI/CT images was assigned a value of at least 2 on a 5-point probability scale. The FDG-PET and MRI/CT images were taken within 1 month of each other. Image fusion between FDG-PET and CT was performed by automatic registration between the two images. During an 18-month period, 34 malignant lesions and seven benign lesions from 32 patients who had undergone either surgical excision or a CT-guided histopathological investigation were included for analysis. Among these cases, image fusion was most frequently required to determine the nature and/or the extent of abdominal and pelvic lesions (28/41, 68%), especially as regards peritoneal seeding (8/41, 20%). Image fusion was most useful in providing better localisation for biopsy (16/41, 39%) and in discriminating between lesions with pathological versus physiological FDG uptake (12/41, 29%). Image fusion changed the original diagnosis based on MRI/CT alone in 9/41 lesions (22%), and the original diagnosis based on FDG-PET alone in 5/41 lesions (12%). It led to alteration of treatment planning (surgery or radiotherapy) in seven of the 32 patients (22%). In patients with gynaecological cancers, the technique of image fusion is helpful in discriminating the nature of FDG-avid lesions, in effectively localising lesions for CT-guided biopsy and in providing better surgical or radiotherapy planning.
---
paper_title: A SPECT-CT image fusion technique for diagnosis of head-neck cancer

paper_content:

A SPECT-CT image fusion technique is developed to provide a more precise anatomical location and an extension of the basal skull involvement of nasopharyngeal carcinoma (NPC) detected by Tc-99m MDP bone scan, and to achieve a better treatment planning. In this technique, both the SPECT images and the head-and-neck CT images were appropriately separated to two sections, the head (upper) part images and the neck (lower) part images. A multimodality image registration procedure was used to individually align the head/neck section of the SPECT images to the head/neck section of the CT images respectively. The procedure requires: generation of simulated SPECT images using the bone component in the CT images, and then alignment of the measured SPECT images to the simulated ones that are based on the CT images. In order to benefit from the advantages of CT and SPECT images, the two aligned images have been fused together. The fused images allows the user to precisely delineate the regions of interest (ROIs) shown to have involvement of NPC which is difficult to be indicated if one imaging modality is used. Moreover, the fused image is also useful for accurate therapeutic planning in radiation oncology and for consistent evaluation of the results between the pre- and post-treatments.
---
paper_title: Single photon emission computed tomography/spiral computed tomography fusion imaging for the diagnosis of bone metastasis in patients with known cancer

paper_content:

Purpose ::: To evaluate single photon emission computed tomography (SPECT)/spiral computed tomography (CT) fusion imaging for the diagnosis of bone metastasis in patients with known cancer and to compare the diagnostic efficacy of SPECT/CT fusion imaging with that of SPECT alone and with SPECT + CT.
---
paper_title: Fusion of Immunoscintigraphy Single Photon Emission Computed Tomography (SPECT) with CT of the Chest in Patients with Non-Small Cell Lung Cancer

paper_content:

In non-small cell lung cancer (NSCLC), accurate staging is critical in deciding between potentially curative surgery and palliative treatment. Image registration, or fusion, combines the unique functional information provided by SPECT imaging with the excellent anatomic detail offered by computed tomography (CT) or magnetic resonance imaging to better characterize the imformation provided by each separate modality. In this study, we explored the role of fusion of immunoscintigraphy SPECT with CT in the staging of NSCLC. We fused chest CT with 99mTc-labeled IMMU-4 anti-carcinoembryonic antigen Fab′ antibody fragment SPECT in 14 patients with NSCLC using a landmark-based algorithm. The algorithm's accuracy was a measure from the center-to-center distance and the percentage overlap of two regions of interest: one drawn on CT and warped onto SPECT, the other drawn directly on the SPECT. We found that the average center-to-center distance was 1.3 ± 0.8 pixels. Average overlap was 46 ± 20%. CT-SPECT fusion helped differentiate tumor from normal blood pool, necrotic areas within viable tumor, tumor recurrence from scar, and malignant lymphadenopathy from hyperplasia. We conclude that fusion of CT and SPECT augments the information provided by each separate modality. Future clinical applications of fusion in NSCLC staging using immunoscintigraphy appear promising.
---
paper_title: Synergistic value of single-photon emission computed tomography/computed tomography fusion to radioimmunoscintigraphic imaging of prostate cancer.

paper_content:

The rationale on which positron emission tomography/computed tomography (PET/CT) imaging is based, combining the functional features of PET with the anatomic detail of CT, provides many advantages that are easily transferable to single-photon emission computed tomography (SPECT)/CT imaging. Our efforts have focused on applying fused SPECT/CT imaging to identify prostate cancer and its metastasis and recurrence through radioimmunoscintigraphy (RIS). This application of RIS to imaging prostate cancer requires 2 key components: (1) a well-defined target associated with the cancer and (2) a "magic bullet" to seek that target. A well-characterized RIS target for prostate cancer is prostate-specific membrane antigen, or PSMA, and finding the bullet to seek this target with high sensitivity and specificity has been the focus of intensive study for nearly two decades. One of the candidate bullets developed is capromab pendetide, which is a monoclonal antibody that seeks PSMA. This antibody is commercially available as ProstaScint, which can be labeled with indium-111 to localize prostate cancer via SPECT imaging. In the course of applying fused SPECT/CT ProstaScint imaging to more than 800 prostate cancer cases, numerous refinements to our protocol have evolved that are aimed at staging the cancer with utmost accuracy. In addition to optimizing the localization of prostate cancer and its metastasis, these refinements also have been extended toward guiding both the implantation of radioactive seeds in brachytherapy and in other types of radiation therapy which is illustrated through 5 case reports. Progress in the therapeutic targeting of PSMA is also being actively explored, which has more universal ramifications because PSMA is found in the neovasculature of other types of cancers.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Medical imaging fusion applications: An overview

paper_content:

Computer aided fusion of multi-modality medical images provides a very promising diagnostic tool with numerous clinical applications. The objective of this paper is to present an overview of medical imaging fusion techniques with an emphasis on the use of neural network algorithms. Case studies derived from oncology (data level fusion), microscopy and ultrasound imaging (feature level and decision level fusion), and lesion placement in pallidotomy (data level fusion) are presented. It is anticipated that these tools will help the physician towards a more realistic and quantitative, assessment of disease.
---
paper_title: 2304 STEREOTACTIC PROSTATE BIOPSY WITH PRE-INTERVENTIONAL MRI AND LIVE US FUSION

paper_content:

INTRODUCTION AND OBJECTIVES: The key challenge for focal therapy of prostate cancer is to identify index lesions. In this context, we describe our initial experience with a new stereotactic prostate biopsy system, which integrates pre-interventional MRI data with periinterventional ultrasound for perineal prostate biopsies. METHODS: 33 patients were stereotactically biopsied (mean age of 64yrs., mean PSA level of 8.2ng/ml and mean prostate vol. of 40ml). 21 of these 33 patients had already had a negative transrectal US-guided biopsy. All men underwent multimodal 3 Tesla MRI without endorectal coil including diffusion weighted imaging and dynamic contrast enhanced sequences as well as MR-spectroscopy. Suspicious lesions were marked by radiologists with over 10 years experience in reading prostate MRI before the obtained data were transferred to the stereotactic biopsy system. Using a custom-made biplane TRUS probe mounted on a stepper device, 3D ultrasound data were generated to set the anatomic landmarks. Then MRI and TRUS imaging data were fused manually. As a result, the suspicious MRI lesions were superimposed onto the TRUS data. Next, 3D biopsy planning was performed including systematic biopsies from the peripheral and transitional zones of the prostate. Perineal biopsies were taken under live US imaging, and the location of each biopsy was documented in 3D. RESULTS: 14 out of 33 patients were diagnosed with prostate cancer. These results showed a positive correlation between MRI findings and histopathology in 22 out of 33 patients. In MRI lesions marked as highly suspicious, the detection rate was 100%. Evaluating the biopsies from lesions marked as highly and as questionable suspicious together, prostate cancer was detected in 27.95%. In comparison, only 8% of the additional systematic biopsies were positive. Target registration error of the first 554 biopsy cores was 1.9 mm. For adverse effects, one patient experienced urinary retention. Postinterventional hemorrhage or urinary tract infection did not occur. CONCLUSIONS: Perineal stereotactic prostate biopsies guided by the combination of MRI and ultrasound enable effective examination of suspicious MRI lesions. Additionally, each biopsy core taken may be documented accurately for its location in 3D. Thus, MRI data may be validated and different treatment options stratified in depth. At the same time the morbidity of the procedure was minimal.
---
paper_title: A volumetric and dosimetric study of LDR brachytherapy prostate implants based on image fusion of ultrasound and computed tomography

paper_content:

Transrectal ultrasound has been the imaging modality of choice for brachytherapy seed implantation. However, CT scanning has been used by some investigators. Here, the authors first examine the differences in the volume of the prostate as defined based on US and CT imaging studies, where the two image sets were fused together. They also investigated the dosimetric implications of the respective plans. All patients in the study had a CT and a transrectal US exam one month after the iodine seed implantation. A catheter with a balloon was placed in the patient's urethra with contrast. The balloon was pulled firmly against the anterior apex of the bladder. The two image sets were then fused together, using the shape of the urethra as the anatomical landmark for the co-registration. The prostate volumes from the two studies were computed as well as the DVH and dose distributions. An over-estimation of the prostate volume as seen on the US of up to 30% was found with the CT images. Spatially, the relative position of the two volumes is uncorrelated. The DVH and dose distributions showed better than 85% coverage for the US volume but mediocre coverage (as low as 50%) for the CT volume. CT imaging of the prostate tends to over-estimate its size. In post implantation studies where a CT is obtained for the localization of seeds, that volumetric discrepancy needs to be taken into account when assessing the quality of the implant.
---
paper_title: Multimodality data fusion aids early detection of breast cancer using conventional technology and advanced digital infrared imaging

paper_content:

The fusion of data from the various screening modalities will lead to a cost effective breast cancer detection system that is certain to improve the early part of early breast cancer detection methods. We explore the integration of data from mammography, ultrasound, clinical evaluation, and CAD, with noninvasive digital infrared imaging data from the BreastScan IR system, to improve early diagnosis. This will address a new solution to the screening of young women and women with dense breasts.
---
paper_title: Medical image fusion by wavelet transform modulus maxima

paper_content:

Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.
---
paper_title: Ultrasound image fusion for external beam radiotherapy for prostate cancer.

paper_content:

PURPOSE ::: To determine whether real-time ultrasound imaging and targeting system for the treatment of prostate cancer was feasible. The initial phase of this project included a study to develop and determine (a) software for the fusion of ultrasound images to standard x-rays obtained during simulation, and (b) the potential reduction in field size with real-time imaging. ::: ::: ::: METHODS AND MATERIALS ::: During 13 patient simulations a transrectal ultrasound image was obtained. Orthogonal x-ray films were acquired with the rectal probe in place. Both the x-ray and ultrasound images were digitized and a fusion image was created of the prostate position in relation to the probe, bladder, and rectum. The two-dimensional area of the rectum, bladder, and prostate was determined in the lateral projection. Potential conformal blocks were designed for the lateral portals in a four-field treatment technique. ::: ::: ::: RESULTS ::: The transrectal ultrasound probe enabled real-time prostate imaging. The lateral field size can be reduced to 6.08 x 5.68 cm2 +/- 0.62 x 0.48 cm2 from the standard 8 x 8 cm2 field. The posterior rectal wall was physically displaced out of the lateral field. The area of the rectum included in the lateral field is 1.75 cm2 +/- 0.85 cm2. ::: ::: ::: CONCLUSION ::: The prostate position can be determined with certainty on a regular basis with transrectal ultrasonography. The amount of normal tissue in the high dose volume can be reduced. This approach may reduce acute and chronic morbidity and allow further dose escalation.
---
paper_title: Design of a simple ultrasound/CT fusion image fusion solution for the evaluation of prostate seed brachytherapy

paper_content:

The purpose of this project is to devise a solution that will enable physicians, medical physicists and dosimetrists to quantitatively evaluate the success of prostate brachytherapy procedures. It is proposed that a simple ultrasound/CT image fusion solution can be developed and implemented to improve the current post-surgical evaluation process.
---
paper_title: Integrated PET/CT Fusion Imaging and Endoscopic Ultrasound in the Pre-operative Staging and Evaluation of Esophageal Cancer

paper_content:

PURPOSE ::: Accurate staging of esophageal cancer (ECA) is critical in determining appropriate therapy. Endoscopic ultrasound (EUS), computed tomography (CT) and positron emission tomography (PET) scanning can be used, but limited data exists regarding the use of combined PET/CT fusion imaging and EUS in ECA staging. The objective of this study is to evaluate the role of integrated PET/CT imaging and EUS in the staging of ECA. ::: ::: ::: PROCEDURES ::: Identification of patients diagnosed with ECA from 2004 to 2007 that underwent staging PET/CT and EUS. Data regarding tumor detection, lymph node identification, presence of metastatic disease, and affect on patient management were collected and compared between PET/CT and EUS. ::: ::: ::: RESULTS ::: Eighty-one patients (65 male, 16 female) were identified with mean age of 63.5 years who underwent EUS and PET/CT to stage known ECA. PET/CT identified the primary tumor in 74/81 (91.4%) of cases, compared to 81/81 (100%) with EUS. Locoregional adenopathy was seen by PET/CT in 29/81 (35.8%) of cases, compared to 49/81 (60.5%) by EUS (p = 0.0001). PET/CT identified celiac axis adenopathy in 8/81 (9.9%) of cases, compared to 11/81 (13.6%) with EUS (p = 0.5050). PET/CT identified 17/81 (21.0%) of patients with distant metastases who subsequently did not undergo attempt at curative surgical resection. ::: ::: ::: CONCLUSIONS ::: In ECA, EUS is superior to PET/CT for T staging and in identifying locoregional nodes, while PET/CT provides M staging. EUS and integrated PET/CT appear to independently affect treatment decisions, indicating complimentary and necessary roles in the staging of ECA.
---
paper_title: Histology and ultrasound fusion of excised prostate tissue using surface registration

paper_content:

We present a method for combining histological data with 3D ultrasound of excised prostate tissue using surface registration. With this technique, 3D prostate cancer lesions can be properly located and visualized within a B-scan volume for tissue characterization comparisons. Three prostate specimens were scanned with a GE Logiq 700 (Expert series) to obtain 2D B-scan sequences. The prostate boundaries were manually segmented from each sequence and reconstructed into 3D volumes. Specimens were fixed, sectioned into slabs, then mounted whole onto slides. Cancerous lesions were outlined by a pathologist. The slides were photographed with an Optronics Spot digital camera using Image-Pro Plus software on a PC. The gland surface was manually segmented from 2D histology images and reconstructed into a volume. Various reconstruction problems were addressed, such as specimen shrinkage due to the fixing and staining processes. The fusion algorithm translates and rotates one surface volume in 3D to find the best surface overlap. The resulting geometric transform is used to reorient the original image volume. The displacement error was determined by measuring the urethra offsets in final volume cross-sections. The offset distance ranged from 1.25 mm to 3.45 mm with an average of 2.36 mm. Another measure to gauge volume alignment is to calculate the ratio of overlapping voxels to total combined voxels(intersection/union). For a perfect case, this ratio will be 1. The ratios for the three cases ranged in value from 0.774 to 0.845.
---
paper_title: CT-ultrasound fusion prostate brachytherapy: a dynamic dosimetry feedback and improvement method. A report of 54 consecutive cases.

paper_content:

PURPOSE ::: The authors describe a prostate brachytherapy technique with dynamic dosimetry feedback, using coregistered CT and ultrasound (US) images, to map initial dosimetry deficiencies and guide remedial source placement. ::: ::: ::: METHODS AND MATERIALS ::: Fifty-four consecutive patients treated with this method were analyzed for coregistration accuracy and dosimetry outcomes by evaluating the prostate V100, V150, D90, and urethral D50 and D10. Dosimetric improvements created by remedial source placement and preplan/postplan prostate D90 agreement were evaluated. ::: ::: ::: RESULTS ::: Median CT-US coregistration discrepancy with this technique ranged from 0 to 4mm, with the posterior midline prostate and base prostate providing the least consistent and the urethra providing the most consistent coregistration agreement. Final prostate V100 values ranged from 96.1% to 99.8% for all patients. The addition of remedial sources directed by CT-US fusion produced V100 and D90 improvements whose magnitude inversely correlated with the initial result and exceeded the effect of adding quantitatively identical randomly distributed increased millicuries. The final prostate D90 result agreed within (-) 5% to (+) 10% of the preplan result in 98% of all patients. ::: ::: ::: CONCLUSIONS ::: CT-US fusion prostate brachytherapy represents a dynamic dosimetry feedback and remediation method that consistently produced high prostate V100 and D90 values with acceptably low urethra D50 and D10 values in our study. The degree of prostate V100 and D90 dosimetry improvement created by remedial source placement effectively matched the degree of initial dosimetry deficiency. This method produced a high level of correlation between the preplan and final prostate D90 values.
---
paper_title: The fusion of anatomic and physiologic imaging in the management of patients with cancer.

paper_content:

Imaging is of major clinical importance in the noninvasive evaluation and management of patients with cancer. Computed tomography (CT) and other anatomic imaging modalities, such as magnetic resonance imaging (MRI) or ultrasound, have a high diagnostic ability by visualizing lesion morphology and by providing the exact localization of malignant sites. Nuclear medicine provides information on the function and metabolism of cancer. Over the last decade, there have been numerous attempts to combine data obtained from different imaging techniques. Fused images of nuclear medicine and CT (or to a lesser extent, MRI) overcome the inherent limitations of both modalities. Valuable physiologic information benefits from a precise topographic localization. Coregistered data have been shown to be useful in the evaluation of patients with cancer at diagnosis and staging, in monitoring the response to treatment, and during follow up, for early detection of recurrence. Time-consuming and difficult realignment and computation for fusion of independent studies have, until now, limited the use of registration techniques to pilot studies performed in a small number of patients. The development of the new technology of single photon emission computed tomography/CT and positron emission tomography/CT that allows for combined functional and anatomic data acquisition has the potential to make fusion an everyday clinical tool.
---
paper_title: Medical imaging fusion applications: An overview

paper_content:

Computer aided fusion of multi-modality medical images provides a very promising diagnostic tool with numerous clinical applications. The objective of this paper is to present an overview of medical imaging fusion techniques with an emphasis on the use of neural network algorithms. Case studies derived from oncology (data level fusion), microscopy and ultrasound imaging (feature level and decision level fusion), and lesion placement in pallidotomy (data level fusion) are presented. It is anticipated that these tools will help the physician towards a more realistic and quantitative, assessment of disease.
---
paper_title: Fusion of Medical Microscopic Images Based on Estimation and Compensation

paper_content:

Image mosaicing can solve the problem that only very limited field of view is available when highly magnifying lens is used in microscopes imaging. The paper puts forward an equation of the system's influence to the grayscale distribution of the medical microscopic images, based on the analysis of the influence. Then, by the system equation and the accurate registration equation, the least-squares optimization error function can be formulated. By these equations, a set of transcendental equations is formed. We can estimate the system's influence to the gray-level of the images by singular-value decomposition, based on the equations set, and compensating for it. This permits image mosaicing seamlessly, without lowering the speed.
---
paper_title: ULTRASONOGRAPHY AND FLUOROSCOPIC FUSION FOR PROSTATE BRACHYTHERAPY DOSIMETRY

paper_content:

Abstract Purpose : To investigate the feasibility of performing postimplant and intraoperative dosimetry for prostate brachytherapy by fusing transrectal ultrasound (TRUS) and fluoroscopic data. Methods and Materials : Registration of ultrasound (prostate boundary) and fluoroscopic (seed) data requires spatial markers that are detectable by both imaging modalities. In this study, the needle tips were considered as such fiducials. Prostate phantoms were implanted with the seeds, and four localization needles were inserted. In the TRUS frame of reference, the longitudinal coordinate of the needle tip was determined by advancing the needle until the echo from its tip just registered at a known probe depth. The tip’s transverse coordinates were determined from the associated TRUS slice. The three-dimensional needle tip positions were also calculated in the fluoroscopic coordinate system using a seed reconstruction method. The transformation between the TRUS and fluoroscopy coordinate systems was established by the least-squares solution using the singular value decomposition. Results : With three of four needle tips as fiducials and the one remaining needle as a test target, the mean fiducial registration error was 0.8 mm and the test target registration error was 2.5 mm. When all four points were used for registration, the errors decreased to 1.1 mm. A comparison between the proposed method and CT-based dosimetry yielded a percentage of prostate volume receiving 100% and 150% of the prescribed minimal peripheral dose and minimal dose received by 90% of the prostate gland that agreed within 0.4%, 2.7%, and 4.2%, respectively. Conclusion : The combination of TRUS and fluoroscopy is a feasible alternative to the currently used CT-based postimplant dosimetry. Furthermore, because of online imaging capability, the method lends itself to real-time intraoperative applications.
---
paper_title: UWB microwave imaging for breast cancer detection: tumor/clutter identification using a time of arrival data fusion method

paper_content:

This paper proposed a novel scheme for early breast cancer detection using UWB microwave imaging. By applying the CLEAN algorithm to process the received signals at each sensor, the IR components can be successfully resolved. The locations and scattering strengths of dominant scatterers are then estimated based on the TOA data fusion concept. Simulation results have indicated the feasibility of detecting a tumor of 2 mm at depth of 2 cm. As compared to the existing techniques, the proposed approach is more computationally-efficient since the scanning process is localized at a few candidate locations. A number of issues that need to be addressed in future include: (i) effectiveness of the proposed approach for 3-D phantoms with realistic breast shapes and tissue compositions; (ii) extension of analysis to more complicated antenna array systems; (iii) other signal processing algorithms that may provide better IR component extraction and scatterer location estimation.
---
paper_title: Fusion Viewer: A New Tool for Fusion and Visualization of Multimodal Medical Data Sets

paper_content:

A new application, Fusion Viewer, available for free, has been designed and implemented with a modular object-oriented design. The viewer provides both traditional and novel tools to fuse 3D data sets such as CT (computed tomography), MRI (magnetic resonance imaging), PET (positron emission tomography), and SPECT (single photon emission tomography) of the same subject, to create maximum intensity projections (MIP) and to adjust dynamic range. In many situations, it is desirable and advantageous to acquire biomedical images in more than one modality. For example, PET can be used to acquire functional data, whereas MRI can be used to acquire morphological data. In some situations, a side-by-side comparison of the images provides enough information, but in most of the cases it may be necessary to have the exact spatial relationship between the modalities presented to the observer. To accomplish this task, the images need to first be registered and then combined (fused) to create a single image. In this paper, we discuss the options for performing such fusion in the context of multimodal breast imaging. Additionally, a novel spline-based dynamic range technique is presented in detail. It has the advantage of obtaining a high level of contrast in the intensity range of interest without discarding the intensity information outside of this range while maintaining a user interface similar to the standard window/level windowing procedure.
---
paper_title: Multimodality data fusion aids early detection of breast cancer using conventional technology and advanced digital infrared imaging

paper_content:

The fusion of data from the various screening modalities will lead to a cost effective breast cancer detection system that is certain to improve the early part of early breast cancer detection methods. We explore the integration of data from mammography, ultrasound, clinical evaluation, and CAD, with noninvasive digital infrared imaging data from the BreastScan IR system, to improve early diagnosis. This will address a new solution to the screening of young women and women with dense breasts.
---
paper_title: MRI-CT fusion to assess postbrachytherapy prostate volume and the effects of prolonged edema on dosimetry following transperineal interstitial permanent prostate brachytherapy.

paper_content:

PURPOSE ::: Quality assurance through postplan assessment is an integral part of permanent seed prostate implants. The use of MRI-CT fusion for 1-month postimplant dosimetry permits accurate assessment of prostate volume without seed induced artifact and uncertainties of prostate contour inherent to CT assessments. Routine use of MRI-CT fusion reveals significant prostate edema may persist several weeks. This study evaluates the effect of edema, and its subsequent resolution, on dosimetry. ::: ::: ::: METHODS AND MATERIALS ::: From May 2001 to June 2003, 241 men were treated with (125)I seed implants based on a transrectal ultrasound (TRUS) preplan. Quality assessment was performed at 1 month by CT-MRI fusion using VariSeed software. Over this 24-month period, 29 patients (12%) with residual edema at 1 month (12-60% >TRUS plan volume), had repeat CT-MRI fusion at 2-4 months to reassess volume and dosimetry. Eleven of the 29 had received prior androgen ablation to shrink the prostate preimplant. ::: ::: ::: RESULTS ::: For the entire group (n = 241), mean preimplant prostate volume was 33.7 cc and median postplan dosimetric parameters were: V100, 92.2%; D90, 153 Gy; and V150, 53%. For the 29 patients with prolonged edema, mean preimplant volume was 34.8 cc and 1-month volume was 46.1 cc (p <0.001). Mean volume reduction between 1 and 2 months was 13%. The decrease in prostate volume had a significant effect on dosimetry with median increase between 1 month and 2 months in calculated V100 of 9.5%, V150 22.6%, V200 30.1%, and D90 11.5%. ::: ::: ::: CONCLUSIONS ::: Significant residual edema is seen 1-month postimplant in 12% of prostates and may have a profound effect on dosimetry. Further study is underway to characterize the time course of resolution of the edema, and to perform integral dosimetry based on the changing volume.
---
paper_title: Fusion of dynamic contrast-enhanced magnetic resonance mammography at 3.0 T with X-ray mammograms: Pilot study evaluation using dedicated semi-automatic registration software

paper_content:

Abstract Rationale and objectives To evaluate the semi-automatic image registration accuracy of X-ray-mammography (XR-M) with high-resolution high-field (3.0 T) MR-mammography (MR-M) in an initial pilot study. Material and methods MR-M was acquired on a high-field clinical scanner at 3.0 T (T1-weighted 3D VIBE ± Gd). XR-M was obtained with state-of-the-art full-field digital systems. Seven patients with clearly delineable mass lesions >10 mm both in XR-M and MR-M were enrolled (exclusion criteria: previous breast surgery; surgical intervention between XR-M and MR-M). XR-M and MR-M were matched using a dedicated image-registration algorithm allowing semi-automatic non-linear deformation of MR-M based on finite-element modeling. To identify registration errors (RE) a virtual craniocaudal 2D mammogram was calculated by the software from MR-M (with and w/o Gadodiamide/Gd) and matched with corresponding XR-M. To quantify REs the geometric center of the lesions in the virtual vs. conventional mammogram were subtracted. The robustness of registration was quantified by registration of X-MRs to both MR-Ms with and w/o Gadodiamide. Results Image registration was performed successfully for all patients. Overall RE was 8.2 mm (1 min after Gd; confidence interval/CI: 2.0–14.4 mm, standard deviation/SD: 6.7 mm) vs. 8.9 mm (no Gd; CI: 4.0–13.9 mm, SD: 5.4 mm). The mean difference between pre- vs. post-contrast was 0.7 mm (SD: 1.9 mm). Conclusion Image registration of high-field 3.0 T MR-mammography with X-ray-mammography is feasible. For this study applying a high-resolution protocol at 3.0 T, the registration was robust and the overall registration error was sufficient for clinical application.
---
paper_title: MRI- Mammography 2D/3D Data Fusion for Breast Pathology Assessment

paper_content:

Increasing use is being made of contrast-enhanced Magnetic Resonance Imaging (Gd-DTPA) for breast cancer assessment since it provides 3D functional information via pharmacokinetic interaction between contrast agent and tumour vascularity, and because it is applicable to women of all ages. Contrast-enhanced MRI (CE-MRI) is complimentary to conventional X-ray mammography since it is a relatively low-resolution functional counterpart of a comparatively high-resolution 2D structural representation. However, despite the additional information provided by MRI, mammography is still an extremely important diagnostic imaging modality, particularly for several common conditions such as ductal carcinoma in-situ (DCIS) where it has been shown that there is a strong correlation between microcalcification clusters and malignancy [1]. Pathological indicators such as calcifications and fine spiculations are not visible in CE-MRI and therefore there is clinical and diagnostic value to fusing the high-resolution structural information available from mammography with the functional data acquired from MRI imaging. This paper presents a novel data fusion technique whereby medio-lateral (ML) and cranio-caudal (CC) mammograms (2D data) are registered to 3D contrast-enhanced MRI volumes. We utilise a combination of pharmacokinetic modelling, projection geometry, wavelet-based landmark detection and thin-plate spline non-rigid registration to transform the coordinates of regions of interest (ROIs) from the 2D mammograms to the spatial reference frame of the contrast-enhanced MRI volume.
---
paper_title: A feedback retina model for improving medical images fusion

paper_content:

Image fusion has become a powerful technique for increasing the interpretation quality of images in medical applications. The diagnostic potential of brain positron emission tomography (PET) imaging is limited by low spatial resolution. For solving this problem, the high-frequency part of the MRI, which would be unrecoverable by the set PET acquisition system, is extracted and added to the PET image. The procedure has the potential of increasing the diagnostic value of a PET image. This paper presents a feedback retina model technique to reduce the spectral distortion and preserve high spatial resolution. Visual and statistical analyses show that the proposed feedback retina model significantly improves the fusion quality compared to non-feedback retina model.
---
paper_title: Evaluation of a Semiautomatic 3D Fusion Technique Applied to Molecular Imaging and MRI Brain/Frame Volume Data Sets

paper_content:

A generally applicable 3D fusion method was evaluated using molecular imaging and MRI volumetric data sets from 15 brain tumor patients with stereotactic frames attached to their skull. Point pairs, placed on the frame only, were chosen, polynomial warping coefficients were generated to map voxels from one coordinate space to the other. The MRI frame was considered the reference structure and the standard for “correct” registration. An ANOVA test (p > 0.05) confirmed the point pair choice to be consistent. The 95% confidence interval for the t-test showed the measured distance difference between the registered volumes was within one MRI voxel. A further experiment was conducted to independently evaluate the brain registration based on testing for consistency of randomly selected interior/exterior points. A t-test result (p < 0.05) showed that the consistency (i.e., both interior or both exterior) before and after volume registration were significantly different. This fusion method may be a viable alternative when other methods fail.
---
paper_title: Information Fusion for Probabilistic Reasoning and Its Application to the Medical Decision Support Systems

paper_content:

Paper deals with the knowledge acquisition process problems. Different experts formulate the rules for decision support system. We assume they have different knowledge about the problem and therefore obtained rules have different qualities. Additional some of them can be generated by machine learning algorithms – of course the quality of information stored in the databases are different. We will formulate the proposition of the confidence measure of knowledge. We will show some of its applications to the decision process. We will propose how use proposed measure for typical probabilistic decision process. The presented concepts will be applied to the real medical decision problem based on boosting idea.
---
paper_title: Multi-level Classifier Design for Tumor Micro-image Based on Multi-feature Fusion

paper_content:

In this paper, we propose a computer aided recognition system. With a series of approaches of image pre-processing and segmentation, extract the whole image feature namely global features, such as the number of cell, also extract the single cell feature namely local features, for example cell area and so on. Then separately do the multi-feature fusion for global feature and local feature. We construct two level of classifier, the first level is global classifier based on the most-short distance which is designed according to the global features; the second level is local classifier based on decision tree which is designed according to the local features design. Firstly according to global classifier to judge whether the stomach epidermis is normal or not, if the image is recognized to be abnormal, the classification ends. Otherwise, we recognize the image by the local classifier once again. After two level of recognitions, The experiment result indicates that this classifier can enormously enhance the classification accuracy rate.
---
paper_title: A Statistical Motion Model Based on Biomechanical Simulations for Data Fusion during Image-Guided Prostate Interventions

paper_content:

A method is described for generating a patient-specific, statistical motion model (SMM) of the prostate gland. Finite element analysis (FEA) is used to simulate the motion of the gland using an ultrasound-based 3D FE model over a range of plausible boundary conditions and soft-tissue properties. By applying principal component analysis to the displacements of the FE mesh node points inside the gland, the simulated deformations are then used as training data to construct the SMM. The SMM is used to both predict the displacement field over the whole gland and constrain a deformable surface registration algorithm, given only a small number of target points on the surface of the deformed gland. Using 3D transrectal ultrasound images of the prostates of five patients, acquired before and after imposing a physical deformation, to evaluate the accuracy of predicted landmark displacements, the mean target registration error was found to be less than 1.9mm.
---
paper_title: Early tumor detection by multiple infrared unsupervised neural nets fusion

paper_content:

The unsupervised classification algorithm called Lagrange constraint neural network (LCNN) has been successfully applied to the sub-pixel multispectral remote sensing, [25]. Here, we apply the LCNN to the early breast cancer detection using two-color mid and long infrared images of the breast. This could be a new paradigm shift that enabled smart neural network algorithm to sort out the underlying malignant heat sources for physician diagnoses. The nonintrusive 2-color passive infrared imaging that could be repeated for record track with no radiation hazard seems to be alternative paradigm shift for the first-line screening against breast cancer. The sub-pixel super-resolution capability of the remote sensing is equivalent to the sub-milimeter scaling of the close-up breast imaging for the vascular and the angiogenesis effects. We demonstrate the potential benefit of the multicolor mid & long infrared imaging capable for detecting the abnormal under-skin thermal textures as well as stage-zero detection of the ductal carcinoma in situ.
---
paper_title: Oral cancer detection in fluorescent image by color image fusion

paper_content:

Biopsy is the main diagnosis method for oral cancer, which is the fifth most prevalent cancer in male population in Taiwan. In order to increase the accuracy of diagnosis, it is required to reveal the distribution of malignancy prior to biopsy. Based on this purpose, we developed the fluorescent image system associated with color image fusion algorithm to indicate the cancer-susceptible area in the oral image. The system therefore can assist physicians to make appropriate biopsy.
---
paper_title: Fusion of Magnetic Resonance and Scintimammography Images for Breast Cancer Evaluation: A Pilot Study

paper_content:

BACKGROUND ::: To achieve a more specific method to estimate the real size of breast cancer, we have developed a method to fuse magnetic resonance imaging (MRI) and scintimammography (SM) images. The aim of this study was to assess the feasibility of this method and to evaluate its accuracy to measure the size of breast cancer compared with MRI alone, mammography, and clinical examination, employing pathologic size as the gold standard. ::: ::: ::: METHODS ::: Twenty consecutive breast cancer women at stages IIA-IIIA, scheduled for mastectomies, underwent SM with (99m)Tc-sestamibi and MRI with gadolinium 2-10 days before surgery. All patients had had recent mammographies and were examined clinically. Software was developed in visual language to perform the fusion between MRI and SM images and tumor measurements (MRI/SM). The tumor size, in 3 diameters (anteroposterior, longitudinal, and transverse), for each examination was correlated with pathological measurements using linear regression. ::: ::: ::: RESULTS ::: The MRI/SM technique was successfully performed in all patients, and the principal tumor was measured by this method. The MRI/SM cancer measurements correlated better with pathology than MRI, mammography, and clinical exam in all diameters analyzed (r = 0.88, 0.81, 0.81; SE = 0.11, 0.14, 0.11 in anteroposterior, longitudinal, and transverse diameters, respectively). ::: ::: ::: CONCLUSIONS ::: The MRI/SM is a feasible technique and appears to be more accurate than other examinations (MRI alone, mammography, and clinical exam) to measure breast cancer size.
---
paper_title: Information Fusion for Perceptual Feedback: A Brain Activity Sonification Approach

paper_content:

When analysing multichannel processes, it is often convenient to use some sort of “visualisation” to help understand and interpret spatio-temporal dependencies between the channels, and to perform input variable selection. This is particularly advantageous when the levels of noise are high, the “active” channel changes its spatial location with time, and also for spatio-temporal processes where several channels contain meaningful information, such as in the case of electroencephalogram (EEG)-based brain activity monitoring. To provide insight into the dynamics of brain electrical responses, spatial sonification of multichannel EEG is performed, whereby the information from active channels is fused into music-like audio. Owing to its “data fusion via fission” mode of operation, empirical mode decomposition (EMD) is employed as a time–frequency analyser, and the brain responses to visual stimuli are sonified to provide audio feedback. Such perceptual feedback has enormous potential in multimodal brain computer and brain machine interfaces (BCI/BMI).
---
paper_title: The importance of postoperative CT image fusion verification of stereotactic interstitial irradiation for brain tumors.

paper_content:

PURPOSE ::: The aim of our study was verification of the position of implanted catheters with (125)I seeds after the catheter implantation for the brachytherapy of brain tumors. ::: ::: ::: METHODS AND MATERIALS ::: The fusion of the CT image used at planning and after the implantation of the catheters enabled us to verify the position of the catheters containing the isotopes. After this, the tumor volume covered by the prescribed dose (TV(PD)) and the normal tissue volume covered by the prescribed dose (NTV(PD)) were compared between the plan and the actual result. The image fusion was performed by the BrainLab-Target 1.19 software on an Alfa 430 (Digital) workstation. ::: ::: ::: RESULTS ::: The position of the catheters was adjusted in 14 (20%) of the 70 image fusion cases being studied. The position of 16 of the 116 catheters (13.8%) required adjustment after the fusion of control images in the 70 cases studied. The Student t probe revealed a significant difference between the TV(PD) values of the reality and the plans (75.8% vs. 92.4%, p < 0.0001). There was a significant difference between values of the real performances and planning for NTV(PD) (86.8% vs. 76%, p = 0.001) and for the conformity index (0.37 vs. 0.54, p = 0.0001), too. ::: ::: ::: CONCLUSION ::: The application of the interstitial irradiation with CT control allows us to identify and correct possible inaccuracies in catheter positioning during the operation. The procedure then becomes far more accurate and reliable, and as such, the irradiation becomes more conformal than without catheter adjustment.
---
paper_title: Fuzzy Information Fusion Scheme Used to Segment Brain Tumor from MR Images

paper_content:

A fuzzy information fusion scheme is proposed in this paper to automatically segment tumor areas of human brain from multispectral magnetic resonance images such as T1-weighted, T2-weighted and Proton Density (PD) feature images. The proposed scheme consists of four stages: data-level fusion, space creation of fuzzy features, fusion of fuzzy features and fuzzy decision. Several fuzzy operators are proposed to create the feature-level fusion. The fuzzy information models describing the characteristics of tumor areas in human brain are also established. A fuzzy region growing based on fuzzy connecting is presented to obtain the final segmentation result. The comparison between the result of our method and the hand-labeled segmentation of a radiology expert shows that this scheme is efficient. The experimental results (based on 4 patients studied) show an average probability of correct detection equal to 96% and an average probability of false detection equal to 5%.
---
paper_title: Quantification of brain tissue volumes using MR/MR fusion

paper_content:

We propose in this article to compute White Matter (WM), Gray Matter (GM) and Cerebrospinal Fluid (CF) volumes from MR images of a human brain using possibilistic logic and a fusion scheme. The fusion process is divided into three steps: fuzzy tissue maps are first computed on all images using a possibilistic clustering algorithm. Fusion is then achieved for all tissues with a context based fusion operator. The final segmentation allows for the computation of brain tissue volumes. Applications on a brain model show a great agreement between the results and the true segmentation, thus affording a relevant tissue classification on clinical data.
---
paper_title: Decoding visual brain states from fMRI using an ensemble of classifiers

paper_content:

Decoding perceptual or cognitive states based on brain activity measured using functional magnetic resonance imaging (fMRI) can be achieved using machine learning algorithms to train classifiers of specific stimuli. However, the high dimensionality and intrinsically low signal to noise ratio (SNR) of fMRI data poses great challenges to such techniques. The problem is aggravated in the case of multiple subject experiments because of the high inter-subject variability in brain function. To address these difficulties, the majority of current approaches uses a single classifier. Since, in many cases, different stimuli activate different brain areas, it makes sense to use a set of classifiers each specialized in a different stimulus. Therefore, we propose in this paper using an ensemble of classifiers for decoding fMRI data. Each classifier in the ensemble has a favorite class or stimulus and uses an optimized feature set for that particular stimulus. The output for each individual stimulus is therefore obtained from the corresponding classifier and the final classification is achieved by simply selecting the best score. The method was applied to three empirical fMRI datasets from multiple subjects performing visual tasks with four classes of stimuli. Ensembles of GNB and k-NN base classifiers were tested. The ensemble of classifiers systematically outperformed a single classifier for the two most challenging datasets. In the remaining dataset, a ceiling effect was observed which probably precluded a clear distinction between the two classification approaches. Our results may be explained by the fact that different visual stimuli elicit specific patterns of brain activation and indicate that an ensemble of classifiers provides an advantageous alternative to commonly used single classifiers, particularly when decoding stimuli associated with specific brain areas.
---
paper_title: A Dynamic Multi-Channel Decision-Fusion Strategy to Classify Differential Brain Activity

paper_content:

A strategy is developed to dynamically fuse classification information from multiple channels in order to accurately classify brain activity elicited by external stimuli. The strategy is dynamic in the sense that different channels are selected at different time-instants. The channels are ranked at different time-instants according to their classification accuracies. Although the brain signals are multivariate signals, the classifiers are simple univariate classifiers. A rule is formulated to dynamically select different channels at different time-instants during the testing phase. The independent decisions of the selected channels are fused into a decision fusion vector. The resulting decision fusion vector is optimally classified using a discrete Bayes classifier. The dynamic decision fusion strategy is tested on 3 evoked potential (EP) data sets of 2 different paradigms using univariate mean and Gaussian classifiers. It is shown that the strategy yields high classification accuracies especially for high noise cases. Furthermore, the generalized formulation of the strategy makes it applicable to a wide range of multi-category classification problems involving multivariate signals collected from multiple sensors.
---
paper_title: Operational Brain Dynamics: Data Fusion Technology for Neurophysiological, Behavioral, and Scenario Context Information in Operational Environments

paper_content:

Classical laboratory studies of human performance have always required some form of data integration, such as the synchronization of stimulus display, behavioral accuracy, and reaction time. Studies of performance in operational environments have typically been limited in the precision of behavioral observations. As improved digital informatics have expanded the laboratory data acquisition from a few bytes to terabytes, there has been a similar expansion in both the opportunities and the challenges for data fusion.
---
paper_title: Automatic segmentation of subcortical brain structures in MR images using information fusion

paper_content:

Reports a new automated method for the segmentation of internal cerebral structures using an information fusion technique. The information is provided both by images and expert knowledge, and consists in morphological, topological, and tissue constitution data. All this ambiguous, complementary and redundant information is managed using a three-step fusion scheme based on fuzzy logic. The information is first modeled into a common theoretical frame managing its imprecision and incertitude. The models are then fused and a decision is taken in order to reduce the imprecision and to increase the certainty in the location of the structures. The whole process is illustrated on the segmentation of thalamus, putamen, and head of the caudate nucleus from expert knowledge and magnetic resonance images, in a protocol involving 14 healthy volunteers. The quantitative validation is achieved by comparing computed, manually segmented structures and published data by means of indexes assessing the accuracy of volume estimation and spatial location. Results suggest a consistent volume estimation with respect to the expert quantification and published data, and a high spatial similarity of the segmented and computed structures. This method is generic and applicable to any structure that can be defined by expert knowledge and morphological images.
---
paper_title: Dual-modality 3D brain PET-CT image segmentation based on probabilistic brain atlas and classification fusion

paper_content:

The increasing prevalence of dual medical imaging modalities, such as PET-CT scanners, poses both challenges and opportunities to image segmentation, as they provide distinct but complementary information. In this paper, we propose a novel segmentation algorithm for 3D brain PET-CT images, which classifies each voxel by fusing the voxel's memberships estimated from four points of view using the PET information, CT information, smoothness prior, and probabilistic brain atlas. All memberships having the same dynamic range greatly facilitates weighting the contribution of the four different information sources. The probabilistic brain atlas estimated for each PET-CT image from a set of training samples provides the anatomical information to the segmentation process. We compared the proposed algorithm to three single-classifier based methods, PET-based SPM algorithm, CT-based Otsu thresholding, and PET-CT based MAP-MRF algorithm. The experimental results in 11 clinical brain PET-CT studies demonstrate that the novel algorithm is capable of providing more accurate and reliable segmentation.
---
paper_title: Unscented Kalman Filter Based Sensor Fusion for Robust Optical and Electromagnetic Tracking in Surgical Navigation

paper_content:

Computer-aided surgery systems provide visual guidance to surgeons by showing the real-time pose of surgical instruments overlaid on preoperative medical images of the patient. Surgical instrument poses are localized in space using mainly optical tracking systems (OTS) and electromagnetic tracking systems (EMTS). OTS systems require clear line-of-sight, which is difficult to ensure in current overcrowded operating rooms. On the other hand, EMTSs provide less accuracy and suffer from magnetic field distortion in the presence of metal objects. In this paper, we propose a sensor fusion algorithm to compensate for the drawbacks of OTS and EMTS, and achieve robust tracking of surgical instruments. Spatial alignment of OTS and EMTS data will be achieved through a calibration procedure. The proposed sensor fusion algorithm uses an unscented Kalman filter (UKF), an extension of the standard Kalman filter based on a deterministic sampling of nonlinear functions. Quaternion representation for rotations is used to avoid singularities of other parameterizations (e.g., Euler angles). In cases of optical marker occlusion, our algorithm will take advantage of the EMTS to estimate the position of the hidden marker(s) and feed it in the UKF. The fusion algorithm will also keep track of an error matrix between OTS and EMTS measured poses, providing an up-to-date estimate of the electromagnetic distortion. This will be used to correct EMTS measurement before using it to compensate for possible optical line-of-sight occlusions. The proposed sensor fusion (SF) method is compared to a predictive UKF based on optical data (NSF). Our results show that the SF effectively compensates for short marker occlusion (nine samples) providing a continuous estimate of the instrument pose with an error significantly lower than the NSF method, and reaching a clinically acceptable accuracy. Furthermore, the proposed algorithm increases the accuracy of EMTS in the presence of magnetic field distortion.
---
paper_title: Hybrid surface- and voxel-based registration for MR-PET brain fusion

paper_content:

In this paper, we propose a novel technique of registration using hybrid approach for MR-PET brain image fusion. Hybrid approach uses merits of surface- and voxel-based registration. Thus, our method measures similarities using voxel intensities in MR images corresponding to the feature points of the brain in PET images. Proposed method selects the brain threshold using histogram accumulation ratio in PET images. And then, we automatically segment the brain using the inverse region growing with pre-calculated threshold and extract the feature points of the brain using sharpening filter in PET images. In order to find the optimal location for registration, we evaluate the Hybrid-based Cross-Correlation using the voxel intensities in MR images corresponding to the feature points in PET images. In our experiments, we evaluate our method using software phantom and clinical datasets in the aspect of visual inspection, accuracy, robustness, and computation time. Experimental results show that our method is dramatically faster than the voxel-based registration and more accurate than the surface-based registration. In particular, our method can robustly align two datasets with large geometrical displacement and noise at optimal location.
---
paper_title: The Application of Image Fusion in Stereotactic Brachytherapy of Brain Tumours

paper_content:

Summary Background. The visualization of any morphological volume (i.e. CT, MRI) together with an additional second morphological volume (i.e. CT, MRI) or functional data set, which may come from SPECT or PET, is a new method for treatment planning, verification and follow-up of interstitial irradiation. Method. The authors present their experience on interstitial irradiation of brain tumours with stereotactically implanted I-125 seeds supported by image fusion. The image fusion was performed by the BrainLab-Target 1.13 software on Alfa 430 (Digital) workstation before, during, and after interstitial irradiation of brain tumours with Iodine125 seeds. Results and Interpretation. On the basis of 20 brachytherapeutic image fusion of stereotactic CT (slices with fiducials) with additional stereotactic CT, MRI, PET and SPECT images provides more accurate and precise target volume, more exact localization of catheters and isotope seeds (verification fusion), differentiation between the localization and amount of the necrotic and proliferating parts of the tumours and shows the volume changes in consequence of interstitial irradiation. The image fusion should help to improve the accuracy and minimize the perifocal morbidity of interstitial irradiation.
---
paper_title: An ensemble based data fusion approach for early diagnosis of Alzheimer’s disease

paper_content:

As the number of the elderly population affected by Alzheimer's disease (AD) rises rapidly, the need to find an accurate, inexpensive and non-intrusive diagnostic procedure that can be made available to community healthcare providers is becoming an increasingly urgent public health concern. Several recent studies have looked at analyzing electroencephalogram (EEG) signals through the use of wavelets and neural networks. While showing great promise, the final outcomes of these studies have been largely inconclusive. This is mostly due to inherent difficulty of the problem, but also - perhaps - due to inefficient use of the available information, as many of these studies have used a single EEG channel for the analysis. In this contribution, we describe an ensemble of classifiers based data fusion approach to combine information from two or more sources, believed to contain complementary information, for early diagnosis of Alzheimer's disease. Our emphasis is on sequentially generating an ensemble of classifiers that explicitly seek the most discriminating information from each data source. Specifically, we use the event related potentials recorded from the Pz, Cz, and Fz electrodes of the EEG, decomposed into different frequency bands using multiresolution wavelet analysis. The proposed data fusion approach includes generating multiple classifiers trained with strategically selected subsets of the training data from each source, which are then combined through a modified weighted majority voting procedure. The implementation details and the promising outcomes of this implementation are presented.
---
paper_title: Discriminative FilterBank selection and EEG information fusion for Brain Computer Interface

paper_content:

Brain Computer Interface (BCI) provides a direct communication pathway between a human and an external device. In this paper, we propose a new ‘Discriminative FilterBank Common Spatial Pattern (DFBCSP)’ algorithm to select the subject-specific filters automatically during training for a motor imagery based BCI. The subject-specific filters are selected using the fisher ratio values of filtered Electroencephalogram (EEG) signal. The channel ‘C3’ alone could give sufficient information to select the discriminative filterbank for the proposed system. We have also explored the possibility of boosting the system performance by including dynamic temporal features. Fusion of static and dynamic features in the proposed DFBCSP frame work gave an average test accuracy of 92.44%, which is significantly better than conventional filterbank based Common Spatial Pattern algorithms.
---
paper_title: Data fusion and fuzzy spatial relationships for locating deep brain stimulation targets in magnetic resonance images

paper_content:

Symptoms of Parkinson's disease can be relieved through Deep Brain Stimulation. This neurosurgical technique relies on high precision positioning of electrodes in specific areas of the basal ganglia and the thalamus. In order to identify these anatomical targets, which are located deep within the brain, we developed a semi-automated method of image analysis, based on data fusion. Information provided by both anatomical magnetic resonance images and expert knowledge is managed in a common possibilistic frame, using a fuzzy logic approach. More specifically, a graph-based virtual atlas modeling theoretical anatomical knowledge is matched to the image data from each patient, through a research algorithm (or strategy) which simultaneously computes an estimation of the location of every structures, thus assisting the neurosurgeon in defining the optimal target. The method was tested on 10 images, with promising results. Location and segmentation results were statistically assessed, opening perspectives for enhancements.
---
paper_title: Fusion of Spatial Relationships for Guiding Recognition, Example of Brain Structure Recognition in 3D MRI

paper_content:

Spatial relations play an important role in recognition of structures embedded in a complex environment and for reasoning under imprecision. Several types of relationships can be modeled in a unified way using fuzzy mathematical morphology. Their combination benefits from the powerful framework of fuzzy set theory for fusion tasks and decision making. This paper presents several methods of fusion of information about spatial relationships and illustrates them on the example of model-based recognition of brain structures in 3D magnetic resonance imaging.
---
paper_title: Multimodal Fusion of Muscle and Brain Signals for a Hybrid-BCI

paper_content:

Practical Brain-Computer Interfaces (BCIs) for disabled people should allow them to use all their remaining functionalities as control possibilities. Sometimes these people have residual activity of their muscles, most likely in the morning when they are not exhausted. In this work we fuse electromyographic (EMG) with electroencephalographic (EEG) activity in the framework of a so called “Hybrid-BCI” (hBCI) approach. Thereby, subjects could achieve a good control of their hBCI independently of their level of muscular fatigue. Furthermore, although EMG alone yields good performance, it is outperformed by the hybrid fusing of EEG and EMG. Two different fusion techniques are explored showing graceful performance degradation in the case of signal attenuation. Such a system allows a very reliable control and a smooth handover if the subjects get exhausted or fatigued during the day.
---
paper_title: A general framework for the fusion of anatomical and functional medical images.

paper_content:

The collection of various data coming from anatomical and functional imagery is becoming very common for the study of a given pathology, and their aggregation generally allows for a better medical decision in clinical studies. A fusion process is described in this article for the modeling of this aggregation. The process is illustrated in the case of anatomical and functional images of the brain, but the general principle may be extended to other organs. The whole three-step fusion process based on possibilistic logic is detailed and a new class of fusion operator is introduced. The use of fuzziness in the process in general and in the operator in particular allows for the management of uncertainty and imprecision inherent to the images. The fusion process is illustrated in two clinical cases: the study of Alzheimer's disease by MR/SPECT fusion and the study of epilepsy by MR/SPECT/PET fusion. Results are presented and evaluated, and a preliminary clinical validation is achieved. The assessment of the method is encouraging, allowing its application on several clinical problems.
---
paper_title: A framework of fuzzy information fusion for the segmentation of brain tumor tissues on MR images

paper_content:

A framework of fuzzy information fusion is proposed in this paper to automatically segment tumor areas of human brain from multispectral magnetic resonance imaging (MRI) such as T1-weighted, T2-weighted and proton density (PD) images. A priori knowledge about tumors described by radiology experts for different types of MRI are very helpful to guide a automatic and a precise segmentation. However, the terminology used by radiology experts are variable in term of image signal. In order to benefit of these descriptions, we propose to modellize them by fuzzy models. One fuzzy model is built for one type of MRI sequence. The segmentation is finally based on a fusion of different fuzzy information obtained from different types of MRI images. Our algorithm consists of four stages: the registration of multispectral MR images, the creation of fuzzy models describing the characteristics of tumor, the fusion based on fuzzy fusion operators and the adjustment by fuzzy region growing based on fuzzy connecting. The comparison between the obtained results and the hand-tracings of a radiology expert shows that the proposed algorithm is efficient. An average probability of correct detection 96% and an average probability of false detection 5% are obtained through studies of four patients.
---
paper_title: Rapid subthalamic nucleus deep brain stimulation lead placement utilising CT/MRI fusion, microelectrode recording and test stimulation.

paper_content:

Subthalamic nucleus (STN) deep brain stimulation (DBS) has become an established treatment strategy for patients with medically refractory Parkinson’s disease (PD). There are however numerous strategies employed for STN lead placement. Variations include method of STN localisation, use of microelectrode recording, number of microelectrode recording passes and time taken for the procedure. We describe a relatively simple and rapid technique of STN lead placement utilising CT / MRI image fusion, microelectrode recording and test stimulation.
---
paper_title: Heterogeneous Data Fusion to Type Brain Tumor Biopsies

paper_content:

Current research in biomedical informatics involves analysis of multiple heterogeneous data sets. This includes patient demographics, clinical and pathology data, treatment history, patient outcomes as well as gene expression, DNA sequences and other information sources such as gene ontology. Analysis of these data sets could lead to better disease diagnosis, prognosis, treatment and drug discovery. In this paper, we use machine learning algorithms to create a novel framework to perform the heterogeneous data fusion on both metabolic and molecular datasets, including state-of-the-art high-resolution magic angle spinning (HRMAS) proton (1H) Magnetic Resonance Spectroscopy and gene transcriptome profiling, to intact brain tumor biopsies and to identify different profiles of brain tumors. Our experimental results show our novel framework outperforms any analysis using individual dataset.
---
paper_title: A surface-projection MMI for the fusion of brain MR and SPECT images

paper_content:

Recently, maximization mutual information (MMI) of image intensities has been proposed as a new matching criterion for automated multimodality image registration. However, the success of the MMI relies on the similarity of the histogram distribution between the images to be fused. This condition is usually hard to be achieved in practical application. Besides, MMI is time consuming because it needs to find an optimal solution about six parameters (three for shifts and three for rotations) during the registration process. To improve the performance of traditional MMI, a novel scheme named surface-projection MMI algorithm (SP-MMI) is proposed here. SP-MMI is a two-stage registration algorithm included MMI of surface projection and the following MMI of axial plane. The experimental results, using MR and SPECT images, to confirm the good performance of the proposed method in comparison with the traditional MMI method are also included.
---
paper_title: FMDIB: a software tool for fusion of MRI and DHC-SPECT images of Brain

paper_content:

Brain disorders are divided into two categories: anatomic and metabolic. Since brain disorders initially appear as metabolic abnormalities and ultimately result in anatomic defects, it is necessary to analyze the metabolic along with the anatomic aspect of the brain tissue in the preliminary stages of diagnosis to greatly increase the chances of a cure. It is possible that metabolic abnormalities exist without an anatomic defect such as Alzheimer disease. The brain anatomic images may achieve from MRI and physiologic and metabolic brain data can achieve from DHC-SPECT or PET system when a radiopharmaceutical such as FDG have been injected to the patient. For fusion of the brain MRI and DHC-SPECT images obtained from individual scanners, we developed the userfriendly FMDIB software in the MATLAB environment. With the help of this software, we resolved the geometrical misalignment of the brain images and subsequently co-registered them with a 1 mm precision. This software enabled us to perform the three dimensional rendering of the image, re-slicing the image with thickness of 1 to 3 mm, slicing the two dimensional DHC-SPECT projections into an arbitrary direction.
---
paper_title: Assistance to planning in deep brain stimulation: data fusion method for locating anatomical targets in MRI.

paper_content:

Symptoms of Parkinson's disease can be relieved through Deep Brain Stimulation. This neurosurgical technique relies on high precision positioning of electrodes in specific areas of the basal ganglia and the thalamus. In order to identify these anatomical targets, which are located deep within the brain, we developed a semi-automated method of image analysis, based on data fusion. Information provided by both anatomical magnetic resonance images and expert knowledge is managed in a common possibilistic frame, using a fuzzy logic approach. More specifically, a graph-based virtual atlas modeling theoretical anatomical knowledge is matched to the image data from each patient, through a research algorithm (or strategy) which simultaneously computes an estimation of the location of every structures, thus assisting the neurosurgeon in defining the optimal target. The method was tested on 10 images, with promising results. Location and segmentation results were statistically assessed, opening perspectives for enhancements.
---
paper_title: Improvement of brain lesions detection using information fusion approach

paper_content:

Automatic segmentation of brain lesions, such as multiple sclerosis in MRI images, is a complex operation. One of the main difficulties is to optimize the dilemma between the false positives and false negatives present in the segmented image. We propose here a new approach to this problem. The idea is to exploit the complementary results from different segmentation algorithms as well as a priori knowledge to reduce false positives. The method starts with modeling inaccuracy about the borders of the segmented regions. The logic rules are then defined in order to combine the white matter image and lesions within the framework of evidence theory. The results show that brain lesion detection is substantially improved using this data fusion approach.
---
paper_title: Evaluation of a Semiautomatic 3D Fusion Technique Applied to Molecular Imaging and MRI Brain/Frame Volume Data Sets

paper_content:

A generally applicable 3D fusion method was evaluated using molecular imaging and MRI volumetric data sets from 15 brain tumor patients with stereotactic frames attached to their skull. Point pairs, placed on the frame only, were chosen, polynomial warping coefficients were generated to map voxels from one coordinate space to the other. The MRI frame was considered the reference structure and the standard for “correct” registration. An ANOVA test (p > 0.05) confirmed the point pair choice to be consistent. The 95% confidence interval for the t-test showed the measured distance difference between the registered volumes was within one MRI voxel. A further experiment was conducted to independently evaluate the brain registration based on testing for consistency of randomly selected interior/exterior points. A t-test result (p < 0.05) showed that the consistency (i.e., both interior or both exterior) before and after volume registration were significantly different. This fusion method may be a viable alternative when other methods fail.
---
paper_title: Emulating the perceptual system of the brain for the purpose of sensor fusion

paper_content:

This work presents a bionic model derived from research findings about the perceptual system of the human brain to build next generation intelligent sensor fusion systems. Therefore, a new information processing principle called neuro-symbolic information processing is introduced. According to this method, sensory data are processed by so-called neuro-symbolic networks. The basic processing units of neuro-symbolic networks are neuro-symbols. Correlations between neuro-symbols of a neuro-symbolic network can be learned from examples. Perception is based on sensor data as well as on interaction with cognitive processes like focus of attention, memory, and knowledge. Additionally, a mechanism for evaluating perception by emotions is suggested.
---
paper_title: Fusion of MRI and DTI to Assist the Treatment Solution of Brain Tumor

paper_content:

With contrast to conventional MRI, diffusion tensor image (DTI) can demonstrate the anatomic structure and pathological process of white matter tracts and it is widely used by clinician. However, at present, DTI and MRI images are diagnosed respectively and clinician gets the image information objectively, and this leads to different diagnosis conclusions. In this article, we develop an integrated approach that aims to maximize the amount of image information for clinician by combining anatomical and functional MR images with nerve fiber bundles derived from diffusion tensor imaging to assist treatment solution for brain tumor. Firstly, segmentation on MRI T2 is operated to decide solid tumor border or edema area et al; and then DTI and MRI images are fused together, which can supplied not only individual information but also fusion information. This new visualization facility which is intuitive for clinician to understand and use can adequately mine this source of image information. Finally, white matter fibers affected by tumor were analyzed quantitatively. The combined visualization and computational methods have the potential to assist in preoperative surgery and radiotherapy planning as well as postoperative treatment evaluation.
---
paper_title: Visualization of the Cortical Potential Field By Medical Imaging Data Fusion

paper_content:

We describe the visualization of the potential field on the scalp and on the cortical surface. The surfaces are derived from magnetic resonance imaging data and the potential fields are reconstructed from electroencephalography data. The visualization tool is validated with clinical and cognitive application studies.
---
paper_title: Multiplanar MRI–CT fusion neuronavigation-guided serial stereotactic biopsy of human brain tumors: proof of a strong correlation between tumor imaging and histopathology by a new technical approach

paper_content:

Purpose ::: Serial stereotactic biopsy is a diagnostic procedure, used when open biopsy or tumor bulk removal seems to be associated with a too high risk of new neurological deficits in tumors of eloquent regions or tumors of deep localizations or in anticipated high surgery related morbidity even in the older patient group. Shortcomings of this method are recognized to be the missed pathohistological information from untargeted areas in heterogeneous tumors. This study shows for the first time a collection of patients with brain tumors with their associated multiplanar MRI–CT fusion imaging during stereotaxis and the histopathological features of serial tumor biopsies along exact trajectorial sites towards the tumor center.
---
paper_title: Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas

paper_content:

A major limitation of the use of endoscopes in minimally invasive surgery is the lack of relative context between the endoscope and its surroundings. The purpose of this work was to fuse images obtained from a tracked endoscope to surfaces derived from three-dimensional (3-D) preoperative magnetic resonance or computed tomography (CT) data, for assistance in surgical planning, training and guidance. We extracted polygonal surfaces from preoperative CT images of a standard brain phantom and digitized endoscopic video images from a tracked neuro-endoscope. The optical properties of the endoscope were characterized using a simple calibration procedure. Registration of the phantom (physical space) and CT images (preoperative image space) was accomplished using fiducial markers that could be identified both on the phantom and within the images. The endoscopic images were corrected for radial lens distortion and then mapped onto the extracted surfaces via a two-dimensional 2-D to 3-D mapping algorithm. The optical tracker has an accuracy of about 0.3 mm at its centroid, which allows the endoscope tip to be localized to within 1.0 mm. The mapping operation allows multiple endoscopic images to be "painted" onto the 3-D brain surfaces, as they are acquired, in the correct anatomical position. This allows panoramic and stereoscopic visualization, as well as navigation of the 3-D surface, painted with multiple endoscopic views, from arbitrary perspectives.
---
paper_title: Brain Tumor Segmentation through Data Fusion of T2-Weighted Image and MR Spectroscopy

paper_content:

Brain tumor segmentation is an important technique in computer aided diagnosis. To improve this, it is necessary to use biochemical information provided by magnetic resonance spectroscopy (MRS). An important issue is how to combine the multimodal signals, such as MRS and structure images, and how to use the combined information to make a decision. A data fusion method is proposed in this paper to perform an automatic segmentation of brain tumor. The combinational data of MRS and T 2 -weighted image should be enhanced by an operation of exponential companding. It consists of five steps: multi-voxel MRS (or CSI) data processing, localization and VOI extraction, data combination, exponential companding, and region growing. Two glioma patients' data provided by Tiantan hospital of China have been used to evaluate our method. Two "ground truth", tumor with edema and tumor only, used for results comparison are manual labels made by neuro radiologists of Tiantan and Nanfang hospitals of China. The segmentation result represents MRS-weighted T 2 structure image in tumor region. Its performance is 99% correct detection for tumor only and 98% for tumor with potential edema, and the false detection are 9% and 6% inside VOI, respectively. The proposed method is also a simple information fusion strategy.
---
paper_title: Medical image pseudo coloring by wavelet fusion

paper_content:

Wavelet image fusion is proposed for medical image pseudo coloring. Wavelet efficiently separates an image into different subspaces that preserve the features of the original images after pseudo coloring. Feature classification information can be directly applied to the coarse approximation of the wavelet decomposed image without segmenting the original image. The proposed method improves the processing speed of the pseudo coloring algorithm and is robust in processing noisy images.
---
paper_title: Data fusion in medical imaging: merging multimodal and multipatient images, identification of structures and 3D display aspects.

paper_content:

Data fusion in medical imaging can be seen into two ways (i) multisensors fusion of anatomical and functional information and (ii) interpatient data fusion by means of warping models. These two aspects set the methodological framework necessary to perform anatomical modelling especially when concerning the modelling of brain structures. The major relevance of the work presented here concerns the interpretation of multimodal 3D neuro-anatomical data bases. Three types of data fusion problems are considered in this paper. The first one concerns the problem of data combination which includes multimodal registration (multisensor fusion applied to CT, MRI, DSA, PET, SPECT, or MEG). In particular, the problem of warping patient data to an anatomical atlas is reviewed and a solution is proposed. The second problem of data fusion addressed in this paper is the identification of anatomical structures by means of image analysis methods. Two techniques have been developed. The first one deals with the analysis of image geometrical features to end up with the determination of a fuzzy mask to label the structure of interest. The second technique consists of labelling major cerebral structures by means of statistical image features associated with relaxation techniques. Finally, the paper presents a review of up to date 3D display techniques with a special emphasis on volume rendering and 3D display of combined data.
---
paper_title: Multiclassifier fusion in human brain MR segmentation: modelling convergence

paper_content:

Segmentations of MR images of the human brain can be generated by propagating an existing atlas label volume to the target image. By fusing multiple propagated label volumes, the segmentation can be improved. We developed a model that predicts the improvement of labelling accuracy and precision based on the number of segmentations used as input. Using a cross-validation study on brain image data as well as numerical simulations, we verified the model. Fit parameters of this model are potential indicators of the quality of a given label propagation method or the consistency of the input segmentations used.
---
paper_title: Evidential segmentation scheme of multi-echo MR images for the detection of brain tumors using neighborhood information

paper_content:

In this paper we propose and study an evidential segmentation scheme of multi-echo MR images for the detection of brain tumors. We show that the modeling by means of evidence theory is well suited to the processing of redundant and complementary data as the MR images. Moreover neighborhood relationship between voxels is taken into account via Dempster's combination rule. We show that using this information improves the classification results previously obtained and leads to a real region-based segmentation. Moreover, the combination of spatial information allows to compute a measure of conflict, which reflects the spatial organization of the data: the conflict is higher at the boundaries between different structures. Thus, it provides a new source of evidence that the specialist can aggregate with the segmentation results to soften its own decision.
---
paper_title: Multichannel fusion models for the parametric classification of differential brain activity

paper_content:

This paper introduces parametric multichannel fusion models to exploit the different but complementary brain activity information recorded from multiple channels in order to accurately classify differential brain activity into their respective categories. A parametric weighted decision fusion model and two parametric weighted data fusion models are introduced for the classification of averaged multichannel evoked potentials (EPs). The decision fusion model combines the independent decisions of each channel classifier into a decision fusion vector and a parametric classifier is designed to determine the EP class from the discrete decision fusion vector. The data fusion models include the weighted EP-sum model in which the fusion vector is a linear combination of the multichannel EPs and the EP-concatenation model in which the fusion vector is a vector-concatenation of the multichannel EPs. The discrete Karhunen-Loeve transform (DKLT) is used to select features for each channel classifier and from each data fusion vector. The difficulty in estimating the probability density function (PDF) parameters from a small number of averaged EPs is identified and the class conditional PDFs of the feature vectors of averaged EPs are, therefore, derived in terms of the PDFs of the single-trial EPs. Multivariate parametric classifiers are developed for each fusion strategy and the performances of the different strategies are compared by classifying 14-channel EPs collected from five subjects involved in making explicit match/mismatch comparisons between sequentially presented stimuli. It is shown that the performance improves by incorporating weights in the fusion rules and that the best performance is obtained using multichannel EP concatenation. It is also noted that the fusion strategies introduced are also applicable to other problems involving the classification of multicategory multivariate signals generated from multiple sources.
---
paper_title: Decision Fusion of Circulating Markers for Breast Cancer Detection in Premenopausal Women

paper_content:

Current mammographic screening for breast cancer is less effective for younger women. To complement mammography for premenopausal women, we investigated the feasibility screening test using 98 blood serum proteins. Because the data set was very noisy and contained only weak features, we used a classifier designed for noisy data: decision fusion. Decision fusion outperformed both a support vector machine (SVM) and linear regression with forward stepwise feature selection on all three two-class classification tasks: normal tissue vs. cancer, normal tissue vs. benign lesions, and benign lesions vs. cancer. Decision fusion detected cancer moderately well (AUC=0.84 on normal vs. cancer), demonstrating promise as a screening tool. Decision fusion also detected benign lesions similarly well (AUC=0.83 on normal vs. benign lesions) and was the only classifier to achieve any success in separating benign from malignant lesions (AUC=0.64 on benign vs. cancer). The classification results suggest that the assayed proteins are more indicative of a secondary effect, such as immune response, rather than specific for breast cancer. In conclusion, the decision fusion classifier demonstrated some promise in detecting breast lesions and outperformed other classifiers, especially for the very noisy classification problem of distinguishing benign from malignant lesions.
---
paper_title: UWB microwave imaging for breast cancer detection: tumor/clutter identification using a time of arrival data fusion method

paper_content:

This paper proposed a novel scheme for early breast cancer detection using UWB microwave imaging. By applying the CLEAN algorithm to process the received signals at each sensor, the IR components can be successfully resolved. The locations and scattering strengths of dominant scatterers are then estimated based on the TOA data fusion concept. Simulation results have indicated the feasibility of detecting a tumor of 2 mm at depth of 2 cm. As compared to the existing techniques, the proposed approach is more computationally-efficient since the scanning process is localized at a few candidate locations. A number of issues that need to be addressed in future include: (i) effectiveness of the proposed approach for 3-D phantoms with realistic breast shapes and tissue compositions; (ii) extension of analysis to more complicated antenna array systems; (iii) other signal processing algorithms that may provide better IR component extraction and scatterer location estimation.
---
paper_title: Multimodality data fusion aids early detection of breast cancer using conventional technology and advanced digital infrared imaging

paper_content:

The fusion of data from the various screening modalities will lead to a cost effective breast cancer detection system that is certain to improve the early part of early breast cancer detection methods. We explore the integration of data from mammography, ultrasound, clinical evaluation, and CAD, with noninvasive digital infrared imaging data from the BreastScan IR system, to improve early diagnosis. This will address a new solution to the screening of young women and women with dense breasts.
---
paper_title: Use of deformable image fusion to allow better definition of tumor bed boost volume after oncoplastic breast surgery

paper_content:

Although the use of boost irradiation is recommended, the standard technique and definition of the boost volume after oncoplastic surgery have not been clearly established. This multidisciplinary study based on image registration was designed to propose practical solutions for the definition of tumor bed boost in this setting.
---
paper_title: Investigation of PET/MRI image fusion schemes for enhanced breast cancer diagnosis

paper_content:

The benefit of registration and fusion of functional images with anatomical images is well appreciated in the advent of PET/CT. There is an increasing interest in expanding this approach to PET/MRI. The focus of much of the related research has been on registering images from different modalities. However, the importance of appropriately jointly displaying (i.e. fusing) the registered images has often been neglected and underestimated. Our aim is to determine which fusion techniques are most useful for enhanced diagnostic performance, ease of use, efficiency, and accuracy for reading registered MRI and PET breast images. Preliminary results indicate that the radiologists were better able to perform a series of tasks when reading the fused PET/MRI data sets using color tables generated by our new genetic algorithm, as compared to commonly used fire/gray or red/gray schemes.
---
paper_title: MRI- Mammography 2D/3D Data Fusion for Breast Pathology Assessment

paper_content:

Increasing use is being made of contrast-enhanced Magnetic Resonance Imaging (Gd-DTPA) for breast cancer assessment since it provides 3D functional information via pharmacokinetic interaction between contrast agent and tumour vascularity, and because it is applicable to women of all ages. Contrast-enhanced MRI (CE-MRI) is complimentary to conventional X-ray mammography since it is a relatively low-resolution functional counterpart of a comparatively high-resolution 2D structural representation. However, despite the additional information provided by MRI, mammography is still an extremely important diagnostic imaging modality, particularly for several common conditions such as ductal carcinoma in-situ (DCIS) where it has been shown that there is a strong correlation between microcalcification clusters and malignancy [1]. Pathological indicators such as calcifications and fine spiculations are not visible in CE-MRI and therefore there is clinical and diagnostic value to fusing the high-resolution structural information available from mammography with the functional data acquired from MRI imaging. This paper presents a novel data fusion technique whereby medio-lateral (ML) and cranio-caudal (CC) mammograms (2D data) are registered to 3D contrast-enhanced MRI volumes. We utilise a combination of pharmacokinetic modelling, projection geometry, wavelet-based landmark detection and thin-plate spline non-rigid registration to transform the coordinates of regions of interest (ROIs) from the 2D mammograms to the spatial reference frame of the contrast-enhanced MRI volume.
---
paper_title: Breast cancer diagnosis using neural-based linear fusion strategies

paper_content:

Breast cancer is one of the leading causes of mortality among women, and the early diagnosis is of significant clinical importance. In this paper, we describe several linear fusion strategies, in particular the Majority Vote, Simple Average, Weighted Average, and Perceptron Average, which are used to combine a group of component multilayer perceptrons with optimal architecture for the classification of breast lesions. In our experiments, we utilize the criteria of mean squared error, absolute classification error, relative error ratio, and Receiver Operating Characteristic (ROC) curve to concretely evaluate and compare the performances of the four fusion strategies. The experimental results demonstrate that the Weighted Average and Perceptron Average strategies can achieve better diagnostic performance compared to the Majority Vote and Simple Average methods.
---
paper_title: Early tumor detection by multiple infrared unsupervised neural nets fusion

paper_content:

The unsupervised classification algorithm called Lagrange constraint neural network (LCNN) has been successfully applied to the sub-pixel multispectral remote sensing, [25]. Here, we apply the LCNN to the early breast cancer detection using two-color mid and long infrared images of the breast. This could be a new paradigm shift that enabled smart neural network algorithm to sort out the underlying malignant heat sources for physician diagnoses. The nonintrusive 2-color passive infrared imaging that could be repeated for record track with no radiation hazard seems to be alternative paradigm shift for the first-line screening against breast cancer. The sub-pixel super-resolution capability of the remote sensing is equivalent to the sub-milimeter scaling of the close-up breast imaging for the vascular and the angiogenesis effects. We demonstrate the potential benefit of the multicolor mid & long infrared imaging capable for detecting the abnormal under-skin thermal textures as well as stage-zero detection of the ductal carcinoma in situ.
---
paper_title: Classifier Fusion Using Dempster-Shafer theory of evidence to Predict Breast Cancer Tumors

paper_content:

In classifier fusion models, classifiers outputs are combined to achieve a group decision. The most often used classifiers fusion models are majority vote, probability schemes, weighted averaging and Bayes approach to name few. We propose a model of classifiers fusion by combining the mathematical belief of classifiers. We used Dempster-shafer theory of evidence to determine the mathematical belief of classifiers. Support Vector Machine (SVM) with linear, polynomial and radial kernel has been employed as classifiers. The output of classifiers used as basis for computing beliefs. We combined these beliefs to arrive at one final decision. Our experimental results have shown that the new proposed classifiers fusion methodology have outperforms single classification models.
---
paper_title: Fusion of Magnetic Resonance and Scintimammography Images for Breast Cancer Evaluation: A Pilot Study

paper_content:

BACKGROUND ::: To achieve a more specific method to estimate the real size of breast cancer, we have developed a method to fuse magnetic resonance imaging (MRI) and scintimammography (SM) images. The aim of this study was to assess the feasibility of this method and to evaluate its accuracy to measure the size of breast cancer compared with MRI alone, mammography, and clinical examination, employing pathologic size as the gold standard. ::: ::: ::: METHODS ::: Twenty consecutive breast cancer women at stages IIA-IIIA, scheduled for mastectomies, underwent SM with (99m)Tc-sestamibi and MRI with gadolinium 2-10 days before surgery. All patients had had recent mammographies and were examined clinically. Software was developed in visual language to perform the fusion between MRI and SM images and tumor measurements (MRI/SM). The tumor size, in 3 diameters (anteroposterior, longitudinal, and transverse), for each examination was correlated with pathological measurements using linear regression. ::: ::: ::: RESULTS ::: The MRI/SM technique was successfully performed in all patients, and the principal tumor was measured by this method. The MRI/SM cancer measurements correlated better with pathology than MRI, mammography, and clinical exam in all diameters analyzed (r = 0.88, 0.81, 0.81; SE = 0.11, 0.14, 0.11 in anteroposterior, longitudinal, and transverse diameters, respectively). ::: ::: ::: CONCLUSIONS ::: The MRI/SM is a feasible technique and appears to be more accurate than other examinations (MRI alone, mammography, and clinical exam) to measure breast cancer size.
---
paper_title: Importance of the CT/MRI fusion method as a learning tool for CT-based postimplant dosimetry in prostate brachytherapy

paper_content:

Abstract Background and purpose To compare the CT-based and CT/MRI fusion-based postimplant dosimetry after permanent prostate brachytherapy and to evaluate the improvement in CT-based dosimetry by physicians with or without experience in using the CT/MRI fusion method. Patients and methods Thirty-eight consecutive patients agreed to participate in a prospective study. The prostate contours from CT/MRI fusion are the gold standard for determining the prostate volume and dose volume histogram (DVH). CT-based postimplant dosimetries were performed by two physicians. Observer 1 was a radiologist who had never used CT/MRI fusion method for postimplant dosimetric analysis. Observer 2 was a radiation oncologist experienced in postimplant analysis using the CT/MRI fusion method. The prostate dosimetry was evaluated by prostate D90 and V100. Results No significant difference was observed in the mean prostate volumes between the two observers and the CT/MRI fusion data. However, the correlation coefficient value for observer 2 ( R 2 =0.932) was greater than that for observer 1 ( R 2 =0.793). The D90 and V100 values as evaluated by the two observers were significantly underestimated in comparison to those evaluated using the CT/MRI fusion methods. The DVH related parameters were underestimated more frequently by observer 1 than by observer 2: (prostate D90: 99.56% for observer 1, 102.97% for observer 2, 109.37% for CT/MRI fusion. Prostate V100: 88.12% for observer 1, 90.14% for observer 2, 91.91% for CT/MRI fusion). Conclusions The difference in the mean value in D90 and V100 by observer 1 was significantly greater than that for observer 2. These findings suggest that the CT/MRI fusion method provides accurate feedback which thereby improves CT-based postimplant dosimetry for prostate brachytherapy.
---
paper_title: Sequential evaluation of prostate edema after permanent seed prostate brachytherapy using CT-MRI fusion

paper_content:

Purpose: To analyze the extent and time course of prostate edema and its effect on dosimetry after permanent seed prostate brachytherapy. Methods and Materials: Twenty patients scheduled for permanent seed 125 I prostate brachytherapy agreed to a prospective study on postimplant edema. Implants were preplanned using transrectal ultrasonography. Postimplant dosimetry was calculated using computed tomography-magnetic resonance imaging (CT-MRI) fusion on the day of the implant (Day 1) and Days 8 and 30. The prostate was contoured on MRI, and the seeds were located on CT. Factors investigated for an influence on edema were the number of seeds and needles, preimplant prostate volume, transitional zone index (transition zone volume divided by prostate volume), age, and prostate-specific antigen level. Prostate dosimetry was evaluated by the percentage of the prostate volume receiving 100% of the prescribed dose (V 100 ) and percentage of prescribed dose received by 90% of the prostate volume (D 90 ). Results: Prostate edema was maximal on Day 1, with the median prostate volume 31% greater than preimplant transrectal ultrasound volume (range, 0.93–1.72; p p = 0.013) and 5% greater on Day 30 ( p p = 0.016) and the preplan prostate volume ( p = 0.003). The median V 100 on Day 1 was 93.6% (range, 86.0–98.2%) and was 96.3% (range, 85.7–99.5%) on Day 30 ( p = 0.079). Patients with a Day 1 V 100 >93% were less affected by edema resolution, showing a median increase in V 100 of 0.67% on Day 30 compared with 2.77% for patients with a V 100 Conclusion: Despite the extreme range of postimplant edema, the effect on dosimetry was less than expected. Dose coverage of the prostate was good for all patients during Days 1–30. Our data indicate that postimplant dosimetry on the day of implant is sufficient for patients with good dose coverage (Day 1 V 100 >93%).
---
paper_title: Prostate seed implant quality assessment using MR and CT image fusion.

paper_content:

PURPOSE ::: After a seed implant of the prostate, computerized tomography (CT) is ideal for determining seed distribution but soft tissue anatomy is frequently not well visualized. Magnetic resonance (MR) images soft tissue anatomy well but seed visualization is problematic. We describe a method of fusing CT and MR images to exploit the advantages of both of these modalities when assessing the quality of a prostate seed implant. ::: ::: ::: METHODS AND MATERIALS ::: Eleven consecutive prostate seed implant patients were imaged with axial MR and CT scans. MR and CT images were fused in three dimensions using the Pinnacle 3.0 version of the ADAC treatment planning system. The urethra and bladder base were used to "line up" MR and CT image sets during image fusion. Alignment was accomplished using translation and rotation in the three ortho-normal planes. Accuracy of image fusion was evaluated by calculating the maximum deviation in millimeters between the center of the urethra on axial MR versus CT images. Implant quality was determined by comparing dosimetric results to previously set parameters. ::: ::: ::: RESULTS ::: Image fusion was performed with a high degree of accuracy. When lining up the urethra and base of bladder, the maximum difference in axial position of the urethra between MR and CT averaged 2.5 mm (range 1.3-4.0 mm, SD 0.9 mm). By projecting CT-derived dose distributions over MR images of soft tissue structures, qualitative and quantitative evaluation of implant quality is straightforward. ::: ::: ::: CONCLUSIONS ::: The image-fusion process we describe provides a sophisticated way of assessing the quality of a prostate seed implant. Commercial software makes the process time-efficient and available to any clinical practice with a high-quality treatment planning system. While we use MR to image soft tissue structures, the process could be used with any imaging modality that is able to visualize the prostatic urethra (e.g., ultrasound).
---
paper_title: ULTRASONOGRAPHY AND FLUOROSCOPIC FUSION FOR PROSTATE BRACHYTHERAPY DOSIMETRY

paper_content:

Abstract Purpose : To investigate the feasibility of performing postimplant and intraoperative dosimetry for prostate brachytherapy by fusing transrectal ultrasound (TRUS) and fluoroscopic data. Methods and Materials : Registration of ultrasound (prostate boundary) and fluoroscopic (seed) data requires spatial markers that are detectable by both imaging modalities. In this study, the needle tips were considered as such fiducials. Prostate phantoms were implanted with the seeds, and four localization needles were inserted. In the TRUS frame of reference, the longitudinal coordinate of the needle tip was determined by advancing the needle until the echo from its tip just registered at a known probe depth. The tip’s transverse coordinates were determined from the associated TRUS slice. The three-dimensional needle tip positions were also calculated in the fluoroscopic coordinate system using a seed reconstruction method. The transformation between the TRUS and fluoroscopy coordinate systems was established by the least-squares solution using the singular value decomposition. Results : With three of four needle tips as fiducials and the one remaining needle as a test target, the mean fiducial registration error was 0.8 mm and the test target registration error was 2.5 mm. When all four points were used for registration, the errors decreased to 1.1 mm. A comparison between the proposed method and CT-based dosimetry yielded a percentage of prostate volume receiving 100% and 150% of the prescribed minimal peripheral dose and minimal dose received by 90% of the prostate gland that agreed within 0.4%, 2.7%, and 4.2%, respectively. Conclusion : The combination of TRUS and fluoroscopy is a feasible alternative to the currently used CT-based postimplant dosimetry. Furthermore, because of online imaging capability, the method lends itself to real-time intraoperative applications.
---
paper_title: Comparison of MRI-based and CT/MRI fusion-based postimplant dosimetric analysis of prostate brachytherapy.

paper_content:

PURPOSE ::: The aim of this study was to compare the outcomes between magnetic resonance imaging (MRI)-based and computed tomography (CT)/MRI fusion-based postimplant dosimetry methods in permanent prostate brachytherapy. ::: ::: ::: METHODS AND MATERIALS ::: Between October 2004 and March 2006, a total of 52 consecutive patients with prostate cancer were treated by brachytherapy, and postimplant dosimetry was performed using CT/MRI fusion. The accuracy and reproducibility were prospectively compared between MRI-based dosimetry and CT/MRI fusion-based dosimetry based on the dose-volume histogram (DVH) related parameters as recommended by the American Brachytherapy Society. ::: ::: ::: RESULTS ::: The prostate volume was 15.97+/-6.17 cc (mean+/-SD) in MRI-based dosimetry, and 15.97+/-6.02 cc in CT/MRI fusion-based dosimetry without statistical difference. The prostate V100 was 94.5% and 93.0% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.002). The prostate D90 was 119.4% and 114.4% in MRI-based and CT/MRI fusion-based dosimetry, respectively, and the difference was statistically significant (p=0.004). ::: ::: ::: CONCLUSION ::: Our current results suggested that, as with fusion images, MR images allowed accurate contouring of the organs, but they tended to overestimate the analysis of postimplant dosimetry in comparison to CT/MRI fusion images. Although this MRI-based dosimetric discrepancy was negligible, MRI-based dosimetry was acceptable and reproducible in comparison to CT-based dosimetry, because the difference between MRI-based and CT/MRI fusion-based results was smaller than that between CT-based and CT/MRI fusion-based results as previously reported.
---
paper_title: Image Registration and Fusion for Interventional MRI Guided Thermal Ablation of the Prostate Cancer

paper_content:

We are investigating interventional MRI (iMRI) guided thermal ablation treatment of the prostate cancer. Functional images such as SPECT can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high resolution MRI volume. Then by registering the high-resolution MR image with iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to iMRI images for improved tumor targeting. For the first step, we used a mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm. Image data were acquired from patients and volunteers. Compared to our volume-to-volume registration that was previously evaluated to be quite accurate, the SV registration accuracy is about 0.5mm for transverse images covering the prostate. With our image registration and fusion software, simulation experiments show that it is feasible to incorporate SPECT and high resolution MRI into the iMRI-guided treatment.
---
paper_title: Initial clinical assessment of CT-MRI image fusion software in localization of the prostate for 3D conformal radiation therapy☆

paper_content:

Abstract Purpose: To assess the utility of image fusion software and compare MRI prostate localization with CT localization in patients undergoing 3D conformal radiation therapy of prostate cancer. Materials and Methods: After a phantom study was performed to ensure the accuracy of image fusion procedure, 22 prostate cancer patients had CT and MRI studies before the start of radiotherapy. Immobilization casts used during radiation treatment were also used for both imaging studies. After the clinical target volume (CTV) (prostate or prostate + seminal vesicles) was defined on CT, slices from the MRI study were reconstructed to precisely match the CT slices by identifying three common bony landmarks on each study. The CTV was separately defined on the matched MRI slices. Data related to the size and location of the prostate were compared between CT and MRI. The spatial relationship between the tip of urethrogram cone on CT and prostate apex seen on MRI was also estimated. Results: The phantom study showed the registration discrepancies between CT and MRI smaller than 1.0 mm in any pair in comparison. The patient study showed a mean image registration error of 0.9 (± 0.6) mm. The average prostate volume was 63.0 (± 25.8) cm 2 and 50.9 (± 22.9) cm 3 determined by CT and MRI, respectively. The difference in prostate location with the two studies usually differed at the base and at the apex of the prostate. On the transverse MRI, the prostate apex was situated 7.1 (± 4.5) mm dorsal and 15.1 (± 4.0) mm cephalad to the tip of urethrogram cone. Conclusions: CT-MRI image fusion study made it possible to compare the two modalities directly. MRI localization of the prostate is more accurate than CT, and indicates the distance from cone to apex is 15 mm. CT-MRI image fusion technique provides valuable supplements to CT technology for more precise targeting of the prostate cancer.
---
paper_title: A volumetric and dosimetric study of LDR brachytherapy prostate implants based on image fusion of ultrasound and computed tomography

paper_content:

Transrectal ultrasound has been the imaging modality of choice for brachytherapy seed implantation. However, CT scanning has been used by some investigators. Here, the authors first examine the differences in the volume of the prostate as defined based on US and CT imaging studies, where the two image sets were fused together. They also investigated the dosimetric implications of the respective plans. All patients in the study had a CT and a transrectal US exam one month after the iodine seed implantation. A catheter with a balloon was placed in the patient's urethra with contrast. The balloon was pulled firmly against the anterior apex of the bladder. The two image sets were then fused together, using the shape of the urethra as the anatomical landmark for the co-registration. The prostate volumes from the two studies were computed as well as the DVH and dose distributions. An over-estimation of the prostate volume as seen on the US of up to 30% was found with the CT images. Spatially, the relative position of the two volumes is uncorrelated. The DVH and dose distributions showed better than 85% coverage for the US volume but mediocre coverage (as low as 50%) for the CT volume. CT imaging of the prostate tends to over-estimate its size. In post implantation studies where a CT is obtained for the localization of seeds, that volumetric discrepancy needs to be taken into account when assessing the quality of the implant.
---
paper_title: Registration and fusion of SPECT, high-resolution MRI, and interventional MRI for thermal ablation of prostate cancer

paper_content:

We are investigating interventional MRI (iMRI) guided radiofrequency thermal ablation for the minimally invasive treatment of the prostate cancer. Nuclear medicine can detect and localize tumor in the prostate not reliably seen in MRI. We intend to combine the advantages of functional images such as nuclear medicine SPECT with iMRI-guided treatments. Our concept is to first register the low-resolution SPECT with a high-resolution MRI volume. Then by registering the high-resolution MR image with live-time iMRI acquisitions, we can, in turn, map the functional data and high-resolution anatomic information to live-time iMRI images for improved tumor targeting. For the first step, we used a three-dimensional mutual information registration method. For the latter, we developed a robust slice to volume (SV) registration algorithm with special features. The concept was tested using image data from three patients and three volunteers. The SV registration accuracy was 0.4 mm /spl plusmn/ 0.2 mm as compared to our volume- to-volume registration that was previously shown to be quite accurate for these image pairs. With our image registration and fusion software, simulation experiments show that it is quite feasible to incorporate SPECT and high-resolution MRI into the iMRI-guided minimally invasive treatment procedures.
---
paper_title: Ultrasound image fusion for external beam radiotherapy for prostate cancer.

paper_content:

PURPOSE ::: To determine whether real-time ultrasound imaging and targeting system for the treatment of prostate cancer was feasible. The initial phase of this project included a study to develop and determine (a) software for the fusion of ultrasound images to standard x-rays obtained during simulation, and (b) the potential reduction in field size with real-time imaging. ::: ::: ::: METHODS AND MATERIALS ::: During 13 patient simulations a transrectal ultrasound image was obtained. Orthogonal x-ray films were acquired with the rectal probe in place. Both the x-ray and ultrasound images were digitized and a fusion image was created of the prostate position in relation to the probe, bladder, and rectum. The two-dimensional area of the rectum, bladder, and prostate was determined in the lateral projection. Potential conformal blocks were designed for the lateral portals in a four-field treatment technique. ::: ::: ::: RESULTS ::: The transrectal ultrasound probe enabled real-time prostate imaging. The lateral field size can be reduced to 6.08 x 5.68 cm2 +/- 0.62 x 0.48 cm2 from the standard 8 x 8 cm2 field. The posterior rectal wall was physically displaced out of the lateral field. The area of the rectum included in the lateral field is 1.75 cm2 +/- 0.85 cm2. ::: ::: ::: CONCLUSION ::: The prostate position can be determined with certainty on a regular basis with transrectal ultrasonography. The amount of normal tissue in the high dose volume can be reduced. This approach may reduce acute and chronic morbidity and allow further dose escalation.
---
paper_title: 11C-Acetate Positron Emission Tomography Imaging and Image Fusion With Computed Tomography and Magnetic Resonance Imaging in Patients With Recurrent Prostate Cancer

paper_content:

Purpose To assess the clinical value of computed tomography (CT) and magnetic resonance imaging (MRI) image fusion with 11 C-acetate (AC) positron emission tomography (PET) imaging for detection and exact location of clinically occult recurrences. Patients and Methods Fifty prostate cancer patients with elevated/increasing serum prostate-specific antigen levels after radical therapy underwent whole-body AC PET. Uptake was initially interpreted as normal, abnormal, or equivocal. In case of abnormal or equivocal uptake, additional conventional imaging techniques, such as CT, MRI, and bone scans, were performed. To precisely define the anatomic location of abnormal uptake and to improve characterization of equivocal lesions, a softwareassisted image fusion (CT-PET, MRI-PET) was performed and evaluated as site-by-site analysis of 51 abnormal (n 37) or equivocal (n 14) sites of all 50 patients. In 17 patients, additional histopathologic evaluation was available. Results In five (10%), 13 (26%), and 32 (64%) of the 50 patients, AC PET studies demonstrated AC uptake judged as normal, equivocal, and abnormal, respectively. Image fusion changed characterization of equivocal lesions as normal in five (10%) of 51 sites and abnormal in nine (18%) of 51 sites. It precisely defined the anatomic location of abnormal uptake in 37 (73%) of 51 sites. AC PET findings did influence patient management in 14 (28%) of 50 patients.
---
paper_title: MRI-CT fusion to assess postbrachytherapy prostate volume and the effects of prolonged edema on dosimetry following transperineal interstitial permanent prostate brachytherapy.

paper_content:

PURPOSE ::: Quality assurance through postplan assessment is an integral part of permanent seed prostate implants. The use of MRI-CT fusion for 1-month postimplant dosimetry permits accurate assessment of prostate volume without seed induced artifact and uncertainties of prostate contour inherent to CT assessments. Routine use of MRI-CT fusion reveals significant prostate edema may persist several weeks. This study evaluates the effect of edema, and its subsequent resolution, on dosimetry. ::: ::: ::: METHODS AND MATERIALS ::: From May 2001 to June 2003, 241 men were treated with (125)I seed implants based on a transrectal ultrasound (TRUS) preplan. Quality assessment was performed at 1 month by CT-MRI fusion using VariSeed software. Over this 24-month period, 29 patients (12%) with residual edema at 1 month (12-60% >TRUS plan volume), had repeat CT-MRI fusion at 2-4 months to reassess volume and dosimetry. Eleven of the 29 had received prior androgen ablation to shrink the prostate preimplant. ::: ::: ::: RESULTS ::: For the entire group (n = 241), mean preimplant prostate volume was 33.7 cc and median postplan dosimetric parameters were: V100, 92.2%; D90, 153 Gy; and V150, 53%. For the 29 patients with prolonged edema, mean preimplant volume was 34.8 cc and 1-month volume was 46.1 cc (p <0.001). Mean volume reduction between 1 and 2 months was 13%. The decrease in prostate volume had a significant effect on dosimetry with median increase between 1 month and 2 months in calculated V100 of 9.5%, V150 22.6%, V200 30.1%, and D90 11.5%. ::: ::: ::: CONCLUSIONS ::: Significant residual edema is seen 1-month postimplant in 12% of prostates and may have a profound effect on dosimetry. Further study is underway to characterize the time course of resolution of the edema, and to perform integral dosimetry based on the changing volume.
---
paper_title: Design of a simple ultrasound/CT fusion image fusion solution for the evaluation of prostate seed brachytherapy

paper_content:

The purpose of this project is to devise a solution that will enable physicians, medical physicists and dosimetrists to quantitatively evaluate the success of prostate brachytherapy procedures. It is proposed that a simple ultrasound/CT image fusion solution can be developed and implemented to improve the current post-surgical evaluation process.
---
paper_title: Histology and ultrasound fusion of excised prostate tissue using surface registration

paper_content:

We present a method for combining histological data with 3D ultrasound of excised prostate tissue using surface registration. With this technique, 3D prostate cancer lesions can be properly located and visualized within a B-scan volume for tissue characterization comparisons. Three prostate specimens were scanned with a GE Logiq 700 (Expert series) to obtain 2D B-scan sequences. The prostate boundaries were manually segmented from each sequence and reconstructed into 3D volumes. Specimens were fixed, sectioned into slabs, then mounted whole onto slides. Cancerous lesions were outlined by a pathologist. The slides were photographed with an Optronics Spot digital camera using Image-Pro Plus software on a PC. The gland surface was manually segmented from 2D histology images and reconstructed into a volume. Various reconstruction problems were addressed, such as specimen shrinkage due to the fixing and staining processes. The fusion algorithm translates and rotates one surface volume in 3D to find the best surface overlap. The resulting geometric transform is used to reorient the original image volume. The displacement error was determined by measuring the urethra offsets in final volume cross-sections. The offset distance ranged from 1.25 mm to 3.45 mm with an average of 2.36 mm. Another measure to gauge volume alignment is to calculate the ratio of overlapping voxels to total combined voxels(intersection/union). For a perfect case, this ratio will be 1. The ratios for the three cases ranged in value from 0.774 to 0.845.
---
paper_title: Synergistic value of single-photon emission computed tomography/computed tomography fusion to radioimmunoscintigraphic imaging of prostate cancer.

paper_content:

The rationale on which positron emission tomography/computed tomography (PET/CT) imaging is based, combining the functional features of PET with the anatomic detail of CT, provides many advantages that are easily transferable to single-photon emission computed tomography (SPECT)/CT imaging. Our efforts have focused on applying fused SPECT/CT imaging to identify prostate cancer and its metastasis and recurrence through radioimmunoscintigraphy (RIS). This application of RIS to imaging prostate cancer requires 2 key components: (1) a well-defined target associated with the cancer and (2) a "magic bullet" to seek that target. A well-characterized RIS target for prostate cancer is prostate-specific membrane antigen, or PSMA, and finding the bullet to seek this target with high sensitivity and specificity has been the focus of intensive study for nearly two decades. One of the candidate bullets developed is capromab pendetide, which is a monoclonal antibody that seeks PSMA. This antibody is commercially available as ProstaScint, which can be labeled with indium-111 to localize prostate cancer via SPECT imaging. In the course of applying fused SPECT/CT ProstaScint imaging to more than 800 prostate cancer cases, numerous refinements to our protocol have evolved that are aimed at staging the cancer with utmost accuracy. In addition to optimizing the localization of prostate cancer and its metastasis, these refinements also have been extended toward guiding both the implantation of radioactive seeds in brachytherapy and in other types of radiation therapy which is illustrated through 5 case reports. Progress in the therapeutic targeting of PSMA is also being actively explored, which has more universal ramifications because PSMA is found in the neovasculature of other types of cancers.
---
paper_title: CT-ultrasound fusion prostate brachytherapy: a dynamic dosimetry feedback and improvement method. A report of 54 consecutive cases.

paper_content:

PURPOSE ::: The authors describe a prostate brachytherapy technique with dynamic dosimetry feedback, using coregistered CT and ultrasound (US) images, to map initial dosimetry deficiencies and guide remedial source placement. ::: ::: ::: METHODS AND MATERIALS ::: Fifty-four consecutive patients treated with this method were analyzed for coregistration accuracy and dosimetry outcomes by evaluating the prostate V100, V150, D90, and urethral D50 and D10. Dosimetric improvements created by remedial source placement and preplan/postplan prostate D90 agreement were evaluated. ::: ::: ::: RESULTS ::: Median CT-US coregistration discrepancy with this technique ranged from 0 to 4mm, with the posterior midline prostate and base prostate providing the least consistent and the urethra providing the most consistent coregistration agreement. Final prostate V100 values ranged from 96.1% to 99.8% for all patients. The addition of remedial sources directed by CT-US fusion produced V100 and D90 improvements whose magnitude inversely correlated with the initial result and exceeded the effect of adding quantitatively identical randomly distributed increased millicuries. The final prostate D90 result agreed within (-) 5% to (+) 10% of the preplan result in 98% of all patients. ::: ::: ::: CONCLUSIONS ::: CT-US fusion prostate brachytherapy represents a dynamic dosimetry feedback and remediation method that consistently produced high prostate V100 and D90 values with acceptably low urethra D50 and D10 values in our study. The degree of prostate V100 and D90 dosimetry improvement created by remedial source placement effectively matched the degree of initial dosimetry deficiency. This method produced a high level of correlation between the preplan and final prostate D90 values.
---
paper_title: Modelling Prostate Motion for Data Fusion During Image-Guided Interventions

paper_content:

There is growing clinical demand for image registration techniques that allow multimodal data fusion for accurate targeting of needle biopsy and ablative prostate cancer treatments. However, during procedures where transrectal ultrasound (TRUS) guidance is used, substantial gland deformation can occur due to TRUS probe pressure. In this paper, the ability of a statistical shape/motion model, trained using finite element simulations, to predict and compensate for this source of motion is investigated. Three-dimensional ultrasound images acquired on five patient prostates, before and after TRUS-probe-induced deformation, were registered using a nonrigid, surface-based method, and the accuracy of different deformation models compared. Registration using a statistical motion model was found to outperform alternative elastic deformation methods in terms of accuracy and robustness, and required substantially fewer target surface points to achieve a successful registration. The mean final target registration error (based on anatomical landmarks) using this method was 1.8 mm. We conclude that a statistical model of prostate deformation provides an accurate, rapid and robust means of predicting prostate deformation from sparse surface data, and is therefore well-suited to a number of interventional applications where there is a need for deformation compensation.
---
paper_title: CT image fusion in the optimization of replanning during the course of 3-dimensional conformal radiotherapy for non-small-cell lung cancer

paper_content:

We utilized CT image fusion technique for replanning during the course of three-dimensional conformal radiotherapy (3D-CRT), and determined its dosimetric effects on both target volumes and normal tissues. Ten patients with non-small-cell lung cancer (NSCLC) were studied. The primary treatment plan was generated from the pre-treatment planning CT scan. Two treatment plans were established in the replanning of the 3D-CRT following the mid-treatment CT scan. Treatment plan A was generated from the fused images with the reference of the primary treatment plan; treatment plan B independently from the second CT scan images by the physicists who were blinded to the primary treatment plan. For the planning target volume, there were no significant differences between treatment plan A and B in D 99 and D 95 of the whole treatment. However, the doses to normal structures in the whole treatment, mean lung dose (MLD) of the lung, Dmax of the spinal cord and esophagus in the treatment plan A were significantly lower than those in the treatment plan B. CT image fusion is needed in replanning during the course of 3D-CRT for NSCLC, which may ensure safe doses to normal tissues.
---
paper_title: Impact of computed tomography and 18F-deoxyglucose coincidence detection emission tomography image fusion for optimization of conformal radiotherapy in non-small-cell lung cancer.

paper_content:

PURPOSE ::: To report a retrospective study concerning the impact of fused 18F-fluoro-deoxy-D-glucose (FDG)-hybrid positron emission tomography (PET) and CT images on three-dimensional conformal radiotherapy planning for patients with non-small-cell lung cancer. ::: ::: ::: METHODS AND MATERIALS ::: A total of 101 patients consecutively treated for Stage I-III non-small-cell lung cancer were studied. Each patient underwent CT and FDG-hybrid PET for simulation treatment in the same treatment position. Images were coregistered using five fiducial markers. Target volume delineation was initially performed on the CT images, and the corresponding FDG-PET data were subsequently used as an overlay to the CT data to define the target volume. ::: ::: ::: RESULTS ::: 18F-fluoro-deoxy-D-glucose-PET identified previously undetected distant metastatic disease in 8 patients, making them ineligible for curative conformal radiotherapy (1 patient presented with some positive uptake corresponding to concomitant pulmonary tuberculosis). Another patient was ineligible for curative treatment because the fused PET-CT images demonstrated excessively extensive intrathoracic disease. The gross tumor volume (GTV) was decreased by CT-PET image fusion in 21 patients (23%) and was increased in 24 patients (26%). The GTV reduction was > or = 25% in 7 patients because CT-PET image fusion reduced the pulmonary GTV in 6 patients (3 patients with atelectasis) and the mediastinal nodal GTV in 1 patient. The GTV increase was > or = 25% in 14 patients owing to an increase in the pulmonary GTV in 11 patients (4 patients with atelectasis) and detection of occult mediastinal lymph node involvement in 3 patients. Of 81 patients receiving a total dose of > or = 60 Gy at the International Commission on Radiation Units and Measurements point, after CT-PET image fusion, the percentage of total lung volume receiving >20 Gy increased in 15 cases and decreased in 22. The percentage of total heart volume receiving >36 Gy increased in 8 patients and decreased in 14. The spinal cord volume receiving at least 45 Gy (2 patients) decreased. Multivariate analysis showed that tumor with atelectasis was the single independent factor that resulted in a significant effect on the modification of the size of the GTV by FDG-PET: tumor with atelectasis (with vs. without atelectasis, p = 0.0001). ::: ::: ::: CONCLUSION ::: The results of our study have confirmed that integrated hybrid PET/CT in the treatment position and coregistered images have an impact on treatment planning and management of non-small-cell lung cancer. However, FDG images using dedicated PET scanners and respiration-gated acquisition protocols could improve the PET-CT image coregistration. Furthermore, the impact on treatment outcome remains to be demonstrated.
---
paper_title: Software-based Fusion of PET and CT Images for Suspected Recurrent Lung Cancer

paper_content:

PURPOSE ::: The purpose of this study was to compare the diagnostic performance of the manual fusion of positron emission tomography (PET) and computed tomography (CT) images with that of CT alone and that of side-by-side PET and CT (PET/CT) in patients with suspected recurrent lung cancer. ::: ::: ::: PROCEDURES ::: Fifty-three patients who had previously had surgery for lung cancer underwent a whole-body 2-deoxy-2-[F-18]fluoro-D-glucose (FDG)-PET scan, followed by a diagnostic CT scan. The PET and CT images were fused on a workstation. CT alone, PET/CT, and fused images were evaluated separately using a five-point grading scale (0=definitely negative, 1=probably negative, 2=equivocal, 3=probably positive, and 4=definitely positive). Lesions of grade 3 or 4 were considered positive, and diagnostic accuracy and certainty were evaluated. ::: ::: ::: RESULTS ::: Overall, 67 lesions in 33 patients were considered true positive pathologically or clinically. Of these 67 lesions, the evaluation of CT, PET/CT, and fused images detected 46, 55, and 66 lesions, respectively, with the number of grade 4 lesions detected being 38, 50, and 63, respectively. The diagnostic accuracy of CT, PET/CT, and fused images according to patients was 75%, 79%, and 87%, respectively. ::: ::: ::: CONCLUSION ::: These results suggest that interpreting fused images increased diagnostic certainty for detecting recurrence and provided more accurate diagnoses.
---
paper_title: CT image fusion in the evaluation of radiation treatment planning for non-small cell lung cancer

paper_content:

Objective ::: We studied the application of CT image fusion in the evaluation of radiation treatment planning for non-small cell lung cancer (NSCLC).
---
paper_title: A phase II comparative study of gross tumor volume definition with or without PET/CT fusion in dosimetric planning for non-small-cell lung cancer (NSCLC): primary analysis of Radiation Therapy Oncology Group (RTOG) 0515.

paper_content:

BACKGROUND ::: Radiation Therapy Oncology Group (RTOG) 0515 is a Phase II prospective trial designed to quantify the impact of positron emission tomography (PET)/computed tomography (CT) compared with CT alone on radiation treatment plans (RTPs) and to determine the rate of elective nodal failure for PET/CT-derived volumes. ::: ::: ::: METHODS ::: Each enrolled patient underwent definitive radiation therapy for non-small-cell lung cancer (≥ 60 Gy) and had two RTP datasets generated: gross tumor volume (GTV) derived with CT alone and with PET/CT. Patients received treatment using the PET/CT-derived plan. The primary end point, the impact of PET/CT fusion on treatment plans was measured by differences of the following variables for each patient: GTV, number of involved nodes, nodal station, mean lung dose (MLD), volume of lung exceeding 20 Gy (V20), and mean esophageal dose (MED). Regional failure rate was a secondary end point. The nonparametric Wilcoxon matched-pairs signed-ranks test was used with Bonferroni adjustment for an overall significance level of 0.05. ::: ::: ::: RESULTS ::: RTOG 0515 accrued 52 patients, 47 of whom are evaluable. The follow-up time for all patients is 12.9 months (2.7-22.2). Tumor staging was as follows: II = 6%; IIIA = 40%; and IIIB = 54%. The GTV was statistically significantly smaller for PET/CT-derived volumes (98.7 vs. 86.2 mL; p < 0.0001). MLDs for PET/CT plans were slightly lower (19 vs. 17.8 Gy; p = 0.06). There was no significant difference in the number of involved nodes (2.1 vs. 2.4), V20 (32% vs. 30.8%), or MED (28.7 vs. 27.1 Gy). Nodal contours were altered by PET/CT for 51% of patients. One patient (2%) has developed an elective nodal failure. ::: ::: ::: CONCLUSIONS ::: PET/CT-derived tumor volumes were smaller than those derived by CT alone. PET/CT changed nodal GTV contours in 51% of patients. The elective nodal failure rate for GTVs derived by PET/CT is quite low, supporting the RTOG standard of limiting the target volume to the primary tumor and involved nodes.
---
paper_title: Fusion of Immunoscintigraphy Single Photon Emission Computed Tomography (SPECT) with CT of the Chest in Patients with Non-Small Cell Lung Cancer

paper_content:

In non-small cell lung cancer (NSCLC), accurate staging is critical in deciding between potentially curative surgery and palliative treatment. Image registration, or fusion, combines the unique functional information provided by SPECT imaging with the excellent anatomic detail offered by computed tomography (CT) or magnetic resonance imaging to better characterize the imformation provided by each separate modality. In this study, we explored the role of fusion of immunoscintigraphy SPECT with CT in the staging of NSCLC. We fused chest CT with 99mTc-labeled IMMU-4 anti-carcinoembryonic antigen Fab′ antibody fragment SPECT in 14 patients with NSCLC using a landmark-based algorithm. The algorithm's accuracy was a measure from the center-to-center distance and the percentage overlap of two regions of interest: one drawn on CT and warped onto SPECT, the other drawn directly on the SPECT. We found that the average center-to-center distance was 1.3 ± 0.8 pixels. Average overlap was 46 ± 20%. CT-SPECT fusion helped differentiate tumor from normal blood pool, necrotic areas within viable tumor, tumor recurrence from scar, and malignant lymphadenopathy from hyperplasia. We conclude that fusion of CT and SPECT augments the information provided by each separate modality. Future clinical applications of fusion in NSCLC staging using immunoscintigraphy appear promising.
---
paper_title: Intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy for lung or liver cancers with a fusion of CT and linear accelerator (FOCAL) unit.

paper_content:

PURPOSE ::: To evaluate intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy (SRT) for lung or liver cancers, we checked repeated CT scanning, with a fusion of CT and linear accelerator (FOCAL) unit. ::: ::: ::: METHODS AND MATERIALS ::: The FOCAL unit is a combination of a linear accelerator (Linac), CT scanner, X-ray simulator (X-S), and carbon table, and is designed to achieve CT-guided SRT with daily CT positioning followed by immediate irradiation while patients keep reduced shallow respirations. To evaluate intrafractional tumor position stability, 50 lung or liver lesions in 20 patients were checked by repeated CT scanning just before and after irradiation, and the obtained images were compared. ::: ::: ::: RESULTS ::: There was no case with the intrafractional error judged to be greater than 10 mm. In 68% of cases, the intrafractional positioning errors were negligible (0-5 mm). ::: ::: ::: CONCLUSIONS ::: Using the FOCAL unit, SRT for lung or liver cancers could be performed with intrafractional positioning errors not greater than 10 mm.
---
paper_title: Fusion imaging using a hybrid SPECT-CT camera improves port perfusion scintigraphy for control of hepatic arterial infusion of chemotherapy in colorectal cancer patients

paper_content:

PURPOSE ::: Exclusive and homogeneous perfusion of the liver is considered essential for the efficacy of hepatic arterial infusion of chemotherapy (HAI). The aim of this study was to evaluate port perfusion scintigraphy in colorectal cancer patients using a hybrid SPECT-CT system for control of minimally invasive intra-arterial port systems within the scope of a phase II trial. ::: ::: ::: METHODS ::: In 24 consecutive patients, the perfusion territories of intra-arterial hepatic port systems were assessed by port scintigraphy with( 99m)Tc-labelled macroaggregated albumin employing planar imaging, SPECT and SPECT-CT (acquired with a hybrid SPECT-CT camera). The results of blinded reading of the scintigraphic modalities concerning the intra- and extrahepatic perfusion pattern were compared with combined image analysis (angiography and contrast-enhanced dedicated CT) and patient history for validation. ::: ::: ::: RESULTS ::: Extrahepatic perfusion was correctly seen in three patients, while suspected extrahepatic perfusion could be excluded in one. In 46 liver lobes, perfusion patterns were correctly visualised by SPECT-CT in 100% of cases (planar, 67%; SPECT, 86%). Assessing the perfusion pattern inside the liver on a segmental basis (segments, n=138), SPECT-CT revealed correct segmental assignment of tracer distribution in 100% and was significantly superior to SPECT alone (accuracy, 84%; p<0.001). The scintigraphic findings resulted in changes in therapeutic management in 8/24 patients (33%); in two of these the relevant findings were visualised only by SPECT-CT. ::: ::: ::: CONCLUSION ::: In patients receiving HAI, port perfusion scintigraphy by fusion imaging with a hybrid SPECT-CT system provides important information for therapy optimisation and appears to be superior to SPECT alone.
---
paper_title: Variability of gross tumor volume delineation in head-and-neck cancer using CT and PET/CT fusion.

paper_content:

PURPOSE ::: To assess the need for gross tumor volume (GTV) delineation protocols in head-and-neck cancer (HNC) treatment planning by use of positron emission tomography (PET)/computed tomography (CT) fusion imaging. Assessment will consist of interobserver and intermodality variation analysis. ::: ::: ::: METHODS AND MATERIALS ::: Sixteen HNC patients were accrued for the study. Four physicians (2 neuroradiologists and 2 radiation oncologists) contoured GTV on 16 patients. Physicians were asked to contour GTV on the basis of the CT alone, and then on PET/CT fusion. Statistical analysis included analysis of variance for interobserver variability and Student's paired sample t test for intermodality and interdisciplinary variability. A Boolean pairwise analysis was included to measure degree of overlap. ::: ::: ::: RESULTS ::: Near-significant variation occurred across physicians' CT volumes (p = 0.09) and significant variation occurred across physicians' PET/CT volumes (p = 0.0002). The Boolean comparison correlates with statistical findings. One radiation oncologist's PET/CT fusion volumes were significantly larger than his CT volumes (p < 0.01). Conversely, the other radiation oncologist's CT volumes tended to be larger than his fusion volumes (p = 0.06). No significant interdisciplinary variation was seen. Significant disagreement occurred between radiation oncologists. ::: ::: ::: CONCLUSION ::: Significant differences in GTV delineation were found between multiple observers contouring on PET/CT fusion. The need for delineation protocol has been confirmed.
---
paper_title: Use of secondary data to estimate instantaneous model parameters of diabetic heart disease: Lemonade Method

paper_content:

With the increasing burden of chronic diseases on the health care system, Markov-type models are becoming popular to predict the long-term outcomes of early intervention and to guide disease management. However, statisticians have not been actively involved in the development of these models. Typically, the models are developed by using secondary data analysis to find a single ''best'' study to estimate each transition in the model. However, due to the nature of secondary data analysis, there frequently are discrepancies between the theoretical model and the design of the studies being used. This paper illustrates a likelihood approach to correctly model the design of clinical studies under the conditions where (1) the theoretical model may include an instantaneous state of distinct interest to the researchers and (2) the study design may be such that study data cannot be used to estimate a single parameter in the theoretical model of interest. For example, a study may ignore intermediary stages of disease. Using our approach, not only can we accommodate the two conditions above, but more than one study may be used to estimate model parameters. In the spirit of ''If life gives you lemon, make lemonade'', we call this method ''Lemonade Method''. Simulation studies are carried out to evaluate the finite sample property of this method. In addition, the method is demonstrated through application to a model of heart disease in diabetes.
---
paper_title: Fuzzy rule base classifier fusion for protein mass spectra based ovarian cancer diagnosis

paper_content:

Fuzzy rule base classification systems have been the focus of increased attention in recent years, due to their unique capability of providing human experts with outcomes by means of linguistic rules. In the same time period classifier fusion approaches have been shown to enhance the performance of pattern recognition systems. In the present study we applied a hybrid random subspace fusion scheme that constructs a set of different fuzzy classifiers utilizing different subsets of both the feature space and the sample domain, combining the results of these classifiers using appropriate decision functions. Experimental results using two protein mass spectra datasets of ovarian cancer demonstrate the usefulness of this approach in comparison to other classifier fusion approaches.
---
paper_title: Photo-realistic representation of anatomical structures for medical education by fusion of volumetric and surface image data

paper_content:

We have produced improved photo-realistic views of anatomical structures for medical education combining data from photographic images of anatomical surfaces with optical, CT and MRI volumetric data such as provided by the NLM Visible Human Project. Volumetric data contains the information needed to construct 3D geometrical models of anatomical structures, but cannot provide a realistic appearance for surfaces. Nieder has captured high quality photographic sequences of anatomy specimens over a range of rotational angles. These have been assembled into QuickTime VR Object movies that can be viewed statically or dynamically. We reuse this surface imagery to produce textures and surface reflectance maps for 3D anatomy models to allow viewing from any orientation and lighting condition. Because the volumetric data comes from different individuals than the surface images, we have to warp these data into alignment. Currently we do not use structured lighting or other direct 3D surface information, so surface shape is recovered from rotational sequences using silhouettes and texture correlations. The results of this work improves the appearance and generality of models, used for anatomy instruction with the PSC Volume Browser.
---
paper_title: Intermodality registration and fusion of liver images for medical diagnosis

paper_content:

SPECT is valuable for the evaluation of hepatic function and CT can offer better anatomical images for liver and its adjacent structures. Image registration from the two modalities, CT and SPECT, acquires both structural and functional information, and therefore is very useful for clinical diagnoses. The registration and fusion of CT and SPECT images are studied. We propose a two stage registration method based on principal axes transformation and surface fitting to register and fuse CT and SPECT images: First, the centroids and principal axes of these landmarks are calculated and aligned. Then the transformation matrix between CT and SPECT images is modified based on the least square error on the distance image. After registration, uniform volume data is reconstructed for image fusion.
---
paper_title: Image Fusion for Computer-Assisted Bone Tumor Surgery

paper_content:

The fusion of computed tomography and magnetic resonance images is a software-dependent processing technique that enables one to integrate and analyze preoperative images for planning complex musculoskeletal tumor resections. By integrating various imaging modalities into one imaging data set we may facilitate preoperative image analysis and planning of navigation computer-assisted bone tumor resection and reconstruction. We performed image fusion for computer-assisted tumor surgery in 13 consecutive patients, seven males and six females, with a mean age of 35.8 years (range, 6–80 years). Visual verification of fused images was accurate in all patients. The mean time for image fusion was 30.6 minutes (range, 8–80 minutes). After intraoperative registration, all tumor resections were performed as planned preoperatively under navigation image guidance. Resections achieved after navigation resection planning were validated by postoperative CT or resected specimens in seven patients. Histologic examination of all resected specimens showed tumor-free margins in patients with bone sarcoma. The fusion of computed tomography and magnetic resonance imaging has the potential to enhance computer-assisted bone tumor surgery. The fusion image, when combined with surgical navigation, helps surgeons reproduce a preoperative plan reliably and may offer substantial clinical benefits.
---
paper_title: Diagnostic performance of fluorodeoxyglucose positron emission tomography/magnetic resonance imaging fusion images of gynecological malignant tumors: comparison with positron emission tomography/computed tomography

paper_content:

Purpose ::: We compared the diagnostic accuracy of fluorodeoxyglucose positron emission tomography/computed tomography (FDG PET/CT) and PET/magnetic resonance imaging (MRI) fusion images for gynecological malignancies.
---
paper_title: Multi-information fusion based tumor cell of bone marrow involvement

paper_content:

The identification of a tumor cell of bone marrow involvement (TCoBMI) is very difficult due to the complexity of hematopoiesis in bone marrow. Based on the demand of medical applications, in this paper, the multi-information fusion method based on the Modified Dempster-Shafer method (MDS) and the research of the identification of TCoBMI is presented with three information sources of the micrograph of bone marrow, blood checks and clinical data. The experiment indicates that this method simulates the identification process of the experienced pathologist successfully. The identification of TCoBiMI based on MDS multi-information fusion method is feasible in medical applications. The result suggests that the accuracy of identification is high and the method of three information source fusion is a powerful aid of diagnosis to identify TCoBMI.
---
paper_title: The impact of image fusion in resolving discrepant findings between FDG-PET and MRI/CT in patients with gynaecological cancers

paper_content:

This study was performed to prospectively investigate the impact of image fusion in resolving discrepant findings between fluorine-18 fluorodeoxyglucose positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) or X-ray computed tomography (CT) in patients with gynaecological cancers. Discrepant findings were defined as lesions where the difference between the FDG-PET and MRI/CT images was assigned a value of at least 2 on a 5-point probability scale. The FDG-PET and MRI/CT images were taken within 1 month of each other. Image fusion between FDG-PET and CT was performed by automatic registration between the two images. During an 18-month period, 34 malignant lesions and seven benign lesions from 32 patients who had undergone either surgical excision or a CT-guided histopathological investigation were included for analysis. Among these cases, image fusion was most frequently required to determine the nature and/or the extent of abdominal and pelvic lesions (28/41, 68%), especially as regards peritoneal seeding (8/41, 20%). Image fusion was most useful in providing better localisation for biopsy (16/41, 39%) and in discriminating between lesions with pathological versus physiological FDG uptake (12/41, 29%). Image fusion changed the original diagnosis based on MRI/CT alone in 9/41 lesions (22%), and the original diagnosis based on FDG-PET alone in 5/41 lesions (12%). It led to alteration of treatment planning (surgery or radiotherapy) in seven of the 32 patients (22%). In patients with gynaecological cancers, the technique of image fusion is helpful in discriminating the nature of FDG-avid lesions, in effectively localising lesions for CT-guided biopsy and in providing better surgical or radiotherapy planning.
---
paper_title: CT-MR image data fusion for computer assisted navigated neurosurgery of temporal bone tumors.

paper_content:

PURPOSE ::: To demonstrate the value of multi detector computed tomography (MDCT) and magnetic resonance imaging (MRI) in the preoperative work up of temporal bone tumors and to present, especially, CT and MR image fusion for surgical planning and performance in computer assisted navigated neurosurgery of temporal bone tumors. ::: ::: ::: MATERIALS AND METHODS ::: Fifteen patients with temporal bone tumors underwent MDCT and MRI. MDCT was performed in high-resolution bone window level setting in axial plane. The reconstructed MDCT slice thickness was 0.8 mm. MRI was performed in axial and coronal plane with T2-weighted fast spin-echo (FSE) sequences, un-enhanced and contrast-enhanced T1-weighted spin-echo (SE) sequences, and coronal T1-weighted SE sequences with fat suppression and with 3D T1-weighted gradient-echo (GE) contrast-enhanced sequences in axial plane. The 3D T1-weighted GE sequence had a slice thickness of 1mm. Image data sets of CT and 3D T1-weighted GE sequences were merged utilizing a workstation to create CT-MR fusion images. MDCT and MR images were separately used to depict and characterize lesions. The fusion images were utilized for interventional planning and intraoperative image guidance. The intraoperative accuracy of the navigation unit was measured, defined as the deviation between the same landmark in the navigation image and the patient. ::: ::: ::: RESULTS ::: Tumorous lesions of bone and soft tissue were well delineated and characterized by CT and MR images. The images played a crucial role in the differentiation of benign and malignant pathologies, which consisted of 13 benign and 2 malignant tumors. The CT-MR fusion images supported the surgeon in preoperative planning and improved surgical performance. The mean intraoperative accuracy of the navigation system was 1.25 mm. ::: ::: ::: CONCLUSION ::: CT and MRI are essential in the preoperative work up of temporal bone tumors. CT-MR image data fusion presents an accurate tool for planning the correct surgical procedure and is a benefit for the operational results in computer assisted navigated neurosurgery of temporal bone tumors.
---
paper_title: Fusion of Immunoscintigraphy Single Photon Emission Computed Tomography (SPECT) with CT of the Chest in Patients with Non-Small Cell Lung Cancer

paper_content:

In non-small cell lung cancer (NSCLC), accurate staging is critical in deciding between potentially curative surgery and palliative treatment. Image registration, or fusion, combines the unique functional information provided by SPECT imaging with the excellent anatomic detail offered by computed tomography (CT) or magnetic resonance imaging to better characterize the imformation provided by each separate modality. In this study, we explored the role of fusion of immunoscintigraphy SPECT with CT in the staging of NSCLC. We fused chest CT with 99mTc-labeled IMMU-4 anti-carcinoembryonic antigen Fab′ antibody fragment SPECT in 14 patients with NSCLC using a landmark-based algorithm. The algorithm's accuracy was a measure from the center-to-center distance and the percentage overlap of two regions of interest: one drawn on CT and warped onto SPECT, the other drawn directly on the SPECT. We found that the average center-to-center distance was 1.3 ± 0.8 pixels. Average overlap was 46 ± 20%. CT-SPECT fusion helped differentiate tumor from normal blood pool, necrotic areas within viable tumor, tumor recurrence from scar, and malignant lymphadenopathy from hyperplasia. We conclude that fusion of CT and SPECT augments the information provided by each separate modality. Future clinical applications of fusion in NSCLC staging using immunoscintigraphy appear promising.
---
paper_title: Oral cancer detection in fluorescent image by color image fusion

paper_content:

Biopsy is the main diagnosis method for oral cancer, which is the fifth most prevalent cancer in male population in Taiwan. In order to increase the accuracy of diagnosis, it is required to reveal the distribution of malignancy prior to biopsy. Based on this purpose, we developed the fluorescent image system associated with color image fusion algorithm to indicate the cancer-susceptible area in the oral image. The system therefore can assist physicians to make appropriate biopsy.
---
paper_title: Intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy for lung or liver cancers with a fusion of CT and linear accelerator (FOCAL) unit.

paper_content:

PURPOSE ::: To evaluate intrafractional tumor position stability during computed tomography (CT)-guided frameless stereotactic radiation therapy (SRT) for lung or liver cancers, we checked repeated CT scanning, with a fusion of CT and linear accelerator (FOCAL) unit. ::: ::: ::: METHODS AND MATERIALS ::: The FOCAL unit is a combination of a linear accelerator (Linac), CT scanner, X-ray simulator (X-S), and carbon table, and is designed to achieve CT-guided SRT with daily CT positioning followed by immediate irradiation while patients keep reduced shallow respirations. To evaluate intrafractional tumor position stability, 50 lung or liver lesions in 20 patients were checked by repeated CT scanning just before and after irradiation, and the obtained images were compared. ::: ::: ::: RESULTS ::: There was no case with the intrafractional error judged to be greater than 10 mm. In 68% of cases, the intrafractional positioning errors were negligible (0-5 mm). ::: ::: ::: CONCLUSIONS ::: Using the FOCAL unit, SRT for lung or liver cancers could be performed with intrafractional positioning errors not greater than 10 mm.
---

Title: Medical Image Fusion: A Survey of the State of the Art

Section 1: Introduction
Description 1: Provide an overview of medical image fusion, its growing appeal, and its relevance to medical diagnostics.

Section 2: Morphological Methods
Description 2: Discuss the application of morphology operators in medical image fusion, highlighting their strengths and weaknesses.

Section 3: Knowledge-Based Methods
Description 3: Explain how domain-specific knowledge from medical practitioners is utilized in designing segmentation, labeling, and registration of images for fusion.

Section 4: Wavelet-Based Methods
Description 4: Describe the use of wavelets in medical image fusion, including various models and applications.

Section 5: Neural Network-Based Methods
Description 5: Outline the application of artificial neural networks (ANN) in image fusion, their advantages, and limitations.

Section 6: Methods Based on Fuzzy Logic
Description 6: Detail the role of fuzzy logic in image fusion, including its conjunction with other techniques for enhanced feature analysis.

Section 7: Other Methods
Description 7: Summarize various other techniques used in medical image fusion including ICA, PCA, and SVM among others.

Section 8: Imaging Modalities
Description 8: Explore different imaging modalities used in medical image fusion, such as MRI, CT, PET, SPECT, and Ultrasound, and discuss their specific applications and combinations.

Section 9: Applications in Various Organs
Description 9: Discuss the application of medical image fusion techniques in studying different organs such as the brain, breast, prostate, lungs, liver, and other organs.

Section 10: Discussions and Conclusions
Description 10: Summarize the technological advances, challenges, and future directions in the field of medical image fusion.