Incorporating prior knowledge in medical image segmentation: a survey
---
paper_title: A Fuzzy Locally Adaptive Bayesian Segmentation Approach for Volume Determination in PET

paper_content:

Accurate volume estimation in positron emission tomography (PET) is crucial for different oncology applications. The objective of our study was to develop a new fuzzy locally adaptive Bayesian (FLAB) segmentation for automatic lesion volume delineation. FLAB was compared with a threshold approach as well as the previously proposed fuzzy hidden Markov chains (FHMC) and the fuzzy C-Means (FCM) algorithms. The performance of the algorithms was assessed on acquired datasets of the IEC phantom, covering a range of spherical lesion sizes (10-37 mm), contrast ratios (4:1 and 8:1), noise levels (1, 2, and 5 min acquisitions), and voxel sizes (8 and 64 mm3). In addition, the performance of the FLAB model was assessed on realistic nonuniform and nonspherical volumes simulated from patient lesions. Results show that FLAB performs better than the other methodologies, particularly for smaller objects. The volume error was 5%-15% for the different sphere sizes (down to 13 mm), contrast and image qualities considered, with a high reproducibility (variation < 4%). By comparison, the thresholding results were greatly dependent on image contrast and noise, whereas FCM results were less dependent on noise but consistently failed to segment lesions < 2 cm. In addition, FLAB performed consistently better for lesions < 2 cm in comparison to the FHMC algorithm. Finally the FLAB model provided errors less than 10% for nonspherical lesions with inhomogeneous activity distributions. Future developments will concentrate on an extension of FLAB in order to allow the segmentation of separate activity distribution regions within the same functional volume as well as a robustness study with respect to different scanners and reconstruction algorithms.
---
paper_title: Segmentation of medical images using adaptive region growing

paper_content:

Interaction increases flexibility of segmentation but it leads to undesirable behavior of an algorithm if knowledge being requested is inappropriate. In region growing, this is the case for defining the homogeneity criterion as its specification depends also on image formation properties that are not known to the user. We developed a region growing algorithm that learns its homogeneity criterion automatically from characteristics of the region to be segmented. The method is based on a model that describes homogeneity and simple shape properties of the region. Parameters of the homogeneity criterion are estimated from sample locations in the region. These locations are selected sequentially in a random walk starting at the seed point, and the homogeneity criterion is updated continuously. The method was tested for segmentation on test images and of structures in CT images. We found the method to work reliable if the model assumption on homogeneity and region characteristics are true. Furthermore, the model is simple but robust, thus allowing for a certain degree of deviation from model constraints and still delivering the expected segmentation result. This approach was extended to a fully automatic and complete segmentation method by using the pixels with the smallest gradient length in the not yet segmented image region as a seed point.
---
paper_title: An Efficient Optimization Framework for Multi-Region Segmentation Based on Lagrangian Duality

paper_content:

We introduce a multi-region model for simultaneous segmentation of medical images. In contrast to many other models, geometric constraints such as inclusion and exclusion between the regions are enforced, which makes it possible to correctly segment different regions even if the intensity distributions are identical. We efficiently optimize the model using a combination of graph cuts and Lagrangian duality which is faster and more memory efficient than current state of the art. As the method is based on global optimization techniques, the resulting segmentations are independent of initialization. We apply our framework to the segmentation of the left and right ventricles, myocardium and the left ventricular papillary muscles in magnetic resonance imaging and to lung segmentation in full-body X-ray computed tomography. We evaluate our approach on a publicly available benchmark with competitive results.
---
paper_title: Seeded region growing

paper_content:

We present here a new algorithm for segmentation of intensity images which is robust, rapid, and free of tuning parameters. The method, however, requires the input of a number of seeds, either individual pixels or regions, which will control the formation of regions into which the image will be segmented. In this correspondence, we present the algorithm, discuss briefly its properties, and suggest two ways in which it can be employed, namely, by using manual seed selection or by automated procedures. >
---
paper_title: Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations

paper_content:

A fast and flexible algorithm for computing watersheds in digital gray-scale images is introduced. A review of watersheds and related motion is first presented, and the major methods to determine watersheds are discussed. The algorithm is based on an immersion process analogy, in which the flooding of the water in the picture is efficiently simulated using of queue of pixel. It is described in detail provided in a pseudo C language. The accuracy of this algorithm is proven to be superior to that of the existing implementations, and it is shown that its adaptation to any kind of digital grid and its generalization to n-dimensional images (and even to graphs) are straightforward. The algorithm is reported to be faster than any other watershed algorithm. Applications of this algorithm with regard to picture segmentation are presented for magnetic resonance (MR) imagery and for digital elevation models. An example of 3-D watershed is also provided. >
---
paper_title: A Bayes-Based Region-Growing Algorithm for Medical Image Segmentation

paper_content:

This paper discusses a new Bayesian-analysis-based region-growing algorithm for medical image segmentation that can robustly and effectively segment medical images. Specifically the approach studies homogeneity criterion parameters in a local neighbor region. Using the multislices Gaussian and anisotropic filters as a preprocess helps reduce an image's noise. The algorithm framework is tested on CT and MRI image segmentation, and experimental results show that the approach is reliable and efficient.
---
paper_title: Joint segmentation of anatomical and functional images: Applications in quantification of lesions from PET, PET-CT, MRI-PET, and MRI-PET-CT images

paper_content:

We present a novel method for the joint segmentation of anatomical and functional images. Our proposed methodology unifies the domains of anatomical and functional images, represents them in a product lattice, and performs simultaneous delineation of regions based on random walk image segmentation. Furthermore, we also propose a simple yet effective object/background seed localization method to make the proposed segmentation process fully automatic. Our study uses PET, PET-CT, MRI-PET, and fused MRI-PET-CT scans (77 studies in all) from 56 patients who had various lesions in different body regions. We validated the effectiveness of the proposed method on different PET phantoms as well as on clinical images with respect to the ground truth segmentation provided by clinicians. Experimental results indicate that the presented method is superior to threshold and Bayesian methods commonly used in PET image segmentation, is more accurate and robust compared to the other PET-CT segmentation methods recently published in the literature, and also it is general in the sense of simultaneously segmenting multiple scans in real-time with high accuracy needed in routine clinical use.
---
paper_title: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration.

paper_content:

OBJECTIVES ::: To investigate a markerless tracking system for real-time stereo-endoscopic visualization of preoperative computed tomographic imaging as an augmented display during robot-assisted laparoscopic partial nephrectomy. ::: ::: ::: METHODS ::: Stereoscopic video segments of a patient undergoing robot-assisted laparoscopic partial nephrectomy for tumor and another for a partial staghorn renal calculus were processed to evaluate the performance of a three-dimensional (3D)-to-3D registration algorithm. After both cases, we registered a segment of the video recording to the corresponding preoperative 3D-computed tomography image. After calibrating the camera and overlay, 3D-to-3D registration was created between the model and the surgical recording using a modified iterative closest point technique. Image-based tracking technology tracked selected fixed points on the kidney surface to augment the image-to-model registration. ::: ::: ::: RESULTS ::: Our investigation has demonstrated that we can identify and track the kidney surface in real time when applied to intraoperative video recordings and overlay the 3D models of the kidney, tumor (or stone), and collecting system semitransparently. Using a basic computer research platform, we achieved an update rate of 10 Hz and an overlay latency of 4 frames. The accuracy of the 3D registration was 1 mm. ::: ::: ::: CONCLUSIONS ::: Augmented reality overlay of reconstructed 3D-computed tomography images onto real-time stereo video footage is possible using iterative closest point and image-based surface tracking technology that does not use external navigation tracking systems or preplaced surface markers. Additional studies are needed to assess the precision and to achieve fully automated registration and display for intraoperative use.
---
paper_title: Watershed segmentation using prior shape and appearance knowledge

paper_content:

Watershed transformation is a common technique for image segmentation. However, its use for automatic medical image segmentation has been limited particularly due to oversegmentation and sensitivity to noise. Employing prior shape knowledge has demonstrated robust improvements to medical image segmentation algorithms. We propose a novel method for enhancing watershed segmentation by utilizing prior shape and appearance knowledge. Our method iteratively aligns a shape histogram with the result of an improved k-means clustering algorithm of the watershed segments. Quantitative validation of magnetic resonance imaging segmentation results supports the robust nature of our method.
---
paper_title: Measuring tortuosity of the intracerebral vasculature from MRA images

paper_content:

The clinical recognition of abnormal vascular tortuosity, or excessive bending, twisting, and winding, is important to the diagnosis of many diseases. Automated detection and quantitation of abnormal vascular tortuosity from three-dimensional (3-D) medical image data would, therefore, be of value. However, previous research has centered primarily upon two-dimensional (2-D) analysis of the special subset of vessels whose paths are normally close to straight. This report provides the first 3-D tortuosity analysis of clusters of vessels within the normally tortuous intracerebral circulation. We define three different clinical patterns of abnormal tortuosity. We extend into 3-D two tortuosity metrics previously reported as useful in analyzing 2-D images and describe a new metric that incorporates counts of minima of total curvature. We extract vessels from MRA data, map corresponding anatomical regions between sets of normal patients and patients with known pathology, and evaluate the three tortuosity metrics for ability to detect each type of abnormality within the region of interest. We conclude that the new tortuosity metric appears to be the most effective in detecting several types of abnormalities. However, one of the other metrics, based on a sum of curvature magnitudes, may be more effective in recognizing tightly coiled, "corkscrew" vessels associated with malignant tumors.
---
paper_title: Multifeature Prostate Cancer Diagnosis and Gleason Grading of Histological Images

paper_content:

We present a study of image features for cancer diagnosis and Gleason grading of the histological images of prostate. In diagnosis, the tissue image is classified into the tumor and nontumor classes. In Gleason grading, which characterizes tumor aggressiveness, the image is classified as containing a low- or high-grade tumor. The image sets used in this paper consisted of 367 and 268 color images for the diagnosis and Gleason grading problems, respectively, and were captured from representative areas of hematoxylin and eosin-stained tissue retrieved from tissue microarray cores or whole sections. The primary contribution of this paper is to aggregate color, texture, and morphometric cues at the global and histological object levels for classification. Features representing different visual cues were combined in a supervised learning framework. We compared the performance of Gaussian, -nearest neighbor, and support vector machine classifiers together with the sequential forward feature selection algorithm. On diagnosis, using a five-fold cross-validation estimate, an accuracy of 96.7% was obtained. On Gleason grading, the achieved accuracy of classification into low- and high-grade classes was 81.0%.
---
paper_title: Detection and Measurement of Fetal Anatomies from Ultrasound Images using a Constrained Probabilistic Boosting Tree

paper_content:

We propose a novel method for the automatic detection and measurement of fetal anatomical structures in ultrasound images. This problem offers a myriad of challenges, including: difficulty of modeling the appearance variations of the visual object of interest, robustness to speckle noise and signal dropout, and large search space of the detection procedure. Previous solutions typically rely on the explicit encoding of prior knowledge and formulation of the problem as a perceptual grouping task solved through clustering or variational approaches. These methods are constrained by the validity of the underlying assumptions and usually are not enough to capture the complex appearances of fetal anatomies. We propose a novel system for fast automatic detection and measurement of fetal anatomies that directly exploits a large database of expert annotated fetal anatomical structures in ultrasound images. Our method learns automatically to distinguish between the appearance of the object of interest and background by training a constrained probabilistic boosting tree classifier. This system is able to produce the automatic segmentation of several fetal anatomies using the same basic detection algorithm. We show results on fully automatic measurement of biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC), femur length (FL), humerus length (HL), and crown rump length (CRL). Notice that our approach is the first in the literature to deal with the HL and CRL measurements. Extensive experiments (with clinical validation) show that our system is, on average, close to the accuracy of experts in terms of segmentation and obstetric measurements. Finally, this system runs under half second on a standard dual-core PC computer.
---
paper_title: An effective visualisation and registration system for image-guided robotic partial nephrectomy

paper_content:

Robotic partial nephrectomy is presently the fastest-growing robotic surgical procedure, and in comparison to traditional techniques it offers reduced tissue trauma and likelihood of post-operative infection, while hastening recovery time and improving cosmesis. It is also an ideal candidate for image guidance technology since soft tissue deformation, while still present, is localised and less problematic compared to other surgical procedures. This work describes the implementation and ongoing development of an effective image guidance system that aims to address some of the remaining challenges in this area. Specific innovations include the introduction of an intuitive, partially automated registration interface, and the use of a hardware platform that makes sophisticated augmented reality overlays practical in real time. Results and examples of image augmentation are presented from both retrospective and live cases. Quantitative analysis of registration error verifies that the proposed registration technique is appropriate for the chosen image guidance targets.
---
paper_title: Improved watershed transform for medical image segmentation using prior information

paper_content:

The watershed transform has interesting properties that make it useful for many different image segmentation applications: it is simple and intuitive, can be parallelized, and always produces a complete division of the image. However, when applied to medical image analysis, it has important drawbacks (oversegmentation, sensitivity to noise, poor detection of thin or low signal to noise ratio structures). We present an improvement to the watershed transform that enables the introduction of prior information in its calculation. We propose to introduce this information via the use of a previous probability calculation. Furthermore, we introduce a method to combine the watershed transform and atlas registration, through the use of markers. We have applied our new algorithm to two challenging applications: knee cartilage and gray matter/white matter segmentation in MR images. Numerical validation of the results is provided, demonstrating the strength of the algorithm for medical image segmentation.
---
paper_title: Segmentation of volumetric MRA images by using Capillary Active Contour

paper_content:

Precise segmentation of three-dimensional (3D) magnetic resonance angiography (MRA) images can be a very useful computer aided diagnosis (CAD) tool for clinical routines. Level sets based evolution schemes, which have been shown to be effective and easy to implement for many segmentation applications, are being applied to MRA data sets. In this paper, we present a segmentation scheme for accurately extracting vasculature from MRA images. Our proposed algorithm models capillary action and derives a capillary active contour for segmentation of thin vessels. The algorithm is implemented using the level set method and has been applied successfully on real 3D MRA images. Compared with other state-of-the-art MRA segmentation algorithms, experiments show that our method facilitates more accurate segmentation of thin blood vessels.
---
paper_title: Ultrasound image segmentation: a survey

paper_content:

This paper reviews ultrasound segmentation methods, in a broad sense, focusing on techniques developed for medical B-mode ultrasound images. First, we present a review of articles by clinical application to highlight the approaches that have been investigated and degree of validation that has been done in different clinical domains. Then, we present a classification of methodology in terms of use of prior information. We conclude by selecting ten papers which have presented original ideas that have demonstrated particular clinical usefulness or potential specific to the ultrasound segmentation problem
---
paper_title: Automated medical image segmentation techniques

paper_content:

Accurate segmentation of medical images is a key step in contouring during radiotherapy planning. Computed topography (CT) and Magnetic resonance (MR) imaging are the most widely used radiographic techniques in diagnosis, clinical studies and treatment planning. This review provides details of automated segmentation methods, specifically discussed in the context of CT and MR images. The motive is to discuss the problems encountered in segmentation of CT and MR images, and the relative merits and limitations of methods currently available for segmentation of medical images.
---
paper_title: Current methods in medical image segmentation.

paper_content:

Image segmentation plays a crucial role in many medical-imaging applications, by automating or facilitating the delineation of anatomical structures and other regions of interest. We present a critical appraisal of the current status of semi-automated and automated methods for the segmentation of anatomical medical images. Terminology and important issues in image segmentation are first presented. Current segmentation approaches are then reviewed with an emphasis on the advantages and disadvantages of these methods for medical imaging applications. We conclude with a discussion on the future of image segmentation methods in biomedical research.
---
paper_title: A review of segmentation methods in short axis cardiac MR images

paper_content:

For the last 15 years, Magnetic Resonance Imaging (MRI) has become a reference examination for cardiac morphology, function and perfusion in humans. Yet, due to the characteristics of cardiac MR images and to the great variability of the images among patients, the problem of heart cavities segmentation in MRI is still open. This paper is a review of fully and semi-automated methods performing segmentation in short axis images using a cardiac cine MRI sequence. Medical background and specific segmentation difficulties associated to these images are presented. For this particularly complex segmentation task, prior knowledge is required. We thus propose an original categorization for cardiac segmentation methods, with a special emphasis on what level of external information is required (weak or strong) and how it is used to constrain segmentation. After reviewing method principles and analyzing segmentation results, we conclude with a discussion and future trends in this field regarding methodological and medical issues.
---
paper_title: Deformable Models in Medical Image Analysis : A Survey

paper_content:

This article surveys deformable models, a promising and vigorously researched computerassisted medical image analysis technique. Among model-based techniques, deformable models offer a unique and powerful approach to image analysis that combines geometry, physics, and approximation theory. They have proven to be effective in segmenting, matching, and tracking anatomic structures by exploiting (bottom-up) constraints derived from the image data together with (top-down) knowledge about the location, size, and shape of these structures. Deformable models are capable of accommodating the significant variability of biological structures over time and across different individuals. Furthermore, they support highly intuitive interaction mechanisms that, when necessary, allow medical scientists and practitioners to bring their expertise to bear on the model-based image interpretation task. This article reviews the rapidly expanding body of work on the development and application of deformable models to problems of fundamental importance in medical image analysis, including segmentation, shape representation, matching, and motion tracking.
---
paper_title: A Survey of Graph Theoretical Approaches to Image Segmentation

paper_content:

Image segmentation is a fundamental problem in computer vision. Despite many years of research, general purpose image segmentation is still a very challenging task because segmentation is inherently ill-posed. Among different segmentation schemes, graph theoretical ones have several good features in practical applications. It explicitly organizes the image elements into mathematically sound structures, and makes the formulation of the problem more flexible and the computation more efficient. In this paper, we conduct a systematic survey of graph theoretical methods for image segmentation, where the problem is modeled in terms of partitioning a graph into several sub-graphs such that each of them represents a meaningful object of interest in the image. These methods are categorized into five classes under a uniform notation: the minimal spanning tree based methods, graph cut based methods with cost functions, graph cut based methods on Markov random field models, the shortest path based methods and the other methods that do not belong to any of these classes. We present motivations and detailed technical descriptions for each category of methods. The quantitative evaluation is carried by using five indices - Probabilistic Rand (PR) index, Normalized Probabilistic Rand (NPR) index, Variation of Information (VI), Global Consistency Error (GCE) and Boundary Displacement Error (BDE) - on some representative automatic and interactive segmentation methods.
---
paper_title: A review of 3D vessel lumen segmentation techniques: Models, features and extraction schemes

paper_content:

Vascular diseases are among the most important public health problems in developed countries. Given the size and complexity of modern angiographic acquisitions, segmentation is a key step toward the accurate visualization, diagnosis and quantification of vascular pathologies. Despite the tremendous amount of past and on-going dedicated research, vascular segmentation remains a challenging task. In this paper, we review state-of-the-art literature on vascular segmentation, with a particular focus on 3D contrast-enhanced imaging modalities (MRA and CTA). We structure our analysis along three axes: models, features and extraction schemes. We first detail model-based assumptions on the vessel appearance and geometry which can embedded in a segmentation approach. We then review the image features that can be extracted to evaluate these models. Finally, we discuss how existing extraction schemes combine model and feature information to perform the segmentation task. Each component (model, feature and extraction scheme) plays a crucial role toward the efficient, robust and accurate segmentation of vessels of interest. Along each axis of study, we discuss the theoretical and practical properties of recent approaches and highlight the most advanced and promising ones.
---
paper_title: Segmentation of medical images using adaptive region growing

paper_content:

Interaction increases flexibility of segmentation but it leads to undesirable behavior of an algorithm if knowledge being requested is inappropriate. In region growing, this is the case for defining the homogeneity criterion as its specification depends also on image formation properties that are not known to the user. We developed a region growing algorithm that learns its homogeneity criterion automatically from characteristics of the region to be segmented. The method is based on a model that describes homogeneity and simple shape properties of the region. Parameters of the homogeneity criterion are estimated from sample locations in the region. These locations are selected sequentially in a random walk starting at the seed point, and the homogeneity criterion is updated continuously. The method was tested for segmentation on test images and of structures in CT images. We found the method to work reliable if the model assumption on homogeneity and region characteristics are true. Furthermore, the model is simple but robust, thus allowing for a certain degree of deviation from model constraints and still delivering the expected segmentation result. This approach was extended to a fully automatic and complete segmentation method by using the pixels with the smallest gradient length in the not yet segmented image region as a seed point.
---
paper_title: Seeded region growing

paper_content:

We present here a new algorithm for segmentation of intensity images which is robust, rapid, and free of tuning parameters. The method, however, requires the input of a number of seeds, either individual pixels or regions, which will control the formation of regions into which the image will be segmented. In this correspondence, we present the algorithm, discuss briefly its properties, and suggest two ways in which it can be employed, namely, by using manual seed selection or by automated procedures. >
---
paper_title: Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations

paper_content:

A fast and flexible algorithm for computing watersheds in digital gray-scale images is introduced. A review of watersheds and related motion is first presented, and the major methods to determine watersheds are discussed. The algorithm is based on an immersion process analogy, in which the flooding of the water in the picture is efficiently simulated using of queue of pixel. It is described in detail provided in a pseudo C language. The accuracy of this algorithm is proven to be superior to that of the existing implementations, and it is shown that its adaptation to any kind of digital grid and its generalization to n-dimensional images (and even to graphs) are straightforward. The algorithm is reported to be faster than any other watershed algorithm. Applications of this algorithm with regard to picture segmentation are presented for magnetic resonance (MR) imagery and for digital elevation models. An example of 3-D watershed is also provided. >
---
paper_title: A Bayes-Based Region-Growing Algorithm for Medical Image Segmentation

paper_content:

This paper discusses a new Bayesian-analysis-based region-growing algorithm for medical image segmentation that can robustly and effectively segment medical images. Specifically the approach studies homogeneity criterion parameters in a local neighbor region. Using the multislices Gaussian and anisotropic filters as a preprocess helps reduce an image's noise. The algorithm framework is tested on CT and MRI image segmentation, and experimental results show that the approach is reliable and efficient.
---
paper_title: Watershed segmentation using prior shape and appearance knowledge

paper_content:

Watershed transformation is a common technique for image segmentation. However, its use for automatic medical image segmentation has been limited particularly due to oversegmentation and sensitivity to noise. Employing prior shape knowledge has demonstrated robust improvements to medical image segmentation algorithms. We propose a novel method for enhancing watershed segmentation by utilizing prior shape and appearance knowledge. Our method iteratively aligns a shape histogram with the result of an improved k-means clustering algorithm of the watershed segments. Quantitative validation of magnetic resonance imaging segmentation results supports the robust nature of our method.
---
paper_title: Improved watershed transform for medical image segmentation using prior information

paper_content:

The watershed transform has interesting properties that make it useful for many different image segmentation applications: it is simple and intuitive, can be parallelized, and always produces a complete division of the image. However, when applied to medical image analysis, it has important drawbacks (oversegmentation, sensitivity to noise, poor detection of thin or low signal to noise ratio structures). We present an improvement to the watershed transform that enables the introduction of prior information in its calculation. We propose to introduce this information via the use of a previous probability calculation. Furthermore, we introduce a method to combine the watershed transform and atlas registration, through the use of markers. We have applied our new algorithm to two challenging applications: knee cartilage and gray matter/white matter segmentation in MR images. Numerical validation of the results is provided, demonstrating the strength of the algorithm for medical image segmentation.
---
paper_title: Diagonal preconditioning for first order primal-dual algorithms in convex optimization

paper_content:

In this paper we study preconditioning techniques for the first-order primal-dual algorithm proposed in [5]. In particular, we propose simple and easy to compute diagonal preconditioners for which convergence of the algorithm is guaranteed without the need to compute any step size parameters. As a by-product, we show that for a certain instance of the preconditioning, the proposed algorithm is equivalent to the old and widely unknown alternating step method for monotropic programming [7]. We show numerical results on general linear programming problems and a few standard computer vision problems. In all examples, the preconditioned algorithm significantly outperforms the algorithm of [5].
---
paper_title: A Distributed Mincut/Maxflow Algorithm Combining Path Augmentation and Push-Relabel

paper_content:

We propose a novel distributed algorithm for the minimum cut problem. Motivated by applications like volumetric segmentation in computer vision, we aim at solving large sparse problems. When the problem does not fully fit in the memory, we need to either process it by parts, looking at one part at a time, or distribute across several computers. Many mincut/maxflow algorithms are designed for the shared memory architecture and do not scale to this setting. We consider algorithms that work on disjoint regions of the problem and exchange messages between the regions. We show that the region push-relabel algorithm of Delong and Boykov (A scalable graph-cut algorithm for N-D grids, in CVPR, 2008) uses Θ(n ::: 2) rounds of message exchange, where n is the number of vertices. Our new algorithm performs path augmentations inside the regions and push-relabel style updates between the regions. It uses asymptotically less message exchanges, $O(\mathcal{B}^{2})$ ::: , where $\mathcal{B}$ ::: is the set of boundary vertices. The sequential and parallel versions of our algorithm are competitive with the state-of-the-art in the shared memory model. By achieving a lower amount of message exchanges (even asymptotically lower in our synthetic experiments), they suit better for solving large problems using a disk storage or a distributed system.
---
paper_title: Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey

paper_content:

In this paper, we present a comprehensive survey of Markov Random Fields (MRFs) in computer vision and image understanding, with respect to the modeling, the inference and the learning. While MRFs were introduced into the computer vision field about two decades ago, they started to become a ubiquitous tool for solving visual perception problems around the turn of the millennium following the emergence of efficient inference methods. During the past decade, a variety of MRF models as well as inference and learning methods have been developed for addressing numerous low, mid and high-level vision problems. While most of the literature concerns pairwise MRFs, in recent years we have also witnessed significant progress in higher-order MRFs, which substantially enhances the expressiveness of graph-based models and expands the domain of solvable problems. This survey provides a compact and informative summary of the major literature in this research topic.
---
paper_title: The maximum flow problem is log space complete for P

paper_content:

The space complexity of the maximum flow problem is investigated. It is shown that the problem is log space complete for deterministic polynomial time. Thus the maximum flow problem probably has no algorithm which needs only O(logk n) storage space for any constant k. Another consequence is that there is probably no fast parallel algorithm for the maximum flow problem.
---
paper_title: Parallel and distributed graph cuts by dual decomposition

paper_content:

Graph cuts methods are at the core of many state-of-the-art algorithms in computer vision due to their efficiency in computing globally optimal solutions. In this paper, we solve the maximum flow/minimum cut problem in parallel by splitting the graph into multiple parts and hence, further increase the computational efficacy of graph cuts. Optimality of the solution is guaranteed by dual decomposition, or more specifically, the solutions to the subproblems are constrained to be equal on the overlap with dual variables. We demonstrate that our approach both allows (i) faster processing on multi-core computers and (ii) the capability to handle larger problems by splitting the graph across multiple computers on a distributed network. Even though our approach does not give a theoretical guarantee of speedup, an extensive empirical evaluation on several applications with many different data sets consistently shows good performance. An open source implementation of the dual decomposition method is also made publicly available.
---
paper_title: Globally minimal surfaces by continuous maximal flows

paper_content:

In this paper, we address the computation of globally minimal curves and surfaces for image segmentation and stereo reconstruction. We present a solution, simulating a continuous maximal flow by a novel system of partial differential equations. Existing methods are either grid-biased (graph-based methods) or suboptimal (active contours and surfaces). The solution simulates the flow of an ideal fluid with isotropic velocity constraints. Velocity constraints are defined by a metric derived from image data. An auxiliary potential function is introduced to create a system of partial differential equations. It is proven that the algorithm produces a globally maximal continuous flow at convergence, and that the globally minimal surface may be obtained trivially from the auxiliary potential. The bias of minimal surface methods toward small objects is also addressed. An efficient implementation is given for the flow simulation. The globally minimal surface algorithm is applied to segmentation in 2D and 3D as well as to stereo matching. Results in 2D agree with an existing minimal contour algorithm for planar images. Results in 3D segmentation and stereo matching demonstrate that the new algorithm is robust and free from grid bias.
---
paper_title: Determining the optimal weights in multiple objective function optimization

paper_content:

An important problem in computer vision is the determination of weights for multiple objective function optimization. This problem arises naturally in many reconstruction problems, where one wishes to reconstruct a function belonging to a constrained class of signals based upon noisy observed data. A common approach is to combine the objective functions into a single total cxt function. The problem then is to determine appropriate weights for the objective functions. In this paper we propose techniques for automatically determining the weights, and discuss their properties. The Min-Max Principle, which avoids the problems of extremely low or high weights, is introduced. ExpresEions are derived relating the optimal weights, objective function values, and total cost.
---
paper_title: G.: Adaptive regularization for image segmentation using local image curvature cues

paper_content:

Image segmentation techniques typically require proper weighting of competing data fidelity and regularization terms. Conventionally, the associated parameters are set through tedious trial and error procedures and kept constant over the image. However, spatially varying structural characteristics, such as object curvature, combined with varying noise and imaging artifacts, significantly complicate the selection process of segmentation parameters. In this work, we propose a novel approach for automating the parameter selection by employing a robust structural cue to prevent excessive regularization of trusted (i.e. low noise) high curvature image regions. Our approach autonomously adapts local regularization weights by combining local measures of image curvature and edge evidence that are gated by a signal reliability measure. We demonstrate the utility and favorable performance of our approach within two major segmentation frameworks, graph cuts and active contours, and present quantitative and qualitative results on a variety of natural and medical images.
---
paper_title: A Survey and Comparison of Discrete and Continuous Multi-label Optimization Approaches for the Potts Model

paper_content:

We present a survey and a comparison of a variety of algorithms that have been proposed over the years to minimize multi-label optimization problems based on the Potts model. Discrete approaches based on Markov Random Fields as well as continuous optimization approaches based on partial differential equations can be applied to the task. In contrast to the case of binary labeling, the multi-label problem is known to be NP hard and thus one can only expect near-optimal solutions. In this paper, we carry out a theoretical comparison and an experimental analysis of existing approaches with respect to accuracy, optimality and runtime, aimed at bringing out the advantages and short-comings of the respective algorithms. Systematic quantitative comparison is done on the Graz interactive image segmentation benchmark. This paper thereby generalizes a previous experimental comparison (Klodt et al. 2008) from the binary to the multi-label case.
---
paper_title: On the Partial Difference Equations of Mathematical Physics

paper_content:

Problems involving the classical linear partial differential equations of mathematical physics can be reduced to algebraic ones of a very much simpler structure by replacing the differentials by difference quotients on some (say rectilinear) mesh. This paper will undertake an elementary discussion of these algebraic problems, in particular of the behavior of the solution as the mesh width tends to zero. For present purposes we limit ourselves mainly to simple but typical cases, and treat them in such a way that the applicability of the method to more general difference equations and to those with arbitrarily many independent variables is made clear.
---
paper_title: Computing geodesics and minimal surfaces via graph cuts

paper_content:

Geodesic active contours and graph cuts are two standard image segmentation techniques. We introduce a new segmentation method combining some of their benefits. Our main intuition is that any cut on a graph embedded in some continuous space can be interpreted as a contour (in 2D) or a surface (in 3D). We show how to build a grid graph and set its edge weights so that the cost of cuts is arbitrarily close to the length (area) of the corresponding contours (surfaces) for any anisotropic Riemannian metric. There are two interesting consequences of this technical result. First, graph cut algorithms can be used to find globally minimum geodesic contours (minimal surfaces in 3D) under arbitrary Riemannian metric for a given set of boundary conditions. Second, we show how to minimize metrication artifacts in existing graph-cut based methods in vision. Theoretically speaking, our work provides an interesting link between several branches of mathematics -differential geometry, integral geometry, and combinatorial optimization. The main technical problem is solved using Cauchy-Crofton formula from integral geometry.
---
paper_title: Parallel graph-cuts by adaptive bottom-up merging

paper_content:

Graph-cuts optimization is prevalent in vision and graphics problems. It is thus of great practical importance to parallelize the graph-cuts optimization using today's ubiquitous multi-core machines. However, the current best serial algorithm by Boykov and Kolmogorov (called the BK algorithm) still has the superior empirical performance. It is non-trivial to parallelize as expensive synchronization overhead easily offsets the advantage of parallelism. In this paper, we propose a novel adaptive bottom-up approach to parallelize the BK algorithm. We first uniformly partition the graph into a number of regularly-shaped disjoint subgraphs and process them in parallel, then we incrementally merge the subgraphs in an adaptive way to obtain the global optimum. The new algorithm has three benefits: 1) it is more cache-friendly within smaller subgraphs; 2) it keeps balanced workloads among computing cores; 3) it causes little overhead and is adaptable to the number of available cores. Extensive experiments in common applications such as 2D/3D image segmentations and 3D surface fitting demonstrate the effectiveness of our approach.
---
paper_title: Is a single energy functional sufficient? adaptive energy functionals and automatic initialization

paper_content:

Energy functional minimization is an increasingly popular technique for image segmentation. However, it is far too commonly applied with hand-tuned parameters and initializations that have only been validated for a few images. Fixing these parameters over a set of images assumes the same parameters are ideal for each image. We highlight the effects of varying the parameters and initialization on segmentation accuracy and propose a framework for attaining improved results using image adaptive parameters and initializations. We provide an analytical definition of optimal weights for functional terms through an examination of segmentation in the context of image manifolds, where nearby images on the manifold require similar parameters and similar initializations. Our results validate that fixed parameters are insufficient in addressing the variability in real clinical data, that similar images require similar parameters, and demonstrate how these parameters correlate with the image manifold. We present significantly improved segmentations for synthetic images and a set of 470 clinical examples.
---
paper_title: Fast approximate energy minimization via graph cuts

paper_content:

In this paper we address the problem of minimizing a large class of energy functions that occur in early vision. The major restriction is that the energy function's smoothness term must only involve pairs of pixels. We propose two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed. The first move we consider is an /spl alpha/-/spl beta/-swap: for a pair of labels /spl alpha/,/spl beta/, this move exchanges the labels between an arbitrary set of pixels labeled a and another arbitrary set labeled /spl beta/. Our first algorithm generates a labeling such that there is no swap move that decreases the energy. The second move we consider is an /spl alpha/-expansion: for a label a, this move assigns an arbitrary set of pixels the label /spl alpha/. Our second algorithm, which requires the smoothness term to be a metric, generates a labeling such that there is no expansion move that decreases the energy. Moreover, this solution is within a known factor of the global minimum. We experimentally demonstrate the effectiveness of our approach on image restoration, stereo and motion.
---
paper_title: A convex formulation of continuous multi-label problems

paper_content:

We propose a spatially continuous formulation of Ishikawa's discrete multi-label problem. We show that the resulting non-convex variational problem can be reformulated as a convex variational problem via embedding in a higher dimensional space. This variational problem can be interpreted as a minimal surface problem in an anisotropic Riemannian space. In several stereo experiments we show that the proposed continuous formulation is superior to its discrete counterpart in terms of computing time, memory efficiency and metrication errors.
---
paper_title: Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models

paper_content:

We show how certain nonconvex optimization problems that arise in image processing and computer vision can be restated as convex minimization problems. This allows, in particular, the finding of global minimizers via standard convex minimization schemes.
---
paper_title: Quasiconvex Optimization and Location Theory

paper_content:

1 Introduction.- 2 Elements of Convexity.- 2.1 Generalities.- 2.2 Convex sets.- 2.2.1 Hulls.- 2.2.2 Topological properties of convex sets.- 2.2.3 Separation of convex sets.- 2.3 Convex functions.- 2.3.1 Continuity of convex functions.- 2.3.2 Lower level sets and the subdifferential.- 2.3.3 Sublinear functions and directional derivatives.- 2.3.4 Support functions and gauges.- 2.3.5 Calculus rules with subdifferentials.- 2.4 Quasiconvex functions.- 2.5 Other directional derivatives.- 3 Convex Programming.- 3.1 Introduction.- 3.2 The ellipsoid method.- 3.2.1 The one dimensional case.- 3.2.2 The multidimensional case.- 3.2.3 Improving the numerical stability.- 3.2.4 Convergence proofs.- 3.2.5 Complexity.- 3.3 Stopping criteria.- 3.3.1 Satisfaction of the stopping rules.- 3.4 Computational experience.- 4 Convexity in Location.- 4.1 Introduction.- 4.2 Measuring convex distances.- 4.3 A general model.- 4.4 A convex location model.- 4.5 Characterizing optimality.- 4.6 Checking optimality in the planar case.- 4.6.1 Solving (D).- 4.6.2 Solving (D' ).- 4.6.3 Computational results.- 4.7 Computational results.- 5 Quasiconvex Programming.- 5.1 Introduction.- 5.2 A separation oracle for quasiconvex functions.- 5.2.1 Descent directions and geometry of lower level sets.- 5.2.2 Computing an element of the normal cone.- 5.3 Easy cases.- 5.3.1 Regular functions.- 5.3.2 Another class of easy functions.- 5.4 When we meet a "bad" point.- 5.5 Convergence proof.- 5.5.1 The unconstrained quasiconvex program.- 5.5.2 The constrained quasiconvex program.- 5.6 An ellipsoid algorithm for quasiconvex programming.- 5.6.1 Ellipsoids and boxes.- 5.6.2 Constructing a localization box.- 5.6.3 New cuts.- 5.6.4 Box cuts.- 5.6.5 Parallel cuts.- 5.6.6 Modified algorithm.- 5.7 Improving the stopping criteria.- 6 Quasiconvexity in Location.- 6.1 Introduction.- 6.2 A quasiconvex location model.- 6.3 Computational results.- 7 Conclusions.
---
paper_title: Exact optimization for markov random fields with convex priors

paper_content:

We introduce a method to solve exactly a first order Markov random field optimization problem in more generality than was previously possible. The MRF has a prior term that is convex in terms of a linearly ordered label set. The method maps the problem into a minimum-cut problem for a directed graph, for which a globally optimal solution can be found in polynomial time. The convexity of the prior function in the energy is shown to be necessary and sufficient for the applicability of the method.
---
paper_title: Multiregion competition: A level set extension of region competition to multiple region image partitioning

paper_content:

The purpose of this study is to investigate a new representation of a partition of an image domain into a fixed but arbitrary number of regions by explicit correspondence between the regions of segmentation and the regions defined by simple closed planar curves and their intersections, and the use of this representation in the context of region competition to provide a level set multiregion competition algorithm. This formulation leads to a system of coupled curve evolution equations which is easily amenable to a level set implementation and the computed solution is one that minimizes the stated functional. An unambiguous segmentation is garanteed because at all time during curve evolution the evolving regions form a partition of the image domain. We present the multiregion competition algorithm for intensity-based image segmentation and we subsequently extend it to motion/disparity. Finally, we consider an extension of the algorithm to account for images with aberrations such as occlusions. The formulation, the ensuing algorithm, and its implementation have been validated in several experiments on gray level, color, and motion segmentation.
---
paper_title: An Efficient Optimization Framework for Multi-Region Segmentation Based on Lagrangian Duality

paper_content:

We introduce a multi-region model for simultaneous segmentation of medical images. In contrast to many other models, geometric constraints such as inclusion and exclusion between the regions are enforced, which makes it possible to correctly segment different regions even if the intensity distributions are identical. We efficiently optimize the model using a combination of graph cuts and Lagrangian duality which is faster and more memory efficient than current state of the art. As the method is based on global optimization techniques, the resulting segmentations are independent of initialization. We apply our framework to the segmentation of the left and right ventricles, myocardium and the left ventricular papillary muscles in magnetic resonance imaging and to lung segmentation in full-body X-ray computed tomography. We evaluate our approach on a publicly available benchmark with competitive results.
---
paper_title: STACS: new active contour scheme for cardiac MR image segmentation

paper_content:

The paper presents a novel stochastic active contour scheme (STACS) for automatic image segmentation designed to overcome some of the unique challenges in cardiac MR images such as problems with low contrast, papillary muscles, and turbulent blood flow. STACS minimizes an energy functional that combines stochastic region-based and edge-based information with shape priors of the heart and local properties of the contour. The minimization algorithm solves, by the level set method, the Euler-Lagrange equation that describes the contour evolution. STACS includes an annealing schedule that balances dynamically the weight of the different terms in the energy functional. Three particularly attractive features of STACS are: 1) ability to segment images with low texture contrast by modeling stochastically the image textures; 2) robustness to initial contour and noise because of the utilization of both edge and region-based information; 3) ability to segment the heart from the chest wall and the undesired papillary muscles due to inclusion of heart shape priors. Application of STACS to a set of 48 real cardiac MR images shows that it can successfully segment the heart from its surroundings such as the chest wall and the heart structures (the left and right ventricles and the epicardium.) We compare STACS' automatically generated contours with manually-traced contours, or the "gold standard," using both area and edge similarity measures. This assessment demonstrates very good and consistent segmentation performance of STACS.
---
paper_title: Multi-Class Segmentation with Relative Location Prior

paper_content:

Multi-class image segmentation has made significant advances in recent years through the combination of local and global features. One important type of global feature is that of inter-class spatial relationships. For example, identifying "tree" pixels indicates that pixels above and to the sides are more likely to be "sky" whereas pixels below are more likely to be "grass." Incorporating such global information across the entire image and between all classes is a computational challenge as it is image-dependent, and hence, cannot be precomputed. ::: ::: In this work we propose a method for capturing global information from inter-class spatial relationships and encoding it as a local feature. We employ a two-stage classification process to label all image pixels. First, we generate predictions which are used to compute a local relative location feature from learned relative location maps. In the second stage, we combine this with appearance-based features to provide a final segmentation. We compare our results to recent published results on several multi-class image segmentation databases and show that the incorporation of relative location information allows us to significantly outperform the current state-of-the-art.
---
paper_title: Learning CRFs using Graph Cuts

paper_content:

Many computer vision problems are naturally formulated as random fields, specifically MRFs or CRFs. The introduction of graph cuts has enabled efficient and optimal inference in associative random fields, greatly advancing applications such as segmentation, stereo reconstruction and many others. However, while fast inference is now widespread, parameter learning in random fields has remained an intractable problem. This paper shows how to apply fast inference algorithms, in particular graph cuts, to learn parameters of random fields with similar efficiency. We find optimal parameter values under standard regularized objective functions that ensure good generalization. Our algorithm enables learning of many parameters in reasonable time, and we explore further speedup techniques. We also discuss extensions to non-associative and multi-class problems. We evaluate the method on image segmentation and geometry recognition.
---
paper_title: Medial-Based Deformable Models in Nonconvex Shape-Spaces for Medical Image Segmentation

paper_content:

We explore the application of genetic algorithms (GA) to deformable models through the proposition of a novel method for medical image segmentation that combines GA with nonconvex, localized, medial-based shape statistics. We replace the more typical gradient descent optimizer used in deformable models with GA, and the convex, implicit, global shape statistics with nonconvex, explicit, localized ones. Specifically, we propose GA to reduce typical deformable model weaknesses pertaining to model initialization, pose estimation and local minima, through the simultaneous evolution of a large number of models. Furthermore, we constrain the evolution, and thus reduce the size of the search-space, by using statistically-based deformable models whose deformations are intuitive (stretch, bulge, bend) and are driven in terms of localized principal modes of variation, instead of modes of variation across the entire shape that often fail to capture localized shape changes. Although GA are not guaranteed to achieve the global optima, our method compares favorably to the prevalent optimization techniques, convex/nonconvex gradient-based optimizers and to globally optimal graph-theoretic combinatorial optimization techniques, when applied to the task of corpus callosum segmentation in 50 mid-sagittal brain magnetic resonance images.
---
paper_title: Prior Shape Level Set Segmentation on Multistep Generated Probability Maps of MR Datasets for Fully Automatic Kidney Parenchyma Volumetry

paper_content:

Fully automatic 3-D segmentation techniques for clinical applications or epidemiological studies have proven to be a very challenging task in the domain of medical image analysis. 3-D organ segmentation on magnetic resonance (MR) datasets requires a well-designed segmentation strategy due to imaging artifacts, partial volume effects, and similar tissue properties of adjacent tissues. We developed a 3-D segmentation framework for fully automatic kidney parenchyma volumetry that uses Bayesian concepts for probability map generation. The probability map quality is improved in a multistep refinement approach. An extended prior shape level set segmentation method is then applied on the refined probability maps. The segmentation quality is improved by incorporating an exterior cortex edge alignment technique using cortex probability maps. In contrast to previous approaches, we combine several relevant kidney parenchyma features in a sequence of segmentation techniques for successful parenchyma delineation on native MR datasets. Furthermore, the proposed method is able to recognize and exclude parenchymal cysts from the parenchymal volume. We analyzed four different quality measures showing better results for right parenchymal tissue than for left parenchymal tissue due to an incorporated liver part removal in the segmentation framework. The results show that the outer cortex edge alignment approach successfully improves the quality measures.
---
paper_title: Graph cut with ordering constraints on labels and its applications

paper_content:

In the last decade, graph-cut optimization has been popular for a variety of pixel labeling problems. Typically graph-cut methods are used to incorporate a smoothness prior on a labeling. Recently several methods incorporated ordering constraints on labels for the application of object segmentation. An example of an ordering constraint is prohibiting a pixel with a ldquocar wheelrdquo label to be above a pixel with a ldquocar roofrdquo label. We observe that the commonly used graph-cut based alpha-expansion is more likely to get stuck in a local minimum when ordering constraints are used. For certain models with ordering constraints, we develop new graph-cut moves which we call order-preserving moves. Order-preserving moves act on all labels, unlike alpha-expansion. Although the global minimum is still not guaranteed, optimization with order-preserving moves performs significantly better than alpha-expansion. We evaluate order-preserving moves for the geometric class scene labeling (introduced by Hoiem et al.) where the goal is to assign each pixel a label such as ldquoskyrdquo, ldquogrounrdquo, etc., so ordering constraints arise naturally. In addition, we use order-preserving moves for certain simple shape priors in graphcut segmentation, which is a novel contribution in itself.
---
paper_title: Tiered scene labeling with dynamic programming

paper_content:

Dynamic programming (DP) has been a useful tool for a variety of computer vision problems. However its application is usually limited to problems with a one dimensional or low treewidth structure, whereas most domains in vision are at least 2D. In this paper we show how to apply DP for pixel labeling of 2D scenes with simple “tiered” structure. While there are many variations possible, for the applications we consider the following tiered structure is appropriate. An image is first divided by horizontal curves into the top, middle, and bottom regions, and the middle region is further subdivided vertically into subregions. Under these constraints a globally optimal labeling can be found using an efficient dynamic programming algorithm. We apply this algorithm to two very different tasks. The first is the problem of geometric class labeling where the goal is to assign each pixel a label such as “sky”, “ground”, and “surface above ground”. The second task involves incorporating simple shape priors for segmentation of an image into the “foreground” and “background” regions.
---
paper_title: An Adaptive Subdivision Scheme for Quadratic Programming in Multi-Label Image Segmentation.

paper_content:

Convex quadratic optimization is one of the most widely used concepts in image segmentation. It facilitates a wide range of information sources, such as edge, intensity, texture and shape. The problem is especially challenging for the multi-label case, even being NP-hard in its most general setting. Therefore, fast "solutions", as the α-expansion of [1], are limited to local optimality. Addressing this problem, several approaches relax the labeling integrality condition, resulting in quadratic programs (QPs) like in [2] and in [4], which can be solved in polynomial time. Although this is efficient in a theoretical sense, large-scale QPs that arise from typical multi-label tasks can rarely be used for image segmentation directly due to either time or space constraints, or both. We address this issue by an adaptive domain subdivision scheme, reducing the problem to a short sequence of spatially smoothed medium-scale QPs, which subsequently better approximate the large-scale program. Our scheme is globally optimal in terms of the approximated problem. Putting our main focus on the subdivision, we restrict ourselves to minimization of the popular but rather simple piecewise constant MumfordShah functional. Therefore, we seek for a labeling that trades off the length of the labeling border and the approximation of image intensity u by known reference intensitites ui for each label i. For discrete domains the associated energy can be written as
---
paper_title: A variational model for object segmentation using boundary information and shape prior driven by the Mumford-Shah functional

paper_content:

In this paper, we propose a variational model to segment an object belonging to a given scale space using the active contour method, a geometric shape prior and the Mumford-Shah functional. We define an energy functional composed by three complementary terms. The first one detects object boundaries from image gradients. The second term constrains the active contour to get a shape compatible with a statistical shape model of the shape of interest. And the third part drives globally the shape prior and the active contour towards a homogeneous intensity region. The segmentation of the object of interest is given by the minimum of our energy functional. This minimum is computed with the calculus of variations and the gradient descent method that provide a system of evolution equations solved with the well-known level set method. We also prove the existence of this minimum in the space of functions with bounded variation. Applications of the proposed model are presented on synthetic and medical images.
---
paper_title: Minimizing sparse higher order energy functions of discrete variables

paper_content:

Higher order energy functions have the ability to encode high level structural dependencies between pixels, which have been shown to be extremely powerful for image labeling problems. Their use, however, is severely hampered in practice by the intractable complexity of representing and minimizing such functions. We observed that higher order functions encountered in computer vision are very often “sparse”, i.e. many labelings of a higher order clique are equally unlikely and hence have the same high cost. In this paper, we address the problem of minimizing such sparse higher order energy functions. Our method works by transforming the problem into an equivalent quadratic function minimization problem. The resulting quadratic function can be minimized using popular message passing or graph cut based algorithms for MAP inference. Although this is primarily a theoretical paper, it also shows how higher order functions can be used to obtain impressive results for the binary texture restoration problem.
---
paper_title: On Parameter Learning in Crf-Based Approaches to Object Class Image Segmentation

paper_content:

Recent progress in per-pixel object class labeling of natural images can be attributed to the use of multiple types of image features and sound statistical learning approaches. Within the latter, Conditional Random Fields (CRF) are prominently used for their ability to represent interactions between random variables. Despite their popularity in computer vision, parameter learning for CRFs has remained difficult, popular approaches being cross-validation and piecewise training. ::: ::: In this work, we propose a simple yet expressive tree-structured CRF based on a recent hierarchical image segmentation method. Our model combines and weights multiple image features within a hierarchical representation and allows simple and efficient globally-optimal learning of ≅ 105 parameters. The tractability of our model allows us to pose and answer some of the open questions regarding parameter learning applying to CRF-based approaches. The key findings for learning CRF models are, from the obvious to the surprising, i) multiple image features always help, ii) the limiting dimension with respect to current models is the amount of training data, iii) piecewise training is competitive, iv) current methods for max-margin training fail for models with many parameters.
---
paper_title: Efficient piecewise learning for conditional random fields

paper_content:

Conditional Random Field models have proved effective for several low-level computer vision problems. Inference in these models involves solving a combinatorial optimization problem, with methods such as graph cuts, belief propagation. Although several methods have been proposed to learn the model parameters from training data, they suffer from various drawbacks. Learning these parameters involves computing the partition function, which is intractable. To overcome this, state-of-the-art structured learning methods frame the problem as one of large margin estimation. Iterative solutions have been proposed to solve the resulting convex optimization problem. Each iteration involves solving an inference problem over all the labels, which limits the efficiency of these structured methods. In this paper we present an efficient large margin piece-wise learning method which is widely applicable. We show how the resulting optimization problem can be reduced to an equivalent convex problem with a small number of constraints, and solve it using an efficient scheme. Our method is both memory and computationally efficient. We show results on publicly available standard datasets.
---
paper_title: Area prior constrained level set evolution for medical image segmentation

paper_content:

The level set framework has proven well suited to medical image segmentation 1-6 thanks to its ability of balancing ::: the contribution of image data and prior knowledge in a principled, flexible and transparent way. It consists ::: of evolving a curve toward the target object boundaries. The curve evolution equation is sought following ::: the optimization of a cost functional containing two types of terms: data terms, which measure the fidelity of ::: segmentation to image intensities, and prior terms, which traduce learned prior knowledge. Without priors many ::: algorithms are likely to fail due to high noise, low contrast and data incompleteness. Different priors have been ::: investigated such as shape 1 and appearance priors. 7 In this study, we propose a simple type of priors: the area ::: prior. This prior embeds knowledge of an approximate object area and has two positive effects. First, It speeds ::: up significantly the evolution when the curve is far from the target object boundaries. Second, it slows down ::: the evolution when the curve is close to the target. Consequently, it reinforces curve stability at the desired ::: boundaries when dealing with low contrast intensity edges. The algorithm is validated with several experiments ::: using Magnetic Resonance (MR) images and Computed Tomography (CT) images. A comparison with another ::: level set method illustrates the positive effects of the area prior.
---
paper_title: A Statistical Overlap Prior for Variational Image Segmentation

paper_content:

This study investigates variational image segmentation with an original data term, referred to as statistical overlap prior, which measures the conformity of overlap between the nonparametric distributions of image data within the segmentation regions to a learned statistical description. This leads to image segmentation and distribution tracking algorithms that relax the assumption of minimal overlap and, as such, are more widely applicable than existing algorithms. We propose to minimize active curve functionals containing the proposed overlap prior, compute the corresponding Euler-Lagrange curve evolution equations, and give an interpretation of how the overlap prior controls such evolution. We model the overlap, measured via the Bhattacharyya coefficient, with a Gaussian prior whose parameters are estimated from a set of relevant training images. Quantitative and comparative performance evaluations of the proposed algorithms over several experiments demonstrate the positive effects of the overlap prior in regard to segmentation accuracy and convergence speed.
---
paper_title: Global Minimization for Continuous Multiphase Partitioning Problems Using a Dual Approach

paper_content:

This paper is devoted to the optimization problem of continuous multi-partitioning, or multi-labeling, which is based on a convex relaxation of the continuous Potts model. In contrast to previous efforts, which are tackling the optimal labeling problem in a direct manner, we first propose a novel dual model and then build up a corresponding duality-based approach. By analyzing the dual formulation, sufficient conditions are derived which show that the relaxation is often exact, i.e. there exists optimal solutions that are also globally optimal to the original nonconvex Potts model. In order to deal with the nonsmooth dual problem, we develop a smoothing method based on the log-sum exponential function and indicate that such a smoothing approach leads to a novel smoothed primal-dual model and suggests labelings with maximum entropy. Such a smoothing method for the dual model also yields a new thresholding scheme to obtain approximate solutions. An expectation maximization like algorithm is proposed based on the smoothed formulation which is shown to be superior in efficiency compared to earlier approaches from continuous optimization. Numerical experiments also show that our method outperforms several competitive approaches in various aspects, such as lower energies and better visual quality.
---
paper_title: Active Volume Models for Medical Image Segmentation

paper_content:

In this paper, we propose a novel predictive model, active volume model (AVM), for object boundary extraction. It is a dynamic “object” model whose manifestation includes a deformable curve or surface representing a shape, a volumetric interior carrying appearance statistics, and an embedded classifier that separates object from background based on current feature information. The model focuses on an accurate representation of the foreground object's attributes, and does not explicitly represent the background. As we will show, however, the model is capable of reasoning about the background statistics thus can detect when is change sufficient to invoke a boundary decision. When applied to object segmentation, the model alternates between two basic operations: 1) deforming according to current region of interest (ROI), which is a binary mask representing the object region predicted by the current model, and 2) predicting ROI according to current appearance statistics of the model. To further improve robustness and accuracy when segmenting multiple objects or an object with multiple parts, we also propose multiple-surface active volume model (MSAVM), which consists of several single-surface AVM models subject to high-level geometric spatial constraints. An AVM's deformation is derived from a linear system based on finite element method (FEM). To keep the model's surface triangulation optimized, surface remeshing is derived from another linear system based on Laplacian mesh optimization (LMO) , . Thus efficient optimization and fast convergence of the model are achieved by solving two linear systems. Segmentation, validation and comparison results are presented from experiments on a variety of 2-D and 3-D medical images.
---
paper_title: Is a single energy functional sufficient? adaptive energy functionals and automatic initialization

paper_content:

Energy functional minimization is an increasingly popular technique for image segmentation. However, it is far too commonly applied with hand-tuned parameters and initializations that have only been validated for a few images. Fixing these parameters over a set of images assumes the same parameters are ideal for each image. We highlight the effects of varying the parameters and initialization on segmentation accuracy and propose a framework for attaining improved results using image adaptive parameters and initializations. We provide an analytical definition of optimal weights for functional terms through an examination of segmentation in the context of image manifolds, where nearby images on the manifold require similar parameters and similar initializations. Our results validate that fixed parameters are insufficient in addressing the variability in real clinical data, that similar images require similar parameters, and demonstrate how these parameters correlate with the image manifold. We present significantly improved segmentations for synthetic images and a set of 470 clinical examples.
---
paper_title: Convex multi-region probabilistic segmentation with shape prior in the isometric log-ratio transformation space

paper_content:

Image segmentation is often performed via the minimization of an energy function over a domain of possible segmentations. The effectiveness and applicability of such methods depends greatly on the properties of the energy function and its domain, and on what information can be encoded by it. Here we propose an energy function that achieves several important goals. Specifically, our energy function is convex and incorporates shape prior information while simultaneously generating a probabilistic segmentation for multiple regions. Our energy function represents multi-region probabilistic segmentations as elements of a vector space using the isometric log-ratio (ILR) transformation. To our knowledge, these four goals (convex, with shape priors, multi-region, and probabilistic) do not exist together in any other method, and this is the first time ILR is used in an image segmentation method. We provide examples demonstrating the usefulness of these features.
---
paper_title: Fast approximate energy minimization via graph cuts

paper_content:

In this paper we address the problem of minimizing a large class of energy functions that occur in early vision. The major restriction is that the energy function's smoothness term must only involve pairs of pixels. We propose two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed. The first move we consider is an /spl alpha/-/spl beta/-swap: for a pair of labels /spl alpha/,/spl beta/, this move exchanges the labels between an arbitrary set of pixels labeled a and another arbitrary set labeled /spl beta/. Our first algorithm generates a labeling such that there is no swap move that decreases the energy. The second move we consider is an /spl alpha/-expansion: for a label a, this move assigns an arbitrary set of pixels the label /spl alpha/. Our second algorithm, which requires the smoothness term to be a metric, generates a labeling such that there is no expansion move that decreases the energy. Moreover, this solution is within a known factor of the global minimum. We experimentally demonstrate the effectiveness of our approach on image restoration, stereo and motion.
---
paper_title: Adaptive segmentation of MRI data

paper_content:

Intensity-based classification of MR images has proven problematic, even when advanced techniques are used. Intrascan and interscan intensity inhomogeneities are a common source of difficulty. While reported methods have had some success in correcting intrascan inhomogeneities, such methods require supervision for the individual scan. This paper describes a new method called adaptive segmentation that uses knowledge of tissue intensity properties and intensity inhomogeneities to correct and segment MR images. Use of the expectation-maximization (EM) algorithm leads to a method that allows for more accurate segmentation of tissue types as well as better visualization of magnetic resonance imaging (MRI) data, that has proven to be effective in a study that includes more than 1000 brain scans. Implementation and results are described for segmenting the brain in the following types of images: axial (dual-echo spin-echo), coronal [three dimensional Fourier transform (3-DFT) gradient-echo T1-weighted] all using a conventional head coil, and a sagittal section acquired using a surface coil. The accuracy of adaptive segmentation was found to be comparable with manual segmentation, and closer to manual segmentation than supervised multivariant classification while segmenting gray and white matter.
---
paper_title: Using the logarithm of odds to define a vector space on probabilistic atlases

paper_content:

The logarithm of the odds ratio (LogOdds) is frequently used in areas such as artificial neural networks, economics, and biology, as an alternative representation of probabilities. Here, we use LogOdds to place probabilistic atlases in a linear vector space. This representation has several useful properties for medical imaging. For example, it not only encodes the shape of multiple anatomical structures but also captures some information concerning uncertainty. We demonstrate that the resulting vector space operations of addition and scalar multiplication have natural probabilistic interpretations. We discuss several examples for placing label maps into the space of LogOdds. First, we relate signed distance maps, a widely used implicit shape representation, to LogOdds and compare it to an alternative that is based on smoothing by spatial Gaussians. We find that the LogOdds approach better preserves shapes in a complex multiple object setting. In the second example, we capture the uncertainty of boundary locations by mapping multiple label maps of the same object into the LogOdds space. Third, we define a framework for nonconvex interpolations among atlases that capture different time points in the aging process of a population. We evaluate the accuracy of our representation by generating a deformable shape atlas that captures the variations of anatomical shapes across a population. The deformable atlas is the result of a principal component analysis within the LogOdds space. This atlas is integrated into an existing segmentation approach for MR images. We compare the performance of the resulting implementation in segmenting 20 test cases to a similar approach that uses a more standard shape model that is based on signed distance maps. On this data set, the Bayesian classification model with our new representation outperformed the other approaches in segmenting subcortical structures.
---
paper_title: Uncertainty and Patient Heterogeneity in Medical Decision Models

paper_content:

Parameter uncertainty, patient heterogeneity, and stochastic uncertainty of outcomes are increasingly important concepts in medical decision models. The purpose of this study is to demonstrate the various methods to analyze uncertainty and patient heterogeneity in a decision model. The authors distinguish various purposes of medical decision modeling, serving various stakeholders. Differences and analogies between the analyses are pointed out, as well as practical issues. The analyses are demonstrated with an example comparing imaging tests for patients with chest pain. For complicated analyses step-by-step algorithms are provided. The focus is on Monte Carlo simulation and value of information analysis. Increasing model complexity is a major challenge for probabilistic sensitivity analysis and value of information analysis. The authors discuss nested analyses that are required in patient-level models, and in nonlinear models for analyses of partial value of information analysis.
---
paper_title: ProbExplorer: Uncertaintyguided exploration and editing of probabilistic medical image segmentation

paper_content:

In this paper, we develop an interactive analysis and visualization tool for probabilistic segmentation results in medical imaging. We provide a systematic approach to analyze, interact and highlight regions of segmentation uncertainty. We introduce a set of visual analysis widgets integrating different approaches to analyze multivariate probabilistic field data with direct volume rendering. We demonstrate the user's ability to identify suspicious regions (e.g. tumors) and correct the misclassification results using a novel uncertainty-based segmentation editing technique. We evaluate our system and demonstrate its usefulness in the context of static and time-varying medical imaging datasets.
---
paper_title: Fuzzy-Snake Segmentation of Anatomical Structures Applied to CT Images

paper_content:

This paper presents a generic strategy to facilitate the segmentation of anatomical structures in medical images. The segmentation is performed using an adapted PDM by fuzzy c-means classification, which also uses the fuzzy decision to evolve PDM into the final contour. Furthermore, the fuzzy reasoning exploits \(\it{a}\) \(\it{priori}\) statistical information from several knowledge sources based on histogram analysis and the intensity values of the structures under consideration. The fuzzy reasoning is also applied and compared to a geometrical active contour model (or level set). The method has been developed to assist clinicians and radiologists in conformal RTP. Experimental results and their quantitative validation to assess the accuracy and efficiency are given segmenting the bladder on CT images. To assess precision, results are also presented in CT images with added Gaussian noise. The fuzzy-snake is free of parameter and it is able to properly segment the structures by using the same initial spline curve for a whole study image-patient set.
---
paper_title: Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm

paper_content:

The finite mixture (FM) model is the most commonly used model for statistical segmentation of brain magnetic resonance (MR) images because of its simple mathematical form and the piecewise constant nature of ideal brain MR images. However, being a histogram-based model, the FM has an intrinsic limitation--no spatial information is taken into account. This causes the FM model to work only on well-defined images with low levels of noise; unfortunately, this is often not the the case due to artifacts such as partial volume effect and bias field distortion. Under these conditions, FM model-based methods produce unreliable results. In this paper, we propose a novel hidden Markov random field (HMRF) model, which is a stochastic process generated by a MRF whose state sequence cannot be observed directly but which can be indirectly estimated through observations. Mathematically, it can be shown that the FM model is a degenerate version of the HMRF model. The advantage of the HMRF model derives from the way in which the spatial information is encoded through the mutual influences of neighboring sites. Although MRF modeling has been employed in MR image segmentation by other researchers, most reported methods are limited to using MRF as a general prior in an FM model-based approach. To fit the HMRF model, an EM algorithm is used. We show that by incorporating both the HMRF model and the EM algorithm into a HMRF-EM framework, an accurate and robust segmentation can be achieved. More importantly, the HMRF-EM framework can easily be combined with other techniques. As an example, we show how the bias field correction algorithm of Guillemaud and Brady (1997) can be incorporated into this framework to achieve a three-dimensional fully automated approach for brain MR image segmentation.
---
paper_title: Probabilistic multi-shape representation using an isometric log-ratio mapping

paper_content:

Several sources of uncertainties in shape boundaries in medical images have motivated the use of probabilistic labeling approaches. Although it is well-known that the sample space for the probabilistic representation of a pixel is the unit simplex, standard techniques of statistical shape analysis (e.g. principal component analysis) have been applied to probabilistic data as if they lie in the unconstrained real Euclidean space. Since these techniques are not constrained to the geometry of the simplex, the statistically feasible data produced end up representing invalid (out of the simplex) shapes. By making use of methods for dealing with what is known as compositional or closed data, we propose a new framework intrinsic to the unit simplex for statistical analysis of probabilistic multi-shape anatomy. In this framework, the isometric logratio (ILR) transformation is used to isometrically and bijectively map the simplex to the Euclidean real space, where data are analyzed in the same way as unconstrained data and then back-transformed to the simplex. We demonstrate favorable properties of ILR over existing mappings (e.g. LogOdds). Our results on synthetic and brain data exhibit a more accurate statistical analysis of probabilistic shapes.
---
paper_title: Go digital, go fuzzy

paper_content:

Abstract In many application areas of imaging sciences, object information captured in multidimensional images needs to be extracted, visualized, manipulated, and analyzed. These four groups of operations have been (and are being) intensively investigated, developed, and applied in a variety of applications. In this paper, after giving a brief overview of the four groups of operations, we put forth two main arguments: (1) Computers are digital, and most image acquisition and communication efforts at present are toward digital approaches. In the same vein, there are considerable advantages to taking an inherently digital approach to the above four groups of operations rather than using concepts based on continuous approximations. (2) Considering the fact that images are inherently fuzzy, to handle uncertainties and heterogeneity of object properties realistically, approaches based on fuzzy sets should be taken to the above four groups of operations. We give two examples in support of these arguments.
---
paper_title: Kinetic Modeling Based Probabilistic Segmentation for Molecular Images

paper_content:

We propose a semi-supervised, kinetic modeling based segmentation technique for molecular imaging applications. It is an iterative, self-learning algorithm based on uncertainty principles, designed to alleviate low signal-to-noise ratio (SNR) and partial volume effect (PVE) problems. Synthetic fluorodeoxyglucose (FDG) and simulated Raclopride dynamic positron emission tomography (dPET) brain images with excessive noise levels are used to validate our algorithm. We show, qualitatively and quantitatively, that our algorithm outperforms state-of-the-art techniques in identifying different functional regions and recovering the kinetic parameters.
---
paper_title: The Isometric Log-Ratio Transform for Probabilistic Multi-Label Anatomical Shape Representation

paper_content:

Sources of uncertainty in the boundaries of structures in medical images have motivated the use of probabilistic labels in segmentation applications. An important component in many medical image segmentation tasks is the use of a shape model, often generated by applying statistical techniques to training data. Standard statistical techniques (e.g., principal component analysis) often assume data lies in an unconstrained vector space, but probabilistic labels are constrained to the unit simplex. If these statistical techniques are used directly on probabilistic labels, relative uncertainty information can be sacrificed. A standard method for facilitating analysis of probabilistic labels is to map them to a vector space using the LogOdds transform. However, the LogOdds transform is asymmetric in one of the labels, which skews results in some applications. The isometric log-ratio (ILR) transform is a symmetrized version of the LogOdds transform, and is so named as it is an isometry between the Aitchison geometry, the inherent geometry of the simplex, and standard Euclidean geometry. We explore how to interpret the Aitchison geometry when applied to probabilistic labels in medical image segmentation applications. We demonstrate the differences when applying the LogOdds transform or the ILR transform to probabilistic labels prior to statistical analysis. Specifically, we show that statistical analysis of ILR transformed data better captures the variability of anatomical shapes in cases where multiple different foreground regions share boundaries (as opposed to foreground-background boundaries).
---
paper_title: The Generalized Log-Ratio Transformation: Learning Shape and Adjacency Priors for Simultaneous Thigh Muscle Segmentation

paper_content:

We present a novel probabilistic shape representation that implicitly includes prior anatomical volume and adjacency information, termed the generalized log-ratio (GLR) representation. We demonstrate the usefulness of this representation in the task of thigh muscle segmentation. Analysis of the shapes and sizes of thigh muscles can lead to a better understanding of the effects of chronic obstructive pulmonary disease (COPD), which often results in skeletal muscle weakness in lower limbs. However, segmenting these muscles from one another is difficult due to a lack of distinctive features and inter-muscular boundaries that are difficult to detect. We overcome these difficulties by building a shape model in the space of GLR representations. We remove pose variability from the model by employing a presegmentation-based alignment scheme. We also design a rotationally invariant random forest boundary detector that learns common appearances of the interface between muscles from training data. We combine the shape model and the boundary detector into a fully automatic globally optimal segmentation technique. Our segmentation technique produces a probabilistic segmentation that can be used to generate uncertainty information, which can be used to aid subsequent analysis. Our experiments on challenging 3D magnetic resonance imaging data sets show that the use of the GLR representation improves the segmentation accuracy, and yields an average Dice similarity coefficient of 0.808 ±0.074, comparable to other state-of-the-art thigh segmentation techniques.
---
paper_title: Random Walks for Image Segmentation

paper_content:

A novel method is proposed for performing multilabel, interactive image segmentation. Given a small number of pixels with user-defined (or predefined) labels, one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels. By assigning each pixel to the label for which the greatest probability is calculated, a high-quality image segmentation may be obtained. Theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits. This algorithm is formulated in discrete space (i.e., on a graph) using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension on arbitrary graphs
---
paper_title: Hands-free interactive image segmentation using eyegaze

paper_content:

This paper explores a novel approach to interactive user-guided image segmentation, using eyegaze information as an input. The method includes three steps: 1) eyegaze tracking for providing user input, such as setting object and background seed pixel selection; 2) an optimization method for image labeling that is constrained or affected by user input; and 3) linking the two previous steps via a graphical user interface for displaying the images and other controls to the user and for providing real-time visual feedback of eyegaze and seed locations, thus enabling the interactive segmentation procedure. We developed a new graphical user interface supported by an eyegaze tracking monitor to capture the user’s eyegaze movement and fixations (as opposed to traditional mouse moving and clicking). The user simply looks at different parts of the screen to select which image to segment, to perform foreground and background seed placement and to set optional segmentation parameters. There is an eyegaze-controlled “zoom” feature for difficult images containing objects with narrow parts, holes or weak boundaries. The image is then segmented using the random walker image segmentation method. We performed a pilot study with 7 subjects who segmented synthetic, natural and real medical images. Our results show that getting used the new interface takes about only 5 minutes. Compared with traditional mouse-based control, the new eyegaze approach provided a 18.6% speed improvement for more than 90% of images with high object-background contrast. However, for low contrast and more difficult images it took longer to place seeds using the eyegaze-based “zoom” to relax the required eyegaze accuracy of seed placement.
---
paper_title: Interactive Live-Wire Boundary Extraction

paper_content:

Abstract Live-wire segmentation is a new interactive tool for efficient, accurate and reproducible boundary extraction which requires minimal user input with a mouse. Optimal boundaries are computed and selected at interactive rates as the user moves the mouse starting from a manually specified seed point. When the mouse position comes into the proximity of an object edge, a ‘live-wire’ boundary snaps to, and wraps around the object of interest. The input of a new seed point ‘freezes’ the selected boundary segment and the process is repeated until the boundary is complete. Two novel enhancements to the basic live-wire methodology include boundary cooling and on-the-fly training. Data-driven boundary cooling generates seed points automatically and further reduces user input. On-the-fly training adapts the dynamic boundary to edges of current interest. Using the live-wire technique, boundaries are extracted in one-fifth of the time required for manual tracing, but with 4.4 times greater accuracy and 4.8 times greater reproducibility. In particular, interobserver reproducibility using the live-wire tool is 3.8 times greater than intraobserver reproducibility using manual tracing.
---
paper_title: Fast Geodesic Active Contours

paper_content:

We use an unconditionally stable numerical scheme to implement a fast version of the geodesic active contour model. The proposed scheme is useful for object segmentation in images, like tracking moving objects in a sequence of images. The method is based on the Weickert-Romeney-Viergever (additive operator splitting) AOS scheme. It is applied at small regions, motivated by the Adalsteinsson-Sethian level set narrow band approach, and uses Sethian's (1996) fast marching method for re-initialization. Experimental results demonstrate the power of the new method for tracking in color movies.
---
paper_title: Exploration and Visualization of Segmentation Uncertainty using Shape and Appearance Prior Information

paper_content:

We develop an interactive analysis and visualization tool for probabilistic segmentation in medical imaging. The originality of our approach is that the data exploration is guided by shape and appearance knowledge learned from expert-segmented images of a training population. We introduce a set of multidimensional transfer function widgets to analyze the multivariate probabilistic field data. These widgets furnish the user with contextual information about conformance or deviation from the population statistics. We demonstrate the user's ability to identify suspicious regions (e.g. tumors) and to correct the misclassification results. We evaluate our system and demonstrate its usefulness in the context of static anatomical and time-varying functional imaging datasets.
---
paper_title: Segmentation from a box

paper_content:

Drawing a box around an intended segmentation target has become both a popular user interface and a common output for learning-driven detection algorithms. Despite the ubiquity of using a box to define a segmentation target, it is unclear in the literature whether a box is sufficient to define a unique segmentation or whether segmentation from a box is ill-posed without higher-level (semantic) knowledge of the intended target. We examine this issue by conducting a study of 14 subjects who are asked to segment a boxed target in a set of 50 real images for which they have no semantic attachment. We find that the subjects do indeed perceive and trace almost the same segmentations as each other, despite the inhomogeneity of the image intensities, irregular shapes of the segmentation targets and weakness of the target boundaries. Since the subjects produce the same segmentation, we conclude that the problem is well-posed and then provide a new segmentation algorithm from a box which achieves results close to the perceived target.
---
paper_title: "GrabCut": interactive foreground extraction using iterated graph cuts

paper_content:

The problem of efficient, interactive foreground/background segmentation in still images is of great practical importance in image editing. Classical image segmentation tools use either texture (colour) information, e.g. Magic Wand, or edge (contrast) information, e.g. Intelligent Scissors. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power of the iterative algorithm is used to simplify substantially the user interaction needed for a given quality of result. Thirdly, a robust algorithm for "border matting" has been developed to estimate simultaneously the alpha-matte around an object boundary and the colours of foreground pixels. We show that for moderately difficult examples the proposed method outperforms competitive tools.
---
paper_title: 3D live-wire-based semi-automatic segmentation of medical images

paper_content:

Segmenting anatomical structures from medical images is usually one of the most important initial steps in many applications, including visualization, computer-aided diagnosis, and morphometric analysis. Manual 2D segmentation suffers from operator variability and is tedious and time-consuming. These disadvantages are accentuated in 3D applications and, the additional requirement of producing intuitive displays to integrate 3D information for the user, makes manual segmentation even less approachable in 3D. Robust, automatic medical image segmentation in 2D to 3D remains an open problem caused particularly by sensitivity to low-level parameters of segmentation algorithms. Semi-automatic techniques present possible balanced solution where automation focuses on low-level computing-intensive tasks that can be hidden from the user, while manual inter- ::: vention captures high-level expert knowledge nontrivial to capture algorithmically. In this paper we present a 3D extension to the 2D semi-automatic live-wire technique. Live-wire based contours generated semi-automatically on a selected set of slices are used as seed points on new unseen slices in different orientations. The seed points are calculated from intersections of user-based live-wire techniques with new slices. Our algorithm includes a step for ordering the live-wire seed points in the new slices, which is essential for subsequent multi-stage optimal path calculation. We present results of automatically detecting contours in new slices in 3D volumes from a variety of medical images.
---
paper_title: Soft scissors: an interactive tool for realtime high quality matting

paper_content:

We present Soft Scissors, an interactive tool for extracting alpha mattes of foreground objects in realtime. We recently proposed a novel offline matting algorithm capable of extracting high-quality mattes for complex foreground objects such as furry animals [Wang and Cohen 2007]. In this paper we both improve the quality of our offline algorithm and give it the ability to incrementally update the matte in an online interactive setting. Our realtime system efficiently estimates foreground color thereby allowing both the matte and the final composite to be revealed instantly as the user roughly paints along the edge of the foreground object. In addition, our system can dynamically adjust the width and boundary conditions of the scissoring paint brush to approximately capture the boundary of the foreground object that lies ahead on the scissor's path. These advantages in both speed and accuracy create the first interactive tool for high quality image matting and compositing.
---
paper_title: User-aided Boundary Delineation through the Propagation of Implicit Representations

paper_content:

In this paper we introduce user-defined segmentation constraints within the level set methods. Snake-driven methods are powerful and widely explored techniques for object extraction. Level set representations is a mathematical framework technique to implement such methods. This formulation is implicit, intrinsic and parameter/topology free. Introducing shape-driven knowledge within the level set method for segmentation is a recently explored topic. User interactive constraints are more simplistic forms of prior shape knowledge. To this end, we propose a simple formulation that converts user interaction to objective function terms that aim to guide the segmentation solution through the user edits.
---
paper_title: Interactive level set segmentation for image-guided therapy

paper_content:

Image-guided therapy procedures require the patient to remain still throughout the image acquisition, data analysis and therapy. This imposes a tight time constraint on the over-all process. Automatic extraction of the pathological regions prior to the therapy can be faster than the customary manual segmentation performed by the physician. However, the image data alone is usually not sufficient for reliable and unambiguous computerized segmentation. Thus, the oversight of an experienced physician remains mandatory. We present a novel segmentation framework, that allows user feedback. A few mouse-clicks of the user, discrete in nature, are represented as a continuous energy term that is incorporated into a level-set functional. We demonstrate the proposed method on MR scans of uterine fibroids acquired prior to focused ultrasound ablation treatment. The experiments show that with a minimal user input, automatic segmentation results become practically identical to manual expert segmentation.
---
paper_title: Image segmentation with a bounding box prior

paper_content:

User-provided object bounding box is a simple and popular interaction paradigm considered by many existing interactive image segmentation frameworks. However, these frameworks tend to exploit the provided bounding box merely to exclude its exterior from consideration and sometimes to initialize the energy minimization. In this paper, we discuss how the bounding box can be further used to impose a powerful topological prior, which prevents the solution from excessive shrinking and ensures that the user-provided box bounds the segmentation in a sufficiently tight way. The prior is expressed using hard constraints incorporated into the global energy minimization framework leading to an NP-hard integer program. We then investigate the possible optimization strategies including linear relaxation as well as a new graph cut algorithm called pinpointing. The latter can be used either as a rounding method for the fractional LP solution, which is provably better than thresholding-based rounding, or as a fast standalone heuristic. We evaluate the proposed algorithms on a publicly available dataset, and demonstrate the practical benefits of the new prior both qualitatively and quantitatively.
---
paper_title: A probabilistic level set formulation for interactive organ segmentation

paper_content:

Level set methods have become increasingly popular as a framework for image segmentation. Yet when used as a generic segmentation tool, they suffer from an important drawback: Current formulations do not allow much user interaction. Upon initialization, boundaries propagate to the final segmentation without the user being able to guide or correct the segmentation. In the present work, we address this limitation by proposing a probabilistic framework for image segmentation which integrates input intensity information and user interaction on equal footings. The resulting algorithm determines the most likely segmentation given the input image and the user input. In order to allow a user interaction in real-time during the segmentation, the algorithm is implemented on a graphics card and in a narrow band formulation.
---
paper_title: Geodesic active contours

paper_content:

A novel scheme for the detection of object boundaries is presented. The technique is based on active contours deforming according to intrinsic geometric measures of the image. The evolving contours naturally split and merge, allowing the simultaneous detection of several objects and both interior and exterior boundaries. The proposed approach is based on the relation between active contours and the computation of geodesics or minimal distance curves. The minimal distance curve lays in a Riemannian space whose metric as defined by the image content. This geodesic approach for object segmentation allows to connect classical "snakes" based on energy minimization and geometric active contours based on the theory of curve evolution. Previous models of geometric active contours are improved as showed by a number of examples. Formal results concerning existence, uniqueness, stability, and correctness of the evolution are presented as well.<<ETX>>
---
paper_title: Snakes: Active contour models

paper_content:

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.
---
paper_title: Ultrasound image segmentation: a survey

paper_content:

This paper reviews ultrasound segmentation methods, in a broad sense, focusing on techniques developed for medical B-mode ultrasound images. First, we present a review of articles by clinical application to highlight the approaches that have been investigated and degree of validation that has been done in different clinical domains. Then, we present a classification of methodology in terms of use of prior information. We conclude by selecting ten papers which have presented original ideas that have demonstrated particular clinical usefulness or potential specific to the ultrasound segmentation problem
---
paper_title: Deformable-model based textured object segmentation

paper_content:

In this paper, we present a deformable-model based solution for segmenting objects with complex texture patterns of all scales. The external image forces in traditional deformable models come primarily from edges or gradient information and it becomes problematic when the object surfaces have complex large-scale texture patterns that generate many local edges within a same region. We introduce a new textured object segmentation algorithm that has both the robustness of model-based approaches and the ability to deal with non-uniform textures of both small and large scales. The main contributions include an information-theoretical approach for computing the natural scale of a “texon” based on model-interior texture, a nonparametric texture statistics comparison technique and the determination of object belongingness through belief propagation. Another important property of the proposed algorithm is in that the texture statistics of an object of interest are learned online from evolving model interiors, requiring no other a priori information. We demonstrate the potential of this model-based framework for texture learning and segmentation using both natural and medical images with various textures of all scales and patterns.
---
paper_title: STACS: new active contour scheme for cardiac MR image segmentation

paper_content:

The paper presents a novel stochastic active contour scheme (STACS) for automatic image segmentation designed to overcome some of the unique challenges in cardiac MR images such as problems with low contrast, papillary muscles, and turbulent blood flow. STACS minimizes an energy functional that combines stochastic region-based and edge-based information with shape priors of the heart and local properties of the contour. The minimization algorithm solves, by the level set method, the Euler-Lagrange equation that describes the contour evolution. STACS includes an annealing schedule that balances dynamically the weight of the different terms in the energy functional. Three particularly attractive features of STACS are: 1) ability to segment images with low texture contrast by modeling stochastically the image textures; 2) robustness to initial contour and noise because of the utilization of both edge and region-based information; 3) ability to segment the heart from the chest wall and the undesired papillary muscles due to inclusion of heart shape priors. Application of STACS to a set of 48 real cardiac MR images shows that it can successfully segment the heart from its surroundings such as the chest wall and the heart structures (the left and right ventricles and the epicardium.) We compare STACS' automatically generated contours with manually-traced contours, or the "gold standard," using both area and edge similarity measures. This assessment demonstrates very good and consistent segmentation performance of STACS.
---
paper_title: Image Segmentation Based on GrabCut Framework Integrating Multiscale Nonlinear Structure Tensor

paper_content:

In this paper, we propose an interactive color natural image segmentation method. The method integrates color feature with multiscale nonlinear structure tensor texture (MSNST) feature and then uses GrabCut method to obtain the segmentations. The MSNST feature is used to describe the texture feature of an image and integrated into GrabCut framework to overcome the problem of the scale difference of textured images. In addition, we extend the Gaussian Mixture Model (GMM) to MSNST feature and GMM based on MSNST is constructed to describe the energy function so that the texture feature can be suitably integrated into GrabCut framework and fused with the color feature to achieve the more superior image segmentation performance than the original GrabCut method. For easier implementation and more efficient computation, the symmetric KL divergence is chosen to produce the estimates of the tensor statistics instead of the Riemannian structure of the space of tensor. The Conjugate norm was employed using Locality Preserving Projections (LPP) technique as the distance measure in the color space for more discriminating power. An adaptive fusing strategy is presented to effectively adjust the mixing factor so that the color and MSNST texture features are efficiently integrated to achieve more robust segmentation performance. Last, an iteration convergence criterion is proposed to reduce the time of the iteration of GrabCut algorithm dramatically with satisfied segmentation accuracy. Experiments using synthesis texture images and real natural scene images demonstrate the superior performance of our proposed method.
---
paper_title: Description of Interest Regions with Local Binary Patterns

paper_content:

This paper presents a novel method for interest region description. We adopted the idea that the appearance of an interest region can be well characterized by the distribution of its local features. The most well-known descriptor built on this idea is the SIFT descriptor that uses gradient as the local feature. Thus far, existing texture features are not widely utilized in the context of region description. In this paper, we introduce a new texture feature called center-symmetric local binary pattern (CS-LBP) that is a modified version of the well-known local binary pattern (LBP) feature. To combine the strengths of the SIFT and LBP, we use the CS-LBP as the local feature in the SIFT algorithm. The resulting descriptor is called the CS-LBP descriptor. In the matching and object category classification experiments, our descriptor performs favorably compared to the SIFT. Furthermore, the CS-LBP descriptor is computationally simpler than the SIFT.
---
paper_title: Multidimensional orientation estimation with applications to texture analysis and optical flow

paper_content:

The problem of detection of orientation in finite dimensional Euclidean spaces is solved in the least squares sense. The theory is developed for the case when such orientation computations are necessary at all local neighborhoods of the n-dimensional Euclidean space. Detection of orientation is shown to correspond to fitting an axis or a plane to the Fourier transform of an n-dimensional structure. The solution of this problem is related to the solution of a well-known matrix eigenvalue problem. The computations can be performed in the spatial domain without actually doing a Fourier transformation. Along with the orientation estimate, a certainty measure, based on the error of the fit, is proposed. Two applications in image analysis are considered: texture segmentation and optical flow. The theory is verified by experiments which confirm accurate orientation estimates and reliable certainty measures in the presence of noise. The comparative results indicate that the theory produces algorithms computing robust texture features as well as optical flow. >
---
paper_title: Lesion Border Detection in Dermoscopy Images

paper_content:

BACKGROUND ::: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, computerized analysis of dermoscopy images has become an important research area. One of the most important steps in dermoscopy image analysis is the automated detection of lesion borders. ::: ::: ::: METHODS ::: In this article, we present a systematic overview of the recent border detection methods in the literature paying particular attention to computational issues and evaluation aspects. ::: ::: ::: CONCLUSION ::: Common problems with the existing approaches include the acquisition, size, and diagnostic distribution of the test image set, the evaluation of the results, and the inadequate description of the employed methods. Border determination by dermatologists appears to depend upon higher-level knowledge, therefore it is likely that the incorporation of domain knowledge in automated methods will enable them to perform better, especially in sets of images with a variety of diagnoses.
---
paper_title: A Graph Cut Approach to Image Segmentation in Tensor Space

paper_content:

This paper proposes a novel method to apply the standard graph cut technique to segmenting multimodal tensor valued images. The Riemannian nature of the tensor space is explicitly taken into account by first mapping the data to a Euclidean space where non-parametric kernel density estimates of the regional distributions may be calculated from user initialized regions. These distributions are then used as regional priors in calculating graph edge weights. Hence this approach utilizes the true variation of the tensor data by respecting its Riemannian structure in calculating distances when forming probability distributions. Further, the non-parametric model generalizes to arbitrary tensor distribution unlike the Gaussian assumption made in previous works. Casting the segmentation problem in a graph cut framework yields a segmentation robust with respect to initialization on the data tested.
---
paper_title: An affine invariant tensor dissimilarity measure and its applications to tensor-valued image segmentation

paper_content:

Tensor fields specifically, matrix valued data sets, have recently attracted increased attention in the fields of image processing, computer vision, visualization and medical imaging. In this paper, we present a novel definition of tensor "distance" grounded in concepts from information theory and incorporate it in the segmentation of tensor-valued images. In some applications, a symmetric positive definite (SPD) tensor at each point of a tensor valued image can be interpreted as the covariance matrix of a local Gaussian distribution. Thus, a natural measure of dissimilarity between SPD tensors would be the KL divergence or its relative. We propose the square root of the J-divergence (symmetrized KL) between two Gaussian distributions corresponding to the tensors being compared that leads to a novel closed form expression. Unlike the traditional Frobenius norm-based tensor distance, our "distance" is affine invariant, a desirable property in many applications. We then incorporate this new tensor "distance" in a region based active contour model for bimodal tensor field segmentation and show its application to the segmentation of diffusion tensor magnetic resonance images (DT-MRI) as well as for the texture segmentation problem in computer vision. Synthetic and real data experiments are shown to depict the performance of the proposed model.
---
paper_title: Discriminative learned dictionaries for local image analysis

paper_content:

Sparse signal models have been the focus of much recent research, leading to (or improving upon) state-of-the-art results in signal, image, and video restoration. This article extends this line of research into a novel framework for local image discrimination tasks, proposing an energy formulation with both sparse reconstruction and class discrimination components, jointly optimized during dictionary learning. This approach improves over the state of the art in texture segmentation experiments using the Brodatz database, and it paves the way for a novel scene analysis and recognition framework based on simultaneously learning discriminative and reconstructive dictionaries. Preliminary results in this direction using examples from the Pascal VOC06 and Graz02 datasets are presented as well.
---
paper_title: Segmentation of 3D Probability Density Fields by Surface Evolution: Application to Diffusion MRI

paper_content:

We propose an original approach for the segmentation of three-dimensional fields of probability density functions. This presents a wide range of applications in medical images processing, in particular for diffusion magnetic resonance imaging where each voxel is assigned with a function describing the average motion of water molecules. Being able to automatically extract relevant anatomical structures of the white matter, such as the corpus callosum, would dramatically improve our current knowledge of the cerebral connectivity as well as allow for their statistical analysis. Our approach relies on the use of the symmetrized Kullback-Leibler distance and on the modelization of its distribution over the subsets of interest in the volume. The variational formulation of the problem yields a level-set evolution converging toward the optimal segmentation.
---
paper_title: Variational Image Segmentation for Endoscopic Human Colonic Aberrant Crypt Foci

paper_content:

The aim of this paper is to introduce a variational image segmentation method for assessing the aberrant crypt foci (ACF) in the human colon captured in vivo by endoscopy. ACF are thought to be precursors for colorectal cancer, and therefore their early detection may play an important clinical role. We enhance the active contours without edges model of Chan and Vese to account for the ACF's particular structure. We employ level sets to represent the segmentation boundaries and discretize in space by finite elements and in (artificial) time by finite differences. The approach is able to identify the ACF, their boundaries, and some of the internal crypts' orifices.
---
paper_title: A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model

paper_content:

We propose a new multiphase level set framework for image segmentation using the Mumford and Shah model, for piecewise constant and piecewise smooth optimal approximations. The proposed method is also a generalization of an active contour model without edges based 2-phase segmentation, developed by the authors earlier in T. Chan and L. Vese (1999. In Scale-Space'99, M. Nilsen et al. (Eds.), LNCS, vol. 1682, pp. 141–151) and T. Chan and L. Vese (2001. IEEE-IP, 10(2):266–277). The multiphase level set formulation is new and of interest on its own: by construction, it automatically avoids the problems of vacuum and overlaps it needs only log n level set functions for n phases in the piecewise constant cases it can represent boundaries with complex topologies, including triple junctionss in the piecewise smooth case, only two level set functions formally suffice to represent any partition, based on The Four-Color Theorem. Finally, we validate the proposed models by numerical results for signal and image denoising and segmentation, implemented using the Osher and Sethian level set method.
---
paper_title: Distinctive Image Features from Scale-Invariant Keypoints

paper_content:

The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.
---
paper_title: Active appearance models

paper_content:

We describe a new method of matching statistical models of appearance to images. A set of model parameters control modes of shape and gray-level variation learned from a training set. We construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors.
---
paper_title: Surf: Speeded up robust features

paper_content:

In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. ::: ::: This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.
---
paper_title: Liver Vessels Segmentation Using a Hybrid Geometrical Moments/Graph Cuts Method

paper_content:

This paper describes a fast and fully automatic method for liver vessel segmentation on computerized tomography scan preoperative images. The basis of this method is the introduction of a 3-D geometrical moment-based detector of cylindrical shapes within the minimum-cut/maximum-flow energy minimization framework. This method represents an original way to introduce a data term as a constraint into the widely used Boykov's graph cuts algorithm, and hence, to automate the segmentation. The method is evaluated and compared with others on a synthetic dataset. Finally, the relevancy of our method regarding the planning of a necessarily accurate percutaneous high-intensity focused ultrasound surgical operation is demonstrated with some examples.
---
paper_title: "GrabCut": interactive foreground extraction using iterated graph cuts

paper_content:

The problem of efficient, interactive foreground/background segmentation in still images is of great practical importance in image editing. Classical image segmentation tools use either texture (colour) information, e.g. Magic Wand, or edge (contrast) information, e.g. Intelligent Scissors. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power of the iterative algorithm is used to simplify substantially the user interaction needed for a given quality of result. Thirdly, a robust algorithm for "border matting" has been developed to estimate simultaneously the alpha-matte around an object boundary and the colours of foreground pixels. We show that for moderately difficult examples the proposed method outperforms competitive tools.
---
paper_title: Decision Forests with Spatio-Temporal Features for Graph-Based Tumor Segmentation in 4D Lung CT

paper_content:

We propose an automatic lung tumor segmentation in dynamic CT images that incorporates the novel use of tumor tissue deformations. In contrast to elastography imaging techniques for measuring tumor tissue properties, which require mechanical compression and thereby interrupt normal breathing, we completely avoid the use of any external physical forces. Instead, we calculate the tissue deformations during normal respiration using deformable registration. We investigate machine learning methods in order to discover the spatio-temporal dynamics that would help distinguish tumor from normal tissue deformation patterns and integrate this information into the segmentation process. Our method adapts an ensemble of decision trees combined with a 3D graph-based optimization that takes into account spatio-temporal consistency. The experimental results on patients with large tumors achieved an average F-measure accuracy of 0.79.
---
paper_title: Abdominal organ segmentation using texture transforms and a Hopfield neural network

paper_content:

Abdominal organ segmentation is highly desirable but difficult, due to large differences between patients and to overlapping grey-scale values of the various tissue types. The first step in automating this process is to cluster together the pixels within each organ or tissue type. The authors propose to form images based on second-order statistical texture transforms (Haralick transforms) of a CT or MRI scan. The original scan plus the suite of texture transforms are then input into a Hopfield neural network (HNN). The network is constructed to solve an optimization problem, where the best solution is the minima of a Lyapunov energy function. On a sample abdominal CT scan, this process successfully clustered 79-100% of the pixels of seven abdominal organs. It is envisioned that this is the first step to automate segmentation. Active contouring (e.g., SNAKE's) or a back-propagation neural network can then be used to assign names to the clusters and fill in the incorrectly clustered pixels.
---
paper_title: Histograms of oriented gradients for human detection

paper_content:

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.
---
paper_title: Classification of tumor histopathology via sparse feature learning

paper_content:

Our goal is to decompose whole slide images (WSI) of histology sections into distinct patches (e.g., viable tumor, necrosis) so that statistics of distinct histopathology can be linked with the outcome. Such an analysis requires a large cohort of histology sections that may originate from different laboratories, which may not use the same protocol in sample preparation. We have evaluated a method based on a variation of the restricted Boltzmann machine (RBM) that learns intrinsic features of the image signature in an unsupervised fashion. Computed code, from the learned representation, is then utilized to classify patches from a curated library of images. The system has been evaluated against a dataset of small image blocks of 1k-by-1k that have been extracted from glioblastoma multiforme (GBM) and clear cell kidney carcinoma (KIRC) from the cancer genome atlas (TCGA) archive. The learned model is then projected on each whole slide image (e.g., of size 20k-by-20k pixels or larger) for characterizing and visualizing tumor architecture. In the case of GBM, each WSI is decomposed into necrotic, transition into necrosis, and viable. In the case of the KIRC, each WSI is decomposed into tumor types, stroma, normal, and others. Evaluation of 1400 and 2500 samples of GBM and KIRC indicates a performance of 84% and 81%, respectively.
---
paper_title: Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

paper_content:

In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.
---
paper_title: Interactive Texture Segmentation using Random Forests and Total Variation

paper_content:

Hypothesis The segmentation quality depends on a strong description for F and B. In order to model hypotheses based on different high-level features, we need an efficient learning algorithm capable of handling arbitrary input data. Random Forests (RFs) are fast to compute while yielding state-of-the-art performance in machine learning and vision problems. Their parallel structure dedicates them to GPU implementations. Recently, an online version of RFs has been proposed [2], which renders retraining of the whole forest upon additional user input unnecessary.
---
paper_title: DT-MRI segmentation using graph cuts

paper_content:

An important problem in medical image analysis is the segmentation of anatomical regions of interest. Once ::: regions of interest are segmented, one can extract shape, appearance, and structural features that can be analyzed ::: for disease diagnosis or treatment evaluation. Diffusion tensor magnetic resonance imaging (DT-MRI) is ::: a relatively new medical imaging modality that captures unique water diffusion properties and fiber orientation ::: information of the imaged tissues. In this paper, we extend the interactive multidimensional graph cuts segmentation ::: technique to operate on DT-MRI data by utilizing latest advances in tensor calculus and diffusion tensor ::: dissimilarity metrics. The user interactively selects certain tensors as object ("obj") or background ("bkg") to ::: provide hard constraints for the segmentation. Additional soft constraints incorporate information about both ::: regional tissue diffusion as well as boundaries between tissues of different diffusion properties. Graph cuts are ::: used to find globally optimal segmentation of the underlying 3D DT-MR image among all segmentations satisfying ::: the constraints. We develop a graph structure from the underlying DT-MR image with the tensor voxels ::: corresponding to the graph vertices and with graph edge weights computed using either Log-Euclidean or the ::: J-divergence tensor dissimilarity metric. The topology of our segmentation is unrestricted and both obj and bkg ::: segments may consist of several isolated parts. We test our method on synthetic DT data and apply it to real ::: 2D and 3D MRI, providing segmentations of the corpus callosum in the brain and the ventricles of the heart.
---
paper_title: A performance evaluation of local descriptors

paper_content:

In this paper we compare the performance of interest point descriptors. Many different descriptors have been proposed in the literature. However, it is unclear which descriptors are more appropriate and how their performance depends on the interest point detector. The descriptors should be distinctive and at the same time robust to changes in viewing conditions as well as to errors of the point detector. Our evaluation uses as criterion detection rate with respect to false positive rate and is carried out for different image transformations. We compare SIFT descriptors (Lowe, 1999), steerable filters (Freeman and Adelson, 1991), differential invariants (Koenderink ad van Doorn, 1987), complex filters (Schaffalitzky and Zisserman, 2002), moment invariants (Van Gool et al., 1996) and cross-correlation for different types of interest points. In this evaluation, we observe that the ranking of the descriptors does not depend on the point detector and that SIFT descriptors perform best. Steerable filters come second ; they can be considered a good choice given the low dimensionality.
---
paper_title: Kinetic Modeling Based Probabilistic Segmentation for Molecular Images

paper_content:

We propose a semi-supervised, kinetic modeling based segmentation technique for molecular imaging applications. It is an iterative, self-learning algorithm based on uncertainty principles, designed to alleviate low signal-to-noise ratio (SNR) and partial volume effect (PVE) problems. Synthetic fluorodeoxyglucose (FDG) and simulated Raclopride dynamic positron emission tomography (dPET) brain images with excessive noise levels are used to validate our algorithm. We show, qualitatively and quantitatively, that our algorithm outperforms state-of-the-art techniques in identifying different functional regions and recovering the kinetic parameters.
---
paper_title: Co-Sparse Textural Similarity for Interactive Segmentation

paper_content:

We propose an algorithm for segmenting natural images based on texture and color information, which leverages the co-sparse analysis model for image segmentation. As a key ingredient of this method, we introduce a novel textural similarity measure, which builds upon the co-sparse representation of image patches. We propose a statistical MAP inference approach to merge textural similarity with information about color and location. Combined with recently developed convex multilabel optimization methods this leads to an efficient algorithm for interactive segmentation, which is easily parallelized on graphics hardware. The provided approach outperforms state-of-the-art interactive segmentation methods on the Graz Benchmark.
---
paper_title: Image Classification using Random Forests and Ferns

paper_content:

We explore the problem of classifying images by the object categories they contain in the case of a large number of object categories. To this end we combine three ingredients: (i) shape and appearance representations that support spatial pyramid matching over a region of interest. This generalizes the representation of Lazebnik et al., (2006) from an image to a region of interest (ROI), and from appearance (visual words) alone to appearance and local shape (edge distributions); (ii) automatic selection of the regions of interest in training. This provides a method of inhibiting background clutter and adding invariance to the object instance 's position; and (iii) the use of random forests (and random ferns) as a multi-way classifier. The advantage of such classifiers (over multi-way SVM for example) is the ease of training and testing. Results are reported for classification of the Caltech-101 and Caltech-256 data sets. We compare the performance of the random forest/ferns classifier with a benchmark multi-way SVM classifier. It is shown that selecting the ROI adds about 5% to the performance and, together with the other improvements, the result is about a 10% improvement over the state of the art for Caltech-256.
---
paper_title: Vessel scale-selection using MRF optimization

paper_content:

Many feature detection algorithms rely on the choice of scale. In this paper, we complement standard scale-selection algorithms with spatial regularization. To this end, we formulate scale-selection as a graph labeling problem and employ Markov random field multi-label optimization. We focus on detecting the scales of vascular structures in medical images. We compare the detected vessel scales using our method to those obtained using the selection approach of the well-known vesselness filter (Frangi et al 1998). We propose and discuss two different approaches for evaluating the goodness of scale-selection. Our results on 40 images from the Digital Retinal Images for Vessel Extraction (DRIVE) database show an average reduction in these error measurements by more than 15%.
---
paper_title: Active unsupervised texture segmentation on a diffusion based feature space

paper_content:

We propose a novel and efficient approach for active unsupervised texture segmentation. First, we show how we can extract a small set of good features for texture segmentation based on the structure tensor and nonlinear diffusion. Then, we propose a variational framework that incorporates these features in a level set based unsupervised segmentation process that adaptively takes into account their estimated statistical information inside and outside the region to segment. The approach has been tested on various textured images, and its performance is favorably compared to recent studies.
---
paper_title: A Statistical Overlap Prior for Variational Image Segmentation

paper_content:

This study investigates variational image segmentation with an original data term, referred to as statistical overlap prior, which measures the conformity of overlap between the nonparametric distributions of image data within the segmentation regions to a learned statistical description. This leads to image segmentation and distribution tracking algorithms that relax the assumption of minimal overlap and, as such, are more widely applicable than existing algorithms. We propose to minimize active curve functionals containing the proposed overlap prior, compute the corresponding Euler-Lagrange curve evolution equations, and give an interpretation of how the overlap prior controls such evolution. We model the overlap, measured via the Bhattacharyya coefficient, with a Gaussian prior whose parameters are estimated from a set of relevant training images. Quantitative and comparative performance evaluations of the proposed algorithms over several experiments demonstrate the positive effects of the overlap prior in regard to segmentation accuracy and convergence speed.
---
paper_title: A variational method for vessels segmentation: algorithm and application to liver vessels visualization

paper_content:

We present a new variational-based method for automatic liver vessels segmentation from abdominal CTA images. The segmentation task is formulated as a functional minimization problem within a variational framework. We introduce a new functional that incorporates both geometrical vesselness measure and vessels surface properties. The functional describes the distance between the desired segmentation and the original image. To minimize the functional, we derive the Euler-Lagrange equation from it and solve it using the conjugate gradients algorithm. Our approach is automatic and improves upon other Hessian-based methods in the detection of bifurcations and complex vessels structures by incorporating a surface term into the functional. To assess our method, we conducted with an expert radiologist two comparative studies on 8 abdominal CTA clinical datasets. In the first study, the radiologist assessed the presence of 11 vascular bifurcations on each dataset, totaling of 73 bifurcations. The radiologist qualitatively compared the bifurcations segmentation of our method and that of a Hessian-based threshold method. Our method correctly segmented 88% of the bifurcations with a higher visibility score of 82%, as compared to only 55% in the Hessian-based method with a visibility score of 33%. In the second study, the radiologist assessed the individual vessels visibility on the 3D segmentation images and on the original CTA slices. Ten main liver vessels were examined in each dataset The overall visibility score was 93%. These results indicate that our method is suitable for the automatic segmentation and visualization of the liver vessels.
---
paper_title: Active Contours without Edges for Vector-Valued Images

paper_content:

In this paper, we propose an active contour algorithm for object detection in vector-valued images (such as RGB or multispectral). The model is an extension of the scalar Chan?Vese algorithm to the vector-valued case 1]. The model minimizes a Mumford?Shah functional over the length of the contour, plus the sum of the fitting error over each component of the vector-valued image. Like the Chan?Vese model, our vector-valued model can detect edges both with or without gradient. We show examples where our model detects vector-valued objects which are undetectable in any scalar representation. For instance, objects with different missing parts in different channels are completely detected (such as occlusion). Also, in color images, objects which are invisible in each channel or in intensity can be detected by our algorithm. Finally, the model is robust with respect to noise, requiring no a priori denoising step.
---
paper_title: A fast local descriptor for dense matching

paper_content:

We introduce a novel local image descriptor designed for dense wide-baseline matching purposes. We feed our descriptors to a graph-cuts based dense depth map estimation algorithm and this yields better wide-baseline performance than the commonly used correlation windows for which the size is hard to tune. As a result, unlike competing techniques that require many high-resolution images to produce good reconstructions, our descriptor can compute them from pairs of low-quality images such as the ones captured by video streams. Our descriptor is inspired from earlier ones such as SIFT and GLOH but can be computed much faster for our purposes. Unlike SURF which can also be computed efficiently at every pixel, it does not introduce artifacts that degrade the matching performance. Our approach was tested with ground truth laser scanned depth maps as well as on a wide variety of image pairs of different resolutions and we show that good reconstructions are achieved even with only two low quality images.
---
paper_title: A tensor-based algorithm for high-order graph matching

paper_content:

This paper addresses the problem of establishing correspondences between two sets of visual features using higher-order constraints instead of the unary or pairwise ones used in classical methods. Concretely, the corresponding hypergraph matching problem is formulated as the maximization of a multilinear objective function over all permutations of the features. This function is defined by a tensor representing the affinity between feature tuples. It is maximized using a generalization of spectral techniques where a relaxed problem is first solved by a multi-dimensional power method, and the solution is then projected onto the closest assignment matrix. The proposed approach has been implemented, and it is compared to state-of-the-art algorithms on both synthetic and real data.
---
paper_title: Level set based segmentation with intensity and curvature priors

paper_content:

A method is presented for segmentation of anatomical structures that incorporates prior information about the intensity and curvature profile of the structure from a training set of images and boundaries. Specifically, we model the intensity distribution as a function of signed distance from the object boundary, instead of modeling only the intensity of the object as a whole. A curvature profile acts as a boundary regularization term specific to the shape being extracted, as opposed to simply penalizing high curvature. Using the prior model, the segmentation process estimates a maximum a posteriori higher dimensional surface whose zero level set converges on the boundary of the object to be segmented. Segmentation results are demonstrated on synthetic data and magnetic resonance imagery.
---
paper_title: Curvature regularity for region-based image segmentation and inpainting: A linear programming relaxation

paper_content:

We consider a class of region-based energies for image segmentation and inpainting which combine region integrals with curvature regularity of the region boundary. To minimize such energies, we formulate an integer linear program which jointly estimates regions and their boundaries. Curvature regularity is imposed by respective costs on pairs of adjacent boundary segments.
---
paper_title: Minimizing Sparse High-Order Energies by Submodular Vertex-cover

paper_content:

Inference in high-order graphical models has become important in recent years. Several approaches are based, for example, on generalized message-passing, or on transformation to a pairwise model with extra 'auxiliary' variables. We focus on a special case where a much more efficient transformation is possible. Instead of adding variables, we transform the original problem into a comparatively small instance of submodular vertex-cover. These vertex-cover instances can then be attacked by existing algorithms (e.g. belief propagation, QPBO), where they often run 4-15 times faster and find better solutions than when applied to the original problem. We evaluate our approach on synthetic data, then we show applications within a fast hierarchical clustering and model-fitting framework.
---
paper_title: Fast global optimization of curvature

paper_content:

Two challenges in computer vision are to accommodate noisy data and missing data. Many problems in computer vision, such as segmentation, filtering, stereo, reconstruction, inpainting and optical flow seek solutions that match the data while satisfying an additional regularization, such as total variation or boundary length. A regularization which has received less attention is to minimize the curvature of the solution. One reason why this regularization has received less attention is due to the difficulty in finding an optimal solution to this image model, since many existing methods are complicated, slow and/or provide a suboptimal solution. Following the recent progress of Schoenemann et al. [28], we provide a simple formulation of curvature regularization which admits a fast optimization which gives globally optimal solutions in practice. We demonstrate the effectiveness of this method by applying this curvature regularization to image segmentation.
---
paper_title: Curvature Regularization for Curves and Surfaces in a Global Optimization Framework

paper_content:

Length and area regularization are commonplace for inverse problems today. It has however turned out to be much more difficult to incorporate a curvature prior. In this paper we propose several improvements to a recently proposed framework based on global optimization. We identify and solve an issue with extraneous arcs in the original formulation by introducing region consistency constraints. The mesh geometry is analyzed both from a theoretical and experimental viewpoint and hexagonal meshes are shown to be superior. We demonstrate that adaptively generated meshes significantly improve the performance. Our final contribution is that we generalize the framework to handle mean curvature regularization for 3D surface completion and segmentation. (Less)
---
paper_title: Snakes: Active contour models

paper_content:

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.
---
paper_title: Shortest Paths with Curvature and Torsion

paper_content:

This paper describes a method of finding thin, elongated structures in images and volumes. We use shortest paths to minimize very general functionals of higher-order curve properties, such as curvature and torsion. Our globally optimal method uses line graphs and its runtime is polynomial in the size of the discretization, often in the order of seconds on a single computer. To our knowledge, we are the first to perform experiments in three dimensions with curvature and torsion regularization. The largest graphs we process have almost one hundred billion arcs. Experiments on medical images and in multi-view reconstruction show the significance and practical usefulness of regularization based on curvature while torsion is still only tractable for small-scale problems.
---
paper_title: Interactive image segmentation via minimization of quadratic energies on directed graphs

paper_content:

We propose a scheme to introduce directionality in the random walker algorithm for image segmentation. In particular, we extend the optimization framework of this algorithm to combinatorial graphs with directed edges. Our scheme is interactive and requires the user to label a few pixels that are representative of a foreground object and of the background. These labeled pixels are used to learn intensity models for the object and the background, which allow us to automatically set the weights of the directed edges. These weights are chosen so that they bias the direction of the object boundary gradients to flow from regions that agree well with the learned object intensity model to regions that do not agree well. We use these weights to define an energy function that associates asymmetric quadratic penalties with the edges in the graph. We show that this energy function is convex, hence it has a unique minimizer. We propose a provably convergent iterative algorithm for minimizing this energy function. We also describe the construction of an equivalent electrical network with diodes and resistors that solves the same segmentation problem as our framework. Finally, our experiments on a database of 69 images show that the use of directional information does improve the segmenting power of the random Walker algorithm.
---
paper_title: An Adaptive Two-Stage Edge Detection Scheme for Digital Color Images

paper_content:

An adaptive two-stage edge detection scheme for digital color images is proposed in this paper. In the first stage of this scheme, each three-dimensional color image is reduced to a one-dimensional gray-level image using the moment-preserving thresholding technique. Then, a new edge detection technique based on the block truncation coding scheme is introduced to detect the edge boundary in the second stage. The edge detection process makes use of the bit plane information of each BTC-encoded block to detect the edge boundary. The experimental results show that the performance of the detected edge image of the proposed scheme is as good as in Yang's scheme and in the Sobel operator. However, the computational cost consumed by the proposed scheme is less than that of Yang's scheme. In addition, the proposed scheme provides an adaptive edge quality decision mechanism. This mechanism can provide different edge images to meet various applications and the subjective evaluation. Moreover, this scheme locates the edge boundaries to the sub-pixel accuracy, which is an advantage to applications such as data hiding and image watermarking.
---
paper_title: Detecting Structure in Diffusion Tensor MR Images

paper_content:

We derive herein first and second-order differential operators for detecting structure in diffusion tensor MRI (DTI). Unlike existing methods, we are able to generate full first and second-order differentials without dimensionality reduction and while respecting the underlying manifold of the data. Further, we extend corner and curvature feature detectors to DTI using our differential operators. Results using the feature detectors on diffusion tensor MR images show the ability to highlight structure within the image that existing methods cannot.
---
paper_title: Geodesic Active Regions and Level Set Methods for Supervised Texture Segmentation

paper_content:

This paper presents a novel variational framework to deal with frame partition problems in Computer Vision. This framework exploits boundary and region-based segmentation modules under a curve-based optimization objective function. The task of supervised texture segmentation is considered to demonstrate the potentials of the proposed framework. The textured feature space is generated by filtering the given textured images using isotropic and anisotropic filters, and analyzing their responses as multi-component conditional probability density functions. The texture segmentation is obtained by unifying region and boundary-based information as an improved Geodesic Active Contour Model. The defined objective function is minimized using a gradient-descent method where a level set approach is used to implement the obtained PDE. According to this PDE, the curve propagation towards the final solution is guided by boundary and region-based segmentation forces, and is constrained by a regularity force. The level set implementation is performed using a fast front propagation algorithm where topological changes are naturally handled. The performance of our method is demonstrated on a variety of synthetic and real textured frames.
---
paper_title: Color lesion boundary detection using live wire

paper_content:

The boundaries of oral lesions in color images were detected using a live-wire method and compared to expert delineations. Multiple cost terms were analyzed for their inclusion in the final total cost function including color gradient magnitude, color gradient direction, Canny edge detection, and Laplacian zero crossing. The gradient magnitude and direction cost terms were implemented so that they acted directly on the three components of the color image, instead of using a single derived color band. The live-wire program was shown to be considerably more accurate and faster compared to manual segmentations by untrained users.
---
paper_title: Quaternion color curvature

paper_content:

In this paper we propose a novel approach to measuring curvature in color or vector-valued images (up to 4-dimensions) based on quaternion singular value decomposition of a Hessian matrix. This approach generalizes the existing scalar-image curvature approach which makes use of the eigenvalues of the Hessian matrix [1]. In the case of vector-valued images, the Hessian is no longer a 2D matrix but rather a rank 3 tensor. We use quaternion curvature to derive vesselness measure for tubular structures in color or vector-valued images by extending Frangi’s [1] vesselness measure for scalar images. Experimental results show the effectiveness of quaternion color curvature in generating a vesselness map.
---
paper_title: Normalized cuts and image segmentation

paper_content:

We propose a novel approach for solving the perceptual grouping problem in vision. Rather than focusing on local features and their consistencies in the image data, our approach aims at extracting the global impression of an image. We treat image segmentation as a graph partitioning problem and propose a novel global criterion, the normalized cut, for segmenting the graph. The normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups. We show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion. We applied this approach to segmenting static images, as well as motion sequences, and found the results to be very encouraging.
---
paper_title: A Computational Approach to Edge Detection

paper_content:

This paper describes a computational approach to edge detection. The success of the approach depends on the definition of a comprehensive set of goals for the computation of edge points. These goals must be precise enough to delimit the desired behavior of the detector while making minimal assumptions about the form of the solution. We define detection and localization criteria for a class of edges, and present mathematical forms for these criteria as functionals on the operator impulse response. A third criterion is then added to ensure that the detector has only one response to a single edge. We use the criteria in numerical optimization to derive detectors for several common image features, including step edges. On specializing the analysis to step edges, we find that there is a natural uncertainty principle between detection and localization performance, which are the two main goals. With this principle we derive a single operator shape which is optimal at any scale. The optimal detector has a simple approximate implementation in which edges are marked at maxima in gradient magnitude of a Gaussian-smoothed image. We extend this simple detector using operators of several widths to cope with different signal-to-noise ratios in the image. We present a general method, called feature synthesis, for the fine-to-coarse integration of information from operators at different scales. Finally we show that step edge detector performance improves considerably as the operator point spread function is extended along the edge.
---
paper_title: Graph Cuts and Efficient N-D Image Segmentation

paper_content:

Combinatorial graph cut algorithms have been successfully applied to a wide range of problems in vision and graphics. This paper focusses on possibly the simplest application of graph-cuts: segmentation of objects in image data. Despite its simplicity, this application epitomizes the best features of combinatorial graph cuts methods in vision: global optima, practical efficiency, numerical robustness, ability to fuse a wide range of visual cues and constraints, unrestricted topological properties of segments, and applicability to N-D problems. Graph cuts based approaches to object extraction have also been shown to have interesting connections with earlier segmentation methods such as snakes, geodesic active contours, and level-sets. The segmentation energies optimized by graph cuts combine boundary regularization with region-based properties in the same fashion as Mumford-Shah style functionals. We present motivation and detailed technical description of the basic combinatorial optimization framework for image segmentation via s/t graph cuts. After the general concept of using binary graph cut algorithms for object segmentation was first proposed and tested in Boykov and Jolly (2001), this idea was widely studied in computer vision and graphics communities. We provide links to a large number of known extensions based on iterative parameter re-estimation and learning, multi-scale or hierarchical approaches, narrow bands, and other techniques for demanding photo, video, and medical applications.
---
paper_title: Geodesic active contours

paper_content:

A novel scheme for the detection of object boundaries is presented. The technique is based on active contours deforming according to intrinsic geometric measures of the image. The evolving contours naturally split and merge, allowing the simultaneous detection of several objects and both interior and exterior boundaries. The proposed approach is based on the relation between active contours and the computation of geodesics or minimal distance curves. The minimal distance curve lays in a Riemannian space whose metric as defined by the image content. This geodesic approach for object segmentation allows to connect classical "snakes" based on energy minimization and geometric active contours based on the theory of curve evolution. Previous models of geometric active contours are improved as showed by a number of examples. Formal results concerning existence, uniqueness, stability, and correctness of the evolution are presented as well.<<ETX>>
---
paper_title: Random Walks for Image Segmentation

paper_content:

A novel method is proposed for performing multilabel, interactive image segmentation. Given a small number of pixels with user-defined (or predefined) labels, one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels. By assigning each pixel to the label for which the greatest probability is calculated, a high-quality image segmentation may be obtained. Theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits. This algorithm is formulated in discrete space (i.e., on a graph) using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension on arbitrary graphs
---
paper_title: Exact optimization for markov random fields with convex priors

paper_content:

We introduce a method to solve exactly a first order Markov random field optimization problem in more generality than was previously possible. The MRF has a prior term that is convex in terms of a linearly ordered label set. The method maps the problem into a minimum-cut problem for a directed graph, for which a globally optimal solution can be found in polynomial time. The convexity of the prior function in the energy is shown to be necessary and sufficient for the applicability of the method.
---
paper_title: Segmentation of Intra-Retinal Layers From Optical Coherence Tomography Images Using an Active Contour Approach

paper_content:

Optical coherence tomography (OCT) is a noninvasive, depth-resolved imaging modality that has become a prominent ophthalmic diagnostic technique. We present a semi-automated segmentation algorithm to detect intra-retinal layers in OCT images acquired from rodent models of retinal degeneration. We adapt Chan-Vese's energy-minimizing active contours without edges for the OCT images, which suffer from low contrast and are highly corrupted by noise. A multiphase framework with a circular shape prior is adopted in order to model the boundaries of retinal layers and estimate the shape parameters using least squares. We use a contextual scheme to balance the weight of different terms in the energy functional. The results from various synthetic experiments and segmentation results on OCT images of rats are presented, demonstrating the strength of our method to detect the desired retinal layers with sufficient accuracy even in the presence of intensity inhomogeneity resulting from blood vessels. Our algorithm achieved an average Dice similarity coefficient of 0.84 over all segmented retinal layers, and of 0.94 for the combined nerve fiber layer, ganglion cell layer, and inner plexiform layer which are the critical layers for glaucomatous degeneration.
---
paper_title: Fast approximate energy minimization via graph cuts

paper_content:

In this paper we address the problem of minimizing a large class of energy functions that occur in early vision. The major restriction is that the energy function's smoothness term must only involve pairs of pixels. We propose two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed. The first move we consider is an /spl alpha/-/spl beta/-swap: for a pair of labels /spl alpha/,/spl beta/, this move exchanges the labels between an arbitrary set of pixels labeled a and another arbitrary set labeled /spl beta/. Our first algorithm generates a labeling such that there is no swap move that decreases the energy. The second move we consider is an /spl alpha/-expansion: for a label a, this move assigns an arbitrary set of pixels the label /spl alpha/. Our second algorithm, which requires the smoothness term to be a metric, generates a labeling such that there is no expansion move that decreases the energy. Moreover, this solution is within a known factor of the global minimum. We experimentally demonstrate the effectiveness of our approach on image restoration, stereo and motion.
---
paper_title: Multiregion competition: A level set extension of region competition to multiple region image partitioning

paper_content:

The purpose of this study is to investigate a new representation of a partition of an image domain into a fixed but arbitrary number of regions by explicit correspondence between the regions of segmentation and the regions defined by simple closed planar curves and their intersections, and the use of this representation in the context of region competition to provide a level set multiregion competition algorithm. This formulation leads to a system of coupled curve evolution equations which is easily amenable to a level set implementation and the computed solution is one that minimizes the stated functional. An unambiguous segmentation is garanteed because at all time during curve evolution the evolving regions form a partition of the image domain. We present the multiregion competition algorithm for intensity-based image segmentation and we subsequently extend it to motion/disparity. Finally, we consider an extension of the algorithm to account for images with aberrations such as occlusions. The formulation, the ensuing algorithm, and its implementation have been validated in several experiments on gray level, color, and motion segmentation.
---
paper_title: A convex formulation of continuous multi-label problems

paper_content:

We propose a spatially continuous formulation of Ishikawa's discrete multi-label problem. We show that the resulting non-convex variational problem can be reformulated as a convex variational problem via embedding in a higher dimensional space. This variational problem can be interpreted as a minimal surface problem in an anisotropic Riemannian space. In several stereo experiments we show that the proposed continuous formulation is superior to its discrete counterpart in terms of computing time, memory efficiency and metrication errors.
---
paper_title: Convex Formulation and Exact Global Solutions for Multi-phase Piecewise Constant Mumford-Shah Image Segmentation

paper_content:

Abstract : Most variational models for multi-phase image segmentation are non-convex and possess multiple local minima, which makes solving for a global solution an extremely difficult task. In this work, we provide a method for computing a global solution for the (non-convex) multi-phase piecewise constant Mumford-Shah (spatially continuous Potts) image segmentation problem. Our approach is based on using a specific representation of the problem due to Lie et al. [27]. We then rewrite this representation using the dual formulation for total variation so that a variational convexification technique due to Pock et al. [30] may be employed. Unlike some recent methods in this direction, our method can guarantee that a global solution is obtained. We believe our method to be the first in the literature that can make this claim. Once we have the convex optimization problem, we give an algorithm to compute a global solution. We demonstrate our algorithm on several multi-phase image segmentation examples, including a medical imaging application.
---
paper_title: A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model

paper_content:

We propose a new multiphase level set framework for image segmentation using the Mumford and Shah model, for piecewise constant and piecewise smooth optimal approximations. The proposed method is also a generalization of an active contour model without edges based 2-phase segmentation, developed by the authors earlier in T. Chan and L. Vese (1999. In Scale-Space'99, M. Nilsen et al. (Eds.), LNCS, vol. 1682, pp. 141–151) and T. Chan and L. Vese (2001. IEEE-IP, 10(2):266–277). The multiphase level set formulation is new and of interest on its own: by construction, it automatically avoids the problems of vacuum and overlaps it needs only log n level set functions for n phases in the piecewise constant cases it can represent boundaries with complex topologies, including triple junctionss in the piecewise smooth case, only two level set functions formally suffice to represent any partition, based on The Four-Color Theorem. Finally, we validate the proposed models by numerical results for signal and image denoising and segmentation, implemented using the Osher and Sethian level set method.
---
paper_title: Star Shape Prior for Graph-Cut Image Segmentation

paper_content:

In recent years, segmentation with graph cuts is increasingly used for a variety of applications, such as photo/video editing, medical image processing, etc. One of the most common applications of graph cut segmentation is extracting an object of interest from its background. If there is any knowledge about the object shape (i.e. a shape prior), incorporating this knowledge helps to achieve a more robust segmentation. In this paper, we show how to implement a star shape prior into graph cut segmentation. This is a generic shape prior, i.e. it is not specific to any particular object, but rather applies to a wide class of objects, in particular to convex objects. Our major assumption is that the center of the star shape is known, for example, it can be provided by the user. The star shape prior has an additional important benefit - it allows an inclusion of a term in the objective function which encourages a longer object boundary. This helps to alleviate the bias of a graph cut towards shorter segmentation boundaries. In fact, we show that in many cases, with this new term we can achieve an accurate object segmentation with only a single pixel, the center of the object, provided by the user, which is rarely possible with standard graph cut interactive segmentation.
---
paper_title: STACS: new active contour scheme for cardiac MR image segmentation

paper_content:

The paper presents a novel stochastic active contour scheme (STACS) for automatic image segmentation designed to overcome some of the unique challenges in cardiac MR images such as problems with low contrast, papillary muscles, and turbulent blood flow. STACS minimizes an energy functional that combines stochastic region-based and edge-based information with shape priors of the heart and local properties of the contour. The minimization algorithm solves, by the level set method, the Euler-Lagrange equation that describes the contour evolution. STACS includes an annealing schedule that balances dynamically the weight of the different terms in the energy functional. Three particularly attractive features of STACS are: 1) ability to segment images with low texture contrast by modeling stochastically the image textures; 2) robustness to initial contour and noise because of the utilization of both edge and region-based information; 3) ability to segment the heart from the chest wall and the undesired papillary muscles due to inclusion of heart shape priors. Application of STACS to a set of 48 real cardiac MR images shows that it can successfully segment the heart from its surroundings such as the chest wall and the heart structures (the left and right ventricles and the epicardium.) We compare STACS' automatically generated contours with manually-traced contours, or the "gold standard," using both area and edge similarity measures. This assessment demonstrates very good and consistent segmentation performance of STACS.
---
paper_title: Using Prior Shapes in Geometric Active Contours in a Variational Framework

paper_content:

In this paper, we report an active contour algorithm that is capable of using prior shapes. The energy functional of the contour is modified so that the energy depends on the image gradient as well as the prior shape. The model provides the segmentation and the transformation that maps the segmented contour to the prior shape. The active contour is able to find boundaries that are similar in shape to the prior, even when the entire boundary is not visible in the image (i.e., when the boundary has gaps). A level set formulation of the active contour is presented. The existence of the solution to the energy minimization is also established. ::: ::: We also report experimental results of the use of this contour on 2d synthetic images, ultrasound images and fMRI images. Classical active contours cannot be used in many of these images.
---
paper_title: Graph-based optimal multi-surface segmentation with a star-shaped prior: Application to the segmentation of the optic disc and cup

paper_content:

A novel graph-based optimal segmentation method which can simultaneously segment multiple star-shaped surfaces is presented in this paper. Minimum and maximum surface distance constraints can be enforced between different surfaces. In addition, the segmented surfaces are ensured to be smooth by incorporating surface smoothness constraints which limit the variation between adjacent surface voxels. A consistent digital ray system is utilized to make sure the segmentation result is star-shaped and consistent, without interpolating image as required by other methods. To the best of our knowledge, the concept of consistent digital rays is for the first time introduced into the field of medical imaging. The problem is formulated as an MRF optimization problem which can be efficiently and exactly solved by computing a single min s-t cut in an appropriately constructed graph. The method is applied to the segmentation of the optic disc and cup on 70 registered fundus and SD-OCT images from glaucoma patients. The result shows improved accuracy by applying the proposed method (versus using a classification-based approach).
---
paper_title: Interactive graph cut based segmentation with shape priors

paper_content:

Interactive or semi-automatic segmentation is a useful alternative to pure automatic segmentation in many applications. While automatic segmentation can be very challenging, a small amount of user input can often resolve ambiguous decisions on the part of the algorithm. In this work, we devise a graph cut algorithm for interactive segmentation which incorporates shape priors. While traditional graph cut approaches to interactive segmentation are often quite successful, they may fail in cases where there are diffuse edges, or multiple similar objects in close proximity to one another. Incorporation of shape priors within this framework mitigates these problems. Positive results on both medical and natural images are demonstrated.
---
paper_title: Nonparametric shape priors for active contour-based image segmentation

paper_content:

When segmenting images of low quality or with missing data, statistical prior information about the shapes of the objects to be segmented can significantly aid the segmentation process. However, defining probability densities in the space of shapes is an open and challenging problem. In this paper, we propose a nonparametric shape prior model for image segmentation problems. In particular, given example training shapes, we estimate the underlying shape distribution by extending a Parzen density estimator to the space of shapes. Such density estimates are expressed in terms of distances between shapes, and we consider the L2 distance between signed distance functions for shape density estimation, in addition to a distance measure based on the template metric. In particular, we consider the case in which the space of shapes is interpreted as a manifold embedded in a Hilbert space. We then incorporate the learned shape prior distribution into a maximum a posteriori (MAP) estimation framework for segmentation. This results in an optimization problem, which we solve using active contours. We demonstrate the effectiveness of the resulting algorithm in segmenting images that involve low-quality data and occlusions. The proposed framework is especially powerful in handling ''multimodal'' shape densities.
---
paper_title: Non-linear statistical models for the 3D reconstruction of human pose and motion from monocular image sequences

paper_content:

This paper presents a model based approach to human body tracking in which the 2D silhouette of a moving human and the corresponding 3D skeletal structure are encapsulated within a non-linear point distribution model. This statistical model allows a direct mapping to be achieved between the external boundary of a human and the anatomical position. It is shown how this information, along with the position of landmark features such as the hands and head can be used to reconstruct information about the pose and structure of the human body from a monocular view of a scene.
---
paper_title: Left ventricle segmentation in MRI via convex relaxed distribution matching

paper_content:

A fundamental step in the diagnosis of cardiovascular diseases, automatic left ventricle (LV) segmentation in cardiac magnetic resonance images (MRIs) is still acknowledged to be a difficult problem. Most of the existing algorithms require either extensive training or intensive user inputs. This study investigates fast detection of the left ventricle (LV) endo- and epicardium surfaces in cardiac MRI via convex relaxation and distribution matching. The algorithm requires a single subject for training and a very simple user input, which amounts to a single point (mouse click) per target region (cavity or myocardium). It seeks cavity and myocardium regions within each 3D phase by optimizing two functionals, each containing two distribution-matching constraints: (1) a distance-based shape prior and (2) an intensity prior. Based on a global measure of similarity between distributions, the shape prior is intrinsically invariant with respect to translation and rotation. We further introduce a scale variable from which we derive a fixed-point equation (FPE), thereby achieving scale-invariance with only few fast computations. The proposed algorithm relaxes the need for costly pose estimation (or registration) procedures and large training sets, and can tolerate shape deformations, unlike template (or atlas) based priors. Our formulation leads to a challenging problem, which is not directly amenable to convex-optimization techniques. For each functional, we split the problem into a sequence of sub-problems, each of which can be solved exactly and globally via a convex relaxation and the augmented Lagrangian method. Unlike related graph-cut approaches, the proposed convex-relaxation solution can be parallelized to reduce substantially the computational time for 3D domains (or higher), extends directly to high dimensions, and does not have the grid-bias problem. Our parallelized implementation on a graphics processing unit (GPU) demonstrates that the proposed algorithm requires about 3.87 s for a typical cardiac MRI volume, a speed-up of about five times compared to a standard implementation. We report a performance evaluation over 400 volumes acquired from 20 subjects, which shows that the obtained 3D surfaces correlate with independent manual delineations. We further demonstrate experimentally that (1) the performance of the algorithm is not significantly affected by the choice of the training subject and (2) the shape description we use does not change significantly from one subject to another. These results support the fact that a single subject is sufficient for training the proposed algorithm.
---
paper_title: A shape-based approach to the segmentation of medical imagery using level sets

paper_content:

We propose a shape-based approach to curve evolution for the segmentation of medical images containing known object types. In particular, motivated by the work of Leventon, Grimson, and Faugeras, we derive a parametric model for an implicit representation of the segmenting curve by applying principal component analysis to a collection of signed distance representations of the training data. The parameters of this representation are then manipulated to minimize an objective function for segmentation. The resulting algorithm is able to handle multidimensional data, can deal with topological changes of the curve, is robust to noise and initial contour placements, and is computationally efficient. At the same time, it avoids the need for point correspondences during the training phase of the algorithm. We demonstrate this technique by applying it to two medical applications; two-dimensional segmentation of cardiac magnetic resonance imaging (MRI) and three-dimensional segmentation of prostate MRI.
---
paper_title: Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation

paper_content:

In this paper, we make two contributions to the field of level set based image segmentation. Firstly, we propose shape dissimilarity measures on the space of level set functions which are analytically invariant under the action of certain transformation groups. The invariance is obtained by an intrinsic registration of the evolving level set function. In contrast to existing approaches to invariance in the level set framework, this closed-form solution removes the need to iteratively optimize explicit pose parameters. The resulting shape gradient is more accurate in that it takes into account the effect of boundary variation on the object's pose. ::: ::: Secondly, based on these invariant shape dissimilarity measures, we propose a statistical shape prior which allows to accurately encode multiple fairly distinct training shapes. This prior constitutes an extension of kernel density estimators to the level set domain. In contrast to the commonly employed Gaussian distribution, such nonparametric density estimators are suited to model aribtrary distributions. ::: ::: We demonstrate the advantages of this multi-modal shape prior applied to the segmentation and tracking of a partially occluded walking person in a video sequence, and on the segmentation of the left ventricle in cardiac ultrasound images. We give quantitative results on segmentation accuracy and on the dependency of segmentation results on the number of training shapes.
---
paper_title: Medial profiles for modeling deformation and statistical analysis of shape and their use in medical image segmentation

paper_content:

We present a novel medial-based, multi-scale approach to shape representation and controlled deformation. We use medial-based profiles for shape representation, which follow the geometry of the structure and describe general, intuitive, and independent shape measures (length, orientation, and thickness). Controlled shape deformations (stretch, bend, and bulge) are obtained either as a result of applying deformation operators at certain locations and scales on the medial profiles, or by varying the weights of the main variation modes obtained from a new hierarchical (multi-scale) and regional (multi-location) principal component analysis of the medial profiles. We demonstrate the ability to produce controlled shape deformations on a medial-based representation of the corpus callosum. We show how this control of shape deformations facilitates the design of a layered framework for image segmentation and present results of segmenting the corpus callosum from 2D mid-sagittal magnetic resonance images of the human brain. Furthermore we show how the medial-based representation facilitates hierarchical, deformation-specific statistical shape analysis of segmented corpora callosa.
---
paper_title: Graph Cuts Segmentation with Statistical Shape Priors for Medical Images

paper_content:

Segmentation of medical images is an important step in many clinical and diagnostic imaging applications. Medical images present many challenges for automated segmentation including poor contrast at tissue boundaries. Traditional segmentation methods based solely on information from the image do not work well in such cases. Statistical shape information for objects in medical images are easy to obtain. In this paper, we propose a graph cuts-based segmentation method for medical images that incorporates statistical shape priors to increase robustness. Our proposed method is able to deal with complex shapes and shape variations while taking advantage of the globally efficient optimization by graph cuts. We demonstrate the effectiveness of our method on kidney images without strong boundaries.
---
paper_title: On the adequacy of principal factor analysis for the study of shape variability

paper_content:

The analysis of shape variability of anatomical structures is of key importance in a number of clinical disciplines, as abnormality in shape can be related to certain diseases. Statistical shape analysis techniques commonly employed in the medical imaging community, such as Active Shape Models or Active Appearance Models rely on Principal Component Analysis (PCA) to decompose shape variability into a reduced set of interpretable components. In this paper we propose Principal Factor Analysis (PFA) as an alternative to PCA and argue that PFA is a better suited technique for medical imaging applications. PFA provides a decomposition into modes of variation that are more easily interpretable, while still being a linear, efficient technique that performs dimensionality reduction (as opposed to Independent Component Analysis, ICA). Both PCA and PFA are described. Examples are provided for 2D landmark data of corpora callosa outlines, as well as vector-valued 3D deformation fields resulting from non-rigid registration of ventricles in MRI. The results show that PFA is a more descriptive tool for shape analysis, at a small cost in size (as in theory more components may be necessary to explain a given percentage of total variance in the data). In conclusion, we argue that it is important to study the potential of factor analysis techniques other than PCA for the application of shape analysis, and defend PFA as a good alternative.
---
paper_title: N.: 3d knowledge-based segmentation using pose-invariant higher-order graphs

paper_content:

Segmentation is a fundamental problem in medical image analysis. The use of prior knowledge is often considered to address the ill-posedness of the process. Such a process consists in bringing all training examples in the same reference pose, and then building statistics. During inference, pose parameters are usually estimated first, and then one seeks a compromise between data-attraction and model-fitness with the prior model. In this paper, we propose a novel higher-order Markov Random Field (MRF) model to encode pose-invariant priors and perform 3D segmentation of challenging data. The approach encodes data support in the singleton terms that are obtained using machine learning, and prior constraints in the higher-order terms. A dual-decomposition-based inference method is used to recover the optimal solution. Promising results on challenging data involving segmentation of tissue classes of the human skeletal muscle demonstrate the potentials of the method.
---
paper_title: Statistical shape influence in geodesic active contours

paper_content:

A novel method of incorporating shape information into the image segmentation process is presented. We introduce a representation for deformable shapes and define a probability distribution over the variances of a set of training shapes. The segmentation process embeds an initial curve as the zero level set of a higher dimensional surface, and evolves the surface such that the zero level set converges on the boundary of the object to be segmented. At each step of the surface evolution, we estimate the maximum a posteriori (MAP) position and shape of the object in the image, based on the prior shape information and the image information. We then evolve the surface globally; towards the MAP estimate, and locally based on image gradients and curvature. Results are demonstrated on synthetic data and medical imagery in 2D min 3D.
---
paper_title: Simultaneous Nonrigid Registration, Segmentation, and Tumor Detection in MRI Guided Cervical Cancer Radiation Therapy

paper_content:

External beam radiation therapy (EBRT) for the treatment of cancer enables accurate placement of radiation dose on the cancerous region. However, the deformation of soft tissue during the course of treatment, such as in cervical cancer, presents significant challenges for the delineation of the target volume and other structures of interest. Furthermore, the presence and regression of pathologies such as tumors may violate registration constraints and cause registration errors. In this paper, automatic segmentation, nonrigid registration and tumor detection in cervical magnetic resonance (MR) data are addressed simultaneously using a unified Bayesian framework. The proposed novel method can generate a tumor probability map while progressively identifying the boundary of an organ of interest based on the achieved nonrigid transformation. The method is able to handle the challenges of significant tumor regression and its effect on surrounding tissues. The new method was compared to various currently existing algorithms on a set of 36 MR data from six patients, each patient has six T2-weighted MR cervical images. The results show that the proposed approach achieves an accuracy comparable to manual segmentation and it significantly outperforms the existing registration algorithms. In addition, the tumor detection result generated by the proposed method has a high agreement with manual delineation by a qualified clinician.
---
paper_title: A Mixture Model for Representing Shape Variation

paper_content:

The shape variation displayed by a class of objects can be represented as probability density function, allowing us to determine plausible and implausible examples of the class. Given a training set of example shapes we can align them into a common co-ordinate frame and use kernel-based density estimation techniques to represent this distribution. Such an estimate is complex and expensive, so we generate a simpler approximation using a mixture of gaussians. We show how to calculate the distribution, and how it can be used in image search to locate examples of the modelled object in new images.
---
paper_title: The Isometric Log-Ratio Transform for Probabilistic Multi-Label Anatomical Shape Representation

paper_content:

Sources of uncertainty in the boundaries of structures in medical images have motivated the use of probabilistic labels in segmentation applications. An important component in many medical image segmentation tasks is the use of a shape model, often generated by applying statistical techniques to training data. Standard statistical techniques (e.g., principal component analysis) often assume data lies in an unconstrained vector space, but probabilistic labels are constrained to the unit simplex. If these statistical techniques are used directly on probabilistic labels, relative uncertainty information can be sacrificed. A standard method for facilitating analysis of probabilistic labels is to map them to a vector space using the LogOdds transform. However, the LogOdds transform is asymmetric in one of the labels, which skews results in some applications. The isometric log-ratio (ILR) transform is a symmetrized version of the LogOdds transform, and is so named as it is an isometry between the Aitchison geometry, the inherent geometry of the simplex, and standard Euclidean geometry. We explore how to interpret the Aitchison geometry when applied to probabilistic labels in medical image segmentation applications. We demonstrate the differences when applying the LogOdds transform or the ILR transform to probabilistic labels prior to statistical analysis. Specifically, we show that statistical analysis of ILR transformed data better captures the variability of anatomical shapes in cases where multiple different foreground regions share boundaries (as opposed to foreground-background boundaries).
---
paper_title: Active shape model segmentation with optimal features

paper_content:

An active shape model segmentation scheme is presented that is steered by optimal local features, contrary to normalized first order derivative profiles, as in the original formulation [Cootes and Taylor, 1995, 1999, and 2001]. A nonlinear kNN-classifier is used, instead of the linear Mahalanobis distance, to find optimal displacements for landmarks. For each of the landmarks that describe the shape, at each resolution level taken into account during the segmentation optimization procedure, a distinct set of optimal features is determined. The selection of features is automatic, using the training images and sequential feature forward and backward selection. The new approach is tested on synthetic data and in four medical segmentation tasks: segmenting the right and left lung fields in a database of 230 chest radiographs, and segmenting the cerebellum and corpus callosum in a database of 90 slices from MRI brain images. In all cases, the new method produces significantly better results in terms of an overlap error measure (p<0.001 using a paired T-test) than the original active shape model scheme.
---
paper_title: Deformable segmentation via sparse representation and dictionary learning

paper_content:

"Shape" and "appearance", the two pillars of a deformable model, complement each other in object segmentation. In many medical imaging applications, while the low-level appearance information is weak or mis-leading, shape priors play a more important role to guide a correct segmentation, thanks to the strong shape characteristics of biological structures. Recently a novel shape prior modeling method has been proposed based on sparse learning theory. Instead of learning a generative shape model, shape priors are incorporated on-the-fly through the sparse shape composition (SSC). SSC is robust to non-Gaussian errors and still preserves individual shape characteristics even when such characteristics is not statistically significant. Although it seems straightforward to incorporate SSC into a deformable segmentation framework as shape priors, the large-scale sparse optimization of SSC has low runtime efficiency, which cannot satisfy clinical requirements. In this paper, we design two strategies to decrease the computational complexity of SSC, making a robust, accurate and efficient deformable segmentation system. (1) When the shape repository contains a large number of instances, which is often the case in 2D problems, K-SVD is used to learn a more compact but still informative shape dictionary. (2) If the derived shape instance has a large number of vertices, which often appears in 3D problems, an affinity propagation method is used to partition the surface into small sub-regions, on which the sparse shape composition is performed locally. Both strategies dramatically decrease the scale of the sparse optimization problem and hence speed up the algorithm. Our method is applied on a diverse set of biomedical image analysis problems. Compared to the original SSC, these two newly-proposed modules not only significant reduce the computational complexity, but also improve the overall accuracy.
---
paper_title: Nonlinear Component Analysis as a Kernel Eigenvalue Problem

paper_content:

A new method for performing a nonlinear form of principal component analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in high-dimensional feature spaces, related to input space by some nonlinear map—for instance, the space of all possible five-pixel products in 16 × 16 images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition.
---
paper_title: Deformable Fourier models for surface finding in 3-D images

paper_content:

This paper describes a new global shape parametrization for smoothly deformable three- dimensional objects, such as those found in biomedical images, whose diversity and irregularity make them difficult to represent in terms of fixed features or parts. This representation is used for geometric surface matching to three-dimensional image data. The parametrization decomposes the surface into sinusoidal basis functions. Four types of surfaces are modeled: tori, open surfaces, closed surfaces, and tubes. This parametrization allows a wide variety of smooth surfaces to be described with a small number of parameters. Surface finding is formulated as an optimization problem. Results of the method applied to synthetic and medical three-dimensional images are presented.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Hierarchical active shape models, using the wavelet transform

paper_content:

Active shape models (ASMs) are often limited by the inability of relatively few eigenvectors to capture the full range of biological shape variability. This paper presents a method that overcomes this limitation, by using a hierarchical formulation of active shape models, using the wavelet transform. The statistical properties of the wavelet transform of a deformable contour are analyzed via principal component analysis, and used as priors in the contour's deformation. Some of these priors reflect relatively global shape characteristics of the object boundaries, whereas, some of them capture local and high-frequency shape characteristics and, thus, serve as local smoothness constraints. This formulation achieves two objectives. First, it is robust when only a limited number of training samples is available. Second, by using local statistics as smoothness constraints, it eliminates the need for adopting ad hoc physical models, such as elasticity or other smoothness models, which do not necessarily reflect true biological variability. Examples on magnetic resonance images of the corpus callosum and hand contours demonstrate that good and fully automated segmentations can be achieved, even with as few as five training samples.
---
paper_title: Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation

paper_content:

The notion of using context information for solving high-level vision and medical image segmentation problems has been increasingly realized in the field. However, how to learn an effective and efficient context model, together with an image appearance model, remains mostly unknown. The current literature using Markov Random Fields (MRFs) and Conditional Random Fields (CRFs) often involves specific algorithm design in which the modeling and computing stages are studied in isolation. In this paper, we propose a learning algorithm, auto-context. Given a set of training images and their corresponding label maps, we first learn a classifier on local image patches. The discriminative probability (or classification confidence) maps created by the learned classifier are then used as context information, in addition to the original image patches, to train a new classifier. The algorithm then iterates until convergence. Auto-context integrates low-level and context information by fusing a large number of low-level appearance features with context and implicit shape information. The resulting discriminative algorithm is general and easy to implement. Under nearly the same parameter settings in training, we apply the algorithm to three challenging vision applications: foreground/background segregation, human body configuration estimation, and scene region labeling. Moreover, context also plays a very important role in medical/brain images where the anatomical structures are mostly constrained to relatively fixed positions. With only some slight changes resulting from using 3D instead of 2D features, the auto-context algorithm applied to brain MRI image segmentation is shown to outperform state-of-the-art algorithms specifically designed for this domain. Furthermore, the scope of the proposed algorithm goes beyond image analysis and it has the potential to be used for a wide variety of problems for structured prediction problems.
---
paper_title: Model-based curve evolution technique for image segmentation

paper_content:

We propose a model-based curve evolution technique for segmentation of images containing known object types. In particular, motivated by the work of Leventon et al. (2000), we derive a parametric model for an implicit representation of the segmenting curve by applying principal component analysis to a collection of signed distance representations of the training data, The parameters of this representation are then calculated to minimize an objective function for segmentation. We found the resulting algorithm to be computationally efficient, able to handle multidimensional data, robust to noise and initial contour placements, while at the same time, avoiding the need for point correspondences during the training phase of the algorithm. We demonstrate this technique by applying it to two medical applications.
---
paper_title: Statistically constrained snake deformations

paper_content:

The authors present a method for constraining the deformations of Snakes (active contour models) when segmenting a known class of objects. The method we propose is similar to both active shape models (ASM) but without the landmark identification and correspondence requirement, and to active contour models (ACM), but armed with a priori information about shape variation. Rather than representing the object boundary by spatial landmarks in a point-by-point fashion, we employ a frequency based boundary representation. In this way, the principal component analysis (PCA), which is central to ASM, is applied to a set of frequency-domain shape descriptors, removing the need for the difficult determination of spatial landmarks. Given a training set of representative images of the object of interest, we extract an average object shape along with a set of significant shape variation modes, explaining most of the shape variation in the training set. Armed with this a priori model of shape variation, we find the boundaries in unknown images by placing an initial ACM and allowing it to deform only according to the examined shape variations. The described methodology was applied to a set of 105 echocardiographic images for locating the left ventricular boundary. The results were particularly encouraging in clinically difficult cases where the ventricular boundary was partly occluded by noise.
---
paper_title: Simulation of Ground-Truth Validation Data Via Physically- and Statistically-Based Warps

paper_content:

The problem of scarcity of ground-truth expert delineations of medical image data is a serious one that impedes the training and validation of medical image analysis techniques. We develop an algorithm for the automatic generation of large databases of annotated images from a single reference dataset. We provide a web-based interface through which the users can upload a reference data set (an image and its corresponding segmentation and landmark points), provide custom setting of parameters, and, following server-side computations, generate and download an arbitrary number of novel ground-truth data, including segmentations, displacement vector fields, intensity non-uniformity maps, and point correspondences. To produce realistic simulated data, we use variational (statistically-based) and vibrational (physically-based) spatial deformations, nonlinear radiometric warps mimicking imaging nonhomogeneity, and additive random noise with different underlying distributions. We outline the algorithmic details, present sample results, and provide the web address to readers for immediate evaluation and usage.
---
paper_title: Globally Optimal Image Segmentation with an Elastic Shape Prior

paper_content:

So far global optimization techniques have been developed independently for the tasks of shape matching and image segmentation. In this paper we show that both tasks can in fact be solved simultaneously using global optimization. By computing cycles of minimal ratio in a large graph spanned by the product of the input image and a shape template, we are able to compute globally optimal segmentations of the image which are similar to a familiar shape and located in places of strong gradient. The presented approach is translation-invariant and robust to local and global scaling and rotation of the given shape. We show how it can be extended to incorporate invariance to similarity transformations. The particular structure of the graph allows for run-time and memory efficient implementations. Highly parallel implementations on graphics cards allow to produce globally optimal solutions in a few seconds only.
---
paper_title: Non-Rigid Motion Analysis in Medical Images: a Physically Based Approach

paper_content:

We present a physically-based deformable model which can be used to track and to analyze non-rigid motion of dynamic structures in time sequences of 2D or 3D medical images. The model considers an object undergoing an elastic deformation as a set of masses linked by springs, where the classical natural lengths of the springs is set equal to zero, and is replaced by a set of constant equilibrium forces, which characterize the shape of the elastic structure in the absence of external forces. This model has the extremely nice property of yielding dynamic equations which are linear and decoupled for each coordinate, whatever the amplitude of the deformation. Compared to the former work of Terzopoulos and his colleagues [12, 27, 26, 15] and Pentland and his colleagues [22, 21, 23, 10], our model can be viewed as a continuation and unification; it provides a reduced algorithmic complexity, and a sound framework for modal analysis, which allows a compact representation of a general deformation by a reduced number of parameters. The power of the approach to segment, track and analyze 2-D and 3-D images is demonstrated by a set of experimental results on various complex medical images (ultrasound and magnetic resonance images).
---
paper_title: Closed-form solutions for physically based shape modeling and recognition

paper_content:

The authors present a closed-form, physically based solution for recovering a three-dimensional (3-D) solid model from collections of 3-D surface measurements. Given a sufficient number of independent measurements, the solution is overconstrained and unique except for rotational symmetries. The proposed approach is based on the finite element method (FEM) and parametric solid modeling using implicit functions. This approach provides both the convenience of parametric modeling and the expressiveness of the physically based mesh formulation and, in addition, can provide great accuracy at physical simulation. A physically based object-recognition method that allows simple, closed-form comparisons of recovered 3-D solid models is presented. The performance of these methods is evaluated using both synthetic range data with various signal-to-noise ratios and using laser rangefinder data. >
---
paper_title: A Finite Element Method for Deformable Models

paper_content:

Deformable models of elastic structures have been proposed for use in image analysis. Previous work has used a variational approach, based on the EulerLagrange theory. In this paper an alternative mathematical treatment is introduced, based on a direct minimisation of the underlying energy integral using the Finite Element Method. The method is outlined and demonstrated, and its principal advantages for modelbased image interpretation are explained.
---
paper_title: Snakes: Active contour models

paper_content:

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.
---
paper_title: Graph cut based image segmentation with connectivity priors

paper_content:

Graph cut is a popular technique for interactive image segmentation. However, it has certain shortcomings. In particular, graph cut has problems with segmenting thin elongated objects due to the ldquoshrinking biasrdquo. To overcome this problem, we propose to impose an additional connectivity prior, which is a very natural assumption about objects. We formulate several versions of the connectivity constraint and show that the corresponding optimization problems are all NP-hard. For some of these versions we propose two optimization algorithms: (i) a practical heuristic technique which we call DijkstraGC, and (ii) a slow method based on problem decomposition which provides a lower bound on the problem. We use the second technique to verify that for some practical examples DijkstraGC is able to find the global minimum.
---
paper_title: Topology Cuts: A Novel Min-Cut/Max-Flow Algorithm for Topology Preserving Segmentation in N-D Images

paper_content:

Topology is an important prior in many image segmentation tasks. In this paper, we design and implement a novel graph-based min-cut/max-flow algorithm that incorporates topology priors as global constraints. We show that the optimization of the energy function we consider here is NP-hard. However, our algorithm is guaranteed to find an approximate solution that conforms to the initialization, which is a desirable property in many applications since the globally optimum solution does not consider any initialization information. The key innovation of our algorithm is the organization of the search for maximum flow in a way that allows consideration of topology constraints. In order to achieve this, we introduce a label attribute for each node to explicitly handle the topology constraints, and we use a distance map to keep track of those nodes that are closest to the boundary. We employ the bucket priority queue data structure that records nodes of equal distance and we efficiently extract the node with minimal distance value. Our methodology of embedding distance functions in a graph-based algorithm is general and can also account for other geometric priors. Experimental results show that our algorithm can efficiently handle segmentation cases that are challenging for graph-cut algorithms. Furthermore, our algorithm is a natural choice for problems with rich topology priors such as object tracking.
---
paper_title: Simple points, topological numbers and geodesic neighborhoods in cubic grids

paper_content:

Abstract We introduce the notion of geodesic neighborhood in order to define some topological numbers associated with a point in a three-dimensional cubic grid. For {6, 26} and {6, 18} connectivities, these numbers lead to a characterization of simple points which consists in only two local conditions.
---
paper_title: A topology preserving level set method for geometric deformable models

paper_content:

Active contour and surface models, also known as deformable models, are powerful image segmentation techniques. Geometric deformable models implemented using level set methods have advantages over parametric models due to their intrinsic behavior, parameterization independence, and ease of implementation. However, a long claimed advantage of geometric deformable models-the ability to automatically handle topology changes-turns out to be a liability in applications where the object to be segmented has a known topology that must be preserved. We present a new class of geometric deformable models designed using a novel topology-preserving level set method, which achieves topology preservation by applying the simple point concept from digital topology. These new models maintain the other advantages of standard geometric deformable models including subpixel accuracy and production of nonintersecting curves or surfaces. Moreover, since the topology-preserving constraint is enforced efficiently through local computations, the resulting algorithm incurs only nominal computational overhead over standard geometric deformable models. Several experiments on simulated and real data are provided to demonstrate the performance of this new deformable model algorithm.
---
paper_title: A convex framework for image segmentation with moment constraints

paper_content:

Convex relaxation techniques have become a popular approach to image segmentation as they allow to compute solutions independent of initialization to a variety of image segmentation problems. In this paper, we will show that shape priors in terms of moment constraints can be imposed within the convex optimization framework, since they give rise to convex constraints. In particular, the lower-order moments correspond to the overall volume, the centroid, and the variance or covariance of the shape and can be easily imposed in interactive segmentation methods. Respective constraints can be imposed as hard constraints or soft constraints. Quantitative segmentation studies on a variety of images demonstrate that the user can easily impose such constraints with a few mouse clicks, giving rise to substantial improvements of the resulting segmentation, and reducing the average segmentation error from 12% to 0:35%. GPU-based computation times of around 1 second allow for interactive segmentation.
---
paper_title: Affine-invariant geometric shape priors for region-based active contours

paper_content:

We present a new way of constraining the evolution of a region-based active contour with respect to a reference shape. Minimizing a shape prior, defined as a distance between shape descriptors based on the Legendre moments of the characteristic function, leads to a geometric flow that can be used with benefits in a two-class segmentation application. The shape model includes intrinsic invariance with regard to pose and affine deformations
---
paper_title: Area prior constrained level set evolution for medical image segmentation

paper_content:

The level set framework has proven well suited to medical image segmentation 1-6 thanks to its ability of balancing ::: the contribution of image data and prior knowledge in a principled, flexible and transparent way. It consists ::: of evolving a curve toward the target object boundaries. The curve evolution equation is sought following ::: the optimization of a cost functional containing two types of terms: data terms, which measure the fidelity of ::: segmentation to image intensities, and prior terms, which traduce learned prior knowledge. Without priors many ::: algorithms are likely to fail due to high noise, low contrast and data incompleteness. Different priors have been ::: investigated such as shape 1 and appearance priors. 7 In this study, we propose a simple type of priors: the area ::: prior. This prior embeds knowledge of an approximate object area and has two positive effects. First, It speeds ::: up significantly the evolution when the curve is far from the target object boundaries. Second, it slows down ::: the evolution when the curve is close to the target. Consequently, it reinforces curve stability at the desired ::: boundaries when dealing with low contrast intensity edges. The algorithm is validated with several experiments ::: using Magnetic Resonance (MR) images and Computed Tomography (CT) images. A comparison with another ::: level set method illustrates the positive effects of the area prior.
---
paper_title: Left ventricle segmentation in MRI via convex relaxed distribution matching

paper_content:

A fundamental step in the diagnosis of cardiovascular diseases, automatic left ventricle (LV) segmentation in cardiac magnetic resonance images (MRIs) is still acknowledged to be a difficult problem. Most of the existing algorithms require either extensive training or intensive user inputs. This study investigates fast detection of the left ventricle (LV) endo- and epicardium surfaces in cardiac MRI via convex relaxation and distribution matching. The algorithm requires a single subject for training and a very simple user input, which amounts to a single point (mouse click) per target region (cavity or myocardium). It seeks cavity and myocardium regions within each 3D phase by optimizing two functionals, each containing two distribution-matching constraints: (1) a distance-based shape prior and (2) an intensity prior. Based on a global measure of similarity between distributions, the shape prior is intrinsically invariant with respect to translation and rotation. We further introduce a scale variable from which we derive a fixed-point equation (FPE), thereby achieving scale-invariance with only few fast computations. The proposed algorithm relaxes the need for costly pose estimation (or registration) procedures and large training sets, and can tolerate shape deformations, unlike template (or atlas) based priors. Our formulation leads to a challenging problem, which is not directly amenable to convex-optimization techniques. For each functional, we split the problem into a sequence of sub-problems, each of which can be solved exactly and globally via a convex relaxation and the augmented Lagrangian method. Unlike related graph-cut approaches, the proposed convex-relaxation solution can be parallelized to reduce substantially the computational time for 3D domains (or higher), extends directly to high dimensions, and does not have the grid-bias problem. Our parallelized implementation on a graphics processing unit (GPU) demonstrates that the proposed algorithm requires about 3.87 s for a typical cardiac MRI volume, a speed-up of about five times compared to a standard implementation. We report a performance evaluation over 400 volumes acquired from 20 subjects, which shows that the obtained 3D surfaces correlate with independent manual delineations. We further demonstrate experimentally that (1) the performance of the algorithm is not significantly affected by the choice of the training subject and (2) the shape description we use does not change significantly from one subject to another. These results support the fact that a single subject is sufficient for training the proposed algorithm.
---
paper_title: Region Detection by Minimizing Intraclass Variance With Geometric Constraints, Global Optimality, and Efficient Approximation

paper_content:

Efficient segmentation of globally optimal surfaces in volumetric images is a central problem in many medical image analysis applications. Intraclass variance has been successfully utilized for object segmentation, for instance, in the Chan-Vese model, especially for images without prominent edges. In this paper, we study the optimization problem of detecting a region (volume) between two coupled smooth surfaces by minimizing the intraclass variance using an efficient polynomial-time algorithm. Our algorithm is based on the shape probing technique in computational geometry and computes a sequence of minimum-cost closed sets in a derived parametric graph. The method has been validated on computer-synthetic volumetric images and in X-ray CT-scanned datasets of plexiglas tubes of known sizes. Its applicability to clinical data sets was also demonstrated. In all cases, the approach yielded highly accurate results. We believe that the developed technique is of interest on its own. We expect that it can shed some light on solving other important optimization problems arising in medical imaging. Furthermore, we report an approximation algorithm which runs much faster than the exact algorithm while yielding highly comparable segmentation accuracy.
---
paper_title: Volumetric layer segmentation using coupled surfaces propagation

paper_content:

The problem of segmenting a volumetric layer of finite thickness is encountered in several important areas within medical image analysis. Key examples include the extraction of the cortical gray matter of the brain and the left ventricle myocardium of the heart. The coupling between the two bounding surfaces of such a layer provides important information that helps to solve the segmentation problem. Here we propose a new approach of coupled surfaces propagation via level set methods, which takes into account coupling as an important constraint. By evolving two embedded surfaces simultaneously, each driven by its own image-derived information while maintaining the coupling, we capture a representation of the two bounding surfaces and achieve automatic segmentation on the layer. Characteristic gray level values, instead of image gradient information alone, are incorporated in deriving the useful image information to drive the surface propagation, which enables our approach to capture the homogeneity inside the layer. The level set implementation offers the advantage of easy initialization, computational efficiency and the ability to capture deep folds of the sulci. As a test example, we apply our approach to unedited 3D Magnetic Resonance (MR) brain images. Our algorithm automatically isolates the brain from non-brain structures and recovers the cortical gray matter.
---
paper_title: Optimal Surface Segmentation in Volumetric Images-A Graph-Theoretic Approach

paper_content:

Efficient segmentation of globally optimal surfaces representing object boundaries in volumetric data sets is important and challenging in many medical image analysis applications. We have developed an optimal surface detection method capable of simultaneously detecting multiple interacting surfaces, in which the optimality is controlled by the cost functions designed for individual surfaces and by several geometric constraints defining the surface smoothness and interrelations. The method solves the surface segmentation problem by transforming it into computing a minimum s{\hbox{-}} t cut in a derived arc-weighted directed graph. The proposed algorithm has a low-order polynomial time complexity and is computationally efficient. It has been extensively validated on more than 300 computer-synthetic volumetric images, 72 CT-scanned data sets of different-sized plexiglas tubes, and tens of medical images spanning various imaging modalities. In all cases, the approach yielded highly accurate results. Our approach can be readily extended to higher-dimensional image segmentation.
---
paper_title: Fuzzy spatial relationships for image processing and interpretation: a review

paper_content:

In spatial reasoning, relationships between spatial entities play a major role. In image interpretation, computer vision and structural recognition, the management of imperfect information and of imprecision constitutes a key point. This calls for the framework of fuzzy sets, which exhibits nice features to represent spatial imprecision at different levels, imprecision in knowledge and knowledge representation, and which provides powerful tools for fusion, decision-making and reasoning. In this paper, we review the main fuzzy approaches for defining spatial relationships including topological (set relationships, adjacency) and metrical relations (distances, directional relative position).
---
paper_title: Globally optimal segmentation of multi-region objects

paper_content:

Many objects contain spatially distinct regions, each with a unique colour/texture model. Mixture models ignore the spatial distribution of colours within an object, and thus cannot distinguish between coherent parts versus randomly distributed colours. We show how to encode geometric interactions between distinct region+boundary models, such as regions being interior/exterior to each other along with preferred distances between their boundaries. With a single graph cut, our method extracts only those multi-region objects that satisfy such a combined model. We show applications in medical segmentation and scene layout estimation. Unlike Li et al. [17] we do not need “domain unwrapping” nor do we have topological limits on shapes.
---
paper_title: A fast convex optimization approach to segmenting 3d scar tissue from delayed-enhancement cardiac MR images

paper_content:

We propose a novel multi-region segmentation approach through a partially-ordered Potts (POP) model to segment myocardial scar tissue solely from 3D cardiac delayed-enhancement MR images (DE-MRI). The algorithm makes use of prior knowledge of anatomical spatial consistency and employs customized label ordering to constrain the segmentation without prior knowledge of geometric representation. The proposed method eliminates the need for regional constraint segmentations, thus reduces processing time and potential sources of error. We solve the proposed optimization problem by means of convex relaxation and introduce its duality: the hierarchical continuous max-flow (HMF) model, which amounts to an efficient numerical solver to the resulting convex optimization problem. Experiments are performed over ten DE-MRI data sets. The results are compared to a FWHM (full-width at half-maximum) method and the inter- and intra-operator variabilities assessed.
---
paper_title: A Variational Approach for the Segmentation of the Left Ventricle in Cardiac Image Analysis

paper_content:

In this paper we propose a level set method to segment MR cardiac images. Our approach is based on a coupled propagation of two cardiac contours and integrates visual information with anatomical constraints. The visual information is expressed through a gradient vector flow-based boundary component and a region term that aims at best separating the cardiac contours/regions according to their global intensity properties. In order to deal with misleading visual support, an anatomical constraint is considered that couples the propagation of the cardiac contours according to their relative distance. The resulting motion equations are implemented using a level set approach and a fast and stable numerical approximation scheme, the Additive Operator Splitting. Encouraging experimental results are provided using real data.
---
paper_title: Efficient Global Optimization Based 3D Carotid AB-LIB MRI Segmentation by Simultaneously Evolving Coupled Surfaces

paper_content:

Magnetic resonance (MR) imaging of carotid atherosclerosis biomarkers are increasingly being investigated for the risk assessment of vulnerable plaques. A fast and robust 3D segmentation of the carotid adventitia (AB) and lumen-intima (LIB) boundaries can greatly alleviate the measurement burden of generating quantitative imaging biomarkers in clinical research. In this paper, we propose a novel global optimization-based approach to segment the carotid AB and LIB from 3D T1-weighted black blood MR images, by simultaneously evolving two coupled surfaces with enforcement of anatomical consistency of the AB and LIB. We show that the evolution of two surfaces at each discrete time-frame can be optimized exactly and globally by means of convex relaxation. Our continuous max-flow based algorithm is implemented in GPUs to achieve high computational performance. The experiment results from 16 carotid MR images show that the algorithm obtained high agreement with manual segmentations and achieved high repeatability in segmentation.
---
paper_title: Optimizing Binary MRFs via Extended Roof Duality

paper_content:

Many computer vision applications rely on the efficient optimization of challenging, so-called non-submodular, binary pairwise MRFs. A promising graph cut based approach for optimizing such MRFs known as "roof duality" was recently introduced into computer vision. We study two methods which extend this approach. First, we discuss an efficient implementation of the "probing" technique introduced recently by Bows et al. (2006). It simplifies the MRF while preserving the global optimum. Our code is 400-700 faster on some graphs than the implementation of the work of Bows et al. (2006). Second, we present a new technique which takes an arbitrary input labeling and tries to improve its energy. We give theoretical characterizations of local minima of this procedure. We applied both techniques to many applications, including image segmentation, new view synthesis, super-resolution, diagram recognition, parameter learning, texture restoration, and image deconvolution. For several applications we see that we are able to find the global minimum very efficiently, and considerably outperform the original roof duality approach. In comparison to existing techniques, such as graph cut, TRW, BP, ICM, and simulated annealing, we nearly always find a lower energy.
---
paper_title: Cortex segmentation - a fast variational geometric approach

paper_content:

An automatic cortical gray matter segmentation from three-dimensional brain images (MR or CT) is a well known problem in medical image processing. We formulate it as a geometric variational problem for propagation of two coupled bounding surfaces. An efficient numerical scheme is used to implement the geodesic active surface model. Experimental results of cortex segmentation on real three-dimensional MR data are provided.
---
paper_title: Multiphase geometric couplings for the segmentation of neural processes

paper_content:

The ability to constrain the geometry of deformable models for image segmentation can be useful when information about the expected shape or positioning of the objects in a scene is known a priori. An example of this occurs when segmenting neural cross sections in electron microscopy. Such images often contain multiple nested boundaries separating regions of homogeneous intensities. For these applications, multiphase level sets provide a partitioning framework that allows for the segmentation of multiple deformable objects by combining several level set functions. Although there has been much effort in the study of statistical shape priors that can be used to constrain the geometry of each partition, none of these methods allow for the direct modeling of geometric arrangements of partitions. In this paper, we show how to define elastic couplings between multiple level set functions to model ribbon-like partitions. We build such couplings using dynamic force fields that can depend on the image content and relative location and shape of the level set functions. To the best of our knowledge, this is the first work that shows a direct way of geometrically constraining multiphase level sets for image segmentation. We demonstrate the robustness of our method by comparing it with previous level set segmentation methods.
---
paper_title: Fast approximate energy minimization via graph cuts

paper_content:

In this paper we address the problem of minimizing a large class of energy functions that occur in early vision. The major restriction is that the energy function's smoothness term must only involve pairs of pixels. We propose two algorithms that use graph cuts to compute a local minimum even when very large moves are allowed. The first move we consider is an /spl alpha/-/spl beta/-swap: for a pair of labels /spl alpha/,/spl beta/, this move exchanges the labels between an arbitrary set of pixels labeled a and another arbitrary set labeled /spl beta/. Our first algorithm generates a labeling such that there is no swap move that decreases the energy. The second move we consider is an /spl alpha/-expansion: for a label a, this move assigns an arbitrary set of pixels the label /spl alpha/. Our second algorithm, which requires the smoothness term to be a metric, generates a labeling such that there is no expansion move that decreases the energy. Moreover, this solution is within a known factor of the global minimum. We experimentally demonstrate the effectiveness of our approach on image restoration, stereo and motion.
---
paper_title: An Efficient Optimization Framework for Multi-Region Segmentation Based on Lagrangian Duality

paper_content:

We introduce a multi-region model for simultaneous segmentation of medical images. In contrast to many other models, geometric constraints such as inclusion and exclusion between the regions are enforced, which makes it possible to correctly segment different regions even if the intensity distributions are identical. We efficiently optimize the model using a combination of graph cuts and Lagrangian duality which is faster and more memory efficient than current state of the art. As the method is based on global optimization techniques, the resulting segmentations are independent of initialization. We apply our framework to the segmentation of the left and right ventricles, myocardium and the left ventricular papillary muscles in magnetic resonance imaging and to lung segmentation in full-body X-ray computed tomography. We evaluate our approach on a publicly available benchmark with competitive results.
---
paper_title: Region Detection by Minimizing Intraclass Variance With Geometric Constraints, Global Optimality, and Efficient Approximation

paper_content:

Efficient segmentation of globally optimal surfaces in volumetric images is a central problem in many medical image analysis applications. Intraclass variance has been successfully utilized for object segmentation, for instance, in the Chan-Vese model, especially for images without prominent edges. In this paper, we study the optimization problem of detecting a region (volume) between two coupled smooth surfaces by minimizing the intraclass variance using an efficient polynomial-time algorithm. Our algorithm is based on the shape probing technique in computational geometry and computes a sequence of minimum-cost closed sets in a derived parametric graph. The method has been validated on computer-synthetic volumetric images and in X-ray CT-scanned datasets of plexiglas tubes of known sizes. Its applicability to clinical data sets was also demonstrated. In all cases, the approach yielded highly accurate results. We believe that the developed technique is of interest on its own. We expect that it can shed some light on solving other important optimization problems arising in medical imaging. Furthermore, we report an approximation algorithm which runs much faster than the exact algorithm while yielding highly comparable segmentation accuracy.
---
paper_title: Volumetric layer segmentation using coupled surfaces propagation

paper_content:

The problem of segmenting a volumetric layer of finite thickness is encountered in several important areas within medical image analysis. Key examples include the extraction of the cortical gray matter of the brain and the left ventricle myocardium of the heart. The coupling between the two bounding surfaces of such a layer provides important information that helps to solve the segmentation problem. Here we propose a new approach of coupled surfaces propagation via level set methods, which takes into account coupling as an important constraint. By evolving two embedded surfaces simultaneously, each driven by its own image-derived information while maintaining the coupling, we capture a representation of the two bounding surfaces and achieve automatic segmentation on the layer. Characteristic gray level values, instead of image gradient information alone, are incorporated in deriving the useful image information to drive the surface propagation, which enables our approach to capture the homogeneity inside the layer. The level set implementation offers the advantage of easy initialization, computational efficiency and the ability to capture deep folds of the sulci. As a test example, we apply our approach to unedited 3D Magnetic Resonance (MR) brain images. Our algorithm automatically isolates the brain from non-brain structures and recovers the cortical gray matter.
---
paper_title: Optimal Surface Segmentation in Volumetric Images-A Graph-Theoretic Approach

paper_content:

Efficient segmentation of globally optimal surfaces representing object boundaries in volumetric data sets is important and challenging in many medical image analysis applications. We have developed an optimal surface detection method capable of simultaneously detecting multiple interacting surfaces, in which the optimality is controlled by the cost functions designed for individual surfaces and by several geometric constraints defining the surface smoothness and interrelations. The method solves the surface segmentation problem by transforming it into computing a minimum s{\hbox{-}} t cut in a derived arc-weighted directed graph. The proposed algorithm has a low-order polynomial time complexity and is computationally efficient. It has been extensively validated on more than 300 computer-synthetic volumetric images, 72 CT-scanned data sets of different-sized plexiglas tubes, and tens of medical images spanning various imaging modalities. In all cases, the approach yielded highly accurate results. Our approach can be readily extended to higher-dimensional image segmentation.
---
paper_title: Globally optimal segmentation of multi-region objects

paper_content:

Many objects contain spatially distinct regions, each with a unique colour/texture model. Mixture models ignore the spatial distribution of colours within an object, and thus cannot distinguish between coherent parts versus randomly distributed colours. We show how to encode geometric interactions between distinct region+boundary models, such as regions being interior/exterior to each other along with preferred distances between their boundaries. With a single graph cut, our method extracts only those multi-region objects that satisfy such a combined model. We show applications in medical segmentation and scene layout estimation. Unlike Li et al. [17] we do not need “domain unwrapping” nor do we have topological limits on shapes.
---
paper_title: A Variational Approach for the Segmentation of the Left Ventricle in Cardiac Image Analysis

paper_content:

In this paper we propose a level set method to segment MR cardiac images. Our approach is based on a coupled propagation of two cardiac contours and integrates visual information with anatomical constraints. The visual information is expressed through a gradient vector flow-based boundary component and a region term that aims at best separating the cardiac contours/regions according to their global intensity properties. In order to deal with misleading visual support, an anatomical constraint is considered that couples the propagation of the cardiac contours according to their relative distance. The resulting motion equations are implemented using a level set approach and a fast and stable numerical approximation scheme, the Additive Operator Splitting. Encouraging experimental results are provided using real data.
---
paper_title: Cortex segmentation - a fast variational geometric approach

paper_content:

An automatic cortical gray matter segmentation from three-dimensional brain images (MR or CT) is a well known problem in medical image processing. We formulate it as a geometric variational problem for propagation of two coupled bounding surfaces. An efficient numerical scheme is used to implement the geodesic active surface model. Experimental results of cortex segmentation on real three-dimensional MR data are provided.
---
paper_title: Multiphase geometric couplings for the segmentation of neural processes

paper_content:

The ability to constrain the geometry of deformable models for image segmentation can be useful when information about the expected shape or positioning of the objects in a scene is known a priori. An example of this occurs when segmenting neural cross sections in electron microscopy. Such images often contain multiple nested boundaries separating regions of homogeneous intensities. For these applications, multiphase level sets provide a partitioning framework that allows for the segmentation of multiple deformable objects by combining several level set functions. Although there has been much effort in the study of statistical shape priors that can be used to constrain the geometry of each partition, none of these methods allow for the direct modeling of geometric arrangements of partitions. In this paper, we show how to define elastic couplings between multiple level set functions to model ribbon-like partitions. We build such couplings using dynamic force fields that can depend on the image content and relative location and shape of the level set functions. To the best of our knowledge, this is the first work that shows a direct way of geometrically constraining multiphase level sets for image segmentation. We demonstrate the robustness of our method by comparing it with previous level set segmentation methods.
---
paper_title: Deformable Organisms and Error Learning for Brain Segmentation

paper_content:

Segmentation methods for medical images may not generalize well to different data sets or tasks, hampering their utility. We attempt to remedy these issues using deformable organisms to create an easily customizable segmentation plan. This plan is developed by borrowing ideas from artificial life to govern a set of deformable models that use control processes such as sensing, proactive planning, reactive behavior, and knowledge representation to segment an image. The image may have landmarks and features specific to that dataset; these may be easily incorporated into the plan. We validate this framework by creating a plan to locate the brain in 3D magnetic resonance images of the head (skull-stripping). This is important for surgical planning, understanding how diseases affect the brain, conducting longitudinal studies, registering brain data, and creating cortical surface models. Our plan dictates how deformable organisms find features in head images and cooperatively work to segment the brain. In addition, we use a method based on Adaboost to learn and correct errors in our segmentation. We tested our method on 630 T1-weighted images from healthy young adults, evaluating results using distance and overlap error metrics based on expert gold standard segmentations. We compare our segmentations with and without the error correction step; we also compare our results to three other widely used methods: BSE, BET, and the Hybrid Watershed algorithm. Our method had the least Hausdorff distance to expert segmentations on this dataset, but included slightly more non-brain voxels (false positives). Our framework captures diverse categories of information needed for skull-stripping, and produces competitive segmentations.
---
paper_title: Morphological Proximity Priors: Spatial Relationships for Semantic Segmentation

paper_content:

The introduction of prior knowledge into image analysis algorithms is a central challenge in computer vision. In this paper, we introduce the concept of proximity priors into semantic segmentation methods in order to penalize the proximity of certain object classes. Proximity priors are a generalization of purely global and purely local co-occurrence priors which have been introduced recently. The key idea is to consider pixels as adjacent if they are within a specified neighborhood of arbitrary size and shape. Respective penalties for the adjacency of various label pairs (the labels ’sheep’ and ’lion’ for example) can be learned statistically from a set of segmented images. We propose a variational approach which integrates morphological operators and derive an exact convex relaxation which can be minimized globally. Extensive numerical validations on an established semantic segmentation benchmark demonstrate that the proposed proximity priors compare favorably to existing approaches.
---
paper_title: Graph cut with ordering constraints on labels and its applications

paper_content:

In the last decade, graph-cut optimization has been popular for a variety of pixel labeling problems. Typically graph-cut methods are used to incorporate a smoothness prior on a labeling. Recently several methods incorporated ordering constraints on labels for the application of object segmentation. An example of an ordering constraint is prohibiting a pixel with a ldquocar wheelrdquo label to be above a pixel with a ldquocar roofrdquo label. We observe that the commonly used graph-cut based alpha-expansion is more likely to get stuck in a local minimum when ordering constraints are used. For certain models with ordering constraints, we develop new graph-cut moves which we call order-preserving moves. Order-preserving moves act on all labels, unlike alpha-expansion. Although the global minimum is still not guaranteed, optimization with order-preserving moves performs significantly better than alpha-expansion. We evaluate order-preserving moves for the geometric class scene labeling (introduced by Hoiem et al.) where the goal is to assign each pixel a label such as ldquoskyrdquo, ldquogrounrdquo, etc., so ordering constraints arise naturally. In addition, we use order-preserving moves for certain simple shape priors in graphcut segmentation, which is a novel contribution in itself.
---
paper_title: Tiered scene labeling with dynamic programming

paper_content:

Dynamic programming (DP) has been a useful tool for a variety of computer vision problems. However its application is usually limited to problems with a one dimensional or low treewidth structure, whereas most domains in vision are at least 2D. In this paper we show how to apply DP for pixel labeling of 2D scenes with simple “tiered” structure. While there are many variations possible, for the applications we consider the following tiered structure is appropriate. An image is first divided by horizontal curves into the top, middle, and bottom regions, and the middle region is further subdivided vertically into subregions. Under these constraints a globally optimal labeling can be found using an efficient dynamic programming algorithm. We apply this algorithm to two very different tasks. The first is the problem of geometric class labeling where the goal is to assign each pixel a label such as “sky”, “ground”, and “surface above ground”. The second task involves incorporating simple shape priors for segmentation of an image into the “foreground” and “background” regions.
---
paper_title: Nonmetric Priors for Continuous Multilabel Optimization

paper_content:

We propose a novel convex prior for multilabel optimization which allows to impose arbitrary distances between labels. Only symmetry, d(i,j)≥0 and d(i,i)=0 are required. In contrast to previous grid based approaches for the nonmetric case, the proposed prior is formulated in the continuous setting avoiding grid artifacts. In particular, the model is easy to implement, provides a convex relaxation for the Mumford-Shah functional and yields comparable or superior results on the MSRC segmentation database comparing to metric or grid based approaches.
---
paper_title: Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multi-band Image Segmentation.

paper_content:

We present a novel statistical and variational approach to image segmentation based on a new algorithm, named region competition. This algorithm is derived by minimizing a generalized Bayes/minimum description length (MDL) criterion using the variational principle. The algorithm is guaranteed to converge to a local minimum and combines aspects of snakes/balloons and region growing. The classic snakes/balloons and region growing algorithms can be directly derived from our approach. We provide theoretical analysis of region competition including accuracy of boundary location, criteria for initial conditions, and the relationship to edge detection using filters. It is straightforward to generalize the algorithm to multiband segmentation and we demonstrate it on gray level images, color images and texture images. The novel color model allows us to eliminate intensity gradients and shadows, thereby obtaining segmentation based on the albedos of objects. It also helps detect highlight regions.
---
paper_title: A Region Merging Prior for Variational Level Set Image Segmentation

paper_content:

In current level set image segmentation methods, the number of regions is assumed to known beforehand. As a result, it remains constant during the optimization of the objective functional. How to allow it to vary is an important question which has been generally avoided. This study investigates a region merging prior related to regions area to allow the number of regions to vary automatically during curve evolution, thereby optimizing the objective functional implicitly with respect to the number of regions. We give a statistical interpretation to the coefficient of this prior to balance its effect systematically against the other functional terms. We demonstrate the validity and efficiency of the method by testing on real images of intensity, color, and motion.
---
paper_title: Unsupervised non-parametric region segmentation using level sets

paper_content:

We present a novel non-parametric unsupervised segmentation algorithm based on region competition (Zhu and Yuille, 1996); but implemented within a level sets framework (Osher and Sethian, 1988). The key novelty of the algorithm is that it can solve N /spl ges/ 2 class segmentation problems using just one embedded surface; this is achieved by controlling the merging and splitting behaviour of the level sets according to a minimum description length (MDL) (Leclerc (1989) and Rissanen (1985)) cost function. This is in contrast to N class region-based level set segmentation methods to date which operate by evolving multiple coupled embedded surfaces in parallel (Chan et al., 2002). Furthermore, it operates in an unsupervised manner; it is necessary neither to specify the value of N nor the class models a-priori. We argue that the level sets methodology provides a more convenient framework for the implementation of the region competition algorithm, which is conventionally implemented using region membership arrays due to the lack of a intrinsic curve representation. Finally, we generalise the Gaussian region model used in standard region competition to the non-parametric case. The region boundary motion and merge equations become simple expressions containing cross-entropy and entropy terms.
---
paper_title: Level Set Segmentation With Multiple Regions

paper_content:

The popularity of level sets for segmentation is mainly based on the sound and convenient treatment of regions and their boundaries. Unfortunately, this convenience is so far not known from level set methods when applied to images with more than two regions. This communication introduces a comparatively simple way how to extend active contours to multiple regions keeping the familiar quality of the two-phase case. We further suggest a strategy to determine the optimum number of regions as well as initializations for the contours.
---
paper_title: Geodesic active regions for motion estimation and tracking

paper_content:

This paper proposes a new front propagation method to deal accurately with the challenging problem of tracking non-rigid moving objects. This is obtained by employing a geodesic active region model where the designed objective function is composed of boundary and region-based terms and optimizes the curve position with respect to motion and intensity properties. The main novelty of our approach is that we deal with the motion estimation (linear models are assumed) and the tracking problem simultaneously. In other words, the optimization problem contains a coupled set of unknown variables; the curve position and the corresponding motion model. The designed objective function is minimized using a gradient descent method; the curve is propagated towards the object boundaries under the influence of boundary, intensity and motion-based forces using a PDE, while given the curve position an incremental analytical solution is obtained for the motion model. Besides, this PDE is implemented using a level set approach where topological changes are naturally handled. Very promising experimental results are provided using real video sequences.
---
paper_title: A level set framework with a shape and motion prior for segmentation and region tracking in echocardiography

paper_content:

We describe a level set formulation using both shape and motion prior, for both segmentation and region tracking in high frame rate echocardiographic image sequences. The proposed approach uses the following steps: registration of the prior shape, level set segmentation constrained through the registered shape and region tracking. Registration of the prior shape is expressed as a rigid or an affine transform problem, where the transform minimizing a global region-based criterion is sought. This criterion is based on image statistics and on the available estimated axial motion data. The segmentation step is then formulated through front propagation, constrained with the registered shape prior. The same region-based criterion is used both for the registration and the segmentation step. Region tracking is based on the motion field estimated from the interframe level set evolution. The proposed approach is applied to high frame rate echocardiographic sequences acquired in vivo. In this particular application, the prior shape is provided by a medical expert and the rigid transform is used for registration. It is shown that this approach provides consistent results in terms of segmentation and stability through the cardiac cycle. In particular, a comparison indicates that the results provided by our approach are very close to the results obtained with manual tracking performed by an expert cardiologist on a Doppler Tissue Imaging (DTI) study. These preliminary results show the ability of the method to perform region tracking and its potential for dynamic parametric imaging of the heart.
---
paper_title: Real-time tracking of the left ventricle in 3D echocardiography using a state estimation approach

paper_content:

In this paper we present a framework for real-time tracking of deformable contours in volumetric datasets. The framework supports composite deformation models, controlled by parameters for contour shape in addition to global pose. Tracking is performed in a sequential state estimation fashion, using an extended Kalman filter, with measurement processing in information space to effectively predict and update contour deformations in real-time. A deformable B-spline surface coupled with a global pose transform is used to model shape changes of the left ventricle of the heart. ::: ::: Successful tracking of global motion and local shape changes without user intervention is demonstrated on a dataset consisting of 21 3D echocardiography recordings. Real-time tracking using the proposed approach requires a modest CPU load of 13% on a modern computer. The segmented volumes compare to a semi-automatic segmentation tool with 95% limits of agreement in the interval 4.1 ± 24.6 ml (r = 0.92).
---
paper_title: PWP3D: Real-Time Segmentation and Tracking of 3D Objects

paper_content:

We formulate a probabilistic framework for simultaneous region-based 2D segmentation and 2D to 3D pose tracking, using a known 3D model. Given such a model, we aim to maximise the discrimination between statistical foreground and background appearance models, via direct optimisation of the 3D pose parameters. The foreground region is delineated by the zero-level-set of a signed distance embedding function, and we define an energy over this region and its immediate background surroundings based on pixel-wise posterior membership probabilities (as opposed to likelihoods). We derive the differentials of this energy with respect to the pose parameters of the 3D object, meaning we can conduct a search for the correct pose using standard gradient-based non-linear minimisation techniques. We propose novel enhancements at the pixel level based on temporal consistency and improved online appearance model adaptation. Furthermore, straightforward extensions of our method lead to multi-camera and multi-object tracking as part of the same framework. The parallel nature of much of the processing in our algorithm means it is amenable to GPU acceleration, and we give details of our real-time implementation, which we use to generate experimental results on both real and artificial video sequences, with a number of 3D models. These experiments demonstrate the benefit of using pixel-wise posteriors rather than likelihoods, and showcase the qualities, such as robustness to occlusions and motion blur (and also some failure modes), of our tracker.
---
paper_title: Animal: Validation and Applications of Nonlinear Registration-Based Segmentation

paper_content:

Magnetic resonance imaging (MRI) has become the modality of choice for neuro-anatomical imaging. Quantitative analysis requires the accurate and reproducible labeling of all voxels in any given structure within the brain. Since manual labeling is prohibitively time-consuming and error-prone we have designed an automated procedure called ANIMAL (Automatic Nonlinear Image Matching and Anatomical Labeling) to objectively segment gross anatomical structures from 3D MRIs of normal brains. The procedure is based on nonlinear registration with a previously labeled target brain, followed by numerical inverse transformation of the labels to the native MRI space. Besides segmentation, ANIMAL has been applied to non-rigid registration and to the analysis of morphometric variability. In this paper, the nonlinear registration approach is validated on five test volumes, produced with simulated deformations. Experiments show that the ANIMAL recovers 64% of the nonlinear residual variability remaining after linear regist...
---
paper_title: Elastically deforming 3D atlas to match anatomical brain images

paper_content:

To evaluate our system for elastically deforming a three-dimensional atlas to match anatomical brain images, six deformed versions of an atlas were generated. The deformed atlases were created by elastically mapping an anatomical brain atlas onto different MR brain image volumes. The mapping matches the edges of the ventricles and the surface of the brain; the resultant deformations are propagated through the atlas volume, deforming the remainder of the structures in the process. The atlas was then elastically matched to its deformed versions. The accuracy of the resultant matches was evaluated by determining the correspondence of 32 cortical and subcortical structures
---
paper_title: Simultaneous monocular 2d segmentation, 3d pose recovery and 3d reconstruction

paper_content:

We propose a novel framework for joint 2D segmentation and 3D pose and 3D shape recovery, for images coming from a single monocular source. In the past, integration of all three has proven difficult, largely because of the high degree of ambiguity in the 2D - 3D mapping. Our solution is to learn nonlinear and probabilistic low dimensional latent spaces, using the Gaussian Process Latent Variable Models dimensionality reduction technique. These act as class or activity constraints to a simultaneous and variational segmentation --- recovery --- reconstruction process. We define an image and level set based energy function, which we minimise with respect to 3D pose and shape, 2D segmentation resulting automatically as the projection of the recovered shape under the recovered pose. We represent 3D shapes as zero levels of 3D level set embedding functions, which we project down directly to probabilistic 2D occupancy maps, without the requirement of an intermediary explicit contour stage. Finally, we detail a fast, open-source, GPU-based implementation of our algorithm, which we use to produce results on both real and artificial video sequences.
---
paper_title: Deformable Medical Image Registration: A Survey

paper_content:

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.
---
paper_title: Medical image registration

paper_content:

Radiological images are increasingly being used in healthcare and medical research. There is, consequently, widespread interest in accurately relating information in the different images for diagnosis, treatment and basic science. This article reviews registration techniques used to solve this problem, and describes the wide variety of applications to which these techniques are applied. Applications of image registration include combining images of the same subject from different modalities, aligning temporal sequences of images to compensate for motion of the subject between scans, image guidance during interventions and aligning images from multiple subjects in cohort studies. Current registration algorithms can, in many cases, automatically register images that are related by a rigid body transformation (i.e. where tissue deformation can be ignored). There has also been substantial progress in non-rigid registration algorithms that can compensate for tissue deformation, or align images from different subjects. Nevertheless many registration problems remain unsolved, and this is likely to continue to be an active field of research in the future.
---
paper_title: A Nonrigid Kernel-Based Framework for 2D-3D Pose Estimation and 2D Image Segmentation

paper_content:

In this work, we present a nonrigid approach to jointly solving the tasks of 2D-3D pose estimation and 2D image segmentation. In general, most frameworks that couple both pose estimation and segmentation assume that one has exact knowledge of the 3D object. However, under nonideal conditions, this assumption may be violated if only a general class to which a given shape belongs is given (e.g., cars, boats, or planes). Thus, we propose to solve the 2D-3D pose estimation and 2D image segmentation via nonlinear manifold learning of 3D embedded shapes for a general class of objects or deformations for which one may not be able to associate a skeleton model. Thus, the novelty of our method is threefold: First, we present and derive a gradient flow for the task of nonrigid pose estimation and segmentation. Second, due to the possible nonlinear structures of one's training set, we evolve the preimage obtained through kernel PCA for the task of shape analysis. Third, we show that the derivation for shape weights is general. This allows us to use various kernels, as well as other statistical learning methodologies, with only minimal changes needing to be made to the overall shape evolution scheme. In contrast with other techniques, we approach the nonrigid problem, which is an infinite-dimensional task, with a finite-dimensional optimization scheme. More importantly, we do not explicitly need to know the interaction between various shapes such as that needed for skeleton models as this is done implicitly through shape learning. We provide experimental results on several challenging pose estimation and segmentation scenarios.
---

Title: Incorporating prior knowledge in medical image segmentation: a survey

Section 1: Introduction
Description 1: This section provides an overview of the process of image segmentation and its applications in medical image analysis, highlighting the need for incorporating prior knowledge.

Section 2: Why yet another survey paper on MIS?
Description 2: This section explores the related surveys and justifies the need for this survey, focusing on the limitations of existing surveys and the advancements covered in this survey.

Section 3: Traditional image segmentation methods
Description 3: This section reviews traditional segmentation methods such as thresholding, region-growing, and watershed, and discusses their limitations in handling medical images.

Section 4: Optimization-based image segmentation
Description 4: This section introduces optimization-based methods for image segmentation, explaining the fundamentals of energy minimization and probability-based approaches.

Section 5: Domain of formulation: continuous vs. discrete
Description 5: This section discusses the differences between continuous and discrete domain formulations in segmentation and compares their advantages and disadvantages.

Section 6: Optimization: convex (submodular) vs. non-convex (nonsubmodular)
Description 6: This section explains the distinctions between convex and non-convex optimization problems and their implications for image segmentation accuracy and feasibility.

Section 7: Fidelity vs. Optimizibility
Description 7: This section elaborates on the trade-off between fidelity and optimizability in energy-based segmentation problems, discussing ways to balance them.

Section 8: Uncertainty and fuzzy / probabilistic vs. crisp labeling
Description 8: This section addresses the role of uncertainty in segmentation, comparing crisp, probabilistic, and fuzzy labeling approaches and their benefits.

Section 9: Sub-pixel accuracy
Description 9: This section discusses achieving sub-pixel accuracy in segmentation, focusing on methods to improve the precision of object boundaries.

Section 10: Prior knowledge for targeted image segmentation
Description 10: This section categorizes and reviews various types of prior knowledge used to improve image segmentation, comparing their effectiveness and methodologies.

Section 11: User interaction
Description 11: This section explains how user input is incorporated into segmentation frameworks to assist in characterizing and delineating the targeted object.

Section 12: Appearance prior
Description 12: This section discusses appearance models, including intensity, color, and texture, and their role in improving segmentation results.

Section 13: Regularization
Description 13: This section elaborates on regularization terms as priors in image segmentation to ensure feasible and stable solutions.

Section 14: Boundary information
Description 14: This section focuses on using boundary and edge information to guide segmentation, including the role of boundary polarity.

Section 15: Extending binary to multi-label segmentation
Description 15: This section discusses techniques for multi-label segmentation, extending from binary segmentation methods to handle multiple objects.

Section 16: Shape prior
Description 16: This section covers the incorporation of shape information as a prior, using geometrical, statistical, and physical models.

Section 17: Topological prior
Description 17: This section addresses the use of topological constraints to maintain specific topologies in segmented objects.

Section 18: Moment prior
Description 18: This section explains the use of moment constraints, including area, centroid, and higher-order moments, as priors in segmentation.

Section 19: Geometrical and region interactions prior
Description 19: This section reviews the role of geometrical and regional interaction constraints in improving segmentation accuracy.

Section 20: Spatial distance prior
Description 20: This section discusses enforcing spatial distance constraints between regions to ensure plausible segmentation results.

Section 21: Adjacency prior
Description 21: This section explains the incorporation of adjacency relationships and ordering constraints between labels in segmentation.

Section 22: Number of regions/labels
Description 22: This section discusses methods to handle cases where the number of regions or labels is not predefined, including label cost priors.

Section 23: Motion prior
Description 23: This section covers the use of motion and tracking information as a prior in segmenting moving objects in videos.

Section 24: Model/Atlas
Description 24: This section discusses atlas-based segmentation methods and their ability to encode spatial relationships between multiple tissues or structures.

Section 25: Summary, discussion, and conclusions
Description 25: This section summarizes the survey, discusses the trade-offs in segmentation methods, and concludes with future research directions.