Interactive lesion segmentation with shape priors from offline and online learning

In medical image segmentation, tumors and other lesions demand the highest levels of accuracy but still call for the highest levels of manual delineation. One factor holding back automatic segmentation is the exemption of pathological regions from shape modelling techniques that rely on high-level shape information not offered by lesions. This paper introduces two new statistical shape models (SSMs) that combine radial shape parameterization with machine learning techniques from the field of nonlinear time series analysis. We then develop two dynamic contour models (DCMs) using the new SSMs as shape priors for tumor and lesion segmentation. From training data, the SSMs learn the lower level shape information of boundary fluctuations, which we prove to be nevertheless highly discriminant. One of the new DCMs also uses online learning to refine the shape prior for the lesion of interest based on user interactions. Classification experiments reveal superior sensitivity and specificity of the new shape priors over those previously used to constrain DCMs. User trials with the new interactive algorithms show that the shape priors are directly responsible for improvements in accuracy and reductions in user demand.     

Modeling Interaction for Segmentation of Neighboring Structures

This paper presents a new method for segmenting medical images by modeling interaction between neighboring structures. Compared to previously reported methods, the proposed approach enables simultaneous segmentation of multiple neighboring structures for improved robustness. During the segmentation process, the object contour evolution and shape prior estimates are influenced by the interactions between neighboring shapes consisting of attraction, repulsion, and competition. Instead of estimating the a priori shape of each structure independently, an interactive maximum a posteriori shape estimation method is used for estimating the shape priors by considering shape prior distribution, neighboring shapes, and image features. Energy functionals are then formulated to model the interaction and segmentation. With the proposed method, neighboring structures with similar intensities and/or textures, and blurred boundaries can be extracted simultaneously. Experimental results obtained on both synthetic data and medical images demonstrate that the introduced interaction between neighboring structures improves segmentation performance compared with other existing approaches.      

Medical image segmentation using minimal path deformable models with implicit shape priors.

This paper presents a new method for segmentation of medical images by extracting organ contours, using minimal path deformable models incorporated with statistical shape priors. In our approach, boundaries of structures are considered as minimal paths, i.e., paths associated with the minimal energy, on weighted graphs. Starting from the theory of minimal path deformable models, an intelligent "worm" algorithm is proposed for segmentation, which is used to evaluate the paths and finally find the minimal path. Prior shape knowledge is incorporated into the segmentation process to achieve more robust segmentation. The shape priors are implicitly represented and the estimated shapes of the structures can be conveniently obtained. The worm evolves under the joint influence of the image features, its internal energy, and the shape priors. The contour of the structure is then extracted as the worm trail. The proposed segmentation framework overcomes the short-comings of existing deformable models and has been successfully applied to segmenting various medical images.     

Statistical approach to segmentation of single-channel cerebral MRimages

A statistical model is presented that represents the distributions of major tissue classes in single-channel magnetic resonance (MR) cerebral images. Using the model, cerebral images are segmented into gray matter, white matter, and cerebrospinal fluid (CSF). The model accounts for random noise, magnetic field inhomogeneities, and biological variations of the tissues. Intensity measurements are modeled by a finite Gaussian mixture. Smoothness and piecewise contiguous nature of the tissue regions are modeled by a three-dimensional (3-D) Markov random field (MRF). A segmentation algorithm, based on the statistical model, approximately finds the maximum a posteriori (MAP) estimation of the segmentation and estimates the model parameters from the image data. The proposed scheme for segmentation is based on the iterative conditional modes (ICM) algorithm in which measurement model parameters are estimated using local information at each site, and the prior model parameters are estimated using the segmentation after each cycle of iterations. Application of the algorithm to a sample of clinical MR brain scans, comparisons of the algorithm with other statistical methods, and a validation study with a phantom are presented. The algorithm constitutes a significant step toward a complete data driven unsupervised approach to segmentation of MR images in the presence of the random noise and intensity inhomogeneities     

Brain anatomical structure segmentation by hybrid discriminative/generative models

In this paper, a hybrid discriminative/generative model for brain anatomical structure segmentation is proposed. The learning aspect of the approach is emphasized. In the discriminative appearance models, various cues such as intensity and curvatures are combined to locally capture the complex appearances of different anatomical structures. A probabilistic boosting tree (PBT) framework is adopted to learn multiclass discriminative models that combine hundreds of features across different scales. On the generative model side, both global and local shape models are used to capture the shape information about each anatomical structure. The parameters to combine the discriminative appearance and generative shape models are also automatically learned. Thus, low-level and high-level information is learned and integrated in a hybrid model. Segmentations are obtained by minimizing an energy function associated with the proposed hybrid model. Finally, a grid-face structure is designed to explicitly represent the 3-D region topology. This representation handles an arbitrary number of regions and facilitates fast surface evolution. Our system was trained and tested on a set of 3-D magnetic resonance imaging (MRI) volumes and the results obtained are encouraging.     

Minimal shape and intensity cost path segmentation

A new generic model-based segmentation algorithm is presented, which can be trained from examples akin to the active shape model (ASM) approach in order to acquire knowledge about the shape to be segmented and about the gray-level appearance of the object in the image. Whereas ASM alternates between shape and intensity information during search, the proposed approach optimizes for shape and intensity characteristics simultaneously. Local gray-level appearance information at the landmark points extracted from feature images is used to automatically detect a number of plausible candidate locations for each landmark. The shape information is described by multiple landmark-specific statistical models that capture local dependencies between adjacent landmarks on the shape. The shape and intensity models are combined in a single cost function that is optimized noniteratively using dynamic programming, without the need for initialization. The algorithm was validated for segmentation of anatomical structures in chest and hand radiographs. In each experiment, the presented method had a significant higher performance when compared to the ASM schemes. As the method is highly effective, optimally suited for pathological cases and easy to implement, it is highly useful for many medical image segmentation tasks.     

Semiautomated segmentation of myocardial contours for fast strain analysis in cine displacement-encoded MRI

The purposes of this study were to develop a semiautomated cardiac contour segmentation method for use with cine displacement-encoded MRI and evaluate its accuracy against manual segmentation. This segmentation model was designed with two distinct phases: preparation and evolution. During the model preparation phase, after manual image cropping and then image intensity standardization, the myocardium is separated from the background based on the difference in their intensity distributions, and the endo- and epi-cardial contours are initialized automatically as zeros of an underlying level set function. During the model evolution phase, the model deformation is driven by the minimization of an energy function consisting of five terms: model intensity, edge attraction, shape prior, contours interaction, and contour smoothness. The energy function is minimized iteratively by adaptively weighting the five terms in the energy function using an annealing algorithm. The validation experiments were performed on a pool of cine data sets of five volunteers. The difference between the semiautomated segmentation and manual segmentation was sufficiently small as to be considered clinically irrelevant. This relatively accurate semiautomated segmentation method can be used to significantly increase the throughput of strain analysis of cine displacement-encoded MR images for clinical applications.      

MR Image Segmentation Using a Power Transformation Approach

This study proposes a segmentation method for brain MR images using a distribution transformation approach. The method extends traditional Gaussian mixtures expectation-maximization segmentation to a power transformed version of mixed intensity distributions, which includes Gaussian mixtures as a special case. As MR intensities tend to exhibit non-Gaussianity due to partial volume effects, the proposed method is designed to fit non-Gaussian tissue intensity distributions. One advantage of the method is that it is intuitively appealing and computationally simple. To avoid performance degradation caused by intensity inhomogeneity, different methods for correcting bias fields were applied prior to image segmentation, and their correction effects on the segmentation results were examined in the empirical study. The partitions of brain tissues (i.e., gray and white matter) resulting from the method were validated and evaluated against manual segmentation results based on 38 real T1-weighted image volumes from the internet brain segmentation repository, and 18 simulated image volumes from BrainWeb. The Jaccard and Dice similarity indexes were computed to evaluate the performance of the proposed approach relative to the expert segmentations. Empirical results suggested that the proposed segmentation method yielded higher similarity measures for both gray matter and white matter as compared with those based on the traditional segmentation using the Gaussian mixtures approach.      

Knowledge-based segmentation of SAR data with learned priors

An approach for the segmentation of still and video synthetic aperture radar (SAR) images is described. A priori knowledge about the objects present in the image, e.g., target, shadow and background terrain, is introduced via Bayes' rule. Posterior probabilities obtained in this way are then anisotropically smoothed, and the image segmentation is obtained via MAP classifications of the smoothed data. When segmenting sequences of images, the smoothed posterior probabilities of past frames are used to learn the prior distributions in the succeeding frame. We show with examples from public data sets that this method provides an efficient and fast technique for addressing the segmentation of SAR data.     

Ultrasound image segmentation with shape priors: application to automatic cattle rib-eye area estimation.

Automatic ultrasound (US) image segmentation is a difficult task due to the quantity of noise present in the images and the lack of information in several zones produced by the acquisition conditions. In this paper, we propose a method that combines shape priors and image information to achieve this task. In particular, we introduce knowledge about the rib-eye shape using a set of images manually segmented by experts. A method is proposed for the automatic segmentation of new samples in which a closed curve is fitted taking into account both the US image information and the geodesic distance between the evolving curve and the estimated mean rib-eye shape in a shape space. This method can be used to solve similar problems that arise when dealing with US images in other fields. The method was successfully tested over a database composed of 610 US images, for which we have the manual segmentations of two experts.     

Medical image segmentation using analysis of isolable-contour maps

A common challenge for automated segmentation techniques is differentiation between images of close objects that have similar intensities, whose boundaries are often blurred due to partial-volume effects. We propose a novel approach to segmentation of two-dimensional images, which addresses this challenge. Our method, which we call intrinsic shape for segmentation (ISeg), analyzes isolabel-contour maps to identify coherent regions that correspond to major objects. ISeg generates an isolabel-contour map for an image by multilevel thresholding with a fine partition of the intensity range. ISeg detects object boundaries by comparing the shape of neighboring isolabel contours from the map. ISeg requires only little effort from users; it does not require construction of shape models of target objects. In a formal validation with computed-tomography angiography data, we showed that ISeg was more robust than conventional thresholding, and that ISeg's results were comparable to results of manual tracing.     

A class-adaptive spatially variant mixture model for image segmentation.

We propose a new approach for image segmentation based on a hierarchical and spatially variant mixture model. According to this model, the pixel labels are random variables and a smoothness prior is imposed on them. The main novelty of this work is a new family of smoothness priors for the label probabilities in spatially variant mixture models. These Gauss-Markov random field-based priors allow all their parameters to be estimated in closed form via the maximum a posteriori (MAP) estimation using the expectation-maximization methodology. Thus, it is possible to introduce priors with multiple parameters that adapt to different aspects of the data. Numerical experiments are presented where the proposed MAP algorithms were tested in various image segmentation scenarios. These experiments demonstrate that the proposed segmentation scheme compares favorably to both standard and previous spatially constrained mixture model-based segmentation.     

iPiano-Net: Nonconvex optimization inspired multi-scale reconstruction network for compressed sensing

Compressed sensing (CS) aims to precisely reconstruct the original signal from under-sampled measurements, which is a typical ill-posed problem. Solving such a problem is challenging and generally needs to incorporate suitable priors about the underlying signals. Traditionally, these priors are hand-crafted and the corresponding approaches generally have limitations in expressive capacity. In this paper, a nonconvex optimization inspired multi-scale reconstruction network is developed for block-based CS, abbreviated as iPiano-Net, by unfolding the classic iPiano algorithm. In iPiano-Net, a block-wise inertial gradient descent interleaves with an image-level network-inducing proximal mapping to exploit the local block and global content information alternately. Therein, network-inducing proximal operators can be adaptively learned in each module, which can efficiently characterize image priors and improve the modeling capacity of iPiano-Net. Such learned image-level priors can suppress blocky artifacts and noises/corruptions while preserving the global information. Different from existing discriminative CS reconstruction models trained with specific measurement ratios, an effective single model is learned to handle CS reconstruction with several measurement ratios even the unseen ones. Experimental results demonstrate that the proposed approach is substantially superior to previous CS methods in terms of Peak Signal to Noise Ratio (PSNR) and visual quality, especially at low measurement ratios. Meanwhile, it is robust to noise while maintaining comparable execution speed.     

Fully Bayesian estimation of Gibbs hyperparameters for emissioncomputed tomography data

In recent years, many investigators have proposed Gibbs prior models to regularize images reconstructed from emission computed tomography data. Unfortunately, hyperparameters used to specify Gibbs priors can greatly influence the degree of regularity imposed by such priors and, as a result, numerous procedures have been proposed to estimate hyperparameter values, from observed image data. Many of these, procedures attempt to maximize the joint posterior distribution on the image scene. To implement these methods, approximations to the joint posterior densities are required, because the dependence of the Gibbs partition function on the hyperparameter values is unknown. Here, the authors use recent results in Markov chain Monte Carlo (MCMC) sampling to estimate the relative values of Gibbs partition functions and using these values, sample from joint posterior distributions on image scenes. This allows for a fully Bayesian procedure which does not fix the hyperparameters at some estimated or specified value, but enables uncertainty about these values to be propagated through to the estimated intensities. The authors utilize realizations from the posterior distribution for determining credible regions for the intensity of the emission source. The authors consider two different Markov random field (MRF) models-the power model and a line-site model. As applications they estimate the posterior distribution of source intensities from computer simulated data as well as data collected from a physical single photon emission computed tomography (SPECT) phantom     

Simultaneous vector-valued image segmentation and intensity nonuniformity correction using variational level set combined with Markov random field modeling

Object quantification requires an image segmentation to make measurements about size, material composition and morphology of the object. In vector-valued or multispectral images, each image channel has its signal characteristics and provides special information that may improve the results of image segmentation method. This paper presents a region-based active contour model for vector-valued image segmentation with a variational level set formulation. In this model, the local image intensities are characterized using Gaussian distributions with different means and variances. Furthermore, by utilizing Markov random field, the spatial correlation between neighboring pixels and voxels is modeled. With incorporation of intensity nonuniformity model, our method is able to deal with brain tissue segmentation from multispectral magnetic resonance (MR) images. Our experiments on synthetic images and multispectral cerebral MR images with different noise and bias level show the advantages of the proposed method.     

STACS: new active contour scheme for cardiac MR image segmentation

The paper presents a novel stochastic active contour scheme (STACS) for automatic image segmentation designed to overcome some of the unique challenges in cardiac MR images such as problems with low contrast, papillary muscles, and turbulent blood flow. STACS minimizes an energy functional that combines stochastic region-based and edge-based information with shape priors of the heart and local properties of the contour. The minimization algorithm solves, by the level set method, the Euler-Lagrange equation that describes the contour evolution. STACS includes an annealing schedule that balances dynamically the weight of the different terms in the energy functional. Three particularly attractive features of STACS are: 1) ability to segment images with low texture contrast by modeling stochastically the image textures; 2) robustness to initial contour and noise because of the utilization of both edge and region-based information; 3) ability to segment the heart from the chest wall and the undesired papillary muscles due to inclusion of heart shape priors. Application of STACS to a set of 48 real cardiac MR images shows that it can successfully segment the heart from its surroundings such as the chest wall and the heart structures (the left and right ventricles and the epicardium.) We compare STACS' automatically generated contours with manually-traced contours, or the "gold standard," using both area and edge similarity measures. This assessment demonstrates very good and consistent segmentation performance of STACS.     

Coupled nonparametric shape and moment-based intershape pose priors for multiple basal ganglia structure segmentation

This paper presents a new active contour-based, statistical method for simultaneous volumetric segmentation of multiple subcortical structures in the brain. In biological tissues, such as the human brain, neighboring structures exhibit co-dependencies which can aid in segmentation, if properly analyzed and modeled. Motivated by this observation, we formulate the segmentation problem as a maximum a posteriori estimation problem, in which we incorporate statistical prior models on the shapes and intershape (relative) poses of the structures of interest. This provides a principled mechanism to bring high level information about the shapes and the relationships of anatomical structures into the segmentation problem. For learning the prior densities we use a nonparametric multivariate kernel density estimation framework. We combine these priors with data in a variational framework and develop an active contour-based iterative segmentation algorithm. We test our method on the problem of volumetric segmentation of basal ganglia structures in magnetic resonance images. We present a set of 2-D and 3-D experiments as well as a quantitative performance analysis. In addition, we perform a comparison to several existent segmentation methods and demonstrate the improvements provided by our approach in terms of segmentation accuracy.     

Brain MR image segmentation based on local Gaussian mixture model and nonlocal spatial regularization

Brain Magnetic Resonance (MR) images often suffer from the inhomogeneous intensities caused by the bias field and heavy noise. The most widely used image segmentation algorithms, which typically rely on the homogeneity of image intensities in different regions, often fail to provide accurate segmentation results due to the existence of bias field and heavy noise. This paper proposes a novel variational approach for brain image segmentation with simultaneous bias correction. We define an energy functional with a local data fitting term and a nonlocal spatial regularization term. The local data fitting term is based on the idea of local Gaussian mixture model (LGMM), which locally models the distribution of each tissue by a linear combination of Gaussian function. By the LGMM, the bias field function in an additive form is embedded to the energy functional, which is helpful for eliminating the influence of the intensity inhomogeneity. For reducing the influence of noise and getting a smooth segmentation, the nonlocal spatial regularization is drawn upon, which is good at preserving fine structures in brain images. Experiments performed on simulated as well as real MR brain data and comparisons with other related methods are given to demonstrate the effectiveness of the proposed method.