Segmentation of intravascular ultrasound images: a knowledge-based approach

Intravascular ultrasound imaging of coronary arteries provides important information about coronary lumen, wall, and plaque characteristics. Quantitative studies of coronary atherosclerosis using intravascular ultrasound and manual identification of wall and plaque borders are limited by the need for observers with substantial experience and the tedious nature of manual border detection. We have developed a method for segmentation of intravascular ultrasound images that identifies the internal and external elastic laminae and the plaque-lumen interface. The border detection algorithm was evaluated in a set of 38 intravascular ultrasound images acquired from fresh cadaveric hearts using a 30 MHz imaging catheter. To assess the performance of our border detection method we compared five quantitative measures of arterial anatomy derived from computer-detected borders with measures derived from borders manually defined by expert observers. Computer-detected and observer-defined lumen areas correlated very well (r=0.96, y=1.02x+0.52), as did plaque areas (r=0.95, y=1.07x-0.48), and percent area stenosis (r=0.93, y=0.99x-1.34.) Computer-derived segmental plaque thickness measurements were highly accurate. Our knowledge-based intravascular ultrasound segmentation method shows substantial promise for the quantitative analysis of in vivo intravascular ultrasound image data.     

A State-of-the-Art Review on Segmentation Algorithms in Intravascular Ultrasound (IVUS) Images

Over the past two decades, intravascular ultrasound (IVUS) image segmentation has remained a challenge for researchers while the use of this imaging modality is rapidly growing in catheterization procedures and in research studies. IVUS provides cross-sectional grayscale images of the arterial wall and the extent of atherosclerotic plaques with high spatial resolution in real time. In this paper, we review recently developed image processing methods for the detection of media-adventitia and luminal borders in IVUS images acquired with different transducers operating at frequencies ranging from 20 to 45 MHz. We discuss methodological challenges, lack of diversity in reported datasets, and weaknesses of quantification metrics that make IVUS segmentation still an open problem despite all efforts. In conclusion, we call for a common reference database, validation metrics, and ground-truth definition with which new and existing algorithms could be benchmarked.     

Segmentation and quantification of human vessels using a 3-D cylindrical intensity model.

We introduce a new approach for 3-D segmentation and quantification of vessels. The approach is based on a 3-D cylindrical parametric intensity model, which is directly fitted to the image intensities through an incremental process based on a Kalman filter. Segmentation results are the vessel centerline and shape, i.e., we estimate the local vessel radius, the 3-D position and 3-D orientation, the contrast, as well as the fitting error. We carried out an extensive validation using 3-D synthetic images and also compared the new approach with an approach based on a Gaussian model. In addition, the new model has been successfully applied to segment vessels from 3-D MRA and computed tomography angiography image data. In particular, we compared our approach with an approach based on the randomized Hough transform. Moreover, a validation of the segmentation results based on ground truth provided by a radiologist confirms the accuracy of the new approach. Our experiments show that the new model yields superior results in estimating the vessel radius compared to previous approaches based on a Gaussian model as well as the Hough transform.     

Combining learning-based intensity distributions with nonparametric shape priors for image segmentation

Integration of shape prior information into level set formulations has led to great improvements in image segmentation in the presence of missing information, occlusion, and noise. However, most shape-based segmentation techniques incorporate image intensity through simplistic data terms. A common underlying assumption of such data terms is that the foreground and the background regions in the image are homogeneous, i.e., intensities are piecewise constant or piecewise smooth. This situation makes integration of shape priors inefficient in the presence of intensity inhomogeneities. In this paper, we propose a new approach for combining information from shape priors with that from image intensities. More specifically, our approach uses shape priors learned by nonparametric density estimation and incorporates image intensity distributions learned in a supervised manner. Such a combination has not been used in previous work. Sample image patches are used to learn the intensity distributions, and segmented training shapes are used to learn the shape priors. We present an active contour algorithm that takes these learned densities into account for image segmentation. Our experiments on synthetic and real images demonstrate the robustness of the proposed approach to complicated intensity distributions, and occlusions, as well as the improvements it provides over existing methods.     

Shape statistics variational approach for the outer contour segmentation of left ventricle MR images.

Segmentation of left ventricles is one of the important research topics in cardiac magnetic resonance (MR) imaging. The segmentation precision influences the authenticity of ventricular motion reconstruction. In left ventricle MR images, the weak and broken boundary increases the difficulty of segmenting the outer contour precisely. In this paper, we present an improved shape statistics variational approach for the outer contour segmentation of left ventricle MR images. We use the Mumford-Shah model in an object feature space and incorporate the shape statistics and an edge image to the variational framework. The introduction of shape statistics can improve the segmentation with broken boundaries. The edge image can enhance the weak boundary and thus improve the segmentation precision. The generation of the object feature image, which has homogenous "intensities" in the left ventricle, facilitates the application of the Mumford-Shah model. A comparison of mean absolute distance analysis between different contours generated with our algorithm and that generated by hand demonstrated that our method can achieve a higher segmentation precision and a better stability than various approaches. It is a semiautomatic way for the segmentation of the outer contour of the left ventricle in clinical applications.     

Estimating coronary artery lumen area with optimization-based contour detection

A modified optimization-based contour detection method was presented to compute the lumen area of the coronary artery from intravascular ultrasound (IVUS) video images. First, the search range for the artery inner wall was determined based on the continuity of IVUS video frames. Next, the internal and external energy were calculated to describe the smoothness of the arterial wall and the grayscale variation of ultrasound images, respectively. Here, a novel form of the external energy which combines the gradient and variance of the intensity of image in the radial direction was used. Finally, the minimal energy path based on the optimum contour of the artery wall was obtained using circular dynamic programming (DP). By the comparison with the typical DP procedure using the traditional external energy form, based only on the image gradient, the reliability of this modified method is considerably improved in the measurement of coronary artery lumen area.     

Accurate automatic analysis of cardiac cine images

Acquisition of noncontrast agent cine cardiac magnetic resonance (CMR) gated images through the cardiac cycle is, at present, a well-established part of examining cardiac global function. However, regional quantification is less well established. We propose a new automated framework for analyzing the wall thickness and thickening function on these images that consists of three main steps. First, inner and outer wall borders are segmented from their surrounding tissues with a geometric deformable model guided by a special stochastic speed relationship. The latter accounts for Markov-Gibbs shape and appearance models of the object-of-interest and its background. In the second step, point-to-point correspondences between the inner and outer borders are found by solving the Laplace equation and provide initial estimates of the local wall thickness and the thickening function index. Finally, the effects of the segmentation error is reduced and a continuity analysis of the LV wall thickening is performed through iterative energy minimization using a generalized Gauss-Markov random field (GGMRF) image model. The framework was evaluated on 26 datasets from clinical cine CMR images that have been collected from patients with eleven independent studies, with chronic ischemic heart disease and heart damage. The performance evaluation of the proposed segmentation approach, based on the receiver operating characteristic (ROC) and Dice similarity coefficients (DSC) between manually drawn and automatically segmented contours, confirmed a high robustness and accuracy of the proposed segmentation approach. Furthermore, the Bland-Altman plot is used to assess the limit of agreement of our measurements of the global function parameters compared to the ground truth. Importantly, comparative results on the publicly available database (MICCAI 2009 Cardiac MR Left Ventricle Segmentation) demonstrated a superior performance of the proposed segmentation approach over published methods.     

Segmenting object with ambiguous boundary using information theoretic rough sets

In image segmentation, image is divided into regions of similar pixels that satisfy a defined notion of similarity. The complexity of image segmentation is further increased when the separation between neighboring regions is ambiguous. In this paper, we propose an approach that uses the information theoretic rough sets concept (ITRS) to model the ambiguous boundary of the object for further segmentation. The advantage of this approach is incorporating the prior knowledge of the object for effective extraction despite its ambiguous boundary. This approach starts with an assumption that seed points of the regions are available. It then computes the probability of association of the pixels with the seed points. Rough sets theory is invoked on this probability or likelihood map to identify positive, negative, and boundary states of the object. Optimal threshold for the boundary region is determined using histogram based segmentation algorithm for final object extraction. The main contribution relies on the application of ITRS in categorizing the object by combining both the prior and image information. The proposed approach, ITRS segmentation, is compared with different image segmentation methods on simulated brain images, and the result is encouraging with its state-of-the-art performance.     

Ultrasound kidney segmentation with a global prior shape

In this paper, we focus on segmentation of ultrasound kidney images. Unlike previous work by using trained prior shapes, we employ a parametric super-ellipse as a global prior shape for a human kidney. The Fisher–Tippett distribution is employed to describe the grey level statistics. Combining the grey level statistics with a global character of a kidney shape, we propose a new active contour model to segment ultrasound kidney images. The proposed model involves two subproblems. One subproblem is to optimize the parameters of a super-ellipse. Another subproblem is to segment an ultrasound kidney image. An alternating minimization scheme is used to optimize the parameters of a super-ellipse and segment an image simultaneously. To segment an image fast, a convex relaxation method is introduced and the split Bregman method is incorporated to propose a fast segmentation algorithm. The efficiency of the proposed method is illustrated by numerical experiments on both simulated images and real ultrasound kidney images.     

Background removal of multiview images by learning shape priors.

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge--to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications.     

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.     

Tissue characterization in intravascular ultrasound images

Intravascular ultrasound (IVUS) imaging permits direct visualization of vascular pathology. It has been used to evaluate lumen and plaque in coronary arteries and its clinical significance for guidance of coronary interventions is increasingly recognized. Conventional manual evaluation is tedious and time-consuming. This paper describes a highly automated approach to segmentation of coronary wall and plaque, and determination of plaque composition in individual IVUS images and pullback image sequences. The determined regions of plaque were classified in one of three classes: soft plaque, hard plaque, or hard plaque shadow. The method's performance was assessed in vitro and in vivo in comparison with observer-defined independent standards. In the analyzed images and image sequences, the mean border positioning error of the wall and plaque borders ranged from 0.13-0.17 mm. Plaque classification correctness was 90%.     

A wrapper-based approach to image segmentation and classification.

The traditional processing flow of segmentation followed by classification in computer vision assumes that the segmentation is able to successfully extract the object of interest from the background image. It is extremely difficult to obtain a reliable segmentation without any prior knowledge about the object that is being extracted from the scene. This is further complicated by the lack of any clearly defined metrics for evaluating the quality of segmentation or for comparing segmentation algorithms. We propose a method of segmentation that addresses both of these issues, by using the object classification subsystem as an integral part of the segmentation. This will provide contextual information regarding the objects to be segmented, as well as allow us to use the probability of correct classification as a metric to determine the quality of the segmentation. We view traditional segmentation as a filter operating on the image that is independent of the classifier, much like the filter methods for feature selection. We propose a new paradigm for segmentation and classification that follows the wrapper methods of feature selection. Our method wraps the segmentation and classification together, and uses the classification accuracy as the metric to determine the best segmentation. By using shape as the classification feature, we are able to develop a segmentation algorithm that relaxes the requirement that the object of interest to be segmented must be homogeneous in some low-level image parameter, such as texture, color, or grayscale. This represents an improvement over other segmentation methods that have used classification information only to modify the segmenter parameters, since these algorithms still require an underlying homogeneity in some parameter space. Rather than considering our method as, yet, another segmentation algorithm, we propose that our wrapper method can be considered as an image segmentation framework, within which existing image segmentation algorithms may be executed. We show the performance of our proposed wrapper-based segmenter on real-world and complex images of automotive vehicle occupants for the purpose of recognizing infants on the passenger seat and disabling the vehicle airbag. This is an interesting application for testing the robustness of our approach, due to the complexity of the images, and, consequently, we believe the algorithm will be suitable for many other real-world applications.     

Medical image segmentation using minimal path deformable models with implicit shape priors.

This paper presents a new method for segmentation of medical images by extracting organ contours, using minimal path deformable models incorporated with statistical shape priors. In our approach, boundaries of structures are considered as minimal paths, i.e., paths associated with the minimal energy, on weighted graphs. Starting from the theory of minimal path deformable models, an intelligent "worm" algorithm is proposed for segmentation, which is used to evaluate the paths and finally find the minimal path. Prior shape knowledge is incorporated into the segmentation process to achieve more robust segmentation. The shape priors are implicitly represented and the estimated shapes of the structures can be conveniently obtained. The worm evolves under the joint influence of the image features, its internal energy, and the shape priors. The contour of the structure is then extracted as the worm trail. The proposed segmentation framework overcomes the short-comings of existing deformable models and has been successfully applied to segmenting various medical images.     

A multiresolution image segmentation technique based on pyramidalsegmentation and fuzzy clustering

In this paper, an unsupervised image segmentation technique is presented, which combines pyramidal image segmentation with the fuzzy c-means clustering algorithm. Each layer of the pyramid is split into a number of regions by a root labeling technique, and then fuzzy c-means is used to merge the regions of the layer with the highest image resolution. A cluster validity functional is used to find the optimal number of objects automatically. Segmentation of a number of synthetic as well as clinical images is illustrated and two fully automatic segmentation approaches are evaluated, which determine the left ventricular volume (LV) in 140 cardiovascular magnetic resonance (MR) images. First fuzzy c-means is applied without pyramids. In the second approach the regions generated by pyramidal segmentation are merged by fuzzy c-means. The correlation coefficients of manually and automatically defined LV lumen of all 140 and 20 end-diastolic images were equal to 0.86 and 0.79, respectively, when images were segmented with fuzzy c-means alone. These coefficients increased to 0.90 and 0.93 when the pyramidal segmentation was combined with fuzzy c-means. This method can be applied to any dimensional representation and at any resolution level of an image series. The evaluation study shows good performance in detecting LV lumen in MR images.     

A Novel Color Image Segmentation Approach Based on Neutrosophic Set and Modified Fuzzy c-Means

Color image segmentation is an important technique in image processing, pattern recognition and computer vision. Many segmentation algorithms have been proposed. However, it is still a complex task especially when there are noises in the images, which have not been studied in much detail. Neutrosophic set (NS) studies the origin, nature, and scope of neutralities. In this paper, we apply NS in the color image and define some new concepts. A directional α-mean operation is proposed to reduce the set indeterminacy. The fuzzy c-means clustering method is improved by integrating with NS and employed for the color image segmentation. The computation of membership and the clustering termination criterion are redefined accordingly. Moreover, a validity criterion is employed to determine the optimal clustering number. Numerical experiments serve to illustrate the effectiveness and reliability of the proposed approach. Experimental results demonstrate that our approach can segment color images automatically and effectively, produce good results as favorably compared to some existing algorithms. The optimal clustering number is determined automatically and no prior knowledge is required. Especially, it can segment both images with the simple and distinct objects and the images with complex and noisy objects, which is the most difficult task for color image segmentation.     

Image segmentation using fuzzy energy-based active contour with shape prior

This paper presents a fuzzy energy-based active contour model with shape prior for image segmentation. The paper proposes a fuzzy energy functional including a data term and a shape prior term. The data term, inspired from the region-based active contour approach proposed by Chan and Vese, evolves the contour relied on image information. The shape term inspired from Chan and Zhu’s work, defined as the distance between the evolving shape and a reference one, constrains the evolving contour with respect to the reference shape. To align the shapes, we exploit the shape normalization procedure which takes into account the affine transformation. In addition, to minimize the energy functional, we utilize a direct method to calculate the energy alterations. The proposed model therefore can deal with images with background clutter and object occlusion, improves the computational speed, and avoids difficulties associated with time step selection issue in gradient descent-based approaches.     

Modeling Interaction for Segmentation of Neighboring Structures

This paper presents a new method for segmenting medical images by modeling interaction between neighboring structures. Compared to previously reported methods, the proposed approach enables simultaneous segmentation of multiple neighboring structures for improved robustness. During the segmentation process, the object contour evolution and shape prior estimates are influenced by the interactions between neighboring shapes consisting of attraction, repulsion, and competition. Instead of estimating the a priori shape of each structure independently, an interactive maximum a posteriori shape estimation method is used for estimating the shape priors by considering shape prior distribution, neighboring shapes, and image features. Energy functionals are then formulated to model the interaction and segmentation. With the proposed method, neighboring structures with similar intensities and/or textures, and blurred boundaries can be extracted simultaneously. Experimental results obtained on both synthetic data and medical images demonstrate that the introduced interaction between neighboring structures improves segmentation performance compared with other existing approaches.      

基于图像分解的稀疏正则化多区域图像分割方法

针对图像具有不同特征的成分,提出一种基于图像分解的多区域图像分割模型和算法.首先将图像分解项引入到图像分割模型中,递减了纹理和噪声对分割的影响;其次使用稀疏正则化方法保持分割区域的边缘几何结构;最后基于增广Lagrange乘子法,给出一种由扩散流引导的小波迭代阈值图像分割算法.一系列实验结果表明,提出的方法抗干扰能力强,对噪声具有更好的鲁棒性.提出的方法不仅能够分割结构图像,并且能够分割较复杂的纹理图像.     

Simultaneous vector-valued image segmentation and intensity nonuniformity correction using variational level set combined with Markov random field modeling

Object quantification requires an image segmentation to make measurements about size, material composition and morphology of the object. In vector-valued or multispectral images, each image channel has its signal characteristics and provides special information that may improve the results of image segmentation method. This paper presents a region-based active contour model for vector-valued image segmentation with a variational level set formulation. In this model, the local image intensities are characterized using Gaussian distributions with different means and variances. Furthermore, by utilizing Markov random field, the spatial correlation between neighboring pixels and voxels is modeled. With incorporation of intensity nonuniformity model, our method is able to deal with brain tissue segmentation from multispectral magnetic resonance (MR) images. Our experiments on synthetic images and multispectral cerebral MR images with different noise and bias level show the advantages of the proposed method.