用于图像分割的局部区域一致性流形约束MRF模型

针对区域马尔可夫随机场(MRF)模型难以有效描述图像复杂先验知识的问题,提出一种基于局部区域一致性流形约束MRF(LRCMC-MRF)模型.首先,所提模型利用高维数据的低维流形分布表征图像局部区域的复杂几何结构先验,建立图像局部区域的流形先验约束;其次,基于Pairwise MRF模型,建立一种包含更多图像局部信息的局部空间自适应MRF模型;最后,基于贝叶斯理论,将复杂局部区域几何结构先验和局部空间自适应统计特征融合,利用Gibbs采样算法对所提出模型进行优化.实验结果表明,与基于常规区域的MRF模型相比,所提出的分割算法具有较好的分割效果.     

Fields of Experts

We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach provides a practical method for learning high-order Markov random field (MRF) models with potential functions that extend over large pixel neighborhoods. These clique potentials are modeled using the Product-of-Experts framework that uses non-linear functions of many linear filter responses. In contrast to previous MRF approaches all parameters, including the linear filters themselves, are learned from training data. We demonstrate the capabilities of this Field-of-Experts model with two example applications, image denoising and image inpainting, which are implemented using a simple, approximate inference scheme. While the model is trained on a generic image database and is not tuned toward a specific application, we obtain results that compete with specialized techniques. The work for this paper was performed while S.R. was at Brown University.     

Make3D: Learning 3D Scene Structure from a Single Still Image

We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov Random Field (MRF) to infer a set of "plane parameters” that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.     

Multiscale Bayesian texture segmentation using neural networks and Markov random fields

This paper presents a wavelet-based texture segmentation method using multilayer perceptron (MLP) networks and Markov random fields (MRF) in a multi-scale Bayesian framework. Inputs and outputs of MLP networks are constructed to estimate a posterior probability. The multi-scale features produced by multi-level wavelet decompositions of textured images are classified at each scale by maximum a posterior (MAP) classification and the posterior probabilities from MLP networks. An MRF model is used in order to model the prior distribution of each texture class, and a factor, which fuses the classification information through scales and acts as a guide for the labeling decision, is incorporated into the MAP classification of each scale. By fusing the multi-scale MAP classifications sequentially from coarse to fine scales, our proposed method gets the final and improved segmentation result at the finest scale. In this fusion process, the MRF model serves as the smoothness constraint and the Gibbs sampler acts as the MAP classifier. Our texture segmentation method was applied to segmentation of gray-level textured images. The proposed segmentation method shows better performance than texture segmentation using the hidden Markov trees (HMT) model and the HMTseg algorithm, which is a multi-scale Bayesian image segmentation algorithm.     

基于多节点拓扑重叠测度高阶MRF模型的图像分割

针对低阶马尔科夫随机场(Markov random field, MRF)模型难以有效表达自然图像中复杂的先验知识而造成误分割问题, 提出一种基于多节点拓扑重叠测度高阶MRF模型(Higher-order MRF model with multi-node topological overlap measure, MTOM-HMRF)的图像分割方法. 首先, 为描述图像局部区域内多像素蕴含的复杂空间拓扑结构信息, 利用多节点拓扑重叠测度建立图像局部区域的高阶先验模型; 其次, 利用较大的局部区域包含更多的标签节点信息能力, 基于Pairwise MRF模型建立基于局部区域的部分二阶Potts先验模型, 提高分割模型的抗噪能力; 再次, 为有效描述观察图像场与其标签场的似然特征分布, 研究利用局部区域内邻接像素的Hamming距离引入图像局部空间相关性, 建立局部空间一致性约束的高斯混合分布; 最后, 基于MRF框架建立用于图像分割的多节点拓扑重叠测度高阶MRF模型, 采用Gibbs采样算法对提出模型进行优化. 实验结果表明, 提出模型不仅能有效抵抗图像强噪声和复杂的纹理突变干扰, 鲁棒性更好, 而且具有更准确的图像分割结果.     

非参数图像分割算法的研究及改进

对传统的Dirichlet过程混合(MDP)非参数算法进行改进,提出一种新的MDP非参数图像分割算法。引入马尔可夫随机场(MRF)空间领域关系,并将其作为空间先验约束条件对图像后验概率加以约束。该算法能够光滑图像中的边缘部分、控制分类数并加快收敛速度。实验结果表明,与传统算法相比,该算法的分割准确度较高。     

Weakly-supervised region annotation for understanding scene images

Scene image understanding has drawn much attention for its intriguing applications in the past years. In this paper, we propose a unified probabilistic graphical model called Topic-based Coherent Region Annotation (TCRA) for weakly-supervised scene region annotation. The multiscale over-segmented regions within a scene image are considered as the “words” of our topic model, which impose neighborhood contextual constraints on topic level through spatial MRF modeling, and incorporate an annotation reasoning mechanism for learning and inferring region labels automatically. Mean field variational inference is provided for model learning. The proposed TCRA has the following two main advantages for understanding natural scene images. First, spatial information of multiscale over-segmented regions is explicitly modeled to obtain coherent region annotations. Second, only image-level labels are needed for automatically inferring the label of every region within the scene. This is particularly helpful in reducing human burden on manually labeling pixel-level semantics in the scene understanding research. Thus, given a scene image that has no textual prior, the regions in it can be automatically labeled using the learned TCRA model. The experimental results conducted on three benchmarks consisting of the MSRCORID image dataset, the UIUC Events image dataset and the SIFT FLOW dataset show that the proposed model outperforms the recent state-of-the-art methods.     

基于马尔可夫随机场的低分辨率车牌图像复原算法*

由于采集到的车牌图像分辨率较小,为获取高分辨率的图像,提出基于马尔可夫随机场模型的车牌图像超分辨率复原算法。首先对图像进行分块;然后利用马尔可夫随机场对这些分块进行建模,通过模型学习训练库中高低分辨率图像的关系,预测待复原的低分辨率车牌图像的高频细节信息。实验结果表明,本算法对车牌图像取得较好的复原效果,算法复原的超分辨率车牌图像更接近于真实图像,具有更高的峰值信噪比。     

Preprocessing of Low-Quality Handwritten Documents Using Markov Random Fields

This paper presents a statistical approach to the preprocessing of degraded handwritten forms including the steps of binarization and form line removal. The degraded image is modeled by a Markov Random Field (MRF) where the hidden-layer prior probability is learned from a training set of high-quality binarized images and the observation probability density is learned on-the-fly from the gray-level histogram of the input image. We have modified the MRF model to drop the preprinted ruling lines from the image. We use the patch-based topology of the MRF and Belief Propagation (BP) for efficiency in processing. To further improve the processing speed, we prune unlikely solutions from the search space while solving the MRF. Experimental results show higher accuracy on two data sets of degraded handwritten images than previously used methods.     

基于多尺度马尔可夫随机场的图像分割

离散马尔可夫随机场(MRF)模型是贝叶斯图像分割中最常用的工具。一般采用双MRF,一个随机场对应于观测图像,另一个随机场对应于未知的分类标号,通过迭代的算法将图像的局部信息逐步传递到整个图像,以求得分割标号的最大后验概率(MAP)或最大后验边缘概率(MPM)估计。近年来提出的多尺度MRF模型(或称因果MRF、分层MRF模     

基于图像片马尔科夫随机场的脑MR图像分割算法

传统的高斯混合模型（Gaussian mixture model,GMM）算法在图像分割中未考虑像素的空间信息,导致其对于噪声十分敏感.马尔科 夫随机场（Markov random field,MRF）模型通过像素类别标记的Gibbs分布先验概率引入了图像的空间信息,能较好地分割含有噪声的图 像,然而MRF模型的分割结果容易出现过平滑现象.为了解决上述缺陷,提出了一种新的基于图像片权重方法的马 尔科夫随机场图像分割模型,对邻域内的不同图像片根据相似度赋予不同的权重,使其在克服噪声影响的同时能 保持图像细节信息.同时,采用KL距离引入先验概率与后验概率关于熵的惩罚项,并对该惩罚项进行平滑,得到 最终的分割结果.实验结果表明,算法具有较强的自适应性,能够有效克服噪声对于分割结果的影响,并获得较高的分割精度.     

基于小波域层次Markov模型的图像分割

针对两个状态的有限高斯混合模型逼近小波系数的不足和小波域隐马尔可夫树标号场相互独立的缺点,提出了一种基于小波域层次马尔可夫模型的图像分割算法,这种模型用有限通用混合模型逼近小波系数的分布,使有限高斯混合模型只是其一种特殊情况;在标号场的先验模型确定上,利用马尔可夫模型描述标号场的局部作用关系,给出标号场的具体表达式,克服了小波域马尔可夫树模型标号场相互独立的不足,然后利用贝叶斯准则,给出相应的分割因果算法。该模型不仅具有空域马尔可夫模型有效的递归算法的优点,同时具有小波域隐马尔可夫树模型中的马尔可夫参数变尺度行为。最后用真实的图像和合成图像同几种分割方法进行了对比实验,实验结果表明了本文算法的有效性和优异性。     

基于边缘约束局部区域MRF的图像分割方法

针对常规马尔科夫随机场（MRF）模型对复杂自然图像分割时,存在对噪声敏感且边缘模糊的问题,构建一种基于边缘约束局部区域MRF（ECLRMRF）的图像分割模型。利用欧氏距离度量局部区域内邻接像素的相似度,依据其相似度构建局部空间来约束高斯混合模型,有效描述丰富的局部区域统计特征,并建立MRF模型的局部区域一致性约束项。利用Canny边缘检测算子提取图像的边缘特征,并在分割过程中建立图像分割区域的边缘约束,通过在MRF模型框架下将局部区域统计特征和图像边缘特征相融合,解决局部区域MRF模型对图像分割边缘模糊的问题,再采用Gibbs采样算法实现对复杂自然图像的准确分割。实验结果表明,该模型能够更好地保留图像边缘信息,并且具有更好的分割效果。     

Bayesian grid matching

A method for locating distorted grid structures in images is presented. The method is based on the theories of template matching and Bayesian image restoration. The grid is modeled as a deformable template. Prior knowledge of the grid is described through a Markov random field (MRF) model which represents the spatial coordinates of the grid nodes. Knowledge of how grid nodes are depicted in the observed image is described through the observation model. The prior consists of a node prior and an arc (edge) prior, both modeled as Gaussian MRFs. The node prior models variations in the positions of grid nodes and the arc prior models variations in row and column spacing across the grid. Grid matching is done by placing an initial rough grid over the image and applying an ensemble annealing scheme to maximize the posterior distribution of the grid. The method can be applied to noisy images with missing grid nodes and grid-node artifacts and the method accommodates a wide range of grid distortions including: large-scale warping, varying row/column spacing, as well as nonrigid random fluctuations of the grid nodes. The methodology is demonstrated in two case studies concerning (1) localization of DNA signals in hybridization filters and (2) localization of knit units in textile samples.     

Bilinear sparse coding for invariant vision

Recent algorithms for sparse coding and independent component analysis (ICA) have demonstrated how localized features can be learned from natural images. However, these approaches do not take image transformations into account. We describe an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model. We show that from an arbitrary set of natural images, the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformations. The learned generative model can be used to translate features to different locations, thereby reducing the need to learn the same feature at multiple locations, a limitation of previous approaches to sparse coding and ICA. Our results suggest that by explicitly modeling the interaction between local image features and their transformations, the sparse bilinear approach can provide a basis for achieving transformation-invariant vision.     

Scale selection for anisotropic diffusion filter by Markov random field model

The selection of stopping time (i.e., scale) significantly affects the performance of anisotropic diffusion filter for image denoising. This paper designs a Markov random field (MRF) scale selection model, which selects scales for image segments, then the denoised image is the composition of segments at their optimal scales in the scale space. Firstly, statistics-based scale selection criteria are proposed for image segments. Then we design a scale selection energy function in the MRF framework by considering the scale coherence between neighboring segments. A segment-based noise estimation algorithm is also developed to estimate the noise statistics efficiently. Experiments show that the performance of MRF scale selection model is much better than the previous global scale selection schemes. Combined with this scale selection model, the anisotropic diffusion filter is comparable to or even outperform the state-of-the-art denoising methods in performance.     

Predictions of the spontaneous symmetry-breaking theory for visual code completeness and spatial scaling in single-cell learning rules

This article shows analytically that single-cell learning rules that give rise to oriented and localized receptive fields, when their synaptic weights are randomly and independently initialized according to a plausible assumption of zero prior information, will generate visual codes that are invariant under two-dimensional translations, rotations, and scale magnifications, provided that the statistics of their training images are sufficiently invariant under these transformations. Such codes span different image locations, orientations, and size scales with equal economy. Thus, single-cell rules could account for the spatial scaling property of the cortical simple-cell code. This prediction is tested computationally by training with natural scenes; it is demonstrated that a single-cell learning rule can give rise to simple-cell receptive fields spanning the full range of orientations, image locations, and spatial frequencies (except at the extreme high and low frequencies at which the scale invariance of the statistics of digitally sampled images must ultimately break down, because of the image boundary and the finite pixel resolution). Thus, no constraint on completeness, or any other coupling between cells, is necessary to induce the visual code to span wide ranges of locations, orientations, and size scales. This prediction is made using the theory of spontaneous symmetry breaking, which we have previously shown can also explain the data-driven self-organization of a wide variety of transformation invariances in neurons' responses, such as the translation invariance of complex cell response.     

User assisted separation of reflections from a single image using a sparsity prior

When we take a picture through transparent glass the image we obtain is often a linear superposition of two images: the image of the scene beyond the glass plus the image of the scene reflected by the glass. Decomposing the single input image into two images is a massively ill-posed problem: in the absence of additional knowledge about the scene being viewed there are an infinite number of valid decompositions. In this paper we focus on an easier problem: user assisted separation in which the user interactively labels a small number of gradients as belonging to one of the layers. Even given labels on part of the gradients, the problem is still ill-posed and additional prior knowledge is needed. Following recent results on the statistics of natural images we use a sparsity prior over derivative filters. This sparsity prior is optimized using the terative reweighted least squares (IRLS) approach. Our results show that using a prior derived from the statistics of natural images gives a far superior performance compared to a Gaussian prior and it enables good separations from a modest number of labeled gradients.     

3-D Depth Reconstruction from a Single Still Image

We consider the task of 3-d depth estimation from a single still image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured indoor and outdoor environments which include forests, sidewalks, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the value of the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a hierarchical, multiscale Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models the depths and the relation between depths at different points in the image. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps. We further propose a model that incorporates both monocular cues and stereo (triangulation) cues, to obtain significantly more accurate depth estimates than is possible using either monocular or stereo cues alone.     

Study on segmentation and interpretation of multi-look polarimetric SAR images

The segmentation and interpretation of multi-look polarimetric synthetic aperture radar (SAR) images is studied. We first introduce a multi-look polarimetric whitening filter (MPWF) to reduce the speckle in multi-look polarimetric SAR images. Then, by utilizing the wavelet multiresolution approach to extract the texture information in different scales and the Markov random field (MRF) model to characterize the spatial constraints between pixels in each scale level, a multiresolution segmentation algorithm (MSA) to segment the speckle-reduced SAR images is presented. The MSA first segments the image at the lowest resolution level and then proceeds to progressively higher resolutions until individual pixels are well classified. An unsupervised step to estimate both the optimal number of texture classes and their model parameters is also included in the MSA so that the segmentation can be implemented without supervision. Finally, in order to interpret the results of the unsupervised segmentation and to understand the whole polarimetric SAR image, we develop an image interpretation approach which jointly utilizes the scattering mechanism identification and target decomposition approaches. Experimental results with the real-world multi-look polarimetric SAR image demonstrate the effectiveness of the segmentation and interpretation approaches.