Adaptive Linear Predictive Coding of Time-Varying Images Using Multidimensional Recursive Least Squares Ladder Filters

This paper presents several adaptive linear predictive coding techniques based upon extension of recursive ladder filters to two and three dimensions (2-D/3-D). A 2-D quarter-plane autoregressive ladder filter is developed using a least square criterion in an exact recursive fashion. The 2-D recursive ladder filter is extended to a 3-D case which can adaptively track the variation of both spatial and temporal changes of moving images. Using the 2-D/3-D ladder filters and a previous frame predictor, two types of adaptive predictor-control schemes are proposed in which the prediction error at each pel can be obtained at or close to a minimum level. We also investigate several modifications of the basic encoding methods. Performance of the 2D/3-D ladder filters, their adaptive control schemes, and variations in coding methods are evaluated by computer simulations on two real sequences and compared to the results of motion compensation and frame differential coders. As a validity test of the ladder filters developed, the error signals for the different predictors are compared and the visual quality of output images is verified.     

On the adaptive three-dimensional transform coding techniques for moving images

In this paper, an adaptive three-dimensional transform coding technique based on the 3-D discrete cosine transform (DCT) for removing the temporal correlation is proposed. Because of the nonstationary nature of the image data, the energy distribution in a 3-D DCT block varies along the vertical, horizontal and temporal directions. Thus, adaptive schemes, such as the 3-D classification, the classified linear scanning technique and the VLC table selection scheme, are used to take local variations into account. Also, in our approach, a hybrid technique, which adaptively combines relatively simple inter-frame coding with intra-frame coding, is presented. Through intensive computer simulations, the performance of the proposed 3-D transform coding technique is evaluated on several well-known moving sequences. The results show that, especially for moving sequences containing slow or moderate motion, the proposed technique provides an improved performance over the scheme with motion compensation (CCITT, 1989) at rates above 0.5 b/pixel (bpp), and a good visual quality of the reconstructed images is also obtained. Thus, the proposed 3-D transform coding technique is believed to be a good candidate for the digital VCR, since motion compensation is not required in the proposed 3-D coding technique.     

Block classification for an adaptive 1-D/2-D DCT video coding

A classification scheme for an adaptive one- or two-dimensional discrete cosine transform (1-D/2-D DCT) technique is described and demonstrated to be a more appropriate strategy than the conventional 2-D DCT for coding motion compensated prediction error images. Two block-based classification methods are introduced and their accuracy in predicting the correct transform type discussed. The accuracy is assessed with a classification measure designed to ascertain the effectiveness of energy compaction when the predicted transform class is applied; vis-a-vis horizontally, vertically or two-dimensionally transformed blocks. Energy compaction is a useful property not only for efficient entropy coding but also for enhancing the resilience of the transform coder to quantisation noise. Improvements against the homogeneous 2-D DCT system both in terms of the peak signal to noise ratio and subjective assessments are achieved. Observable ringing artifacts along edges, which are usual in conventional transform coding, are reduced     

2-D mesh-based video object segmentation and tracking with occlusion resolution

This paper integrates fully automatic video object segmentation and tracking including detection and assignment of uncovered regions in a 2-D mesh-based framework. Particular contributions of this work are (i) a novel video object segmentation method that is posed as a constrained maximum contrast path search problem along the edges of a 2-D triangular mesh, and (ii) a 2-D mesh-based uncovered region detection method along the object boundary as well as within the object. At the first frame, an optimal number of feature points are selected as nodes of a 2-D content-based mesh. These points are classified as moving (foreground) and stationary nodes based on multi-frame node motion analysis, yielding a coarse estimate of the foreground object boundary. Color differences across triangles near the coarse boundary are employed for a maximum contrast path search along the edges of the 2-D mesh to refine the boundary of the video object. Next, we propagate the refined boundary to the subsequent frame by using motion vectors of the node points to form the coarse boundary at the next frame. We detect occluded regions by using motion-compensated frame differences and range filtered edge maps. The boundaries of detected uncovered regions are then refined by using the search procedure. These regions are either appended to the foreground object or tracked as new objects. The segmentation procedure is re-initialized when unreliable motion vectors exceed a certain number. The proposed scheme is demonstrated on several video sequences.     

VIDEO OBJECT SEGMENTATION BY 2-D MESH-BASED MOTION ANALYSIS

Video object extraction is a key technology in content-based video coding.A novel video object extracting algorithm by two Dimensional (2-D) mesh-based motion analysis is proposed in this paper.Firstly,a 2-D mesh fitting the original frame image is obtained via feature detection algorithm. Then,higher order statistics motion analysis is applied on the 2-D mesh representation to get an initial motion detection mask.After post-processing,the final segmenting mask is quickly obtained.And hence the video object is effectively extracted.Experimental results show that the proposed algorithm combines the merits of mesh-based segmenting algorithms and pixel-based segmenting algorithms,and hereby achieves satisfactory subjective and objective performance while dramatically increasing the segmenting speed.     

Two-dimensional mesh-based mosaic representation for manipulationof video objects with occlusion

We present a two-dimensional (2-D) mesh-based mosaic representation, consisting of an object mesh and a mosaic mesh for each frame and a final mosaic image, for video objects with mildly deformable motion in the presence of self and/or object-to-object (external) occlusion. Unlike classical mosaic representations where successive frames are registered using global motion models, we map the uncovered regions in the successive frames onto the mosaic reference frame using local affine models, i.e., those of the neighboring mesh patches. The proposed method to compute this mosaic representation is tightly coupled with an occlusion adaptive 2-D mesh tracking procedure, which consist of propagating the object mesh frame to frame, and updating of both object and mosaic meshes to optimize texture mapping from the mosaic to each instance of the object. The proposed representation has been applied to video object rendering and editing, including self transfiguration, synthetic transfiguration, and 2-D augmented reality in the presence of self and/or external occlusion. We also provide an algorithm to determine the minimum number of still views needed to reconstruct a replacement mosaic which is needed for synthetic transfiguration. Experimental results are provided to demonstrate both the 2-D mesh-based mosaic synthesis and two different video object editing applications on real video sequences.     

Three-dimensional encoding/two-dimensional decoding of medical data

We propose a fully three-dimensional (3-D) wavelet-based coding system featuring 3-D encoding/two-dimensional (2-D) decoding functionalities. A fully 3-D transform is combined with context adaptive arithmetic coding; 2-D decoding is enabled by encoding every 2-D subband image independently. The system allows a finely graded up to lossless quality scalability on any 2-D image of the dataset. Fast access to 2-D images is obtained by decoding only the corresponding information thus avoiding the reconstruction of the entire volume. The performance has been evaluated on a set of volumetric data and compared to that provided by other 3-D as well as 2-D coding systems. Results show a substantial improvement in coding efficiency (up to 33%) on volumes featuring good correlation properties along the z axis. Even though we did not address the complexity issue, we expect a decoding time of the order of one second/image after optimization. In summary, the proposed 3-D/2-D multidimensional layered zero coding system provides the improvement in compression efficiency attainable with 3-D systems without sacrificing the effectiveness in accessing the single images characteristic of 2-D ones.     

网格运动分析与形态学滤波相结合的视频对象分割

提出了一种基于二维网格运动分析与改进形态学滤波空域自动分割策略相结合的视频对象时空分割算法。该算法首先利用高阶统计方法对视频图像的二维网格表示进行运动分析，快速得到前景对象区域，通过后处理有效获得前景对象运动检测掩膜。然后，用一种结合交变序列重建滤波算法和自适应阈值判别算法的改进分水岭分割策略有效获得前景对象的精确边缘。最后，用区域基时空融合算法将时域分割结果和空域分割结果结合起来提取出边缘精细的视频对象。实验结果表明，本算法综合了多种算法的优点，主客观分割效果理想。     

Motion compensated lossy-to-lossless compression of 4-D medical images using integer wavelet transforms.

This paper proposes a method for progressive lossy-to-lossless compression of four-dimensional (4-D) medical images (sequences of volumetric images over time) by using a combination of three-dimensional (3-D) integer wavelet transform (IWT) and 3-D motion compensation. A 3-D extension of the set-partitioning in hierarchical trees (SPIHT) algorithm is employed for coding the wavelet coefficients. To effectively exploit the redundancy between consecutive 3-D images, the concepts of key and residual frames from video coding is used. A fast 3-D cube matching algorithm is employed to do motion estimation. The key and the residual volumes are then coded using 3-D IWT and the modified 3-D SPIHT. The experimental results presented in this paper show that our proposed compression scheme achieves better lossy and lossless compression performance on 4-D medical images when compared with JPEG-2000 and volumetric compression based on 3-D SPIHT.     

Occlusion-adaptive, content-based mesh design and forward tracking

Two-dimensional (2-D) mesh-based motion compensation preserves neighboring relations (through connectivity of the mesh) as well as allowing warping transformations between pairs of frames; thus, it effectively eliminates blocking artifacts that are common in motion compensation by block matching. However, available 2-D mesh models, whether uniform or non-uniform, enforce connectivity everywhere within a frame, which is clearly not suitable across occlusion boundaries. To this effect, we hereby propose an occlusion-adaptive forward-tracking mesh model, where connectivity of the mesh elements (patches) across covered and uncovered region boundaries are broken. This is achieved by allowing no node points within the background to be covered (BTBC) and refining the mesh structure within the model failure (MF) region(s) at each frame. The proposed content-based mesh structure enables better rendition of the motion (compared to a uniform or a hierarchical mesh), while tracking is necessary to avoid transmission of all node locations at each frame. Experimental results show successful motion compensation and tracking.     

Adaptive entropy-coded predictive vector quantization of images

The authors consider 2-D predictive vector quantization (PVQ) of images subject to an entropy constraint and demonstrate the substantial performance improvements over existing unconstrained approaches. They describe a simple adaptive buffer-instrumented implementation of this 2-D entropy-coded PVQ scheme which can accommodate the associated variable-length entropy coding while completely eliminating buffer overflow/underflow problems at the expense of only a slight degradation in performance. This scheme, called 2-D PVQ/AECQ (adaptive entropy-coded quantization), is shown to result in excellent rate-distortion performance and impressive quality reconstructions of real-world images. Indeed, the real-world coding results shown demonstrate little distortion at rates as low as 0.5 b/pixel     

Model-based image coding advanced video coding techniques for verylow bit-rate applications

The paper gives an overview of model-based approaches applied to image coding, by looking at image source models. In these model-based schemes, which are different from the various conventional waveform coding methods, the 3-D properties of the scenes are taken into consideration. They can achieve very low bit rate image transmission. The 2-D model and 3-D model based approaches are explained. Among them, a 3-D model based method using a 3-D facial model and a 2-D model based method utilizing 2-D deformable triangular patches are described. Works related to 3-D model-based coding of facial images and some of the remaining problems are also described     

Adaptive entropy coded subband coding of images

The authors describe a design approach, called 2-D entropy-constrained subband coding (ECSBC), based upon recently developed 2-D entropy-constrained vector quantization (ECVQ) schemes. The output indexes of the embedded quantizers are further compressed by use of noiseless entropy coding schemes, such as Huffman or arithmetic codes, resulting in variable-rate outputs. Depending upon the specific configurations of the ECVQ and the ECPVQ over the subbands, many different types of SBC schemes can be derived within the generic 2-D ECSBC framework. Among these, the authors concentrate on three representative types of 2-D ECSBC schemes and provide relative performance evaluations. They also describe an adaptive buffer instrumented version of 2-D ECSBC, called 2-D ECSBC/AEC, for use with fixed-rate channels which completely eliminates buffer overflow/underflow problems. This adaptive scheme achieves performance quite close to the corresponding ideal 2-D ECSBC system.     

Two-dimensional mesh-based visual-object representation forinteractive synthetic/natural digital video

This paper first provides an overview of two-dimensional (2-D) and three-dimensional mesh models for digital video processing. It then introduces 2-D mesh-based modeling of video objects as a compact representation of motion and shape for interactive, synthetic/natural video manipulation, compression, and indexing. The 2-D mesh representation and the mesh geometry and motion compression have been included in the visual tools of the upcoming MPEG-4 standard. Functionalities enabled by 2-D mesh-based visual-object representation include animation of still texture maps, transfiguration of video overlays, video morphing, and shape-and motion-based retrieval of video objects     

Adaptive fuzzy segmentation of magnetic resonance images

An algorithm is presented for the fuzzy segmentation of two-dimensional (2-D) and three-dimensional (3-D) multispectral magnetic resonance (MR) images that have been corrupted by intensity inhomogeneities, also known as shading artifacts. The algorithm is an extension of the 2-D adaptive fuzzy C-means algorithm (2-D AFCM) presented in previous work by the authors. This algorithm models the intensity inhomogeneities as a gain field that causes image intensities to smoothly and slowly vary through the image space. It iteratively adapts to the intensity inhomogeneities and is completely automated. In this paper, we fully generalize 2-D AFCM to three-dimensional (3-D) multispectral images. Because of the potential size of 3-D image data, we also describe a new faster multigrid-based algorithm for its implementation. We show, using simulated MR data, that 3-D AFCM yields lower error rates than both the standard fuzzy C-means (FCM) algorithm and two other competing methods, when segmenting corrupted images. Its efficacy is further demonstrated using real 3-D scalar and multispectral MR brain images.     

Adaptive enhancement of compressed SAR images

Synthetic aperture radar (SAR) imagery is an important global all-weather surveillance and mapping satellite imagery system. As space-borne systems have a limited storage capacity, it is imperative to heavily compress SAR images, possible with lossy compression schemes. As a result, SAR images need to be enhanced in earth stations. The work reported in this paper aims to address the issue of compression artefact removal of SAR images in an adaptive manner. The SAR images, compressed using the JPEG utility at significantly low bit rates, are enhanced by adaptively removing coding artefacts and speckle noise. As edges carry significant information in satellite imagery, a significant edge image is used for edge enhancement with selective removal of noisy edges. Further, an image sharpness metric is proposed in this work to serve as an objective no-reference metric for measuring the sharpness of SAR images.     

Local image registration by adaptive filtering.

We propose a new adaptive filtering framework for local image registration, which compensates for the effect of local distortions/displacements without explicitly estimating a distortion/displacement field. To this effect, we formulate local image registration as a two-dimensional (2-D) system identification problem with spatially varying system parameters. We utilize a 2-D adaptive filtering framework to identify the locally varying system parameters, where a new block adaptive filtering scheme is introduced. We discuss the conditions under which the adaptive filter coefficients conform to a local displacement vector at each pixel. Experimental results demonstrate that the proposed 2-D adaptive filtering framework is very successful in modeling and compensation of both local distortions, such as Stirmark attacks, and local motion, such as in the presence of a parallax field. In particular, we show that the proposed method can provide image registration to: a) enable reliable detection of watermarks following a Stirmark attack in nonblind detection scenarios, b) compensate for lens distortions, and c) align multiview images with nonparametric local motion.     

Orientation adaptive subband coding of images

In the subband coding of images, directionality of image features has thus far been exploited very little. The proposed subband coding scheme utilizes orientation of local image features to avoid the highly objectionable Gibbs-like phenomena observed at reconstructed image edges with conventional subband schemes at low bit rates, At comparable bit rates, the subjective image quality obtained by our orientation adaptive scheme is considerably enhanced over a conventional separable subband coding scheme, as well as other separable approaches such as the JPEG compression standard.     

Markerless real-time 3-D target region tracking by motion backprojection from projection images

Accurate and fast localization of a predefined target region inside the patient is an important component of many image-guided therapy procedures. This problem is commonly solved by registration of intraoperative 2-D projection images to 3-D preoperative images. If the patient is not fixed during the intervention, the 2-D image acquisition is repeated several times during the procedure, and the registration problem can be cast instead as a 3-D tracking problem. To solve the 3-D problem, we propose in this paper to apply 2-D region tracking to first recover the components of the transformation that are in-plane to the projections. The 2-D motion estimates of all projections are backprojected into 3-D space, where they are then combined into a consistent estimate of the 3-D motion. We compare this method to intensity-based 2-D to 3-D registration and a combination of 2-D motion backprojection followed by a 2-D to 3-D registration stage. Using clinical data with a fiducial marker-based gold-standard transformation, we show that our method is capable of accurately tracking vertebral targets in 3-D from 2-D motion measured in X-ray projection images. Using a standard tracking algorithm (hyperplane tracking), tracking is achieved at video frame rates but fails relatively often (32% of all frames tracked with target registration error (TRE) better than 1.2 mm, 82% of all frames tracked with TRE better than 2.4 mm). With intensity-based 2-D to 2-D image registration using normalized mutual information (NMI) and pattern intensity (PI), accuracy and robustness are substantially improved. NMI tracked 82% of all frames in our data with TRE better than 1.2 mm and 96% of all frames with TRE better than 2.4 mm. This comes at the cost of a reduced frame rate, 1.7 s average processing time per frame and projection device. Results using PI were slightly more accurate, but required on average 5.4 s time per frame. These results are still substantially faster than 2-D to 3-D registration. We conclude that motion backprojection from 2-D motion tracking is an accurate and efficient method for tracking 3-D target motion, but tracking 2-D motion accurately and robustly remains a challenge.     

Encoding Probabilistic Brain Atlases Using Bayesian Inference

This paper addresses the problem of creating probabilistic brain atlases from manually labeled training data. Probabilistic atlases are typically constructed by counting the relative frequency of occurrence of labels in corresponding locations across the training images. However, such an “averaging” approach generalizes poorly to unseen cases when the number of training images is limited, and provides no principled way of aligning the training datasets using deformable registration. In this paper, we generalize the generative image model implicitly underlying standard “average” atlases, using mesh-based representations endowed with an explicit deformation model. Bayesian inference is used to infer the optimal model parameters from the training data, leading to a simultaneous group-wise registration and atlas estimation scheme that encompasses standard averaging as a special case. We also use Bayesian inference to compare alternative atlas models in light of the training data, and show how this leads to a data compression problem that is intuitive to interpret and computationally feasible. Using this technique, we automatically determine the optimal amount of spatial blurring, the best deformation field flexibility, and the most compact mesh representation. We demonstrate, using 2-D training datasets, that the resulting models are better at capturing the structure in the training data than conventional probabilistic atlases. We also present experiments of the proposed atlas construction technique in 3-D, and show the resulting atlases' potential in fully-automated, pulse sequence-adaptive segmentation of 36 neuroanatomical structures in brain MRI scans.