PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosaicking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed “denoising first and demosaicking later” scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA) based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existed in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosaicking and denoising schemes, in terms of both objective measurement and visual evaluation.      

Color reproduction from noisy CFA data of single sensor digital cameras.

Single sensor digital color still/video cameras capture images using a color filter array (CFA) and require color interpolation (demosaicking) to reconstruct full color images. The color reproduction has to combat sensor noises which are channel dependent. If untreated in demosaicking, sensor noises can cause color artifacts that are hard to remove later by a separate denoising process, because the demosaicking process complicates the noise characteristics by blending noises of different color channels. This paper presents a joint demosaicking-denoising approach to overcome this difficulty. The color image is restored from noisy mosaic data in two steps. First, the difference signals of color channels are estimated by linear minimum mean square-error estimation. This process exploits both spectral and spatial correlations to simultaneously suppress sensor noise and interpolation error. With the estimated difference signals, the full resolution green channel is recovered. The second step involves in a wavelet-based denoising process to remove the CFA channel-dependent noises from the reconstructed green channel. The red and blue channels are subsequently recovered. Simulated and real CFA mosaic data are used to evaluate the performance of the proposed joint demosaicking-denoising scheme and compare it with many recently developed sophisticated demosaicking and denoising schemes.     

Color filter array demosaicking: new method and performance measures

Single-sensor digital cameras capture imagery by covering the sensor surface with a color filter array (CFA) such that each sensor pixel only samples one of three primary color values. To render a full-color image, an interpolation process, commonly referred to as CFA demosaicking, is required to estimate the other two missing color values at each pixel. In this paper, we present two contributions to the CFA demosaicking: a new and improved CFA demosaicking method for producing high quality color images and new image measures for quantifying the performance of demosaicking methods. The proposed demosaicking method consists of two successive steps: an interpolation step that estimates missing color values by exploiting spatial and spectral correlations among neighboring pixels, and a post-processing step that suppresses noticeable demosaicking artifacts by adaptive median filtering. Moreover, in recognition of the limitations of current image measures, we propose two types of image measures to quantify the performance of different demosaicking methods; the first type evaluates the fidelity of demosaicked images by computing the peak signal-to-noise ratio and CIELAB /spl utri/E/sup *//sub ab/ for edge and smooth regions separately, and the second type accounts for one major demosaicking artifact-zipper effect. We gauge the proposed demosaicking method and image measures using several existing methods as benchmarks, and demonstrate their efficacy using a variety of test images.     

Adaptive filtering for color filter array demosaicking.

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: (1) the high frequencies are similar across three color components, and (2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost.     

Improvement of color video demosaicking in temporal domain.

Color demosaicking is critical to the image quality of digital still and video cameras that use a single-sensor array. Limited by the mosaic sampling pattern of the color filter array (CFA), color artifacts may occur in a demosaicked image in areas of high-frequency and/or sharp color transition structures. However, a color digital video camera captures a sequence of mosaic images and the temporal dimension of the color signals provides a rich source of information about the scene via camera and object motions. This paper proposes an inter-frame demosaicking approach to take advantage of all three forms of pixel correlations: spatial, spectral, and temporal. By motion estimation and statistical data fusion between adjacent mosaic frames, the new approach can remove much of the color artifacts that survive intra-frame demosaicking and also improve tone reproduction accuracy. Empirical results show that the proposed inter-frame demosaicking approach consistently outperforms its intra-frame counterparts both in peak signal-to-noise measure and subjective visual quality.     

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.     

基于压缩感知的正六边形CFA模式彩色图像去马赛克方法

针对基于四边形排列的去马赛克(Demosaicking)的 传统方法存在拉链现象和虚假色等问题,本文尝试将更加符合人眼视觉特性的六边形采 样方式应用于彩色图像成像 系统,并从图像稀疏特性角度出发,提出基于压缩感知(Compressive sensing,CS)框架的 彩色图像去马赛克方法。本文方法 充分挖掘了彩色分量间和分量内的稀疏特性,可使复原图像的纹理细节与色彩更加逼真,有 效地避免了拉链现象和虚假色现象。实验结果验证了本文方法的有效性。     

A low-complexity joint color demosaicking and zooming algorithm for digital camera.

This paper presents a low complexity joint color demosaicking and digital zooming algorithm for single-sensor digital cameras. The proposed algorithm directly extracts edge information from raw sensor data for interpolation in both demosaicking and zooming to preserve edge features in its output. This allows the extracted information to be exploited consistently in both stages and also efficiently, as no separate extraction process is required in different stages. The proposed algorithm can produce a zoomed full-color image as well as a zoomed Bayer color filter array image with outstanding performance as compared with conventional approaches which generally combine separate color demosaicking and digital zooming schemes.     

Spatio-spectral color filter array design for optimal image recovery

In digital imaging applications, data are typically obtained via a spatial subsampling procedure implemented as a color filter array-a physical construction whereby only a single color value is measured at each pixel location. Owing to the growing ubiquity of color imaging and display devices, much recent work has focused on the implications of such arrays for subsequent digital processing, including in particular the canonical demosaicking task of reconstructing a full color image from spatially subsampled and incomplete color data acquired under a particular choice of array pattern. In contrast to the majority of the demosaicking literature, we consider here the problem of color filter array design and its implications for spatial reconstruction quality. We pose this problem formally as one of simultaneously maximizing the spectral radii of luminance and chrominance channels subject to perfect reconstruction, and-after proving sub-optimality of a wide class of existing array patterns-provide a constructive method for its solution that yields robust, new panchromatic designs implementable as subtractive colors. Empirical evaluations on multiple color image test sets support our theoretical results, and indicate the potential of these patterns to increase spatial resolution for fixed sensor size, and to contribute to improved reconstruction fidelity as well as significantly reduced hardware complexity.     

Secure single-sensor digital camera

An efficient secure single-sensor camera for captured image copyright protection and ownership authentication is introduced. Visible watermarks are embedded during the colour filter array (CFA) data acquisition process. Subsequent demosaicking of the watermarked CFA grey-scale images generates secure full-colour images readily available for storage or distribution.     

Joint demosaicing and denoising.

The output image of a digital camera is subject to a severe degradation due to noise in the image sensor. This paper proposes a novel technique to combine demosaicing and denoising procedures systematically into a single operation by exploiting their obvious similarities. We first design a filter as if we are optimally estimating a pixel value from a noisy single-color (sensor) image. With additional constraints, we show that the same filter coefficients are appropriate for color filter array interpolation (demosaicing) given noisy sensor data. The proposed technique can combine many existing denoising algorithms with the demosaicing operation. In this paper, a total least squares denoising method is used to demonstrate the concept. The algorithm is tested on color images with pseudorandom noise and on raw sensor data from a real CMOS digital camera that we calibrated. The experimental results confirm that the proposed method suppresses noise (CMOS/CCD image sensor noise model) while effectively interpolating the missing pixel components, demonstrating a significant improvement in image quality when compared to treating demosaicing and denoising problems independently.     

Reversing demosaicking and compression in color filter array image processing: performance analysis and modeling.

In the conventional processing chain of single-sensor digital still cameras (DSCs), the images are captured with color filter arrays (CFAs) and the CFA samples are demosaicked into a full color image before compression. To avoid additional data redundancy created by the demosaicking process, an alternative processing chain has been proposed to move the compression process before the demosaicking. Recent empirical studies have shown that the alternative chain can outperform the conventional one in terms of image quality at low compression ratios. To provide a theoretically sound basis for such conclusion, we propose analytical models for the reconstruction errors of the two processing chains. The models developed confirm the results of existing empirical studies and provide better understanding of DSC processing chains. The modeling also allows performance predictions for more advanced compression and demosaicking methods, thus providing important cues for future development in this area.     

Binary tree-based generic demosaicking algorithm for multispectral filter arrays.

In this paper, we extend the idea of using mosaicked color filter array (CFA) in color imaging, which has been widely adopted in the digital color camera industry, to the use of multispectral filter array (MSFA) in multispectral imaging. The filter array technique can help reduce the cost, achieve exact registration, and improve the robustness of the imaging system. However, the extension from CFA to MSFA is not straightforward. First, most CFAs only deal with a few bands (3 or 4) within the narrow visual spectral region, while the design of MSFA needs to handle the arrangement of multiple bands (more than 3) across a much wider spectral range. Second, most existing CFA demosaicking algorithms assume the fixed Bayer CFA and are confined to properties only existed in the color domain. Therefore, they cannot be directly applied to multispectral demosaicking. The main challenges faced in multispectral demosaicking is how to design a generic algorithm that can handle the more diversified MSFA patterns, and how to improve performance with a coarser spatial resolution and a less degree of spectral correlation. In this paper, we present a binary tree based generic demosaicking method. Two metrics are used to evaluate the generic algorithm, including the root mean-square error (RMSE) for reconstruction performance and the classification accuracy for target discrimination performance. Experimental results show that the demosaicked images present low RMSE (less than 7) and comparable classification performance as original images. These results support that MSFA technique can be applied to multispectral imaging with unique advantages.     

Fabrication and performance of color filter arrays for solid-state imagers

A technique has been developed for the fabrication of a color filter array (CFA) to be used in conjunction with a solid-state area sensor to provide three-color image information from a single sensor. The fabrication technique employs sublimable dyes which are heat-transferred through photoresist windows onto a polymer receiving layer. Good edge sharpness and a low dye penetration depth into the polymer have been achieved. The predicted and measured spectral response of the color channels of the composite device are in good agreement. The pattern noise associated with CFA transmittance fluctuations from element to element is on the order of 10 percent.     

基于色差空间的Bayer 图像的迭代插值算法

单CCD/CMOS 传感器相机捕捉图像信息靠在传感器表面覆盖一层颜色滤波阵列(CFA),经过CFA 后每个像素点只能获得物理三基色(红,绿,蓝)其中一种分量。另外缺少的两种颜色分量,需要通过周围像素的值来估算。首先利用55 模板内的像素来估计插值的方向并用最优的权重系数来插值G 分量。其次利用了基于有理函数的二维插值算子在色差空间插值R(B)处缺少的B(R)分量。再次利用色差插值G 处缺少的R 和B 分量。最后,使用方差约束条件,迭代插值过程被重复多次直到达到了最优的插值结果。通过在24 幅柯达图片以及笔者相机拍摄的图片上的Matlab 仿真实验,结果显示,被提出的算法无论是在视觉方面还是在量化的数据方面都表现出了优势。     

A lossless compression scheme for Bayer color filter array images.

In most digital cameras, Bayer color filter array (CFA) images are captured and demosaicing is generally carried out before compression. Recently, it was found that compression-first schemes outperform the conventional demosaicing-first schemes in terms of output image quality. An efficient prediction-based lossless compression scheme for Bayer CFA images is proposed in this paper. It exploits a context matching technique to rank the neighboring pixels when predicting a pixel, an adaptive color difference estimation scheme to remove the color spectral redundancy when handling red and blue samples, and an adaptive codeword generation technique to adjust the divisor of Rice code for encoding the prediction residues. Simulation results show that the proposed compression scheme can achieve a better compression performance than conventional lossless CFA image coding schemes.     

Regularization Approaches to Demosaicking

Demosaicking is the process of reconstructing a full resolution color image from the sampled data acquired by a digital camera that apply a color filter array to a single sensor. In this paper, we propose a regularization approach to demosaicking, making use of some prior knowledge about natural color images, such as smoothness of each single color component and correlation between the different color channels. Initially, a quadratic strategy is considered and a general approach is reported. Then, an adaptive technique is analyzed, in order to improve the reconstruction near the edges and the discontinuities of the image. This is performed using a novel strategy that avoids computational demanding iterations. The proposed approach provides good performances and candidates itself for many applications. Moreover, since the response of the pixel sensors can be taken into account, it can handle nonideal acquisition devices.     

引入双边滤波器优化的彩色滤波阵列插值

基于彩色滤波阵列(CFA)的图像传感器在每个像素 位置获得三原色红(R)、绿(G)和蓝(B)中的一种分量 ,其缺失 分量需要根据周围像素插值得到。目前提出的许多种插值算法,绝大部分采用Bayer排列模 式。本文在色 差恒定假设基础上,提出一种基于双边滤波器的自适应Bayer模型插值算法,对G通道的估计采 用自适应滤波器进行插值,对R和B通道的插值采用双边滤波器。算法利用待插值像素 与不同距离像 素相关性不同的思想,根据图像边缘自适应设定滤波模板,能较准确估计G、B和R、G通 道之间的色差值。 实验结果表明,对比多尺度色差梯度算法和边缘强度滤波等算法,插值后的图像不仅 主观视觉, 且客观评价指标(彩色峰值信噪比,CPSNR)均优于这些算法。     

Analysis and measurement of pattern noise in color-filter arrays for image sensors

Pattern-noise measurements on a color-filter array (CFA) for an image sensor are valuable for quantifying the pixel-to-pixel transmittance uniformity important for good image quality. The pattern noise, however, changes with test conditions and is not an intrinsic array property. We develop here a relationship between the pattern noise and the fluctuations in density parameters that produce it. Determining these fundamental parameters makes clear the source of the noise and permits calculation of the filter-array performance for any conditions of interest. The analysis shows how the filter-array spectral shapes influence the pattern noise and how it can be reduced. The pattern noise of CFA's of the dyed photopolymer type is reported. Monochromatic measurements, using microdensitometry and an image sensor with an RGBG striped array, confirm a predicted increase in pattern noise with dye density. Fitting the data to the model yields fluctuations in dye coverage of 0.2-0.4 percent, depending on the dye. For a frame-transfer CCD in white light (5500 K) this corresponds to pattern noise of 0.4 percent or less. Power spectra show that these variations in dye concentration occur largely over sensor dimensions, not pixel spacings.     

Fabrication and Performance of Color Filter Arrays for Solid-State Imagers

A technique has been developed for the fabrication of a color filter array (CFA) to be used in conjunction with a solid-state area sensor to provide three-color image information from a single sensor. The fabrication technique employs sublimable dyes which are heat-transferred through photoresist windows onto a polymer receiving layer. Good edge sharpness and a low dye penetration depth into the polymer have been achieved. The predicted and measured spectral response of the color channels of the composite device are in good agreement. The pattern noise associated with CFA transmittance fluctuations from element to element is on the order of 10 percent.