Scale‐adaptive Structure‐preserving Texture Filtering

This paper proposes a scale‐adaptive filtering method to improve the performance of structure‐preserving texture filtering for image smoothing. With classical texture filters, it usually is challenging to smooth texture at multiple scales while preserving salient structures in an image. We address this issue in the concept of adaptive bilateral filtering, where the scales of Gaussian range kernels are allowed to vary from pixel to pixel. Based on direction‐wise statistics, our method distinguishes texture from structure effectively, identifies appropriate scope around a pixel to be smoothed and thus infers an optimal smoothing scale for it. Filtering an image with varying‐scale kernels, the image is smoothed according to the distribution of texture adaptively. With commendable experimental results, we show that, needing less iterations, our proposed scheme boosts texture filtering performance in terms of preserving the geometric structures of multiple scales even after aggressive smoothing of the original image.     

Structure‐Texture Decomposition of Images with Interval Gradient

This paper presents a novel filtering‐based method for decomposing an image into structures and textures. Unlike previous filtering algorithms, our method adaptively smooths image gradients to filter out textures from images. A new gradient operator, the interval gradient, is proposed for adaptive gradient smoothing. Using interval gradients, textures can be distinguished from structure edges and smoothly varying shadings. We also propose an effective gradient‐guided algorithm to produce high‐quality image filtering results from filtered gradients. Our method avoids gradient reversal in the filtering results and preserves sharp features better than existing filtering approaches, while retaining simplicity and highly parallel implementation. The proposed method can be utilized for various applications that require accurate structure‐texture decomposition of images.     

面向纹理平滑的方向性滤波尺度预测模型

目的 传统图像处理的纹理滤波方法难以区分梯度较强的纹理与物体的结构,而深度学习方法使用的训练集生成方式不够合理,且模型表示方式比较粗糙,为此本文设计了一种面向纹理平滑的方向性滤波尺度预测模型,并生成了含有标签的新的纹理滤波数据集。方法 在现有结构图像中逐连通区域填充多种纹理图,生成有利于模型训练的纹理滤波数据集。设计了方向性滤波尺度预测模型,该模型包含尺度感知子网络和图像平滑子网络。前者预测得到的滤波尺度图不但体现了该像素与周围像素是否为同一纹理,而且还隐含了该像素是否为结构像素的信息。后者以滤波尺度图和原图的堆叠作为输入,凭借少量的卷积层快速得出纹理滤波的结果。结果 在本文的纹理滤波数据集上与7个算法进行比较,峰值信噪比(peak signal to noise ratio,PSNR)与结构相似度(structural similarity,SSIM)分别高于第2名2.79 dB、0.0133,均方误差(mean squared error,MSE)低于第2名6.863 8,运算速度快于第2名0.002 s。在其他数据集上的实验对比也显示出本文算法更好地保持结构与平滑纹理。通过比较不同数据集上训练的同一网络模型,证实了本文的纹理滤波数据集有助于增强模型对于强梯度纹理与物体结构的区分能力。结论 本文制作的纹理滤波数据集使模型更好地区分强梯度纹理与物体结构并增强模型的泛化能力。本文设计的方向性滤波尺度预测模型在性能上超越了已有的大多数纹理平滑方法,尤其在强梯度纹理的抑制和弱梯度结构的保持两个方面表现优异。     

Image and Video Abstraction by Coherence‐Enhancing Filtering

In this work, we present a non‐photorealistic rendering technique to create stylized abstractions from color images and videos. Our approach is based on adaptive line integral convolution in combination with directional shock filtering. The smoothing process regularizes directional image features while the shock filter provides a sharpening effect. Both operations are guided by a flow field derived from the structure tensor. To obtain a high‐quality flow field, we present a novel smoothing scheme for the structure tensor based on Poisson's equation. Our approach effectively regularizes anisotropic image regions while preserving the overall image structure and achieving a consistent level of abstraction. Moreover, it is suitable for per‐frame filtering of video and can be efficiently implemented to process content in real‐time.     

Topology‐based Smoothing of 2D Scalar Fields with C1‐Continuity

Data sets coming from simulations or sampling of real‐world phenomena often contain noise that hinders their processing and analysis. Automatic filtering and denoising can be challenging: when the nature of the noise is unknown, it is difficult to distinguish between noise and actual data features; in addition, the filtering process itself may introduce “artificial” features into the data set that were not originally present. In this paper, we propose a smoothing method for 2D scalar fields that gives the user explicit control over the data features. We define features as critical points of the given scalar function, and the topological structure they induce (i.e., the Morse‐Smale complex). Feature significance is rated according to topological persistence. Our method allows filtering out spurious features that arise due to noise by means of topological simplification, providing the user with a simple interface that defines the significance threshold, coupled with immediate visual feedback of the remaining data features. In contrast to previous work, our smoothing method guarantees a C¹‐continuous output scalar field with the exact specified features and topological structures.     

自适应尺度的双边纹理滤波方法

目前已有的结构保持的纹理平滑方法主要是利用矩形片内的统计量来区分纹理和结构,但是所用的矩形片边长是单一尺度的,这将导致含有尖锐结构或结构在多个尺度上的图像出现纹理过平滑或未平滑的现象。为此,提出一种自适应尺度的双边纹理滤波方法。首先,通过对局部区域进行统计分析,从给定候选值中自适应地为每个像素选取合适的矩形片边长,对于均匀的纹理区域,选取较大的矩形片边长,对于邻近特征边的区域选取较小边长;其次,利用自适应的矩形片边长计算引导图像;最后,对原始图像进行引导双边滤波。实验结果表明,所提方法能够在保持图像结构的同时更好地平滑纹理。     

Contrast‐Enhanced Black and White Images

This paper investigates contrast enhancement as an approach to tone reduction, aiming to convert a photograph to black and white. Using a filter‐based approach to strengthen contrast, we avoid making a hard decision about how to assign tones to segmented regions. Our method is inspired by sticks filtering, used to enhance medical images but not previously used in non‐photorealistic rendering. We amplify contrast of pixels along the direction of greatest local difference from the mean, strengthening even weak features if they are most prominent. A final thresholding step converts the contrast‐enhanced image to black and white. Local smoothing and contrast enhancement balances abstraction and structure preservation; the main advantage of our method is its faithful depiction of image detail. Our method can create a set of effects: line drawing, hatching, and black and white, all having superior details to previous black and white methods.     

Parameterization‐Aware MIP‐Mapping

We present a method of generating mipmaps that takes into account the distortions due to the parameterization of a surface. Existing algorithms for generating mipmaps assume that the texture is isometrically mapped to the surface and ignore the actual surface parameterization. Our method correctly downsamples warped textures by assigning texels weights proportional to their area on a surface. We also provide a least‐squares approach to filtering over these warped domains that takes into account the postfilter used by the GPU. Our method improves texture filtering for most models but only modifies mipmap generation, requires no modification of art assets or rasterization algorithms, and does not affect run‐time performance.     

结构识别引导下的纹理抑制图像平滑

目的 针对目前已有的纹理平滑方法难以在抑制强梯度和尺度变化纹理的同时保持完整结构的问题,提出一种结构识别引导下的纹理抑制图像平滑算法。方法 首先,结构与纹理的根本区别在于重复模式,结构应该是稀疏的,而纹理应该是一个有重复模式的区域,因此,通过对结构/纹理的多尺度分析,提取了对于结构/纹理具有辨别力的多尺度内变差特征;然后,借助支持向量机,对提取的特征样本点训练出一个结构/纹理分类器;就分类结果中存在的结构较粗、毛刺等问题,进一步对分类结果进行细化和剔除毛刺与孤立点的后处理操作,以获得最终的更为精细的结构识别结果;最后,提出结构引导下的自适应双边图像滤波算法,达到既能抑制强梯度和尺度变化的纹理又能保持结构完整性的图像平滑效果。结果 本文提出的多尺度内变差特征在支持向量机训练中达到了96.12%的正确率,结构引导下的图像滤波能够在保持结构的同时,有效地抑制强梯度和尺度变化的纹理细节。结论 本文算法在兼顾结构的保持和强梯度以及尺度变化纹理的抑制方面超越了已有的方法,对于结构提取、细节增强、图像分割、色调映射、图像融合和目标识别等众多技术领域的发展将具有较强的促进作用,体现了潜在的实际应用价值。     

Striped-texture image segmentation with application to multimedia security

In today’s rapid growth of volume of multimedia data, security is important yet challenging problem in multimedia applications. Image, which covers the highest percentage of the multimedia data, it is very important for multimedia security. Image segmentation is utilized as a fundamental preprocessing of various multimedia applications such as surveillance for security by breaking a given image into multiple salient regions. In this paper, we present a new image segmentation approach based on frequency-domain filtering for images with stripe texture, and generalize it to lattice fence images. Our method significantly reduces the impact of stripes on segmentation performance. The approach proposed in this paper consists of three phases. Given the images, we weaken the effect of stripe texture by filtering in the frequency domain automatically. Then, structure-preserving image smoothing is employed to remove texture details and extract the main image structures. Last, we use an effective threshold method to produce segmentation results. Our method achieves very promising results for the test image dataset and could benefit a number of new multimedia applications such as public security.

     

细节感知的纹理滤波算法

目的 基于现有的研究提出一种细节感知的纹理去除算法,在去除图像纹理时,能够很好地保持图像的结构信息,尤其是诸如细长结构和边角信息等在其他方法中容易被模糊化的特殊细节。方法 首先,本文提出一种能够识别细长结构的结构检测方法,对细长结构进行检测并增强其结构特征。其次,为了估计每个像素点的最优滤波核尺度,改进原有的相对总变差模型,多方向寻找最小相对总变差,使它能够更好地区分纹理和边界,并且将边角信息从纹理中区分出来。然后,将检测出来的细长结构归一到改进的相对总变差的度量尺度上,估计滤波核尺度,生成引导滤波图像。这样就能够在平坦或有纹理的区域运用大尺度的滤波核,并在结构边缘和边角附近减小滤波核。最后,通过联合双边滤波器得到纹理去除后的图像。结果 实验测试了马赛克图像和艺术画作,对比了相对总变差和尺度敏感的结构保护滤波等方法,本文方法在去除纹理的同时保留了细长结构和边角细节,并且具有良好的普适性和鲁棒性。利用本文算法处理一幅含10万像素的图像,仅通过一次迭代计算就能够去除大量纹理且效果优于已有的方法,本算法的计算时间为3.37 s,其他算法为0.07~3.29 s。结论 本文设计的纹理滤波器不仅在保持诸如细长结构方面的性能更好,而且使纹理去除后的图像在边角细节处更尖锐,为图像的后续处理提供了一种强有力的图像预处理方式。     

Parallel four‐dimensional Haralick texture analysis for disk‐resident image datasets

Texture analysis is one possible method of detecting features in biomedical images. During texture analysis, texture‐related information is found by examining local variations in image brightness. Four‐dimensional (4D) Haralick texture analysis is a method that extracts local variations along space and time dimensions and represents them as a collection of 14 statistical parameters. However, application of the 4D Haralick method on large time‐dependent image datasets is hindered by data retrieval, computation, and memory requirements. This paper describes a parallel implementation using a distributed component‐based framework of 4D Haralick texture analysis on PC clusters. The experimental performance results show that good performance can be achieved for this application via combined use of task‐ and data‐parallelism. In addition, we show that our 4D texture analysis implementation can be used to classify imaged tissues. Copyright © 2006 John Wiley & Sons, Ltd.     

An Efficient Structure‐Aware Bilateral Texture Filtering for Image Smoothing

Photos contain well‐structured and plentiful visual information. Edges are active and expressive stimuli for human visual perception. However, it is hard to separate structure from details because edge strength and object scale are entirely different concepts. This paper proposes a structure‐aware bilateral texture algorithm to remove texture patterns and preserve structures. Our proposed method is simple and fast, as well as effective in removing textures. Instead of patch shift, smaller patches represent pixels located at structure edges, and original patches represent the texture regions. Specifically, this paper also improves joint bilateral filter to preserve small structures. Moreover, a windowed inherent variation is adapted to distinguish textures and structures for detecting structure edges. Finally, the proposed method produces excellent experimental results. These results are compared to some results of previous studies. Besides, structure‐preserving filtering is a critical operation in many image processing applications. Our proposed filter is also demonstrated in many attractive applications, such as seam carving, detail enhancement, artistic rendering, etc.     

Visual Coherence for Large‐Scale Line‐Plot Visualizations

Displaying a large number of lines within a limited amount of screen space is a task that is common to many different classes of visualization techniques such as time‐series visualizations, parallel coordinates, link‐node diagrams, and phase‐space diagrams. This paper addresses the challenging problems of cluttering and overdraw inherent to such visualizations. We generate a 2×2 tensor field during line rasterization that encodes the distribution of line orientations through each image pixel. Anisotropic diffusion of a noise texture is then used to generate a dense, coherent visualization of line orientation. In order to represent features of different scales, we employ a multi‐resolution representation of the tensor field. The resulting technique can easily be applied to a wide variety of line‐based visualizations. We demonstrate this for parallel coordinates, a time‐series visualization, and a phase‐space diagram. Furthermore, we demonstrate how to integrate a focus+context approach by incorporating a second tensor field. Our approach achieves interactive rendering performance for large data sets containing millions of data items, due to its image‐based nature and ease of implementation on GPUs. Simulation results from computational fluid dynamics are used to evaluate the performance and usefulness of the proposed method.     

基于梯度曲面面积与稀疏约束的图像平滑方法

针对纹理图像在平滑过程中低对比度边缘易丢失和纹理细节抑制不彻底等问题,提出基于梯度曲面面积与稀疏约束的图像平滑方法。首先,将图像视作三维空间中的二维嵌入曲面,再在此基础上分析图像的几何特征并提出梯度曲面面积约束正则化项,以提高纹理抑制性能;其次,根据图像的统计特性,建立L₀梯度稀疏与自适应梯度曲面面积约束的混合正则化约束图像平滑模型;最后,采用交替方向乘子法对非凸非光滑的优化模型进行高效求解。通过纹理抑制、边缘检测、纹理增强和图像融合等方面的实验结果可知,所提出的图像平滑算法克服了L₀梯度最小化平滑方法易造成的阶梯效应和欠滤波等缺陷,能够在去除大量纹理信息的同时保持并锐化图像显著的边缘轮廓。     

Peek‐in‐the‐Pic: Flying Through Architectural Scenes From a Single Image*

Many casually taken ‘tourist’ photographs comprise of architectural objects like houses, buildings, etc. Reconstructing such 3D scenes captured in a single photograph is a very challenging problem. We propose a novel approach to reconstruct such architectural scenes with minimal and simple user interaction, with the goal of providing 3D navigational capability to an image rather than acquiring accurate geometric detail. Our system, Peek‐in‐the‐Pic, is based on a sketch‐based geometry reconstruction paradigm. Given an image, the user simply traces out objects from it. Our system regards these as perspective line drawings, automatically completes them and reconstructs geometry from them. We make basic assumptions about the structure of traced objects and provide simple gestures for placing additional constraints. We also provide a simple sketching tool to progressively complete parts of the reconstructed buildings that are not visible in the image and cannot be automatically completed. Finally, we fill holes created in the original image when reconstructed buildings are removed from it, by automatic texture synthesis. Users can spend more time using interactive texture synthesis for further refining the image. Thus, instead of looking at flat images, a user can fly through them after some simple processing. Minimal manual work, ease of use and interactivity are the salient features of our approach.     

Bilinear Accelerated Filter Approximation

Our method approximates exact texture filtering for arbitrary scales and translations of an image while taking into account the performance characteristics of modern GPUs. Our algorithm is fast because it accesses textures with a high degree of spatial locality. Using bilinear samples guarantees that the texels we read are in a regular pattern and that we use a hardware accelerated path. We control the texel weights by manipulating the u, v parameters of each sample and the blend factor between the samples. Our method is similar in quality to Cardinality‐Constrained Texture Filtering [ MS13 ] but runs two times faster.     

Adaptive Image‐Space Sampling for Gaze‐Contingent Real‐time Rendering

With ever‐increasing display resolution for wide field‐of‐view displays—such as head‐mounted displays or 8k projectors—shading has become the major computational cost in rasterization. To reduce computational effort, we propose an algorithm that only shades visible features of the image while cost‐effectively interpolating the remaining features without affecting perceived quality. In contrast to previous approaches we do not only simulate acuity falloff but also introduce a sampling scheme that incorporates multiple aspects of the human visual system: acuity, eye motion, contrast (stemming from geometry, material or lighting properties), and brightness adaptation. Our sampling scheme is incorporated into a deferred shading pipeline to shade the image's perceptually relevant fragments while a pull‐push algorithm interpolates the radiance for the rest of the image. Our approach does not impose any restrictions on the performed shading. We conduct a number of psycho‐visual experiments to validate scene‐ and task‐independence of our approach. The number of fragments that need to be shaded is reduced by 50 % to 80 %. Our algorithm scales favorably with increasing resolution and field‐of‐view, rendering it well‐suited for head‐mounted displays and wide‐field‐of‐view projection.