Medical image categorization and retrieval for PACS using the GMM-KL framework.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.     

基于相关反馈的一种加权距离法

在基于内容的图像检索系统中我们经常使用一些底层特征,如表现图像的颜色和文本信息.如果给出了这些特征的内容,如特征向量,我们就可以通过计算特征空间的距离测量出图像之间的相似度.然而,这些底层特征并不一定能反映人的视觉中高层概念的相似度.相关反馈技术就是通过交互检索来提高检索性能,在数据库的搜索中参考了用户的反馈信息.因此...     

A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback.

A content-based image retrieval (CBIR) framework for diverse collection of medical images of different imaging modalities, anatomic regions with different orientations and biological systems is proposed. Organization of images in such a database (DB) is well defined with predefined semantic categories; hence, it can be useful for category-specific searching. The proposed framework consists of machine learning methods for image prefiltering, similarity matching using statistical distance measures, and a relevance feedback (RF) scheme. To narrow down the semantic gap and increase the retrieval efficiency, we investigate both supervised and unsupervised learning techniques to associate low-level global image features (e.g., color, texture, and edge) in the projected PCA-based eigenspace with their high-level semantic and visual categories. Specially, we explore the use of a probabilistic multiclass support vector machine (SVM) and fuzzy c-mean (FCM) clustering for categorization and prefiltering of images to reduce the search space. A category-specific statistical similarity matching is proposed in a finer level on the prefiltered images. To incorporate a better perception subjectivity, an RF mechanism is also added to update the query parameters dynamically and adjust the proposed matching functions. Experiments are based on a ground-truth DB consisting of 5000 diverse medical images of 20 predefined categories. Analysis of results based on cross-validation (CV) accuracy and precision-recall for image categorization and retrieval is reported. It demonstrates the improvement, effectiveness, and efficiency achieved by the proposed framework.     

Learning an identity distinguishable space for large scale face recognition

Implementing face recognition efficiently to real world large scale dataset presents great challenges to existing approaches.The methodin this paper was proposed to learn an identity distinguishable space for large scale face recognition in MSR-Bing image recognition challenge(IRC). Firstly, a deep convolutional neural network (CNN) was used to optimize a 128 B embedding for large scale face retrieval. The embedding was trained via using triplets of aligned face patches from FaceScrub and CASIA-WebFace datasets. Secondly, the evaluation of MSR-Bing IRC was conducted according to a cross-domain retrieval scheme. The real-time retrievalin thispaperwas benefited from the K-means clustering performed on the feature space of training data. Furthermore, a large scale similarity learning (LSSL) was applied on the relevant face images for learning a better identity space. A novel method for selecting similar pairs was proposed for LSSL. Compared with many existing networks of face recognition, the proposed model was lightweight and the retrieval method was promising as well.     

Similarity-based online feature selection in content-based image retrieval.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.     

Indexing and retrieval of images by spatial constraints

Many multimedia applications require retrieval of spatially similar images against a given query image. Existing work on image retrieval and indexing either requires extensive low-level computations or elaborate human interaction. In this paper, we introduce a new symbolic image representation technique to eliminate repetitive tasks of image understanding and object processing. Our symbolic image representation scheme is based on the concept of hierarchical decomposition of image space into spatial arrangements of features while preserving the spatial relationships among the image objects. Quadtrees are used to manage the decomposition hierarchy and play an important role in defining the similarity measure. This scheme is incremental in nature, can be adopted to accommodate varying levels of details in a wide range of application domains, and provides geometric variance independence. While ensuring that there are no false negatives, our approach also discriminates against non-matching entities by eliminating them as soon as possible, during the coarser matching phases. A hierarchical indexing scheme based on the concept of image signatures and efficient quadtree matching has been devised. Each level of the hierarchy tends to reduce the search space, allowing more involved comparisons only for potentially matching candidate database images. For a given query image, a facility is provided to rank-order the retrieved spatially similar images from the image database for subsequent browsing and selection by the user.     

CLUE: cluster-based retrieval of images by unsupervised learning.

In a typical content-based image retrieval (CBIR) system, target images (images in the database) are sorted by feature similarities with respect to the query. Similarities among target images are usually ignored. This paper introduces a new technique, cluster-based retrieval of images by unsupervised learning (CLUE), for improving user interaction with image retrieval systems by fully exploiting the similarity information. CLUE retrieves image clusters by applying a graph-theoretic clustering algorithm to a collection of images in the vicinity of the query. Clustering in CLUE is dynamic. In particular, clusters formed depend on which images are retrieved in response to the query. CLUE can be combined with any real-valued symmetric similarity measure (metric or nonmetric). Thus, it may be embedded in many current CBIR systems, including relevance feedback systems. The performance of an experimental image retrieval system using CLUE is evaluated on a database of around 60,000 images from COREL. Empirical results demonstrate improved performance compared with a CBIR system using the same image similarity measure. In addition, results on images returned by Google's Image Search reveal the potential of applying CLUE to real-world image data and integrating CLUE as a part of the interface for keyword-based image retrieval systems.     

An efficient signature representation for retrieval of spatially similar images

Multimedia applications involving image retrieval demand fast and efficient response. Efficiency of search and retrieval of information in a database system is index dependent. Generally, a two-level indexing scheme in an image database can help to reduce the search space against a given query image. In such type of indexing scheme, the first level is required to significantly reduce the search space for second stage of comparisons and must be computationally efficient. It is also required to guarantee that no false negatives may result. The second level of indexing involves more detailed analysis and comparison of potentially relevant images. In this paper, we present an efficient signature representation scheme for first level of a two-level image indexing scheme that is based on hierarchical decomposition of image space into spatial arrangement of image features. Experimental results demonstrate that our signature representation scheme results in fewer number of matching signatures in the first level and significantly improves the overall computational time. As this scheme relies on corner points as the salient feature points in an image to describe its contents, we also compare results using several different contemporary corner detection methods. Further, we formally prove that the proposed signature representation scheme not only results in fewer number of signatures but also does not result in any false negative.     

Efficient privacy-preserving image retrieval scheme over outsourced data with multi-user

The traditional privacy-preserving image retrieval schemes not only bring large computational and communication overhead,but also cannot protect the image and query privacy in multi-user scenarios.To solve above problems,an efficient privacy-preserving content-based image retrieval scheme was proposed in multi-user scenarios.The scheme used Euclidean distance comparison technique to rank the pictures according to similarity of picture feature vectors and return top-k returned.Meanwhile,the efficient key conversion protocol designed in proposed image retrieval scheme allowed each search user to generate queries based on his own private key so that he can retrieval encrypted images generated by different data owners.Strict security analysis shows that the user privacy and cloud data security can be well protected during the image retrieval process,and the performance analysis using real-world dataset shows that the proposed image retrieval scheme is efficient and feasible in practical applications.     

Image annotation using high order statistics in non-Euclidean spaces

Automatic image annotation is a promising way to achieve more effective image retrieval and image analysis by using keywords associated to the image content. Due to the semantic gap between low-level visual features and high-level semantic concepts of an image, however, the performances of many existing algorithms are not so satisfactory. In this paper, a novel image classification scheme, named high order statistics based maximum a posterior (HOS-MAP), is proposed to deal with the issue of image annotation. To bridge the gap between human judgment and machine intelligence, the proposed scheme first constructs a dissimilarity representation for each image in a non-Euclidean space; then, the information of dissimilarity diffusion distribution for each image is achieved with respect to the high-order statistics of a triplet of nearest neighbor images; finally, a maximum a posteriori algorithm with the information of Gaussian Mixture Model and dissimilarity diffusion distribution is adopted to estimate the relevance between each annotation and an input un-annotated image. Experimental results on a general-purpose image database demonstrate the effectiveness and efficiency of the proposed automatic image annotation scheme.     

An Indexing and Retrieval Mechanism for Complex Similarity Queries in Image Databases

A content-based image retrieval mechanism to support complex similarity queries is presented. The image content is defined by three kinds of features: quantifiable features describing the visual information, nonquantifiable features describing the semantic information, and keywords describing more abstract semantic information. In correspondence with these feature sets, we construct three types of indexes: visual indexes, semantic indexes, and keyword indexes. Index structures are elaborated to provide effective and efficient retrieval of images based on their contents. The underlying index structure used for all indexes is the HG-tree. In addition to the HG-tree, the signature file and hashing technique are also employed to index keywords and semantic features. The proposed indexing scheme combines and extends the HG-tree, the signature file, and the hashing scheme to support complex similarity queries. We also propose a new evaluation strategy to process the complex similarity queries. Experiments have been carried out on large image collections to demonstrate the effectiveness of the proposed retrieval mechanism.     

Clustering of hierarchical image database to reduce inter-and intra-semantic gaps in visual space for finding specific image semantics

Empowering content based systems to assign image semantics is an interesting concept. This work explores semantically categorized image database and forms a hierarchical visual search space. Overlapping of visual features of images from different categories and subcategories are possible reasons behind inter-semantic and intra-semantic gaps. Usually each category/node in the image database has a single representation, but variability and broadness of semantic limit the usage of such representation. This work explores the application of agglomerative hierarchical clustering to automatically identify groups within a semantic in the visual space. Visual signatures of dominant clusters corresponding to a node represent its semantic. Adaptive selection of branches on this clustered data facilitates efficient semantic assignment to query image in reduced search cost. Based on the concept, content based semantic retrieval system is developed and tested on hierarchical and non-hierarchical databases. Results showcase capability of the proposed system to reduce inter- and intra-semantic gaps.     

一种改进的综合BTC颜色矩和DT-CWT的图像检索算法

摘 要：特征提取是基于内容的图像检索中的关键技术。针对基于单一特征检索效果不理想的问题,提出一种改进的综合颜色和纹理特征的图像检索算法。该算法在YIQ颜色空间中进行特征提取,首先结合方块编码（BTC）的思想,提取颜色矩作为颜色特征;采用双树复小波变换（DT-CWT）提取纹理特征,融合两种特征并利用相似性度量方式进行图像检索。实验结果表明算法所提取的颜色、纹理特征更利于检索,使用综合特征检索的平均查准率比同类算法更高。     

Histological image retrieval based on semantic content analysis

The demand for automatic recognition and retrieval of medical images for screening, reference, and management is increasing. We present an intelligent content-based image retrieval system called I-Browse, which integrates both iconic and semantic content for histological image analysis. The I-Browse system combines low-level image processing technology with high-level semantic analysis of medical image content through different processing modules in the proposed system architecture. Similarity measures are proposed and their performance is evaluated. Furthermore, as a byproduct of semantic analysis, I-Browse allows textual annotations to be generated for unknown images. As an image browser, apart from retrieving images by image example, it also supports query by natural language.     

Using binarization and hashing for efficient SIFT matching

The well-known SIFT is capable of extracting distinctive features for image retrieval. However, its matching is time consuming and slows down the entire process. In the SIFT matching, the Euclidean distance is used to measure the similarity of two features, which is expensive because it involves taking square root. Moreover, the scale of the image database is usually too large to adopt linear search for image retrieval. To improve the SIFT matching, this paper proposes a fast image retrieval scheme transforms the SIFT features to binary representations. The complexity of the distance calculation is reduced to bit-wise operation and the retrieval time is greatly decreased. Moreover, the proposed scheme utilizes hashing for retrieving similar images according to the binarized features and further speeds up the retrieval process. The experiment results show the proposed scheme can retrieve images efficiently with only a little sacrifice of accuracy as compared to SIFT.     

Design and analysis of a content-based pathology image retrieval system

A prototype, content-based image retrieval system has been built employing a client/server architecture to access supercomputing power from the physician's desktop. The system retrieves images and their associated annotations from a networked microscopic pathology image database based on content similarity to user supplied query images. Similarity is evaluated based on four image feature types: color histogram, image texture, Fourier coefficients, and wavelet coefficients, using the vector dot product as a distance metric. Current retrieval accuracy varies across pathological categories depending on the number of available training samples and the effectiveness of the feature set. The distance measure of the search algorithm was validated by agglomerative cluster analysis in light of the medical domain knowledge. Results show a correlation between pathological significance and the image document distance value generated by the computer algorithm. This correlation agrees with observed visual similarity. This validation method has an advantage over traditional statistical evaluation methods when sample size is small and where domain knowledge is important. A multi-dimensional scaling analysis shows a low dimensionality nature of the embedded space for the current test set.     

A new way for multidimensional medical data management: volume of interest (VOI)-based retrieval of medical images with visual and functional features.

The advances in digital medical imaging and storage in integrated databases are resulting in growing demands for efficient image retrieval and management. Content-based image retrieval (CBIR) refers to the retrieval of images from a database, using the visual features derived from the information in the image, and has become an attractive approach to managing large medical image archives. In conventional CBIR systems for medical images, images are often segmented into regions which are used to derive two-dimensional visual features for region-based queries. Although such approach has the advantage of including only relevant regions in the formulation of a query, medical images that are inherently multidimensional can potentially benefit from the multidimensional feature extraction which could open up new opportunities in visual feature extraction and retrieval. In this study, we present a volume of interest (VOI) based content-based retrieval of four-dimensional (three spatial and one temporal) dynamic PET images. By segmenting the images into VOIs consisting of functionally similar voxels (e.g., a tumor structure), multidimensional visual and functional features were extracted and used as region-based query features. A prototype VOI-based functional image retrieval system (VOI-FIRS) has been designed to demonstrate the proposed multidimensional feature extraction and retrieval. Experimental results show that the proposed system allows for the retrieval of related images that constitute similar visual and functional VOI features, and can find potential applications in medical data management, such as to aid in education, diagnosis, and statistical analysis.