Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Many computer vision and pattern recognition algorithms are very sensitive to the choice of an appropriate distance metric. Some recent research sought to address a variant of the conventional clustering problem called semi-supervised clustering, which performs clustering in the presence of some background knowledge or supervisory information expressed as pairwise similarity or dissimilarity constraints. However, existing metric learning methods for semi-supervised clustering mostly perform global metric learning through a linear transformation. In this paper, we propose a new metric learning method that performs nonlinear transformation globally but linear transformation locally. In particular, we formulate the learning problem as an optimization problem and present three methods for solving it. Through some toy data sets, we show empirically that our locally linear metric adaptation (LLMA) method can handle some difficult cases that cannot be handled satisfactorily by previous methods. We also demonstrate the effectiveness of our method on some UCI data sets. Besides applying LLMA to semi-supervised clustering, we have also used it to improve the performance of content-based image retrieval systems through metric learning. Experimental results based on two real-world image databases show that LLMA significantly outperforms other methods in boosting the image retrieval performance.     

Relaxational metric adaptation and its application to semi-supervised clustering and content-based image retrieval

The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering tasks, however, class label information is not available. In order to adapt the metric to improve the clustering results, some background knowledge or side information is needed. One useful type of side information is in the form of pairwise similarity or dissimilarity information. Recently, some novel methods (e.g., the parametric method proposed by Xing et al.) for learning global metrics based on pairwise side information have been shown to demonstrate promising results. In this paper, we propose a nonparametric method, called relaxational metric adaptation (RMA), for the same metric adaptation problem. While RMA is local in the sense that it allows locally adaptive metrics, it is also global because even patterns not in the vicinity can have long-range effects on the metric adaptation process. Experimental results for semi-supervised clustering based on both simulated and real-world data sets show that RMA outperforms Xing et al.'s method under most situations. Besides applying RMA to semi-supervised learning, we have also used it to improve the performance of content-based image retrieval systems through metric adaptation. Experimental results based on two real-world image databases show that RMA significantly outperforms other methods in improving the image retrieval performance.     

Neighborhood preserving regression for image retrieval

In content-based image retrieval (CBIR), relevance feedback has been proven to be a powerful tool for bridging the gap between low level visual features and high level semantic concepts. Traditionally, relevance feedback driven CBIR is often considered as a supervised learning problem where the user provided feedbacks are used to learn a distance metric or classification function. However, CBIR is intrinsically a semi-supervised learning problem in which the testing samples (images in the database) are present during the learning process. Moreover, when there are no sufficient feedbacks, these methods may suffer from the overfitting problem. In this paper, we propose a novel neighborhood preserving regression algorithm which makes efficient use of both labeled and unlabeled images. By using the unlabeled images, the geometrical structure of the image space can be incorporated into the learning system through a regularizer. Specifically, from all the functions which minimize the empirical loss on the labeled images, we select the one which best preserves the local neighborhood structure of the image space. In this way, our method can obtain a regression function which respects both semantic and geometrical structures of the image database. We present experimental evidence suggesting that our algorithm is able to use unlabeled data effectively for image retrieval.     

A semi-supervised probabilistic model for clustering large databases of complex images

Image content clustering is an effective way to organize large databases thereby making the content based image retrieval process much easier. However, clustering of images with varied background and foreground is quite challenging. In this paper, we propose a novel image content clustering paradigm suitable for clustering large and diverse image databases. In our approach images are represented in a continuous domain based on a probabilistic Gaussian Mixture Model (GMM) with the images modeled as mixture of Gaussian distributions in the selected feature space. The distance metric between the Gaussian distributions is defined in the sense of Kullback–Leibler (KL) divergence. The clustering is done using a semi-supervised learning framework where labeled data in the form of cluster templates is used to classify the unlabelled data. The clusters are formed around initially chosen seeds and are updated in the due course based on user inputs. In our clustering approach the user interaction is done in a structured way as to get maximum inputs from the user in a limited time. We propose two methods to carry out the structured user interaction using which the cluster templates are updated to improve the quality of the clusters formed. The proposed method is experimentally evaluated on benchmark datasets that are specifically chosen to include a wide variation of images around a common theme that is typically encountered in applications like photo-summarization and poses a major semantic gap challenge to conventional clustering approaches. The experimental results presented demonstrate the effectiveness of the proposed approach.     

Distance metric learning guided adaptive subspace semi-supervised clustering

Most existing semi-supervised clustering algorithms are not designed for handling high-dimensional data. On the other hand, semi-supervised dimensionality reduction methods may not necessarily improve the clustering performance, due to the fact that the inherent relationship between subspace selection and clustering is ignored. In order to mitigate the above problems, we present a semi-supervised clustering algorithm using adaptive distance metric learning (SCADM) which performs semi-supervised clustering and distance metric learning simultaneously. SCADM applies the clustering results to learn a distance metric and then projects the data onto a low-dimensional space where the separability of the data is maximized. Experimental results on real-world data sets show that the proposed method can effectively deal with high-dimensional data and provides an appealing clustering performance.     

基于混合距离学习的鲁棒的模糊C均值聚类算法

距离度量对模糊聚类算法FCM的聚类结果有关键性的影响。实际应用中存在这样一种场景,聚类的数据集中存在着一定量的带标签的成对约束集合的辅助信息。为了充分利用这些辅助信息,首先提出了一种基于混合距离学习方法,它能利用这样的辅助信息来学习出数据集合的距离度量公式。然后,提出了一种基于混合距离学习的鲁棒的模糊C均值聚类算法（HR-FCM算法）,它是一种半监督的聚类算法。算法HR-FCM既保留了GIFP-FCM（Generalized FCM algorithm with improved fuzzy partitions）算法的鲁棒性等性能,也因为所采用更为合适的距离度量而具有更好的聚类性能。实验结果证明了所提算法的有效性。     

Active hashing and its application to image and text retrieval

In recent years, hashing-based methods for large-scale similarity search have sparked considerable research interests in the data mining and machine learning communities. While unsupervised hashing-based methods have achieved promising successes for metric similarity, they cannot handle semantic similarity which is usually given in the form of labeled point pairs. To overcome this limitation, some attempts have recently been made on semi-supervised hashing which aims at learning hash functions from both metric and semantic similarity simultaneously. Existing semi-supervised hashing methods can be regarded as passive hashing since they assume that the labeled pairs are provided in advance. In this paper, we propose a novel framework, called active hashing, which can actively select the most informative labeled pairs for hash function learning. Specifically, it identifies the most informative points to label and constructs labeled pairs accordingly. Under this framework, we use data uncertainty as a measure of informativeness and develop a batch mode algorithm to speed up active selection. We empirically compare our method with a state-of-the-art passive hashing method on two benchmark data sets, showing that the proposed method can reduce labeling cost as well as overcome the limitations of passive hashing.     

Structure from Motion with Line Segments Under Relaxed Endpoint Constraints

This paper is concerned with the problem of learning a distance metric by considering meaningful and discriminative distance constraints in some contexts where rich information between data is provided. Classic metric learning approaches focus on constraints that involve pairs or triplets of images. We propose a general Mahalanobis-like distance metric learning framework that exploits distance constraints over up to four different images. We show how the integration of such constraints can lead to unsupervised or semi-supervised learning tasks in some applications. We also show the benefit on recognition performance of this type of constraints, in rich contexts such as relative attributes, class taxonomies and temporal webpage analysis.     

Brain image segmentation using semi-supervised clustering

The objective of brain image segmentation is to partition the brain images into different non-overlapping homogeneous regions representing the different anatomical structures. Magnetic resonance brain image segmentation has large number of applications in diagnosis of neurological disorders like Alzheimer diseases, Parkinson related syndrome etc. But automatically segmenting the MR brain image is not an easy task. To solve this problem, several unsupervised and supervised based classification techniques have been developed in the literature. But supervised classification techniques are more time consuming and cost-sensitive due to the requirement of sufficient labeled data. In contrast, unsupervised classification techniques work without using any prior information but it suffers from the local trap problems. So, to overcome the problems associated with unsupervised and supervised classification techniques, we have proposed a new semi-supervised clustering technique using the concepts of multiobjective optimization and applied this technique for automatic segmentation of MR brain images in the intensity space. Multiple centers are used to encode a cluster in the form of a string. The proposed clustering technique utilizes intensity values of the brain pixels as the features. Additionally it also assumes that the actual class label information of 10% points of a particular image data set is also known. Three cluster validity indices are utilized as the objective functions, which are simultaneously optimized using AMOSA, a modern multiobjective optimization technique based on the concepts of simulated annealing. First two cluster validity indices are symmetry distance based Sym-index and Euclidean distance based I-index, which are based on unsupervised properties. Last one is a supervised information based cluster validity index, Minkowski Index. The effectiveness of this proposed semi-supervised clustering technique is demonstrated on several simulated MR normal brain images and MR brain images having some multiple sclerosis lesions. The performance of the proposed semi-supervised clustering technique is compared with some other popular image segmentation techniques like Fuzzy C-means, Expectation Maximization and some recent image clustering techniques like multi-objective based MCMOClust technique, and Fuzzy-VGAPS clustering techniques.     

半监督类保持局部线性嵌入方法

为使局部线性嵌入（local linear embedding, LLE）这一无监督高维数据的非线性特征提取方法提取出的特征在分类或聚类学习上更优,提出一种半监督类保持局部线性嵌入(semi-supervised class preserving local linear embedding, SSCLLE)的非线性特征提取方法。该方法将半监督信息融入到LLE中,首先对标记样本近邻赋予伪标签,增大标记样本数量。其次,对标记样本之间的距离进行局部调整,缩小同类样本间距,扩大异类样本间距。同时在局部线性嵌入优化目标函数中增加全局同类样本间距和异类样本间距的约束项,使得提取出的低维特征可以确保同类样本点互相靠近,而异类样本点彼此分离。在一系列实验中,其聚类精确度以及可视化效果明显高于无监督LLE和现有半监督流特征提取方法,表明该方法提取出的特征具有很好的类保持特性。     

Non-linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data

The problem of clustering with side information has received much recent attention and metric learning has been considered as a powerful approach to this problem. Until now, various metric learning methods have been proposed for semi-supervised clustering. Although some of the existing methods can use both positive (must-link) and negative (cannot-link) constraints, they are usually limited to learning a linear transformation (i.e., finding a global Mahalanobis metric). In this paper, we propose a framework for learning linear and non-linear transformations efficiently. We use both positive and negative constraints and also the intrinsic topological structure of data. We formulate our metric learning method as an appropriate optimization problem and find the global optimum of this problem. The proposed non-linear method can be considered as an efficient kernel learning method that yields an explicit non-linear transformation and thus shows out-of-sample generalization ability. Experimental results on synthetic and real-world data sets show the effectiveness of our metric learning method for semi-supervised clustering tasks.     

Semi-supervised clustering with metric learning: An adaptive kernel method

Most existing representative works in semi-supervised clustering do not sufficiently solve the violation problem of pairwise constraints. On the other hand, traditional kernel methods for semi-supervised clustering not only face the problem of manually tuning the kernel parameters due to the fact that no sufficient supervision is provided, but also lack a measure that achieves better effectiveness of clustering. In this paper, we propose an adaptive Semi-supervised Clustering Kernel Method based on Metric learning (SCKMM) to mitigate the above problems. Specifically, we first construct an objective function from pairwise constraints to automatically estimate the parameter of the Gaussian kernel. Then, we use pairwise constraint-based K-means approach to solve the violation issue of constraints and to cluster the data. Furthermore, we introduce metric learning into nonlinear semi-supervised clustering to improve separability of the data for clustering. Finally, we perform clustering and metric learning simultaneously. Experimental results on a number of real-world data sets validate the effectiveness of the proposed method.     

基于双模型交互学习的半监督医学图像分割

在医学图像中,器官或病变区域的精准分割对疾病诊断等临床应用有着至关重要的作用,然而分割模型的训练依赖于大量标注数据.为减少对标注数据的需求,本文主要研究针对医学图像分割的半监督学习任务.现有半监督学习方法广泛采用平均教师模型,其缺点在于,基于指数移动平均(Exponential moving average, EMA)的参数更新方式使得老师模型累积学生模型的错误知识.为避免上述问题,提出一种双模型交互学习方法,引入像素稳定性判断机制,利用一个模型中预测结果更稳定的像素监督另一个模型的学习,从而缓解了单个模型的错误经验的累积和传播.提出的方法在心脏结构分割、肝脏肿瘤分割和脑肿瘤分割三个数据集中取得优于前沿半监督方法的结果.在仅采用30%的标注比例时,该方法在三个数据集上的戴斯相似指标(Dice similarity coefficient, DSC)分别达到89.13%, 94.15%, 87.02%.     

一种基于谱聚类的半监督聚类方法

半监督聚类利用少部分标签的数据辅助大量未标签的数据进行非监督的学习,从而提高聚类的性能。提出一种基于谱聚类的半监督聚类算法,其利用标签数据的信息,调整点与点之间的距离所形成的距离矩阵,而后基于被调整的距离矩阵进行谱聚类。实验表明,该算法较之于已提出的半监督聚类算法,获得了更好的聚类性能。     

Enhancing remote sensing image retrieval using a triplet deep metric learning network

ABSTRACT

With the rapid growing of remotely sensed imagery data, there is a high demand for effective and efficient image retrieval tools to manage and exploit such data. In this letter, we present a novel content-based remote sensing image retrieval (RSIR) method based on Triplet deep metric learning convolutional neural network (CNN). By constructing a Triplet network with metric learning objective function, we extract the representative features of the images in a semantic space in which images from the same class are close to each other while those from different classes are far apart. In such a semantic space, simple metric measures such as Euclidean distance can be used directly to compare the similarity of images and effectively retrieve images of the same class. We also investigate a supervised and an unsupervised learning methods for reducing the dimensionality of the learned semantic features. We present comprehensive experimental results on two public RSIR datasets and show that our method significantly outperforms state-of-the-art.     

有约束的半监督聚类方法

在数据挖掘领域的很多实际应用中,获取大量的无标签样本非常容易,而获取有标签的样本通常需要付出较大的代价,并且有时不可能得到所有的数据的标签,半监督聚类就是使用一小部分的标签数据对无标签数据的聚类过程进行指导。提出了一种新的半监督聚类算法,它利用标签数据提供的信息来初步确定数据的相似性和不相似性标准,并在聚类过程中对其进行自动调整,利用它们对聚类过程进行约束和指导。通过在标准数据集高斯数据集上的测试,该算法相对于无指导聚类来说有更高的精度和更快的速度。     

Collaborative image retrieval via regularized metric learning

In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval.” In this paper, we present a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semidefinite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than two alternative algorithms, which exploit log data for image retrieval.     

Multi-source shared nearest neighbours for multi-modal image clustering

Shared Nearest Neighbours (SNN) techniques are well known to overcome several shortcomings of traditional clustering approaches, notably high dimensionality and metric limitations. However, previous methods were limited to a single information source whereas such methods appear to be very well suited for heterogeneous data, typically in multi-modal contexts. In this paper, we propose a new technique to accelerate the calculation of shared neighbours and we introduce a new multi-source shared neighbours scheme applied to multi-modal image clustering. We first extend existing SNN-based similarity measures to the case of multiple sources and we introduce an original automatic source selection step when building candidate clusters. The key point is that each resulting cluster is built with its own optimal subset of modalities which improves the robustness to noisy or outlier information sources. We experiment our method in the scope of multi-modal search result clustering, visual search mining and subspace clustering. Experimental results on both synthetic and real data involving different information sources and several datasets show the effectiveness of our method.     

Learning multi-task local metrics for image annotation

The goal of image annotation is to automatically assign a set of textual labels to an image to describe the visual contents thereof. Recently, with the rapid increase in the number of web images, nearest neighbor (NN) based methods have become more attractive and have shown exciting results for image annotation. One of the key challenges of these methods is to define an appropriate similarity measure between images for neighbor selection. Several distance metric learning (DML) algorithms derived from traditional image classification problems have been applied to annotation tasks. However, a fundamental limitation of applying DML to image annotation is that it learns a single global distance metric over the entire image collection and measures the distance between image pairs in the image-level. For multi-label annotation problems, it may be more reasonable to measure similarity of image pairs in the label-level. In this paper, we develop a novel label prediction scheme utilizing multiple label-specific local metrics for label-level similarity measure, and propose two different local metric learning methods in a multi-task learning (MTL) framework. Extensive experimental results on two challenging annotation datasets demonstrate that 1) utilizing multiple local distance metrics to learn label-level distances is superior to using a single global metric in label prediction, and 2) the proposed methods using the MTL framework to learn multiple local metrics simultaneously can model the commonalities of labels, thereby facilitating label prediction results to achieve state-of-the-art annotation performance.     

基于半监督编码生成对抗网络的图像分类模型

在实际应用中,为分类模型提供大量的人工标签越来越困难,因此,近几年基于半监督的图像分类问题获得了越来越多的关注.而大量实验表明,在生成对抗网络(Generative adversarial network,GANs)的训练过程中,引入少量的标签数据能获得更好的分类效果,但在该类模型的框架中并没有考虑用于提取图像特征的结构,为了进一步利用其模型的学习能力,本文提出一种新的半监督分类模型.该模型在原生成对抗网络模型中添加了一个编码器结构,用于直接提取图像特征,并构造了一种新的半监督训练方式,获得了突出的分类效果.本模型分别在标准的手写体识别数据库MNIST、街牌号数据库SVHN和自然图像数据库CIFAR-10上完成了数值实验,并与其他半监督模型进行了对比,结果表明本文所提模型在使用少量带标数据情况下得到了更高的分类精度.