| 深度对比学习综述 |
| |
| 引用本文： | 张重生,陈杰,李岐龙,邓斌权,王杰,陈承功.深度对比学习综述[J].自动化学报,2023,49(1):15-39. |
| |
| 作者姓名： | 张重生 陈杰 李岐龙 邓斌权 王杰 陈承功 |
| |
| 作者单位： | 1.河南大学 河南省大数据分析与处理重点实验室 开封 475001 |
| |
| 基金项目： | 科技部高端外国专家项目(G2021026016L)资助 |
| |
| 摘 要： | 在深度学习中,如何利用大量、易获取的无标注数据增强神经网络模型的特征表达能力,是一个具有重要意义的研究问题,而对比学习是解决该问题的有效方法之一,近年来得到了学术界的广泛关注,涌现出一大批新的研究方法和成果.本文综合考察对比学习近年的发展和进步,提出一种新的面向对比学习的归类方法,该方法将现有对比学习方法归纳为5类,包括:1)样本对构造; 2)图像增广; 3)网络架构; 4)损失函数; 5)应用.基于提出的归类方法,对现有对比研究成果进行系统综述,并评述代表性方法的技术特点和区别,系统对比分析现有对比学习方法在不同基准数据集上的性能表现.本文还将梳理对比学习的学术发展史,并探讨对比学习与自监督学习、度量学习的区别和联系.最后,本文将讨论对比学习的现存挑战,并展望未来发展方向和趋势.
|
| 关 键 词： | 对比学习 深度学习 特征提取 自监督学习 度量学习 |
| 收稿时间： | 2022-05-22 |
Deep Contrastive Learning: A Survey |
| |
| Affiliation: | 1.Henan Key Lab of Big Data Analysis and Processing, Henan University, Kaifeng 475001 |
| |
| Abstract: | In deep learning, it has been a crucial research concern on how to make use of the vast amount of unlabeled data to enhance the feature extraction capability of deep neural networks, for which contrastive learning is an effective approach. It has attracted significant research effort in the past few years, and a large number of contrastive learning methods have been proposed. In this paper, we survey recent advances and progress in contrastive learning in a comprehensive way. We first propose a new taxonomy for contrastive learning, in which we divide existing methods into 5 categories, including 1) sample pair construction methods, 2) image augmentation methods, 3) network architecture level methods, 4) loss function level methods, and 5) applications. Based on our proposed taxonomy, we systematically review the methods in each category, and analyze the characteristics and differences of representative methods. Moreover, we report and compare the performance of different contrastive learning methods on the benchmark datasets. We also retrospect the history of contrastive learning and discuss the differences and connections among contrastive learning, self-supervised learning, and metric learning. Finally, we discuss remaining issues and challenges in contrastive learning and outlook its future directions. |
| |
| Keywords: | |
|
| 点击此处可从《自动化学报》浏览原始摘要信息 |
|
点击此处可从《自动化学报》下载全文 |
|
