基于BERT的短文本相似度判别模型

短文本的表示方法和特征提取方法是自然语言处理基础研究的一个重要方向,具有广泛的应用价值。本文提出了BERT_BLSTM_TCNN模型,该神经网络模型利用BERT的迁移学习,并在词向量编码阶段引入对抗训练方法,训练出包括句的语义和结构特征的且泛化性能更优的句特征,并将这些特征输入BLSTM_TCNN层中进行特征抽取以完成对短文本的语义层面上的相似判定。在相关数据集上的实验结果表明:与最先进的预训练模型相比,该模型在有着不错的判定准确率的同时还有参数量小易于训练的优点。

Short Text Similarity Discrimination Model based on BERT

Short text representation methods and feature extraction methods are an important direction of basic research in natural language processing,and have a wide range of applications.This paper proposes the BERT_BLSTM_TCNN model.The neural net?work model uses BERT's transfer learning and introduces an adversarial training method in the word vector encoding stage to train sentence features that include the semantic and structural features of the sentence and have better generalization performance,and combine these The feature is input into the BLSTM_TCNN layer for feature extraction to complete the similarity determination on the semantic level of the short text.The experimental results on the relevant data set show that:compared with the most advanced pre-training model,this model has a good judgment accuracy rate and also has the advantages of small parameters and easy train?ing.