结合预训练模型和语言知识库的文本匹配方法

针对文本匹配任务,该文提出一种大规模预训练模型融合外部语言知识库的方法。该方法在大规模预训练模型的基础上,通过生成基于WordNet的同义-反义词汇知识学习任务和词组-搭配知识学习任务引入外部语言学知识。进而,与MT-DNN多任务学习模型进行联合训练,以进一步提高模型性能。最后利用文本匹配标注数据进行微调。在MRPC和QQP两个公开数据集的实验结果显示,该方法可以在大规模预训练模型和微调的框架基础上,通过引入外部语言知识进行联合训练有效提升文本匹配性能。

A Text Matching Method by Combining Pre-trained Model and Language Knowledge Base

Aiming at text matching task, this paper proposes a method to incorporate large-scale pre-training model and external language knowledge base. On the basis of large-scale pre-training model, this method introduces external linguistic knowledge by generating synonym-antonym knowledge learning task and phrase-collocation knowledge learning task based on WordNet, respectively. Then, the two new generated tasks are joint trained with MT-DNN multi task learning model to further improve the model performance. Finally, the annotated text matching data is used to fine tune. The experimental results on two open datasets, MRPC and QQP, show that the proposed method can effectively improve the performance of text matching by introducing external language knowledge for joint training on the basis of the framework of large-scale pre-training model and fine-tuning.