基于BERT嵌入BiLSTM-CRF模型的中文专业术语抽取研究

专业术语的识别与自动抽取对于提升专业信息检索精度,构建领域知识图谱发挥着重要基础性作用。为进一步提升中文专业术语识别的精确率和召回率,提出一种端到端的不依赖人工特征选择和领域知识,基于谷歌BERT预训练语言模型及中文预训练字嵌入向量,融合BiLSTM和CRF的中文专业术语抽取模型。以自建的1278条深度学习语料数据为实验对象,该模型对术语提取的F1值为92.96%,相对于传统的浅层机器学习模型(如左右熵与互信息算法、word2vec相似词算法等)和BiLSTM-CRF深度神经网络模型的性能有较为显著的提升。本文也给出了模型应用的具体流程,能够为中文专业术语库的构建提供实践指南。

Automatic Extraction of Chinese Terminology Based on BERT Embedding and BiLSTM-CRF Model

High quality professional term recognition and its extraction play an important role in the fields of domain information retrieval and knowledge graph building. To improve the precision and recall rate of terminology recognition, we propose a Chinese terminology recognition and extraction approach that does not rely on specific domain knowledge or artificial features. Using the latest developments in representation learning, this study introduces BERT embedding as a character-based pre-trained model and incorporates it with a bi-directional long short-term memory(BiLSTM) and a conditional random field(CRF) to extract deep learning terminologies from 1278 annotated datasets collected from domain e-books.The experimental results show that the proposed model reaches 92.96% in F-score and outperforms other competing algorithms, such as left and right entropy, mutual information, a word2 vec based similar terminology recognition algorithm,and a BiLSTM-CRF model. The best practices and related procedures for the implementation of the proposed model are also provided to guide its users in its further improvement.