基于BERT-CRF的领域词向量生成研究

如何在中文BERT字向量基础上进一步得到高质量的领域词向量表示，用于各种以领域分词为基础的文本分析任务是一个亟待解决的问题。提出了一种基于BERT的领域词向量生成方法。建立一个BERT-CRF领域分词器，在预训练BERT字向量基础上结合领域文本进行fine-tuning和领域分词学习；通过领域分词解码结果进一步得到领域词向量表示。实验表明，该方法仅利用少量的领域文本就可以学习出符合领域任务需求的分词器模型，并能获得相比原始BERT更高质量的领域词向量。

Research on Domain-Specific Word Vector Generation Based on BERT-CRF

How to obtain a high-quality domain-specific word vector representation based on the Chinese BERT word vector for various text analysis tasks based on domain word segmentation is an urgent problem to be solved. This paper proposes a domain-specific word vector generation method based on BERT. A BERT-CRF domain-specific word segmenter is established, and the domain text is combined with the domain text to perform fine-tuning and domain word segmentation learning based on the pre-trained BERT word vector. The domain-specific word vector representation is further obtained through the domain-specific word segmentation decoding results. Experiments show that this method can learn a tokenizer model that meets the requirements of the domain task using only a small amount of domain text, and can obtain a higher-quality domain-specific word vector than the original BERT.