基于BERT的三阶段式问答模型

预训练语言模型的发展极大地推动了机器阅读理解任务的进步.为了充分利用预训练语言模型中的浅层特征,并进一步提升问答模型预测答案的准确性,提出了一种基于BERT的三阶段式问答模型.首先,基于BERT设计了预回答、再回答及答案调整三个阶段;然后,在预回答阶段将BERT嵌入层的输入视作浅层特征来进行答案预生成;接着,在再回答阶...

Three-stage question answering model based on BERT

The development of pre-trained language models has greatly promoted the progress of machine reading comprehension tasks. In order to make full use of shallow features of the pre-trained language model and further improve the accuracy of predictive answer of question answering model， a three-stage question answering model based on Bidirectional Encoder Representation from Transformers （BERT） was proposed. Firstly， the three stages of pre-answering， re-answering and answer-adjusting were designed based on BERT. Secondly， the inputs of embedding layer of BERT were treated as shallow features to pre-generate an answer in pre-answering stage. Then， the deep features fully encoded by BERT were used to re-generate another answer in re-answering stage. Finally， the final prediction result was generated by combining the previous two answers in answer-adjusting stage. Experimental results on English dataset Stanford Question Answering Dataset 2.0 （SQuAD2.0） and Chinese dataset Chinese Machine Reading Comprehension 2018 （CMRC2018） of span-extraction question answering task show that the Exact Match （EM） and F1 score （F1） of the proposed model are improved by the average of 1 to 3 percentage points compared with those of the similar baseline models， and the model has the extracted answer fragments more accurate. By combining shallow features of BERT with deep features， this three-stage model extends the abstract representation ability of BERT， and explores the application of shallow features of BERT in question answering models， and has the characteristics of simple structure， accurate prediction， and fast speed of training and inference.