基于指针生成网络的中文对话文本摘要模型

针对传统Seq2Seq序列模型在文本摘要任务中无法准确地提取到文本中的关键信息、无法处理单词表之外的单词等问题,本文提出一种基于Fastformer的指针生成网络(pointer generator network, PGN)模型,且该模型结合了抽取式和生成式两种文本摘要方法.模型首先利用Fastformer模型高效的获取具有上下文信息的单词嵌入向量,然后利用指针生成网络模型选择从源文本中复制单词或利用词汇表来生成新的摘要信息,以解决文本摘要任务中常出现的OOV(out of vocabulary)问题,同时模型使用覆盖机制来追踪过去时间步的注意力分布,动态的调整单词的重要性,解决了重复词问题,最后,在解码阶段引入了Beam Search优化算法,使得解码器能够获得更加准确的摘要结果.实验在百度AI Studio中汽车大师所提供的汽车诊断对话数据集中进行,结果表明本文提出的FastformerPGN模型在中文文本摘要任务中达到的效果要优于基准模型,具有更好的效果.

Chinese Dialogue Text Summarization Model Based on Pointer Generator Network

Considering the problems that the traditional Seq2Seq model cannot accurately extract key information from texts and process words outside the word list in text summarization tasks, this study proposes a pointer generator network (PGN) model based on Fastformer. The model combines the text summarization methods of extraction and generation. Specifically, the Fastformer model is used to efficiently obtain the word embedding vector with context information, and then PGN helps choose to copy words from the source text or use vocabulary to generate new summary information, so as to solve the out-of-vocabulary (OOV) problem that often occurs in text summarization tasks. At the same time, the model uses the coverage mechanism to track the attention distribution of the past time step and dynamically adjust the importance of words to solve the problem of repeated words. Finally, the Beam Search algorithm is introduced in the decoding stage to make the decoder obtain more accurate summary results. The experiments on the dataset of auto-diagnosis dialogues provided by Auto Master in AI Studio of Baidu show that the Fastformer-PGN model proposed in this study achieves better performance in text summarization tasks of Chinese dialogues than the benchmark model.