基于改进Apriori算法的问题模板无监督抽取方法

在面向限定领域的事实型问答系统中,基于模板匹配的问答是一种有效且稳定的方法。然而,现有的问题模板构建方法通常是在有监督场景下进行的,导致其严重依赖于人工标注数据,同时领域间可扩展性较差。因此,该文提出了一种改进Apriori算法的无监督模板抽取方法。对于限定领域问题样本,加入短语有序特征来挖掘频繁项集,将频繁项作为问题模板的框架词;同时,使用TF-IDF来度量模板的信息量,去除信息量小的模板;特别地,为了获取项数较长的模板,为Apriori算法引入了支持度自适应更新机制;最终,借助命名实体识别进行槽位识别,并组合框架词和槽,得到问题模板。实验表明,该方法可以在限定领域的问答数据集上有效挖掘问题模板,并取得了比基线模型更好的抽取效果。

Unsupervised Question Template Extraction Based on Improved Apriori Algorithm

For domain-specific question answering (QA) systems, question retrieval via template matching proves to be effective and stable. However, existing template extraction methods usually work in a supervised manner, resulting in heavy dependence on manually annotated data and poor extensibility among different domains. To address this issue, this paper proposes an unsupervised template extraction method based on an improved Apriori algorithm. For given samples of question utterances, the frequently occurred phrases are first orderly extracted as frame words of candidate templates. The information inhabited in candidate templates is measured via TF-IDF, and candidates with low information are filtered out. In particular, to allow longer templates, an adaptive updating mechanism for support threshold is proposed. Finally, NER methods are adopted to locate slots, and question templates are obtained by combining frame words and the corresponding slots. Experimental results show that our method can effectively extract question templates for specific domains and obtain better results than baseline models.