面向问句复述识别的多卷积自交互匹配方法研究

问句复述识别旨在识别两个自然问句是否语义一致。目前,基于表示学习和深度神经网络架构的复述识别技术已取得较好效果。但是,这类方法往往面临复杂度较高且训练难度较大的瓶颈。针对这一问题,该文提出一种快速的多卷积自交互匹配方法。该方法融合了多种句子特征和词义特征,并由此形成分布式表示。在此基础上,这一方法利用卷积神经网络获取短语级的句子向量表示,并利用自交互融合技术将词级与短语级的向量表示进行充分融合,借以获得多粒度句子向量表示。该文将这一方法应用于判定自然语句是否互为复述的二元分类任务中,利用Quora标准问句复述识别语料进行测试。实验结果证明,在不引入外部数据的情况下,该文所提方法与基于双向多视角匹配的基准模型具有可比的性能,但在空间复杂度上具有较高的优越性,并且获得更快训练速度。具体地,该方法训练所需的物理显存比基准模型方法下降80%,训练迭代速度快19倍。     

基于领域知识融合的复述识别方法研究

复述识别任务，即判断两个句子是否表达相同的语义。传统的复述识别任务针对的是通用领域，模型通过理解两个句子的语义，比较句子的语义相似度从而进行复述判断。而在特定领域的复述识别任务中，模型必须结合该领域的专业知识，才能准确地理解两个句子的语义，并进一步判断出它们的区别与联系。该文针对特定领域提出了一种基于领域知识融合的复述识别方法。方法首先为句子检索专业知识，再将专业知识融入到每个句子的语义中，最后实现更准确的语义相似度判断。该文在计算机科学领域的复述识别数据集PARADE上进行了相关实验，实验结果显示，该文方法在F₁指标上达到了73.9,比基线方法提升了3.1。     

结合注意力机制的循环神经网络复述识别模型

传统基于深度学习的复述识别模型通常以关注文本表示为核心,忽略了对多粒度交互特征的挖掘与匹配.为此,建模文本交互空间,分别利用双向长短时记忆网络对两个候选复述句按条件编码,基于迭代隐状态的输出,通过逐词软对齐的方式从词、短语、句子等多个粒度层次推理并获取句子对的语义表示,最后综合不同视角的语义表达利用softmax实现二元分类.为解决复述标注训练语料不足,在超过580000句子对的数据集上利用语言建模任务对模型参数无监督预训练,再使用预训练好的参数在标准数据集上有监督微调.与先前最佳的神经网络模型相比,所提出模型在标准数据集MSRP上准确率提高2.96%,$F_1$值改善2%.所提出模型综合文本全局和局部匹配信息,多粒度、多视角地描述文本交互匹配模式,能够降低对人工特征工程的需求,具有良好的实用性.     

面向问题复述识别的定向数据增强方法

问题复述识别旨在召回“同质异构”的问句对子(语义相同表述迥异的问句)和摒弃语义无关的噪声问句,对输入的问句对进行“是复述”和“非复述”的二相判别。现有预训练语言模型(如BERT、RoBERTa和MacBERT)被广泛应用于自然语言的语义编码,并取得了显著的性能优势。然而,其优势并未在问句复述问题的求解中得到充分的体现,原因在于: ①预训练语言模型对特定任务中精细的语义表示需求并不敏感; ②复述样本的“是与非”往往取决于极为微妙的语义差异。微调预训练语言模型成为提高其任务适应性的关键步骤,但其极大地依赖训练数据的数量(多样性)与质量(可靠性)。为此,该文提出一种基于生成模型的定向数据增强方法(DDA)。该方法能够利用诱导标签对神经生成网络进行引导,借以自动生成多样的复述和非复述的增强样本(即高迷惑性的异构样本),促进训练数据的自动扩展。此外,该文设计了一种多模型集成的标签投票机制,并用其修正增强样本的潜在标签错误,以此提高扩展数据的可靠性。在中文问题复述数据集LCQMC上的实验结果证明,与传统数据增强方法相比,该文方法生成的样本质量更高,且语义表达更加多元化。     

基于Transformer和重要词识别的句子融合方法

句子融合是为多个句子生成言简意赅、符合语法的句子,可应用到自动摘要、复述生成等自然语言处理任务。目前句子融合方法已取得一定成效,但还存在重要信息缺失、语义偏离原句等问题。该文提出基于Transformer和重要词识别的方法来缓解上述问题。该方法包括两个模块,(1)重要词识别模块：利用序列标注模型识别原句重要词;(2)句子融合模块：将重要词与原句输入到Transformer框架并利用BERT进行语义表示,然后在全连接层引入基于原句和词表获得的向量作为先验知识进行句子融合。基于NLPCC2017摘要任务集构建句子融合数据集,并进行相关实验,结果表明所提方法的性能明显优于基线系统。     

基于语义空间距离的相似问句识别方法研究

前沿相关研究将相似问句识别转化为二元问句匹配识别并取得很大进展。但是在自动问答系统的实际应用场景中存在大量数据,这些方法受限于二元问句匹配识别模式,导致时效性不高。针对这一问题,受人脸识别相关研究的启发,该文提出基于语义空间距离衡量的相似问句识别方法（Semantic Space Distance Method,SSDM）。该方法将相似问句识别作为多分类问题进行训练,通过利用人脸识别任务中Margin Softmax损失函数得到语义编码模型。该语义编码模型能够将相似问句在语义空间中聚合,不相似问句在语义空间中远离。SSDM方法将相似问句识别转化成语义空间中的向量距离计算,突破二元问句匹配的方式,保证了一定的高时效性,并且仍然能够在深层语义层面对相似问句进行识别。该方法在Biendata的ASQD数据集中实验测试,取得了比基线方法更优的性能,验证了SSDM方法的有效性。     

基于多角度信息交互的文本语义匹配模型

现有的文本语义匹配方法大多基于简单的注意力机制进行交互,较少考虑文本自身结构信息和文本之间原始信息的的交互。针对2个中文文本的语义匹配问题,构建一个多角度信息交互的文本匹配模型MAII。分别从颗粒、局部、全局3个角度计算2个文本深层次的语义交互矩阵,同时考虑语序信息之间和结构信息之间的交互以及文本内部的依赖关系,从而得到含有丰富信息的语义向量,并通过语义推理计算出两文本之间的语义匹配度。实验结果表明,相比在英文数据集上表现良好的DSSM、ESIM和DIIN模型,MAII模型在CCKS 2018问句匹配大赛的中文数据集上达到77.77%的准确率,表现出更好的匹配性能。     

面向工业生产的中文Text-to-SQL模型

英文自然语言查询转SQL语句（Text-to-SQL）任务的模型迁移到中文工业Text-to-SQL任务时,由于工业数据集的可解释差且比较分散,会出现数据库的表名列名等信息与问句中关键信息的表示形式不一致以及问句中的列名隐含在语义中等问题导致模型精确匹配率变低。针对迁移过程中出现的问题,提出了对应的解决方法并构建修改后的模型。首先,在数据使用过程中融入工厂元数据信息以解决表示形式不一致以及列名隐含在语义中的问题;然后,根据中文语言表达方式的特性,使用基于相对位置的自注意力模型直接通过问句以及数据库模式信息识别出where子句的value值;最后,根据工业问句查询内容的特性,使用微调后的基于变换器的双向编码器表示技术（BERT）对问句进行分类以提高模型对SQL语句结构预测的准确率。构建了一个基于铝冶炼行业的工业数据集,并在该数据集上进行实验验证。结果表明所提模型在工业测试集上的精确匹配率为74.2%,对比英文数据集Spider上各阶段主流模型的效果后可以看出,所提模型能有效处理中文工业Text-to-SQL任务。     

基于词典和字形特征的中文命名实体识别

命名实体识别是自然语言处理中的一项基础任务。通过基于词典的方法增强词内语义和词边界信息是中文命名实体识别的主流做法。然而,汉字由象形字演变而来,汉字字形中包含着丰富的实体信息,这些信息在该任务中却很少被使用。该文提出了一个基于词典和字形特征的中文命名实体识别模型,将词信息和结构信息统一地结合起来,提高了实体匹配的准确性。该文首先通过SoftLexicon方法丰富语义信息,并使用改进的部首级嵌入优化字符表示;然后通过门卷积网络加强了对潜在词和上下文信息的提取;最后在四个基准数据集上实验,结果表明与传统模型和最新模型相比,基于词典和字形特征的模型取得了显著的性能提升。     

融合高低层语义信息的自然语言句子匹配方法

针对目前自然语言句子匹配方法未能融合公共语义信息以及难以捕获深层次语义信息的问题,提出了一种融合高低层语义信息的自然语言句子匹配算法。该算法首先采用预训练词向量GloVe和字符级别的词向量获取句子P和句子Q的词嵌入表示;其次,使用双向LSTM(long-short term memory)对P和Q进行编码,再初步融合P和Q获取低层语义信息;然后,计算P与Q的双向注意力,拼接得到语义表征,再计算其自注意力获取高层语义信息;最后,采用启发式的融合函数将低层语义信息和高层语义信息进行融合,得到最终的语义表征,使用卷积神经网络预测得到答案。在文本蕴涵识别和释义识别两个任务上评估了该模型。在文本蕴涵识别任务SNLI数据集和释义识别任务Quora数据集上进行了实验,实验结果表明该算法在SNLI测试集上的准确率为87.1%,在Quora测试集上的准确率为86.8%,验证了算法在自然语言句子匹配任务上的有效性。     

关系代数派生算子语义表达式间等价性证明

关系代数的派生算子在关系数据库查询语言中得到了广泛应用。它们的语义有两种常见的表示方式,一种是基于原始算子的表达式,一种是基于一阶逻辑的表达式。但有关的文献资料都没有给出这两种表达式等价性的严格证明。文章尝试通过一系列等价变换,证明派生算子语义的这两种表达式间的等价性。从派生算子(主要是除算子)语义的原始算子表达式出发,根据关系代数表达式的特点,通过一步步的等价变换,得到派生算子语义的一阶逻辑表达式。所使用的变换方法能为关系代数表达式的正确性证明打下基础。     

融合深度图像的卷积神经网络语义分割方法

该文提出了一种基于深度学习框架的图像语义分割方法,通过使用由相对深度点对标注训练的网络模型,实现了基于彩色图像的深度图像预测,并将其与原彩色图像共同输入到包含带孔卷积的全卷积神经网络中。考虑到彩色图像与深度图像作为物体不同的属性表征,在特征图上用合并连接操 作而非传统的相加操作对其进行融合,为后续卷积层提供特征图输入时保持了两种表征的差异。在两个数据集上的实验结果表明,该法可以有效提升语义分割的性能。     

Learning classification rules from database in the context ofknowledge acquisition and representation

A method for learning knowledge from a database is used to address the bottleneck of manual knowledge acquisition. An attempt is made to improve representation with the assistance of experts and from computer resident knowledge. The knowledge representation is described in the framework of a conceptual schema consisting of a semantic model and an event model. A concept classifies a domain into different subdomains. As a method of knowledge acquisition, inductive learning techniques are used for rule generation. The theory of rough sets is used in designing the learning algorithm. Examples of certain concepts are used to induce general specifications of the concepts called classification rules. The basic approach is to partition the information into equivalence classes and to derive conclusions based on equivalence relations. In a sense, what is involved is a data-reduction process, where the goal is to reduce a large database of information to a small number of rules describing the domain. This completely integrated approach includes user interface, semantics, constraints, representations of temporal events, induction, etc     

A survey on question answering technology from an information retrieval perspective

This article provides a comprehensive and comparative overview of question answering technology. It presents the question answering task from an information retrieval perspective and emphasises the importance of retrieval models, i.e., representations of queries and information documents, and retrieval functions which are used for estimating the relevance between a query and an answer candidate. The survey suggests a general question answering architecture that steadily increases the complexity of the representation level of questions and information objects. On the one hand, natural language queries are reduced to keyword-based searches, on the other hand, knowledge bases are queried with structured or logical queries obtained from the natural language questions, and answers are obtained through reasoning. We discuss different levels of processing yielding bag-of-words-based and more complex representations integrating part-of-speech tags, classification of the expected answer type, semantic roles, discourse analysis, translation into a SQL-like language and logical representations.     

Semantic consistent adversarial cross-modal retrieval exploiting semantic similarity

Cross-modal retrieval aims to search the semantically similar instances from the other modalities given a query from one modality. However, the differences of the distributions and representations between different modalities make that the similarity of different modalities can not be measured directly. To address this problem, in this paper, we propose a novel semantic consistent adversarial cross-modal retrieval (SC-ACMR), which learns semantic consistent representation for different modalities under adversarial learning framework by considering the semantic similarity from intra-modality and inter-modality. Specifically, for intra-modality, we minimize the intra-class distances. For the inter-modality, we require class center of different modalities with same semantic label to be as close as possible, and also minimize the distances between the samples and the class center with same semantic label from different modalities. Furthermore, we preserve the semantic similarity of transformed features of different modalities through a semantic similarity matrix. Comprehensive experiments on two benchmark datasets are conducted and the experimental results show that the proposed method have learned more compact semantic representations and achieved better performance than many existing methods in cross-modal retrieval.

     

Equivalence of rational representations of behaviors

This article deals with the equivalence of representations of behaviors of linear differential systems. In general, the behavior of a given linear differential system has many different representations. In this paper we restrict ourselves to kernel and image representations. Two kernel representations are called equivalent if they represent one and the same behavior. For kernel representations defined by polynomial matrices, necessary and sufficient conditions for equivalence are well known. In this paper, we deal with the equivalence of rational representations, i. e. kernel and image representations that are defined in terms of rational matrices. As the first main result of this paper, we will derive a new condition for the equivalence of rational kernel representations of possibly noncontrollable behaviors. Secondly we will derive conditions for the equivalence of rational representations of a given behavior in terms of the polynomial modules generated by the rows of the rational matrices. We will also establish conditions for the equivalence of rational image representations. Finally, we will derive conditions under which a given rational kernel representation is equivalent to a given rational image representation.     

Global viewing of heterogeneous data sources

The problem of defining global views of heterogeneous data sources to support querying and cooperation activities is becoming more and more important due to the availability of multiple data sources within complex organizations and in global information systems. Global views are defined to provide a unified representation of the information in the different sources by analyzing conceptual schemas associated with them and resolving possible semantic heterogeneity. We propose an affinity based unification method for global view construction. In the method: (1) the concept of affinity is introduced to assess the level of semantic relationship between elements in different schemas by taking into account semantic heterogeneity; (2) schema elements are classified by affinity levels using clustering procedures so that their different representations can be analyzed for unification; (3) global views are constructed starting from selected clusters by unifying representations of their elements. Experiences of applying the proposed unification method and the associated tool environment ARTEMIS on databases of the Italian Public Administration information systems are described     

基于注意力机制的混合神经网络关系分类方法

关系分类是自然语言处理领域的一项重要语义处理任务。传统的关系分类方法通过人工设计各类特征以及各类核函数来对句子内部2个实体之间的关系进行判断。近年来,关系分类方法的主要工作集中于通过各类神经网络获取句子的语义特征表示来进行分类,以减少手动构造各类特征。在句子中,不同关键词对关系分类任务的贡献程度是不同的,然而重要的词义有可能出现在句子中的任意位置。为此,提出了一种基于注意力的混合神经网络关系分类模型来捕获重要的语义信息,用来进行关系分类,该方法是一种端到端的方法。实验结果表明了该方法的有效性。     

Regrouping parallel processes

Two techniques are introduced for achieving semantics preserving transformations of process interconnection patterns described by algebraic parallel behaviour expressions. The semantic relation to be preserved is strong bisimulation equivalence. Convenient process network representations are identified as unique representatives of equivalence classes of parallel expressions. The first transformation technique is very simple, but can only be applied to a restricted class of expressions. The second technique is general, and achieves the desired process regrouping, whenever it exists, by inscribing the multi-arcs of the underlying process network into the binary syntax tree of a target expression pattern. The described transformation techniques have been implemented in the LOTOS Integrated Tool Environment (LITE) of ESPRIT Project LotoSphere.This work has been partially funded by ESPRIT Project 2304-LotoSphere.     

融合语义信息的矩阵分解词向量学习模型

词向量在自然语言处理中起着重要的作用,近年来受到越来越多研究者的关注。然而,传统词向量学习方法往往依赖于大量未经标注的文本语料库,却忽略了单词的语义信息如单词间的语义关系。为了充分利用已有领域知识库（包含丰富的词语义信息）,文中提出一种融合语义信息的词向量学习方法（KbEMF）,该方法在矩阵分解学习词向量的模型上加入领域知识约束项,使得拥有强语义关系的词对获得的词向量相对近似。在实际数据上进行的单词类比推理任务和单词相似度量任务结果表明,KbEMF比已有模型具有明显的性能提升。