基于深度学习的命名实体识别研究综述

命名实体识别是自然语言处理领域的一项关键任务,其目的在于从自然语言文本中识别出具有特定含义的实体,如人名、地名、机构名和专有名词等。在命名实体识别任务中,研究人员提出过多种方法,包括基于知识和有监督的机器学习方法。近年来,随着互联网文本数据规模的快速扩大和深度学习技术的快速发展,深度学习模型已成为命名实体识别的研究热点,并在该领域取得显著进展。文中全面回顾现有的命名实体识别深度学习技术,主要分为四类：基于卷积神经网络模型、基于循环神经网络模型、基于Transformer模型和基于图神经网络模型的命名实体识别。此外,对深度学习的命名实体识别架构进行了介绍。最后,探讨命名实体识别所面临的挑战以及未来可能的研究方向,以期推动命名实体识别领域的进一步发展。     

基于预训练模型的医疗命名实体识别

文中针对生物医学实体识别中存在的边界识别不准确和鲁棒性差的问题,提出了一种融合了预训练语言模型BERT与跨度标签网络的命名实体识别模型。该模型利用BERT获取文本的上下文信息,并结合跨度标签网络进行实体分类及边界判定,显著提升了实体识别的准确性。为增强模型的鲁棒性,引入对抗训练策略,通过迭代训练正常样本与对抗样本,以优化模型参数。基于CCKS2019评测数据集的实验表明,应用对抗训练方法后,其精准率、召回率及F1值均有所提升,验证了对抗训练能对提高模型的预测能力和鲁棒性的有效性。     

融合知识的中文医疗实体识别模型

从医疗文本中抽取知识对构建医疗辅助诊断系统等应用具有重要意义.实体识别是其中的核心步骤.现有的实体识别模型大都是基于标注数据的深度学习模型,非常依赖高质量大规模的标注数据.为了充分利用已有的医疗领域词典和预训练语言模型,本文提出了融合知识的中文医疗实体识别模型.一方面基于领域词典提取领域知识,另一方面,引入预训练语言模型BERT作为通用知识,然后将领域知识和通用知识融入到模型中.此外,本文引入了卷积神经网络来提高模型的上下文建模能力.本文在多个数据集上进行实验,实验结果表明,将知识融合到模型中能够有效提高中文医疗实体识别的效果.     

一种联合文本和图像信息的行人检测方法

针对纯视觉行人检测方法存在的误检、漏检率高,遮挡目标以及小尺度目标检测精度低等问题,提出一种联合文本和图像信息的行人检测方法.该方法首先利用图像分析的方法初步获取图像目标的候选框,其次通过文本分析的方法获取文本中有关图像目标的实体表达,并提出一种基于马尔科夫随机场的模型用于推断图像候选框与文本实体表达之间的共指关系（Coreference Relation）,以此达到联合图像和文本信息以辅助机器视觉提高交通场景下行人检测精度的目的.在增加了图像文本描述的加州理工大学行人检测数据集上进行的测评结果表明,该方法不仅可以在图像信息的基础上联合文本信息提高交通场景中的行人检测精度,也能在文本信息的基础上联合图像信息提高文本中的指代消解（Anaphora Resolution）精度.     

基于Faster-RCNN的极验点选式验证码识别

针对传统方法难以识别极验点选式验证码的问题,文中提出一种基于Faster-RCNN目标检测模型和卷积神经网络的识别方法。通过简化的RPN提高Faster-RCNN对于背景图片文本定位的精度,再设计卷积神经网络对文本进行分类识别,并训练Tesseract-OCR识别库对信息提示文本进行识别,实现背景图片文本识别结果与信息提示文本识别结果一一对应,达到识别此类验证码的目的。实验结果表明,该方法识别此类验证码的准确率达到72.4%。     

基于XLNet-BiLSTM的中文电子病历命名实体识别方法

中文临床电子病历命名实体识别是实现智慧医疗的基本任务之一.本文针对传统的词向量模型文本语义表示不充分,以及循环神经网络(RNN)模型无法解决长时间依赖等问题,提出一个基于XLNet的中文临床电子病历命名实体识别模型XLNet-BiLSTM-MHA-CRF,将XLNet预训练语言模型作为嵌入层,对病历文本进行向量化表示,解决一词多义等问题;利用双向长短时记忆网络(BiLSTM)门控制单元获取句子的前向和后向语义特征信息,将特征序列输入到多头注意力层(multi-head attention,MHA);利用MHA获得特征序列不同子空间表示的信息,增强上下文语义的关联性,同时剔除噪声;最后输入条件随机场CRF识别全局最优序列.实验结果表明,XLNet-BiLSTM-Attention-CRF模型在CCKS-2017命名实体识别数据集上取得了良好的效果.     

命名实体识别技术综述

命名实体识别是自然语言处理中的热点研究方向之一,目的是识别文本中的命名实体并将其归纳到相应的实体类型中。首先阐述了命名实体识别任务的定义、目标和意义,分析提出了命名实体识别的主要难点在于领域命名实体识别局限性、命名实体表述多样性和歧义性、命名实体的复杂性和开放性;然后介绍了命名实体识别研究的发展进程,从最初的规则和字典方法到传统的统计学习方法再到现在的深度学习方法,不断地将新技术应用到命名实体识别研究中以提高性能;接着系统梳理了当下命名实体识别任务中的若干热门研究点,分别是匮乏资源下的命名实体识别、细粒度命名实体识别、嵌套命名实体识别以及命名实体链接;最后针对评判命名实体识别模型的好坏,总结了常用的若干数据集和实验测评指标,并给出了未来的研究建议。     

融合注意力机制的RoBERTa-BiLSTM-CRF语音对话文本命名实体识别

<正>本文基于国网客服中心95598语音转文本数据，采用Ro BERTa语言模型对语句进行向量表示与特征提取，通过添加机制，模型可以关注与当前输出相关的信息，并获得文本中的潜在语义信息。通过对照组实验，本文构造的模型能够有效提高实体识别精准程度，且该模型在文本数据集上的抽取效果明显优于其他模型。命名实体识别(Named Entity Recognition,NER)获取有关子任务的信息，搜索分配给文本的对象，并将其分类到预定义的类别中。命名实体识别是自然语言处理中的热点研究方向之一，用于识别和分类文本中的相关实体的类别。命名实体识别的准确度，     

基于循环卷积多任务学习的多领域文本分类方法

文本分类任务中,不同领域的文本很多表达相似,具有相关性的特点,可以解决有标签训练数据不足的问题.采用多任务学习的方法联合学习能够将不同领域的文本利用起来,提升模型的训练准确率和速度.该文提出循环卷积多任务学习(MTL-RC)模型用于文本多分类,将多个任务的文本共同建模,分别利用多任务学习、循环神经网络(RNN)和卷积神经网络(CNN)模型的优势获取多领域文本间的相关性、文本长期依赖关系、提取文本的局部特征.基于多领域文本分类数据集进行丰富的实验,该文提出的循环卷积多任务学习模型(MTL-LC)不同领域的文本分类平均准确率达到90.1％,比单任务学习模型循环卷积单任务学习模型(STL-LC)提升了6.5％,与当前热门的多任务学习模型完全共享多任务学习模型(FS-MTL)、对抗多任务学习模型(ASP-MTL)、间接交流多任务学习框架(IC-MTL)相比分别提升了5.4％,?4％和2.8％.     

基于字符级联合网络特征融合的中文文本情感分析

针对传统卷积神经网络(CNN)同层神经元之间信息不能互传,无法充分利用同一层次上的特征信息,以及无法提取长距离上下文相关特征的问题.该文针对中文文本,提出字符级联合网络特征融合的模型进行情感分析,在字符级的基础上采用BiGRU和CNN-BiGRU并行的联合网络提取特征,利用CNN的强学习能力提取深层次特征,再利用双向门限循环神经网络(BiGRU)进行深度学习,加强模型对特征的学习能力.另一方面,利用BiGRU提取上下文相关的特征,丰富特征信息.最后在单方面上引入注意力机制进行特征权重分配,降低噪声干扰.在数据集上进行多组对比实验,该方法取得92.36%的F1值,结果表明本文提出的模型能有效的提高文本分类的准确率.     

Simple and effective neural coreference resolution for Korean language

We propose an end-to-end neural coreference resolution for the Korean language that uses an attention mechanism to point to the same entity. Because Korean is a head-final language, we focused on a method that uses a pointer network based on the head. The key idea is to consider all nouns in the document as candidates based on the head-final characteristics of the Korean language and learn distributions over the referenced entity positions for each noun. Given the recent success of applications using bidirectional encoder representation from transformer (BERT) in natural language-processing tasks, we employed BERT in the proposed model to create word representations based on contextual information. The experimental results indicated that the proposed model achieved state-of-the-art performance in Korean language coreference resolution.     

Deep recurrent neural networks with word embeddings for Urdu named entity recognition

Named entity recognition (NER) continues to be an important task in natural language processing because it is featured as a subtask and/or subproblem in information extraction and machine translation. In Urdu language processing, it is a very difficult task. This paper proposes various deep recurrent neural network (DRNN) learning models with word embedding. Experimental results demonstrate that they improve upon current state‐of‐the‐art NER approaches for Urdu. The DRRN models evaluated include forward and bidirectional extensions of the long short‐term memory and back propagation through time approaches. The proposed models consider both language‐dependent features, such as part‐of‐speech tags, and language‐independent features, such as the “context windows” of words. The effectiveness of the DRNN models with word embedding for NER in Urdu is demonstrated using three datasets. The results reveal that the proposed approach significantly outperforms previous conditional random field and artificial neural network approaches. The best f‐measure values achieved on the three benchmark datasets using the proposed deep learning approaches are 81.1%, 79.94%, and 63.21%, respectively.     

Multi-task learning with contextual hierarchical attention for Korean coreference resolution

Coreference resolution is a task in discourse analysis that links several headwords used in any document object. We suggest pointer networks-based coreference resolution for Korean using multi-task learning (MTL) with an attention mechanism for a hierarchical structure. As Korean is a head-final language, the head can easily be found. Our model learns the distribution by referring to the same entity position and utilizes a pointer network to conduct coreference resolution depending on the input headword. As the input is a document, the input sequence is very long. Thus, the core idea is to learn the word- and sentence-level distributions in parallel with MTL, while using a shared representation to address the long sequence problem. The suggested technique is used to generate word representations for Korean based on contextual information using pre-trained language models for Korean. In the same experimental conditions, our model performed roughly 1.8% better on CoNLL F1 than previous research without hierarchical structure.     

Research on coreference resolution technology of entity in information security

To solve the problem of coreference resolution in information security,a hybrid method was proposed.Based on the BiLSTM-attention-CRF model,the domain-dictionary matching mechanism was introduced and combined with the attention mechanism at the document level.As a new dictionary-based attention mechanism,the word features were calculated to solve the problem of weak recognition ability of rare entities and entities with long length when extracting candidates from text.And by summarizing the features of the domain texts,the candidates were coreferenced by rules and machine learning according to the part of speech to improve the accuracy.Through the experiments on security data set,the superiority of the method is proved from the aspects of coreference resolution and extraction of candidates from text .     

中英命名实体识别及对齐中的中文分词优化

中文分词结果对中英命名实体识别及对齐有着直接的影响,本文提出了一种命名实体识别及对齐中的中文分词优化方法.该方法利用实体词汇的对齐信息,首先修正命名实体识别结果,然后根据实体对齐结果调整分词粒度、修正错误分词.分词优化后的结果使得双语命名实体尽可能多地实现一一对应,进而提高中英命名实体翻译抽取和统计机器翻译的效果.实验结果表明了本文优化方法的有效性.     

Rank‐weighted reconstruction feature for a robust deep neural network‐based acoustic model

In this paper, we propose a rank‐weighted reconstruction feature to improve the robustness of a feed‐forward deep neural network (FFDNN)‐based acoustic model. In the FFDNN‐based acoustic model, an input feature is constructed by vectorizing a submatrix that is created by slicing the feature vectors of frames within a context window. In this type of feature construction, the appropriate context window size is important because it determines the amount of trivial or discriminative information, such as redundancy, or temporal context of the input features. However, we ascertained whether a single parameter is sufficiently able to control the quantity of information. Therefore, we investigated the input feature construction from the perspectives of rank and nullity, and proposed a rank‐weighted reconstruction feature herein, that allows for the retention of speech information components and the reduction in trivial components. The proposed method was evaluated in the TIMIT phone recognition and Wall Street Journal (WSJ) domains. The proposed method reduced the phone error rate of the TIMIT domain from 18.4% to 18.0%, and the word error rate of the WSJ domain from 4.70% to 4.43%.     

S2-aware network for visual recognition

Capturing the comprehensive information of various sizes and shapes of images in the same convolution layer is typically a challenging task in computer vision. There are two main kinds of methods for capturing those features. The first uses the inception structure and its variants. The second utilizes larger convolution kernels on specific layers or stacks with more convolution blocks. However, these methods can result in computationally intensive or vanishing gradients. In this paper, to accommodate feature distributions with different sizes, shapes and reduce computational cost, we propose a width- and depth-aware module named the WD-module to match feature distributions. Moreover, the proposed WD-module consumes less computational cost and parameters compared with traditional residual convolution layers. To verify the effectiveness of our proposed method, a size- and shape-aware backbone network named S²A-Net was built, which was obtained by stacking the WD-modules. By visualizing heat maps and features, the proposed S²A-Net can adapt to objects with different sizes and shapes in visual recognition tasks and learn more comprehensive characteristics. Experimental results show that the proposed method has higher accuracy in image recognition and outperforms other state-of-the-art networks with the same numbers of layers.     

基于层叠隐马尔可夫模型的中文命名实体识别

提出了一种基于层叠隐马尔可夫模型的中文命名实体一体化识别方法,旨在将人名识别、地名识别以及机构名识别等命名实体识别融合到一个相对统一的理论模型中。首先在词语粗切分的结果集上采用底层隐马尔可夫模型识别出普通无嵌套的人名、地名和机构名等,然后依次采取高层隐马尔可夫模型识别出嵌套了人名、地名的复杂地名和机构名。在对大规模真实语料库的封闭测试中,人名、地名和机构识别的F-1值分别达到92.55%、94.53%、86.51%。采用该方法的系统ICTCLAS在2003年5月SIGHAN举办的第一届汉语分词大赛中名列前茅。