融合预训练模型和注意力的实体关系抽取方法

实体关系抽取旨在从无结构的文档中检测出实体和实体对的关系,是构建领域知识图谱的重要步骤。针对现有抽取模型语义表达能力差、重叠三元组抽取准确率低的情况,研究了融合预训练模型和注意力的实体关系联合抽取问题,将实体关系抽取任务分解为两个标记模块。头实体标记模块采用预训练模型对句子进行编码,为了进一步学习句子的内在特征,利用双向长短时记忆网络（BiLSTM）和自注意力机制组成特征加强层。采用二进制分类器作为模型的解码器,标记出头实体在句子中的起止位置。为了加深两个标记模块之间的联系,在尾实体标记任务前设置特征融合层,将头实体特征与句子向量通过卷积神经网络（CNN）和注意力机制进行特征融合,通过多个相同且独立的二进制分类器判定实体间关系并标记尾实体,构建出融合预训练模型和注意力的联合抽取模型（JPEA）。实验结果表明,该方法能显著提升抽取的效果,对比不同预训练模型下抽取任务的性能,进一步说明了模型的优越性。     

基于上下文语义增强的实体关系联合抽取

基于span的联合抽取模型在实体和关系抽取（RE）任务中共享实体span的语义表示,能有效降低流水线模型带来的级联误差,但现有模型无法充分地将上下文信息融入实体和关系的表示中。针对上述问题,提出一个基于上下文语义增强的实体关系联合抽取（JERCE）模型。首先通过对比学习的方法获取句子级文本和实体间文本的语义特征表示;然后,将该表示加入实体和关系的表示中,对实体关系进行联合预测;最后,动态调整两个任务的损失以使联合模型的整体性能最优化。在公共数据集CoNLL04、ADE和ACE05上进行实验,结果显示JERCE模型与触发器感知记忆流框架（TriMF）相比,实体识别F1值分别提升了1.04、0.13和2.12个百分点,RE的F1值则分别提升了1.19、1.14和0.44个百分点。实验结果表明,JERCE模型可以充分获取上下文中的语义信息。     

用于关系抽取的注意力图长短时记忆神经网络

关系抽取是信息获取中一项关键技术。句子结构树能够捕获单词之间的长距离依赖关系,已被广泛用于关系抽取任务中。但是,现有方法存在过度依赖句子结构树本身信息而忽略外部信息的不足。本文提出一种新型的图神经网络模型,即注意力图长短时记忆神经网络(attention graph long short term memory neural network, AGLSTM)。该模型采用一种软修剪策略自动学习对关系抽取有用的句子结构信息;通过引入注意力机制,结合句法图信息学习句子的结构特征;并设计一种新型的图长短时记忆神经网络,使得模型能够更好地融合句法图信息和句子的时序信息。与10种典型的关系抽取方法进行对比,实验验证了该模型的优异性能。     

基于图卷积神经网络的中文实体关系联合抽取

现有实体关系联合抽取方法未充分考虑中文句子中实体关系的复杂结构特征,为此,提出一种基于图卷积神经网络（GCN）的中文实体关系联合抽取方法。在双向长短时记忆网络抽取序列特征的基础上,利用GCN编码依存分析结果中的语法结构信息,借鉴改进的实体标注策略构建端到端的中文实体关系联合抽取模型。实验结果表明,该方法的F值可达61.4%,相比LSTM-LSTM模型提高了4.1%,GCN能有效编码文本的先验词间关系并提升实体关系抽取性能。     

基于BiLSTM和ResCNN的实体关系抽取方法

当前大多数实体关系抽取方法无法获取较长句子中的远距离依赖信息,并且由于远程监督数据噪声的干扰导致关系抽取性能下降。因此本文提出一种基于双向长短期记忆网络（BiLSTM）和残差卷积神经网络（ResCNN）的实体关系抽取模型,该模型在向量表示阶段采用BiLSTM获取词语的上下文信息向量,利用残差网络将卷积神经网络中低层的特征传递到高层,有效解决梯度消失问题。同时将挤压-激励块嵌入残差网络中,能大幅降低数据噪声,强化特征传递,在池化阶段采用分段最大化池化方法来捕捉实体对的结构信息。设计在NYT-Freebase数据集上的验证实验,实验结果表明,该模型能够充分学习特征,显著提升实体关系抽取的效果。     

基于藏文Albert预训练语言模型的图采样与聚合实体关系抽取

实体关系抽取任务是对句子中实体对间的语义关系进行识别。该文提出了一种基于Albert预训练语言模型结合图采样与聚合算法(Graph Sampling and Aggregation, GraphSAGE)的实体关系抽取方法,并在藏文实体关系抽取数据集上实验。该文针对藏文句子特征表示匮乏、传统藏文实体关系抽取模型准确率不高等问题,提出以下方案：(1)使用预先训练的藏文Albert模型获得高质量的藏文句子动态词向量特征;(2)使用提出的图结构数据构建与表示方法生成GraphSAGE模型的输入数据,并通过实验证明了该方法的有效性;(3)借鉴GraphSAGE模型的优势,利用其图采样与聚合操作进行关系抽取。实验结果表明,该文方法有效提高了藏文实体关系抽取模型的准确率,且优于基线实验效果。     

基于Span方法和多叉解码树的实体关系抽取

实体关系抽取作为自然语言处理领域的一项关键技术,在构建知识图谱、信息检索等领域有着极为重要的意义。然实体关系抽取模型普遍存在词与词之间依赖性运用不足、实体识别效果低下以及单解码带来的三元组强行执行某种不必要顺序的问题。为了解决这三个方面的问题,提升模型的性能,提出了一种新的实体关系抽取模型。该模型首先运用提取特征能力更强的BERT预训练模型获取句子表征,然后采用图卷积神经网络来增强实体与关系之间的依赖关系,再使用对实体提取能力更强的Span方法(识别实体的神经网络方法)进行实体抽取,最后采用深度多叉解码树实施并行解码得到相应的关系三元组。在CoNLL04、ADE数据集上的实验结果表明,与其他的关系抽取基线模型相比,该模型的F1值具有较好的提升,同时也验证了该文模型的有效性与泛化能力。     

基于强化学习的实体关系联合抽取模型

针对现有的基于远程监督的实体和关系抽取方法存在着标签噪声问题，提出了一种基于强化学习的实体关系联合抽取方法。该模型有两个模块：句子选择器模块和实体关系联合抽取模块。首先，句子选择器模块选择没有标签噪声的高质量句子，将所选句子输入到实体关系联合抽取模型；然后，实体关系联合抽取模块采用序列标注方法对输入的句子进行预测，并向句子选择器模块提供反馈，指导句子选择器模块挑选高质量的句子；最后，句子选择器模块和实体关系联合抽取模块同时训练，将句子选择与序列标注一起优化。实验结果表明，该模型在实体关系联合抽取中的F1值为47.3%，与CoType为代表的联合抽取模型相比，所提模型的F1值提升了1%；与LINE为代表的串行模型相比，所提模型的F1值提升了14%。结果表明强化学习结合实体关系联合抽取模型能够有效地提高序列标注模型的F1值，其中句子选择器能有效地处理数据的噪声。     

实体驱动的双向LSTM篇章连贯性建模

篇章连贯性建模是自然语言处理研究领域的一个基础问题。主流的篇章连贯性模型分为两大类,分别是基于实体网格的连贯性模型和基于神经网络的篇章连贯性模型。其中,基于实体网格的篇章连贯性模型需要进行特征提取,而基于深度学习的模型没有充分考虑篇章中句子间的实体链接对连贯性建模的重要作用。基于此,该文首先抽取篇章中相邻句子的实体信息,将其进行分布式表示,然后将此信息通过多种简单且有效的向量操作融合至句子级的双向LSTM深度学习模型之中。在汉语和英语篇章语料上的句子排序和中英文机器翻译连贯性检测两种任务上的实验表明该文提出的模型性能和现有模型相比有所提升,尤其在中文上有显著提升。     

基于联合模型的藏文实体关系抽取方法研究

从无结构文本中抽取实体与实体之间的关系是自然语言处理领域的重要研究内容,同时也为构建知识图谱、问答系统等应用提供重要支撑。基于联合模型的实体关系抽取任务将实体识别和关系抽取同时进行,克服了传统实体关系抽取任务中先识别句子中的实体,然后再进行实体关系判断这两次任务中的错误累加。该文针对藏文语料匮乏、实体识别准确率不高等问题,提出了基于联合模型抽取藏文实体关系的方法。基于藏文实体关系抽取任务,提出以下方案: ①针对藏文分词准确率不高的问题,对藏文进行字级和词级两种方式进行预处理,并给出对比实验,结果表明采用字级处理方式较词级处理方式效果有所提高。②藏文是一种语法规则比较强的语言,名词、格助词等能明确指示句子各组块之间的语法和语义结构关系,因此该文将藏文的词性标注特征加入到藏文的字词向量中,实验结果证明了方法的有效性。③该文借鉴了联合模型处理的优势,提出基于联合模型处理方式,采用端到端的BiLSTM框架将藏文实体关系抽取任务转变为藏文序列标注的问题,实验结果表明,该文的方法较传统的基于藏文处理方式,如SVM算法和LR算法,准确率提高了30%～40%。     

European Community policy and the market

Abstract This paper starts with some reflections on the policy considerations and priorities which are shaping European Commission (EC) research programmes. Then it attempts to position the current projects which seek to capitalise on information and communications technologies for learning in relation to these priorities and the apparent realities of the marketplace. It concludes that while there are grounds to be optimistic about the contribution EC programmes can make to the efficiency and standard of education and training, they are still too technology driven.     

一种自适应子融合集成多分类器方法

融合集成方法已经广泛应用在模式识别领域,然而一些基分类器实时性能稳定性较差,导致多分类器融合性能差,针对上述问题本文提出了一种新的基于多分类器的子融合集成分类器系统。该方法考虑在度量层融合层次之上通过对各类基多分类器进行动态选择,票数最多的类别作为融合系统中对特征向量识别的类别,构成一种新的自适应子融合集成分类器方法。实验表明,该方法比传统的分类器以及分类融合方法识别准确率明显更高,具有更好的鲁棒性。     

Explanation and prediction: an architecture for default and abductive reasoning

Although there are many arguments that logic is an appropriate tool for artificial intelligence, there has been a perceived problem with the monotonicity of classical logic. This paper elaborates on the idea that reasoning should be viewed as theory formation where logic tells us the consequences of our assumptions. The two activities of predicting what is expected to be true and explaining observations are considered in a simple theory formation framework. Properties of each activity are discussed, along with a number of proposals as to what should be predicted or accepted as reasonable explanations. An architecture is proposed to combine explanation and prediction into one coherent framework. Algorithms used to implement the system as well as examples from a running implementation are given.     

Three Process Perspectives: Organizations, Teams, and People

This paper provides the author's personal views and perspectives on software process improvement. Starting with his first work on technology assessment in IBM over 20 years ago, Watts Humphrey describes the process improvement work he has been directly involved in. This includes the development of the early process assessment methods, the original design of the CMM, and the introduction of the Personal Software Process (PSP)^SM and Team Software Process (TSP){^SM}. In addition to describing the original motivation for this work, the author also reviews many of the problems he and his associates encountered and why they solved them the way they did. He also comments on the outstanding issues and likely directions for future work. Finally, this work has built on the experiences and contributions of many people. Mr. Humphrey only describes work that he was personally involved in and he names many of the key contributors. However, so many people have been involved in this work that a full list of the important participants would be impractical.     

基于复小波噪声方差显著修正的SAR图像去噪

提出了一种基于复小波域统计建模与噪声方差估计显著性修正相结合的合成孔径雷达(Synthetic Aperture Radar,SAR)图像斑点噪声滤波方法。该方法首先通过对数变换将乘性噪声模型转化为加性噪声模型,然后对变换后的图像进行双树复小波变换(Dualtree Complex Wavelet Transform,DCWT),并对复数小波系数的统计分布进行建模。在此先验分布的基础上,通过运用贝叶斯估计方法从含噪系数中恢复原始系数,达到滤除噪声的目的。实验结果表明该方法在去除噪声的同时保留了图像的细节信息,取得了很好的降噪效果。     

How do children do mathematics with LOGO?

Abstract  This paper considers some results of a study designed to investigate the kinds of mathematical activity undertaken by children (aged between 8 and 11) as they learned to program in LOGO. A model of learning modes is proposed, which attempts to describe the ways in which children used and acquired understanding of the programming/mathematical concepts involved. The remainder of the paper is concerned with discussing the validity and limitations of the model, and its implications for further research and curriculum development.     

Editorial: Special issue on recent advances in hybrid control theory and applications

正The demands of a rapidly advancing technology for faster and more accurate controllers have always had a strong influence on the progress of automatic control theory.In recent years control problems have been arising with increasing frequency in widely different areas,which cannot be addressed using conventional control techniques.The principal reason for this is the fact that a highly competitive economy is forcing systems to operate in regimes where     

Information for Authors

正Aim The Journals of Zhejiang University-SCIENCE(A/B/C)areedited by the international board of distinguished Chinese andforeign scientists,and are aimed to present the latest devel-opments and achievements in scientific research in China andoverseas to the world’s scientific circles,especially to stimulateand promote academic exchange between Chinese and for-eign scientists everywhere.     

Enhancing a leaf radiative transfer model to estimate concentrations and in vivo specific absorption coefficients of total carotenoids and chlorophylls a and b from single-needle reflectance and transmittance

The relative concentrations of different pigments within a leaf have significant physiological and spectral consequences. Photosynthesis, light use efficiency, mass and energy exchange, and stress response are dependent on relationships among an ensemble of pigments. This ensemble also determines the visible characteristics of a leaf, which can be measured remotely and used to quantify leaf biochemistry and structure. But current remote sensing approaches are limited in their ability to resolve individual pigments. This paper focuses on the incorporation of three pigments—chlorophyll a, chlorophyll b, and total carotenoids—into the LIBERTY leaf radiative transfer model to better understand relationships between leaf biochemical, biophysical, and spectral properties.Pinus ponderosa and Pinus jeffreyi needles were collected from three sites in the California Sierra Nevada. Hemispheric single-leaf visible reflectance and transmittance and concentrations of chlorophylls a and b and total carotenoids of fresh needles were measured. These data were input to the enhanced LIBERTY model to estimate optical and biochemical properties of pine needles. The enhanced model successfully estimated reflectance (RMSE = 0.0255, BIAS = 0.00477, RMS%E = 16.7%), had variable success estimating transmittance (RMSE = 0.0442, BIAS = 0.0294, RMS%E = 181%), and generated very good estimates of carotenoid concentrations (RMSE = 2.48 µg/cm², BIAS = 0.143 µg/cm², RMS%E = 20.4%), good estimates of chlorophyll a concentrations (RMSE = 10.7 µg/cm², BIAS = − 0.992 µg/cm², RMS%E = 21.1%), and fair estimates of chlorophyll b concentrations (RMSE = 7.49 µg/cm², BIAS = − 2.12 µg/cm², RMS%E = 43.7%). Overall root mean squared errors of reflectance, transmittance, and pigment concentration estimates were lower for the three-pigment model than for the single-pigment model. The algorithm to estimate three in vivo specific absorption coefficients is robust, although estimated values are distorted by inconsistencies in model biophysics. The capacity to invert the model from single-leaf reflectance and transmittance was added to the model so it could be coupled with vegetation canopy models to estimate canopy biochemistry from remotely sensed data.     

Mr. Twinn’s bombes

This article discusses the history and design of the special versions of the bombe key-finding machines used by Britain’s Government Code & Cypher School (GC&CS) during World War II to attack the Enigma traffic of the Abwehr (the German military intelligence service). These special bombes were based on the design of their more numerous counterparts used against the traffic of the German armed services, but differed from them in important ways that highlight the adaptability of the British bombe design, and the power and flexibility of the diagonal board. Also discussed are the changes in the Abwehr indicating system that drove the development of these machines, the ingenious ways in which they were used, and some related developments involving the bombes used by the U.S. Navy’s cryptanalytic unit (OP-20-G).