A framework for differentially-private knowledge graph embeddings

Knowledge graph (KG) embedding methods are at the basis of many KG-based data mining tasks, such as link prediction and node clustering. However, graphs may contain confidential information about people or organizations, which may be leaked via embeddings. Research recently studied how to apply differential privacy to a number of graphs (and KG) analyses, but embedding methods have not been considered so far. This study moves a step toward filling such a gap, by proposing the Differential Private Knowledge Graph Embedding (DPKGE) framework.DPKGE extends existing KG embedding methods (e.g., TransE, TransM, RESCAL, and DistMult) and processes KGs containing both confidential and unrestricted statements. The resulting embeddings protect the presence of any of the former statements in the embedding space using differential privacy. Our experiments identify the cases where DPKGE produces useful embeddings, by analyzing the training process and tasks executed on top of the resulting embeddings.     

Learning classification rules for telecom customer call data under concept drift

The application of the CD3 decision tree induction algorithm to telecommunications customer call data to obtain classification rules is described. CD3 is robust against drift in the underlying rules over time (concept drift): it both detects drift and protects the induction process from its effects. Specifically, the task is to data mine customer details and call records to determine whether the profile of customers registering for a friends and family service is changing over time and to maintain a rule set profiling such customers. CD3 and the rationale behind it are described and experimental results on customer data are presented.     

面向概念漂移问题的渐进多核学习方法

针对概念漂移问题,构建数据特性随时间发生渐进变化特点的分类学习模型,提出一种基于渐进支持向量机（G-SVM）的渐进多核学习方法（G-MKL）。该方法采用支持向量机（SVM）为基本分类器,进行多区间上的子分类器耦合训练,并通过约束子分类器增量方式使模型适应数据渐进变化特性,最终将多个核函数以线性组合方式融入SVM求解框架。该方法综合发挥了各个核函数的优势,大大提高了模型适应性和有效性。在具有渐变特性的模拟数据集和真实数据集上将所提算法与多种经典算法进行了对比,验证了所提算法在处理非静态数据问题的有效性。     

Exponentially weighted moving average charts for detecting concept drift

Classifying streaming data requires the development of methods which are computationally efficient and able to cope with changes in the underlying distribution of the stream, a phenomenon known in the literature as concept drift. We propose a new method for detecting concept drift which uses an exponentially weighted moving average (EWMA) chart to monitor the misclassification rate of an streaming classifier. Our approach is modular and can hence be run in parallel with any underlying classifier to provide an additional layer of concept drift detection. Moreover our method is computationally efficient with overhead O(1) and works in a fully online manner with no need to store data points in memory. Unlike many existing approaches to concept drift detection, our method allows the rate of false positive detections to be controlled and kept constant over time.     

Translational relation embeddings for multi-hop knowledge base question answering

Multi-hop Knowledge Base Question Answering (KBQA) aims to predict answers that require multi-hop reasoning from the topic entity in the question over the Knowledge Base (KB). Relation extraction is a core step in KBQA, which extracts the relation path from the topic entity to the answer entity. Compared with single-hop questions, multi-hop ones have more complex syntactic structures to understand, and multi-hop relation paths lead to a larger search space, which makes it much more challenging to extract the correct relation paths. To tackle the above challenges, most existing relation extraction approaches focus on the semantic similarity between questions and relation paths. However, those approaches only consider the word semantics of the relation names but ignore the graph semantics inside the knowledge base. As a result, their generalization ability relying on the naming rules of the relations, making it more difficult to generalize over large knowledge bases.To address the current limitations and take advantage of the graph semantics of relations, we propose a novel translational embedding-based relation extractor that utilizes pretrained embeddings from TransE. In particular, we treat the multi-hop relation path as a translation from the first relation to the last one in the semantic space of TransE. Then we map the question into this space under the supervision of the path embeddings. To take full advantage of the pretrained graph semantics in TransE, we propose a KBQA framework that leverages pretrained relation semantics in relation extraction and pretrained entity semantics in answer selection. Our approach achieves state-of-the-art performance on two benchmark datasets, WebQuestionSP and MetaQA, demonstrating its effectiveness on the multi-hop KBQA task.     

基于自学习近邻图策略的短文本匹配方法

针对自然语言处理中的文本匹配问题,提出一种基于自学习文本近邻图框架的深度学习模型,以处理短文本匹配问题。文本近邻图可使用词嵌入将文本转换为向量形式,再通过构建文本相似度关系矩阵获得,可表达文本样本的近邻关系。现有方法通常构造静态的近邻图,这些方法一方面依赖先验知识,另一方面难以获得句子对的最优表示。因此,提出了利用孪生卷积神经网络学习更优的动态更新的近邻图。该模型在Quora数据集上的准确率和◢F◣▼1▽值分别是84.15%和79.88%,在MSRP数据集上的准确率和◢F◣▼1▽值分别是74.55%和81.63%。实验表明,提出的模型能有效地提高文本识别和匹配的准确率。     

DetectA: abrupt concept drift detection in non-stationary environments

Almost all drift detection mechanisms designed for classification problems work reactively: after receiving the complete data set (input patterns and class labels) they apply a sequence of procedures to identify some change in the class-conditional distribution – a concept drift. However, detecting changes after its occurrence can be in some situations harmful to the process under analysis. This paper proposes a proactive approach for abrupt drift detection, called DetectA (Detect Abrupt Drift). Briefly, this method is composed of three steps: (i) label the patterns from the test set (an unlabelled data block), using an unsupervised method; (ii) compute some statistics from the train and test sets, conditioned to the given class labels for train set; and (iii) compare the training and testing statistics using a multivariate hypothesis test. Based on the results of the hypothesis tests, we attempt to detect the drift on the test set, before the real labels are obtained. A procedure for creating datasets with abrupt drift has been proposed to perform a sensitivity analysis of the DetectA model. The result of the sensitivity analysis suggests that the detector is efficient and suitable for datasets of high-dimensionality, blocks with any proportion of drifts, and datasets with class imbalance. The performance of the DetectA method, with different configurations, was also evaluated on real and artificial datasets, using an MLP as a classifier. The best results were obtained using one of the detection methods, being the proactive manner a top contender regarding improving the underlying base classifier accuracy.     

融合层次类型信息的双向图注意力机制的知识图谱嵌入模型

知识图谱嵌入旨在将实体与关系映射到低维且稠密的向量空间中。目前已有的嵌入模型仍存在以下两个方面的缺陷：现有的模型大多只关注知识图谱的语义信息,而忽略了大量三元组的隐藏信息;现有的模型仅关注了实体的单向信息,而忽略了双向的潜在信息。针对以上问题,提出了一种融合层次类型信息的双向图注意力机制的知识图谱嵌入模型Bi-HTGAT,该模型设计了层次类型注意力机制,考虑不同关系下每种类型的不同实体对中心实体的贡献。同时引入了关系的方向注意力机制,通过融合不同方向的邻居信息来更新实体和关系嵌入,最终聚合两部分信息以得到实体的最终嵌入。在基准数据集上的实验证明,Bi-HTGAT在链接预测任务上性能明显优于其他基线模型,充分证明了Bi-HTGAT能够进一步提高嵌入结果的精准度。     

Meta-cognitive Recurrent Recursive Kernel OS-ELM for concept drift handling

In this paper, a Meta-cognitive Recurrent Recursive Kernel Online Sequential Extreme Learning Machine with Drift Detector Mechanism (meta-RRKOS-ELM-DDM) is proposed. It combines the strengths of Recurrent Kernel Online Sequential Extreme Learning Machine with a new modified Drift Detector Mechanism (DDM) and Approximate Linear Dependency Kernel Filter (ALD) in solving concept drift problems and reducing complex computations in the learning. The recursive kernel method successfully replaces the normal kernel method in Recurrent Kernel Online Sequential Extreme Learning Machine with DDM (RKOS-ELM-DDM) and generates a fixed reservoir with optimized information in enhancing the forecasting performance. Meta-cognitive learning strategy decides when the incoming data needs to be updated, retrained, or discarded during learning and automatically finding ALD threshold that reduces the learning time of prediction model. In the experiment, six synthetic and three real-world time series datasets are used to evaluate the ability of recursive kernel method, the performance of concept drift detectors, and meta-cognitive learning strategy in time series prediction. Experimental results indicate the meta-RRKOS-ELM with DDM has superior prediction ability in the different predicting horizons as compared with other algorithms.     

支持企业知识共享的企业本体论及其获取

企业本体论是实现企业知识共享的基础。论文通过分析企业本体概念的结构层次和体现形式,提出了一种基于企业组织层次和业务流程的由底向上获取本体论的方法,增强了本体获取的完整性和简洁性。     

Knowledge representation for mathematical discovery: Three experiments in graph theory

This paper describes the nature of mathematical discovery (including concept definition and exploration, example generation, and theorem conjecture and proof), and considers how such an intelligent process can be simulated by a machine. Although the material is drawn primarily from graph theory, the results are immediately relevant to research in mathematical discovery and learning.The thought experiment, a protocol paradigm for the empirical study of mathematical discovery, highlights behavioral objectives for machine simulation. This thought experiment provides an insightful account of the discovery process, motivates a framework for describing mathematical knowledge in terms of object classes, and is a rich source of advice on the design of a system to perform discovery in graph theory. The evaluation criteria for a discovery system, it is argued, must include both a set of behavior to display (behavioral objectives) and a target set of facts to be discovered (factual knowledge).Cues from the thought experiment are used to formulate two hierarchies of representational languages for graphy theory. The first hierarchy is based on the superficial terminology and knowledge of the thought experiment. Generated by formal grammars with set-theoretic semantics, this eminently reasonable approach ultimately fails to meet the factual knowledge criteria. The second hierarchy uses declarative expressions, each of which has a semantic interpretation as a stylized, recursive algorithm that defines a class by generating it correctly and completely. A simple version of one such representation is validated by a successful, implemented system called Graph Theorist (GT) for mathematical research in graph theory. GT generates correct examples, defines and explores new graph theory properties, and conjectures and proves theorems.Several themes run through this paper. The first is the dual goals, behavioral objectives and factural knowledge to be discovered, and the multiplicity of their demands on a discovery system. The second theme is the central role of object classes to knowledge representation. The third is the increased power and flexibility of a constructive (generator) definition over the more traditional predicate (tester) definition. The final theme is the importance of examples and recursion in mathematical knowledge. The results provide important guidance for further research in the simulation of mathematical discovery.     

Benchmarking knowledge-driven zero-shot learning

External knowledge (a.k.a. side information) plays a critical role in zero-shot learning (ZSL) which aims to predict with unseen classes that have never appeared in training data. Several kinds of external knowledge, such as text and attribute, have been widely investigated, but they alone are limited with incomplete semantics. Some very recent studies thus propose to use Knowledge Graph (KG) due to its high expressivity and compatibility for representing kinds of knowledge. However, the ZSL community is still in short of standard benchmarks for studying and comparing different external knowledge settings and different KG-based ZSL methods. In this paper, we proposed six resources covering three tasks, i.e., zero-shot image classification (ZS-IMGC), zero-shot relation extraction (ZS-RE), and zero-shot KG completion (ZS-KGC). Each resource has a normal ZSL benchmark and a KG containing semantics ranging from text to attribute, from relational knowledge to logical expressions. We have clearly presented these resources including their construction, statistics, data formats and usage cases w.r.t. different ZSL methods. More importantly, we have conducted a comprehensive benchmarking study, with a few classic and state-of-the-art methods for each task, including a method with KG augmented explanation. We discussed and compared different ZSL paradigms w.r.t. different external knowledge settings, and found that our resources have great potential for developing more advanced ZSL methods and more solutions for applying KGs for augmenting machine learning. All the resources are available at https://github.com/China-UK-ZSL/Resources_for_KZSL.     

融合实体概念描述与图像特征的知识图谱表示学习研究

知识图谱表示学习将实体和关系映射到一个连续的低维空间.传统学习方法是从结构化的三元组学习知识表示,忽略了三元组之外与实体相关的丰富多源信息.针对该问题,提出一种将实体概念描述和图像特征与事实三元组相结合的知识图谱表示学习模型DIRL.首先,利用BERT模型进行实体概念描述的语义表示;其次,使用CNN编码器对图像总体特征进行提取,然后通过基于注意力的方法表示图像特征;最后,将基于概念描述的表示和基于图像特征的表示与翻译模型TransR结合起来进行知识图谱表示学习.通过实验验证,DIRL模型优于现有方法,提高了多源信息知识图谱表示的有效性.     

基于知识图谱驱动的图神经网络推荐模型

知识图谱(KG)蕴含丰富的结构与关联信息,不仅可以缓解推荐系统中数据稀疏、冷启动等问题,还可以更准确地进行个性化推荐,因此提出一种基于知识图谱驱动的端到端图神经网络推荐模型KGLN.首先使用单层神经网络框架对图中单个节点进行特征融合,并加入影响因子来改变不同邻居实体的聚合权重;然后通过迭代的方式将单层扩展到多层,使实体...     

A robust incremental learning method for non-stationary environments

Recent machine learning challenges require the capability of learning in non-stationary environments. These challenges imply the development of new algorithms that are able to deal with changes in the underlying problem to be learnt. These changes can be gradual or trend changes, abrupt changes and recurring contexts. As the dynamics of the changes can be very different, existing machine learning algorithms exhibit difficulties to cope with them. Several methods using, for instance, ensembles or variable length windowing have been proposed to approach this task.In this work we propose a new method, for single-layer neural networks, that is based on the introduction of a forgetting function in an incremental online learning algorithm. This forgetting function gives a monotonically increasing importance to new data. Due to the combination of incremental learning and increasing importance assignment the network forgets rapidly in the presence of changes while maintaining a stable behavior when the context is stationary.The performance of the method has been tested over several regression and classification problems and its results compared with those of previous works. The proposed algorithm has demonstrated high adaptation to changes while maintaining a low consumption of computational resources.