Empirical prediction limit estimation methods for feed-forward neural networks

Feed-forward neural networks (FFNs) have gained a lot of interest in the last decade as empirical models for their powerful representational capacity, non-parametric nature and multivariate characteristics. While these neural network models focus primarily on accurate prediction of output values, often outperforming their statistical counterparts in dealing with sparse date sets, they usually do not provide any information regarding the confidence with which they make these predictions. Since prediction limits (PLs) indicate the extent to which one can rely on predictions for making future decisions, it is of paramount importance to estimate these limits. Two empirical PL estimation methods for FFNs are presented. The two methods differ in one fundamental aspect: the method employed for modeling the properties of the neural network model residuals. While one method uses a local approximation scheme, the other utilizes a global approximation scheme. Simulation results reveal that both methods have their relative strengths and weaknesses.     

基于多模态表示学习的知识库补全

目前大多数知识图谱表示学习只考虑实体和关系之间的结构知识,性能受存储知识的限制,造成知识库补全能力不稳定,而融入外部信息的知识表示方法大多只针对某一特定的外部模态信息建模,适用范围有限.因此,文中提出带有注意力模块的卷积神经网络模型.首先,考虑文本和图像两种外部模态信息,提出三种融合外部模态信息和实体的方案,获得实体的多模态表示.再通过结合通道注意力模块和空间注意力模块,增强卷积的表现力,提高知识表示的质量,提升模型的补全能力.在多个公开的多模态数据集上进行链路预测和三元组分类实验,结果表明文中模型性能较优.     

Prediction of Resource Availability in Fine-Grained Cycle Sharing Systems Empirical Evaluation

Fine-Grained Cycle Sharing (FGCS) systems aim at utilizing the large amount of computational resources available on the Internet. In FGCS, host computers allow guest jobs to utilize the CPU cycles if the jobs do not significantly impact the local users. Such resources are generally provided voluntarily and their availability fluctuates highly. Guest jobs may fail unexpectedly, as resources become unavailable. To improve this situation, we consider methods to predict resource availability. This paper presents empirical studies on resource availability in FGCS systems and a prediction method. From studies on resource contention among guest jobs and local users, we derive a multi-state availability model. The model enables us to detect resource unavailability in a non-intrusive way. We analyzed the traces collected from a production FGCS system for 3 months. The results suggest the feasibility of predicting resource availability, and motivate our method of applying semi-Markov Process models for the prediction. We describe the prediction framework and its implementation in a production FGCS system, named iShare. Through the experiments on an iShare testbed, we demonstrate that the prediction achieves an accuracy of 86% on average and outperforms linear time series models, while the computational cost is negligible. Our experimental results also show that the prediction is robust in the presence of irregular resource availability. We tested the effectiveness of the prediction in a proactive scheduler. Initial results show that applying availability prediction to job scheduling reduces the number of jobs failed due to resource unavailability. This work was supported, in part, by the National Science Foundation under Grants No. 0103582-EIA, 0429535-CCF, and 0650016-CNS. We thank Ruben Torres for his help with the reference prediction algorithms used in our experiments.     

FOXBASE与其他语言的混合编程综述

ＦＯＸＢＡＳＥ、汇编及各种高级语言各有所长，如采用ＦＯＸＢＡＳＥ与其他语言混合编程就可以兼得双方之长，继承优秀软件成果，充分利用软硬件资源。本文综述了ＦＯＸＢＡＳＥ与高级语言、汇编语言的混合编程技术，以及它们之间的数据通讯方法。阐明了高级语言程序、宏汇编程序如何调用ＦＯＸＢＡＳＥ程序；ＦＯＸＢＡＳＥ又如何调用宏汇编和Ｃ语言程序；ＦＯＸＢＡＳＥ和高级语言在文本文件和桥的基础上如何互相通讯；高级语言如     

Character-level arabic text generation from sign language video using encoder–decoder model

Video to text conversion is a vital activity in the field of computer vision. In recent years, deep learning algorithms have dominated automatic text generation in English, but there are a few research works available for other languages. In this paper, we propose a novel encoding–decoding system that generates character-level Arabic sentences from isolated RGB videos of Moroccan sign language. The video sequence was encoded by a spatiotemporal feature extraction using pose estimation models, while the label text of the video is transmitted to a sequence of representative vectors. Both the features and the label vector are joined and treated by a decoder layer to derive a final prediction. We trained the proposed system on an isolated Moroccan Sign Language dataset (MoSLD), composed of RGB videos from 125 MoSL signs. The experimental results reveal that the proposed model attains the best performance under several evaluation metrics.     

Text classification method based on self-training and LDA topic models

Supervised text classification methods are efficient when they can learn with reasonably sized labeled sets. On the other hand, when only a small set of labeled documents is available, semi-supervised methods become more appropriate. These methods are based on comparing distributions between labeled and unlabeled instances, therefore it is important to focus on the representation and its discrimination abilities. In this paper we present the ST LDA method for text classification in a semi-supervised manner with representations based on topic models. The proposed method comprises a semi-supervised text classification algorithm based on self-training and a model, which determines parameter settings for any new document collection. Self-training is used to enlarge the small initial labeled set with the help of information from unlabeled data. We investigate how topic-based representation affects prediction accuracy by performing NBMN and SVM classification algorithms on an enlarged labeled set and then compare the results with the same method on a typical TF-IDF representation. We also compare ST LDA with supervised classification methods and other well-known semi-supervised methods. Experiments were conducted on 11 very small initial labeled sets sampled from six publicly available document collections. The results show that our ST LDA method, when used in combination with NBMN, performed significantly better in terms of classification accuracy than other comparable methods and variations. In this manner, the ST LDA method proved to be a competitive classification method for different text collections when only a small set of labeled instances is available. As such, the proposed ST LDA method may well help to improve text classification tasks, which are essential in many advanced expert and intelligent systems, especially in the case of a scarcity of labeled texts.     

多标签文本分类研究进展

文本分类作为自然语言处理中一个基本任务,在20世纪50年代就已经对其算法进行了研究,现在单标签文本分类算法已经趋向成熟,但是对于多标签文本分类的研究还有很大的提升空间。介绍了多标签文本分类的基本概念以及基本流程,包括数据集获取、文本预处理、模型训练和预测结果。介绍了多标签文本分类的方法。这些方法主要分为两大类：传统机器学习方法和基于深度学习的方法。传统机器学习方法主要包括问题转换方法和算法自适应方法。基于深度学习的方法是利用各种神经网络模型来处理多标签文本分类问题,根据模型结构,将其分为基于CNN结构、基于RNN结构和基于Transfomer结构的多标签文本分类方法。对多标签文本分类常用的数据集进行了梳理总结。对未来的发展趋势进行了分析与展望。     

基于图论的文本数字水印技术

由于文本自身的一些特点,相对于其他的媒体,在文本中嵌入数字水印更加困难.这造成了当前文本数字水印技术的发展远远落后于其他数字水印技术.提出了一种新的基于图论的文本数字水印技术,通过适当改变字符或字符串的拓扑结构,设计出语义上相同的字符或字符串的多种字形,并将这些字形映射为图论中的“图”,对“图”或者“图”的特征量进行恰当的编码,利用这些编码来表示数字水印;描述了水印嵌入、检测方法的数学模型;给出了鲁棒性和视觉影响试验方法与结果;提出了文本数字水印系统通用去除攻击准则,并分析了这种水印的抗攻击性能与方法.通过实验与分析可知,该文本数字水印技术具有水印容量大、鲁棒性强、视觉影响小、抗攻击能力强的特点.使用这种技术,在字符中嵌入水印对于汉语、韩语等象形文字具有一定的优势,而英语、法语等字母文字适于在字符串中嵌入水印.     

赛事分析与预测的KDD建模

把KDD用于赛事决策支持,见之报道的还很少,本文针对一类非对垒式的,以个体成绩为主进行名次排序的体育和娱乐赛事,给出了一个该类赛事分析与预测的KDD建模方法,用于帮助分析和确定对每个参赛者的比赛成绩有影响的各种重要因素,并对比赛结果作出预测。我们的方法以轻量的、多模型的和多种技术的组合／结合为策略;兼顾到了问题的各个方面和主要特征：可以透过对几个参数的简单处理,自动、方便地对已建立的模型进行不断的修正;同时还能主动建议对模型的改进或重建,并为模型的改进或重建提供有用的帮助信息,从而能够较好地解决参赛群体多样性、多变性、影响因素的复杂性和预测的困难性等问题。     

A sample entropy based prognostics method for lithium-ion batteries using relevance vector machine

Over the increasing number of charging and discharging cycling processes of lithium-ion batteries, the aging and even failure of lithium-ion batteries may occur. If anomalies are not detected in time, lithium-ion batteries could cause major safety accidents. In this paper, a prognostics method integrating the sample entropies and relevance vector machine (RVM) is proposed to estimate the remaining useful life (RUL) of lithium-ion batteries. First, RUL prediction using multiple inputs, including the voltage sample entropy and the current sample entropy, are compared with prediction methods based on a single entropy input. The multiple entropy input method indicates better capability of describing the battery degradation process. In addition, the wavelet denoising method is used to pre-process the inputs to remove sudden and unusual changes in the battery capacity degradation data. A prediction model using the denoised entropy inputs is constructed through linearly weighting the entropy inputs in the RVM model. The weight for each input is assigned according to the individual contribution to the prediction accuracy. Experimental data from lithium-ion battery testing are applied to three prediction models with different entropy inputs. The results indicate that the proposed method has higher prediction accuracy than those in existing models only using a single sample entropy. The proposed method has potentials for the RUL estimation of industrial machinery in manufacturing.     

THE LANGUAGE COMPONENT OF THE FASTY TEXT PREDICTION SYSTEM

ABSTRACT

This paper describes the language component of FASTY, a text prediction system designed to improve text input efficiency for disabled users. The FASTY language component is based on state-of-the-art n-gram-based word-level and part-of-speech-level prediction and on a number of innovative modules (morphological analysis, collocation-based prediction, compound prediction) that are meant to enhance performance in languages other than English. Together with its modular architecture, these novel techniques make it adaptable to a wide range of languages without sacrificing performance. Currently, versions for Dutch, German, French, Italian, and Swedish are supported. The system can be parameterized to be used with different user interfaces and for a range of different applications. In this paper, we discuss each of the modules in detail and we present a series of experimental evaluations of the system.     

Composing domain-specific physical models with general-purpose software modules in embedded control software

A considerable portion of software systems today are adopted in the embedded control domain. Embedded control software deals with controlling a physical system, and as such models of physical characteristics become part of the embedded control software. In current practices, usually general-purpose languages (GPL), such as C/C++ are used for embedded systems development. Although a GPL is suitable for expressing general-purpose computation, it falls short in expressing the models of physical characteristics as desired. This reduces not only the readability of the code but also hampers reuse due to the lack of dedicated abstractions and composition operators. Moreover, domain-specific static and dynamic checks may not be applied effectively. There exist domain-specific modeling languages (DSML) and tools to specify models of physical characteristics. Although they are commonly used for simulation and documentation of physical systems, they are often not used to implement embedded control software. This is due to the fact that these DSMLs are not suitable to express the general-purpose computation and they cannot be easily composed with other software modules that are implemented in GPL. This paper presents a novel approach to combine a DSML to model physical characteristics and a GPL to implement general-purpose computation. The composition filters model is used to compose models specified in the DSML with modules specified in the GPL at the abstraction level of both languages. As such, this approach combines the benefits of using a DSML to model physical characteristics with the freedom of a GPL to implement general-purpose computation. The approach is illustrated using two industrial case studies from the printing systems domain.     

Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis

Sentiment analysis is the natural language processing task dealing with sentiment detection and classification from texts. In recent years, due to the growth in the quantity and fast spreading of user-generated contents online and the impact such information has on events, people and companies worldwide, this task has been approached in an important body of research in the field. Despite different methods having been proposed for distinct types of text, the research community has concentrated less on developing methods for languages other than English. In the above-mentioned context, the present work studies the possibility to employ machine translation systems and supervised methods to build models able to detect and classify sentiment in languages for which less/no resources are available for this task when compared to English, stressing upon the impact of translation quality on the sentiment classification performance. Our extensive evaluation scenarios show that machine translation systems are approaching a good level of maturity and that they can, in combination to appropriate machine learning algorithms and carefully chosen features, be used to build sentiment analysis systems that can obtain comparable performances to the one obtained for English.     

Fast sampling methods for Bayesian max-margin models

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampling for these models still remains challenging, especially for applications that involve large-scale datasets. In this paper, we present the stochastic subgradient Hamiltonian Monte Carlo (HMC) methods, which are easy to implement and computationally efficient. We show the approximate detailed balance property of subgradient HMC which reveals a natural and validated generalization of the ordinary HMC. Furthermore, we investigate the variants that use stochastic subsampling and thermostats for better scalability and mixing. Using stochastic subgradient Markov Chain Monte Carlo (MCMC), we efficiently solve the posterior inference task of various Bayesian max-margin models and extensive experimental results demonstrate the effectiveness of our approach.     

Models and technologies for the extraction of aggregated knowledge to control processes of the retrieval of non-structured information

To control the retrieval and to arrange the retrieval of non-structured information in large-scale collections of texts in natural languages, crucially new models of aggregated representation of knowledge about their topic and content (semiotic and semantic characteristics of the text) are proposed. Based on the proposed models, techniques of computational extraction and use of knowledge about the topic and content of text collections are developed. The proposed models and techniques are based on formation of secondary information resources characterizing the topic and content of particular texts in natural languages.     

基于LSTM及分位数回归理论的配电变压器重过载概率预测

配电变压器的重过载是导致变压器故障和损坏的主要原因之一。因此,准确地预测配电变压器的运行情况对于电力系统的安全和可靠运行至关重要。由于配电台区负荷受到诸多复杂变量的影响,同时这些复杂变量的影响往往无法可靠建模估计,故最终预测结果表现出一定的不确定性。传统单点预测为预测单一最优值,无法充分量化预测的不确定性。本文融合多维特征与变压器历史运行数据,采用分位数回归方法对台区负荷情况进行建模,通过将条件分位数与一般线性或非线性模型结合来构建概率预测模型,分位数回归能够对整个条件分布建模,相对于标准回归方法其可以提供更多信息。传统的点预测由于其无法估计预测结果的不确定性,故对业务部门的决策具有一定的风险,而概率预测不仅可以像点预测一样提供未来最优预测点,也可以提供未来预测值的分布情况,概率预测以预测区间或分位数的形式可以更好地估计重过载情况。     

小时间粒度网络流量自回归预测分析

网络流量测量和预测是网络QoS管理和流量工程中一个重要的组成部分,尤其是对于为了保证网络QoS而引入的一些实时方法,比如接纳控制,资源预留等。较好的网络流量预测效果,能有效地提高这些方法的工作效率,从而有效提高网络带宽的利用率,保证网络QoS。所以高效的网络流量预测不仅是值得的,而且是必要的。由于许多文献研究的是网络流量在大时间粒度(天、周、月等)上的自回归特性,不能用于这种以秒级为单位的接纳控制、资源预留等实时方法,所以本文具体分析了网络流量在小时间粒度的自相似特性,并提出了其自回归预测模型。在模拟实验中采用了实际网络流量,并证明了在大多数情况下预测误差小于15%的概率为90%,它可有效地应用到接纳控制等方面的网络流量预测中。     

Detecting both superimposed and scene text with multiple languages and multiple alignments in video

Video text often contains highly useful semantic information that can contribute significantly to video retrieval and understanding. Video text can be classified into scene text and superimposed text. Most of the previous methods detect superimposed or scene text separately due to different text alignments. Moreover, because different language characters have different edge and texture features, it is very difficult to detect the multilingual text. In this paper, we first perform a detailed analysis of motion patterns of video text, and show that the superimposed and scene text exhibit different motion patterns on consecutive frames, which is insensitive to multiple language characters and multiple text alignments. Based on our analysis, we define Motion Perception Field (MPF) to represent the text motion patterns. Finally, we propose a text detection algorithms using MPF for both superimposed and scene text with multiple languages and multiple alignments. Experimental results on diverse videos demonstrate that our algorithms are robust, and outperform previous methods for detecting both superimposed and scene texts with multiple languages and multiple alignments.     

面向法律文书的自然语言理解

法律人工智能因其高效、便捷的特点,近年来受到社会各界的广泛关注。法律文书是法律在社会生活中最常见的表现形式,应用自然语言理解方法智能地处理法律文书内容是一个重要的研究和应用方向。该文梳理与总结面向法律文书的自然语言理解技术,首先介绍了五类面向法律文书的自然语言理解任务形式: 法律文书信息提取、类案检索、司法问答、法律文书摘要和判决预测。然后,该文探讨了运用现有自然语言理解技术应对法律文书理解的主要挑战,指出需要解决好法律文书与日常生活语言之间的表述差异性、建模好法律文书中特有的推理与论辩结构,并且需要将法条、推理模式等法律知识融入自然语言理解模型。