Time series forecasting with a neuro-fuzzy modeling scheme

Time series forecasting concerns the prediction of future values based on the observations previously taken at equally spaced time points. Statistical methods have been extensively applied in the forecasting community for the past decades. Recently, machine learning techniques have drawn attention and useful forecasting systems based on these techniques have been developed. In this paper, we propose an approach based on neuro-fuzzy modeling for time series prediction. Given a predicting sequence, the local context of the sequence is located in the series of the observed data. Proper lags of relevant variables are selected and training patterns are extracted. Based on the extracted training patterns, a set of TSK fuzzy rules are constructed and the parameters involved in the rules are refined by a hybrid learning algorithm. The refined fuzzy rules are then used for prediction. Our approach has several advantages. It can produce adaptive forecasting models. It works for univariate and multivariate prediction. It also works for one-step as well as multi-step prediction. Several experiments are conducted to demonstrate the effectiveness of the proposed approach.     

基于分量属性近邻传播的多元时间序列数据聚类方法

鉴于传统方法不能直接有效地对多元时间序列数据进行聚类分析,提出一种基于分量属性近邻传播的多元时间序列数据聚类方法.通过动态时间弯曲方法度量多元时间序列数据之间的总体距离,利用近邻传播聚类算法分别对数据之间的总体距离矩阵和分量近似距离矩阵进行聚类分析,综合考虑这两种视角下序列数据之间的关联关系,使用近邻传播方法对反映原始多元时间序列数据的综合关系矩阵实现较高质量的聚类.数值实验结果表明,与传统聚类方法相比,所提出方法不仅能够有效地反映总体数据特征之间的关系,而且通过重要分量属性序列之间的关联关系分析能够提高原始时间序列数据的聚类效果.     

An analysis of alignment and integral based kernels for machine learning from vessel trajectories

In this paper we present an analysis of the application of the two most important types of similarity measures for moving object trajectories in machine learning from vessel movement data. These similarities are applied in the tasks of clustering, classification and outlier detection. The first similarity type are alignment measures, such as dynamic time warping and edit distance. The second type are based on the integral over time between two trajectories. Following earlier work we define these measures in the context of kernel methods, which provide state-of-the-art, robust algorithms for the tasks studied. Furthermore, we include the influence of applying piecewise linear segmentation as pre-processing to the vessel trajectories when computing alignment measures, since this has been shown to give a positive effect in computation time and performance.In our experiments the alignment based measures show the best performance. Regular versions of edit distance give the best performance in clustering and classification, whereas the softmax variant of dynamic time warping works best in outlier detection. Moreover, piecewise linear segmentation has a positive effect on alignments, due to the fact that salient points in a trajectory, especially important in clustering and outlier detection, are highlighted by the segmentation and have a large influence in the alignments. Based on our experiments, integral over time based similarity measures are not well-suited for learning from vessel trajectories.     

基于DTW 距离的探地雷达数据可视化

提出一种基于动态时间弯曲算法距离度量的探地雷达数据可视化方法,利用动态 时间弯曲算法在时间轴方向上伸缩的优越性,结合可指定类数的聚类算法对探地雷达数据进行 聚类和可视化分析。可用于实测的探地雷达数据集,实验结果表明,相对于传统的聚类算法, 本文算法能得到更好的聚类结果。     

A non-linear clustering method for fuzzy time series: Histogram damping partition under the optimized cluster paradox

The aim of this paper is to investigate the problem of finding the efficient number of clusters in fuzzy time series. The clustering process has been discussed in the existing literature, and a number of methods have been suggested. These methods have several drawbacks, especially the lack of cluster shape and quantity optimization. There are two critical dimensions in a fuzzy time series clustering: the selection of a proper interval for fuzzy clusters and the optimization of the membership degrees among the fuzzy cluster set. The existing methods for the interval selection assume that the intended data has a short-tailed distribution, and the cluster intervals are established in identical lengths (e.g. Song and Chissom, 1994; Chen, 1996; Yolcu et al., 2009). However, the time series data (particularly in economic research) is rarely short-tailed and mostly converges to long-tail distribution because of the boom-bust market behavior. This paper proposes a novel clustering method named histogram damping partition (HDP) to define sub-clusters on the standard deviation intervals and truncate the histogram of the data by a constraint based on the coefficient of variation. The HDP approach can be used for many different kinds of fuzzy time series models at the clustering stage.     

基于主动学习的动态模糊聚类算法

聚类问题是近几年来机器学习和数据挖掘领域研究的热点问题,由于获取大量监督信息费时费力,目前国内外研究的重点是如何获得少量但对聚类性能提高显著的监督信息,再加上实际问题中存在的动态模糊性,故本文提出一种结合主动学习的动态模糊聚类算法DF-DBSCAN,通过引入动态模糊等价关系、动态模糊信任测度和动态模糊似然测度这3个约束信息来指导DBSCAN的聚类过程,以提高聚类的性能。实验结果表明,DF-DBSCAN算法不仅解决了实际问题中存在的动态模糊性数据的描述和表示问题,而且能够高效地进行数据聚类,显著地提高聚类性能。      

DENFIS: dynamic evolving neural-fuzzy inference system and itsapplication for time-series prediction

This paper introduces a new type of fuzzy inference systems, denoted as dynamic evolving neural-fuzzy inference system (DENFIS), for adaptive online and offline learning, and their application for dynamic time series prediction. DENFIS evolve through incremental, hybrid (supervised/unsupervised), learning, and accommodate new input data, including new features, new classes, etc., through local element tuning. New fuzzy rules are created and updated during the operation of the system. At each time moment, the output of DENFIS is calculated through a fuzzy inference system based on m-most activated fuzzy rules which are dynamically chosen from a fuzzy rule set. Two approaches are proposed: (1) dynamic creation of a first-order Takagi-Sugeno-type fuzzy rule set for a DENFIS online model; and (2) creation of a first-order Takagi-Sugeno-type fuzzy rule set, or an expanded high-order one, for a DENFIS offline model. A set of fuzzy rules can be inserted into DENFIS before or during its learning process. Fuzzy rules can also be extracted during or after the learning process. An evolving clustering method (ECM), which is employed in both online and offline DENFIS models, is also introduced. It is demonstrated that DENFIS can effectively learn complex temporal sequences in an adaptive way and outperform some well-known, existing models     

面向数据挖掘的时间序列聚类方法研究

一、引言自然界以及我们社会生活中的各种事物都在运动、变化和发展着,将它们按时间顺序记录下来,我们就可以得到各种各样的“时间序列”数据。对时间序列进行分析,可以揭示事物运动、变化和发展的内在规律,对于人们正确认识事物并据此作出科学的决策具有重要的现实意义。     

凝聚中心犹豫度恒定的模糊层次聚类算法

传统的模糊方法已无法解决数据本身不确定性的问题,犹豫模糊集方法却行之有效.原有的犹豫模糊层次聚类算法没有考虑犹豫模糊集对权值的影响,缺乏合理的权重计算方法,并且算法的时间复杂度和空间复杂度都为指数级.为了更有效地解决聚类分析问题,本文提出一种凝聚中心犹豫度恒定的模糊层次聚类算法(FHCA),首先设计了一种基于数据集本身...     

基于轨迹聚类的公共安全异常检测

公共安全异常检测的需求越来越迫切,监控中基于轨迹聚类的检测方法越来越流行,但是现有方法在处理高维不等长轨迹数据时效果并不理想。提出一个新的轨迹聚类方法,该方法通过组合动态时间弯曲和密度峰算法实现。动态时间弯曲用于度量轨迹间的距离,密度峰算法根据距离进行聚类。前者可直接度量不等长轨迹聚类,后者是近年提出的非球体分布数据聚类算法,以局部密度和最近邻聚类组合实现。实验在PETS2006监控视频数据集上进行,测试结果表明该方法有效地发现了异常的轨迹行为模式。     

Dynamic parameter optimization of evolutionary computation for on-line prediction of time series with changing dynamics

This paper introduces a dynamic evolving computation system (DECS) model, for adaptive on-line learning, and its application for dynamic time series prediction. DECS evolve through evolving clustering method and evolutionary computation for structure learning, Levenberg–Marquardt method for parameter learning, learning and accommodate new input data. DECS is created and updated during the operation of the system. At each time moment the output of DECS is calculated through a knowledge rule inference system based on m-most activated fuzzy rules which are dynamically chosen from a fuzzy rule set. An approach is proposed for a dynamic creation of a first order Takagi–Sugeno type fuzzy rule set for the DECS model. The fuzzy rules can be inserted into DECS before, or during its learning process, and the rules can also be extracted from DECS during, or after its learning process. It is demonstrated that DECS can effectively learn complex temporal sequences in an adaptive way and outperform some existing models.     

结合模糊控制的时序数据预测循环神经网络

时序数据处理任务中,循环神经网络模型以及相关衍生模型有较好的性能,如长短期记忆模型(LSTM),门限循环单元(GRU)等.模型的记忆层能够保存每个时间步的信息,但是无法高效处理某些领域的时序数据中的非等时间间隔和不规律的数据波动,如金融数据.本文提出了一种基于模糊控制的新型门限循环单元(GRU-Fuzzy)来解决这些问题.本文在GRU的基础上对记忆层增加了一个子空间分解,由模糊控制模块和一个启发式的失效函数组成,根据数据波动和时间间隔决定记忆层保留的信息量,从而提升模型性能.实验表明,相比于其他的循环神经网络模型,在标普500和上证50中选出股票的股价预测任务中,本文提出的模型有较好的表现.     

Pair-copula based mixture models and their application in clustering

Finite mixtures are often used to perform model based clustering of multivariate data sets. In real life applications, such data may exhibit complex nonlinear form of dependence among the variables. Also, the individual variables (margins) may follow different families of distributions. Most of the existing mixture models are unable to accommodate these two aspects of the data. This paper presents a finite mixture model that involves a pair-copula based construction of a multivariate distribution. Such a model de-couples the margins and the dependence structures. Hence, the margins can be modeled using different families. Again, many possible dependence structures can also be studied using different copulas. The resulting mixture model (called DVMM) is then capable of capturing a broad family of distributions including non-Gaussian models. Here we study DVMM in the context of clustering of multivariate data. We design an expectation maximization procedure for estimating the mixture parameters. We perform extensive experiments on the basis of a number of well-known data sets. A detailed evaluation of the clustering quality obtained by DVMM in comparison to other mixture models is presented. The experimental results show that the performance of DVMM is quite satisfactory.     

Development of a systematic methodology of fuzzy logic modeling

This paper proposes a systematic methodology of fuzzy logic modeling for complex system modeling. It has a unified parameterized reasoning formulation, an improved fuzzy clustering algorithm, and an efficient strategy of selecting significant system inputs and their membership functions. The reasoning mechanism introduces 4 parameters whose variation provides a continuous range of inference operation. As a result, we are no longer restricted to standard extremes in any step of reasoning. The fuzzy model itself can then adjust the reasoning process by optimizing the inference parameters based on input-output data. The fuzzy rules are generated through fuzzy c-means (FCM) clustering. Major bottlenecks are addressed and analytical solutions are suggested. We also address the classification process to extend the derived fuzzy partition to the entire output space. In order to select suitable input variables among a finite number of candidates (unlike traditional approaches) we suggest a new strategy through which dominant input parameters are assigned in one step and no iteration process is required. Furthermore, a clustering technique called fuzzy fine clustering is introduced to assign the input membership functions. In order to evaluate the proposed methodology, two examples-a nonlinear function and a gas furnace dynamic procedure-are investigated in detail. The significant improvement of the model is concluded compared to other fuzzy modeling approaches     

基于密度聚类的出租车异常轨迹检测

出租车GPS装备的普及使用产生了大量轨迹数据。出租车异常轨迹的检测和分析，可为惩罚具有欺诈行为的出租车司机提供有益支撑。针对出租车稀疏轨迹，基于轨迹相对相似度检测异常轨迹，由于其具有不对称性，类似于DBSCAN的传统密度聚类方法无法适应此种情况，本文提出基于密度RDBSCAN算法用于出租车异常轨迹聚类检测。对于聚类得出的候选异常轨迹，结合轨迹密度异常值和轨迹长度异常值的概念，利用证据理论综合前述2个因素来判别轨迹的异常程度，进而得到异常程度最高的TOP-N异常轨迹。使用旧金山真实的出租车数据，通过提取相同起点和终点（Origin-Destination, OD）的轨迹集进行实验，实验结果表明本文提出的方法能够有效地检测到异常轨迹，并成功给出异常程度最高的TOP-N异常轨迹。     

一种基于动态聚类的模糊分类规则的生成方法

介绍了一种基于动态聚类的模糊分类规则的生成方法，这种方法能决定规则数目，隶属函数的位置及形状．首先，介绍了基于超圆雏体隶属函数的模糊分类规则的基本形式；然后，介绍动态聚类算法，该算法能将每一类训练模式动态的分为成簇，对于每簇，则建立一个模糊规则；通过调整隶属函数的斜度，来提高对训练模式分类识别率，达到对模糊分类规则进行优化调整的目的；用两个典型的数据集评测了这篇文章研究的方法，这种方法构成的分类系统在识别率与多层神经网络分类器相当，但训练时间远少于多层神经网络分类器的训练时间．     

基于AR模型的动态模糊聚类算法

与传统的硬划分聚类相比,模糊聚类算法(以FCM为例)对数据的比例变化具有鲁棒性,能够更准确地反映数据点与类中心的实际关系,目前已得到广泛应用.然而对于时序基因表达数据来说,传统的聚类算法往往不能充分利用到数据中时间上的动态关联信息.因此可以在模糊聚类算法的基础上引入自回归(AR)模型,将时序基因表达数据作为一组时间序列进行动态的聚类分析.这样不仅可以充分利用到时序基因表达数据的内部自相关性,并且可以进一步利用隶属度函数对AR模型的预测过程进行模糊化调整,从而得到更为理想的聚类结果.     

基于动态时间弯曲的股票时间序列联动性研究

对于股票联动性的研究,传统时间序列分析方法及目前数据挖 掘技术主要使用国内或者国外股票指数来研究市场、板块或行业之间的联动关系,并得到一 些较为宏观的结论,存在着缺少直接分析与挖掘个股数据之间的联动性的问题。鉴于此,本文提出一种基于动态时间弯曲的股票时间序列联动性研究方法。通过动态时间弯曲找出若干只形态相似的股票,并在此基础上获得相关的重要信息,再提出基于动态时间弯曲的k-means聚类方法实现股票聚类,进而得到具有相同波动趋势的股票簇。实验结果表 明,新方法能从大量股票中准确找到具有联动关系的个股,区分开不同波动趋势的股票簇,具有一定的优越性。     

A data clustering algorithm for stratified data partitioning in artificial neural network

The statistical properties of training, validation and test data play an important role in assuring optimal performance in artificial neural networks (ANNs). Researchers have proposed optimized data partitioning (ODP) and stratified data partitioning (SDP) methods to partition of input data into training, validation and test datasets. ODP methods based on genetic algorithm (GA) are computationally expensive as the random search space can be in the power of twenty or more for an average sized dataset. For SDP methods, clustering algorithms such as self organizing map (SOM) and fuzzy clustering (FC) are used to form strata. It is assumed that data points in any individual stratum are in close statistical agreement. Reported clustering algorithms are designed to form natural clusters. In the case of large multivariate datasets, some of these natural clusters can be big enough such that the furthest data vectors are statistically far away from the mean. Further, these algorithms are computationally expensive as well. We propose a custom design clustering algorithm (CDCA) to overcome these shortcomings. Comparisons are made using three benchmark case studies, one each from classification, function approximation and prediction domains. The proposed CDCA data partitioning method is evaluated in comparison with SOM, FC and GA based data partitioning methods. It is found that the CDCA data partitioning method not only perform well but also reduces the average CPU time.     

Recognition of human grasps by time-clustering and fuzzy modeling

In this paper, we address the problem of recognition of human grasps for five-fingered robotic hands and industrial robots in the context of programming-by-demonstration. The robot is instructed by a human operator wearing a data glove capturing the hand poses. For a number of human grasps, the corresponding fingertip trajectories are modeled in time and space by fuzzy clustering and Takagi–Sugeno (TS) modeling. This so-called time-clustering leads to grasp models using time as an input parameter and fingertip positions as outputs. For a sequence of grasps, the control system of the robot hand identifies the grasp segments, classifies the grasps and generates the sequence of grasps shown before. For this purpose, each grasp is correlated with a training sequence. By means of a hybrid fuzzy model, the demonstrated grasp sequence can be reconstructed.