Stochastic Iterative Learning Control With Faded Signals

Stochastic iterative learning control (ILC) is designed for solving the tracking problem of stochastic linear systems through fading channels. Consequently, the signals used in learning control algorithms are faded in the sense that a random variable is multiplied by the original signal. To achieve the tracking objective, a two-dimensional Kalman filtering method is used in this study to derive a learning gain matrix varying along both time and iteration axes. The learning gain matrix minimizes the trace of input error covariance. The asymptotic convergence of the generated input sequence to the desired input value is strictly proved in the mean-square sense. Both output and input fading are accounted for separately in turn, followed by a general formulation that both input and output fading coexists. Illustrative examples are provided to verify the effectiveness of the proposed schemes.      

Adaptive multiple minor directions extraction in parallel using a PCA neural network

A principal component analysis (PCA) neural network is developed for online extraction of the multiple minor directions of an input signal. The neural network can extract the multiple minor directions in parallel by computing the principal directions of the transformed input signal so that the stability-speed problem of directly computing the minor directions can be avoided to a certain extent. On the other hand, the learning algorithms for updating the net weights use constant learning rates. This overcomes the shortcoming of the learning rates approaching zero. In addition, the proposed algorithms are globally convergent so that it is very simple to choose the initial values of the learning parameters. This paper presents the convergence analysis of the proposed algorithms by studying the corresponding deterministic discrete time (DDT) equations. Rigorous mathematical proof is given to prove the global convergence. The theoretical results are further confirmed via simulations.     

On reference governor in iterative learning control for dynamic systems with input saturation

Input saturation is inevitable in many engineering applications. Most existing iterative learning control (ILC) algorithms that can deal with input saturation require that the reference signal is realizable within the saturation bound. For engineering systems without precise models, it is hard to verify this requirement. In this note, a “reference governor” (RG) is introduced and is incorporated with the available ILC algorithms (primary ILC algorithms). The role of the RG is to re-design the reference signal so that the modified reference signal is realizable. Two types of the RG are proposed: one modifies the amplitude of the reference signal and the other modifies the frequency. Our main results provide design guidelines for two RGs. Moreover, a design trade-off between the convergence speed and tracking performance is also discussed. A simple simulation result verifies the effectiveness of the proposed methods.     

Neural networks that learn from fuzzy if-then rules

An architecture for neural networks that can handle fuzzy input vectors is proposed, and learning algorithms that utilize fuzzy if-then rules as well as numerical data in neural network learning for classification problems and for fuzzy control problems are derived. The learning algorithms can be viewed as an extension of the backpropagation algorithm to the case of fuzzy input vectors and fuzzy target outputs. Using the proposed methods, linguistic knowledge from human experts represented by fuzzy if-then rules and numerical data from measuring instruments can be integrated into a single information processing system (classification system or fuzzy control system). It is shown that the scheme works well for simple examples     

即时学习算法在非线性系统迭代学习控制中的应用

运用即时学习算法来解决一类非线性系统的迭代学习控制初值问题。对于任何类型的迭代学习控制算法，即时学习算法都能有效地估计初始控制量，减小了初始输出误差，加快了算法的收敛速度，使得经过有限次迭代后系统输出能严格跟踪理想信号。对机器人系统的仿真结果表明了该方法的有效性。     

Human–machine interface in bioprosthesis control using EMG signal classification

Abstract: We present a concept of human–machine interface intended for the task of bioprosthesis decision control by means of sequential recognition of the patient's intent based on the electromyography (EMG) signal acquired from his/her body. The EMG signal characteristics, the problem of processing the signals including acquisition and feature extraction and their classification are discussed. The contextual (sequential) recognition via fuzzy relations for the classification of the patient's intent is considered and the implied decision algorithms are presented. In the proposed method, the fuzzy relation is determined on the basis of the learning set as a solution of an appropriate optimization problem and then this relation is used in the form of a matrix of membership degrees at successive instants of the sequential decision process. Three algorithms of sequential classification which differ from one another in the sets of input data and procedure are described. The proposed algorithms were experimentally tested in the recognition of phases of the grasping process of the hand on the basis of the EMG signal, where the real-coded genetic algorithm was used as an optimization procedure. The concept of the measurement stand which was the source of information exploited in the experimental investigations of the algorithms is also described.     

基于强化学习的交通情景问题决策优化

在复杂交通情景中求解出租车路径规划决策问题和交通信号灯控制问题时,传统强化学习算法在收敛速度和求解精度上存在局限性;因此提出一种改进的强化学习算法求解该类问题。首先,通过优化的贝尔曼公式和快速Q学习（SQL）机制,以及引入经验池技术和直接策略,提出一种改进的强化学习算法GSQL-DSEP;然后,利用GSQL-DSEP算法分别优化出租车路径规划决策问题中的路径长度与交通信号灯控制问题中的车辆总等待时间。相较于Q学习、快速Q学习（SQL）、、广义快速Q学习（GSQL）、Dyna-Q算法,GSQL-DSEP算法在性能测试中降低了至少18.7%的误差,在出租车路径规划决策问题中使决策路径长度至少缩短了17.4%,在交通信号灯控制问题中使车辆总等待时间最多减少了51.5%。实验结果表明,相较于对比算法,GSQL-DSEP算法对解决交通情景问题更具优势。     

Early event detection in a deep-learning driven quality prediction model for ultrasonic welding

A goal in ultrasonic welding (USW) process monitoring is to accurately predict quality outcomes based on monitored signals. However, in most cases, knowing only that the USW process has failed is insufficient. Modern process automation should assess signal information and intercede to rectify process problems. Identification of when a process signal deviates from an acceptable final quality outcome, i.e., the time at which an abnormal event starts, facilitates control action or root cause analysis to bring it back to compliance. A long short-term memory (LSTM) recurrent neural network is proposed to monitor USW and other time-series signals and identify this point. This deep neural network is trained to classify quality outcomes from continuous signals. The process monitoring signals and their sampling time are divided into finite segments as input to this network. The time segment at which the process signal first converges to the final quality class prediction is identified using cross-entropy of the classification probabilities. This procedure is demonstrated using USW quality monitoring algorithms and robot motion failure detection. The examples show an LSTM network not only provides high accuracy for USW quality prediction, but also that the time of classification convergence is consistent with variance observed in USW weld quality factors. Moreover, classification convergence time was shown to be associated to specific robot motion failures, useful as input to adaptive learning. This work realizes deep-learning driven quality prediction and early event detection for quality classification problems, and provides the information necessary for adaptive control algorithms.     

基于非线性系统相对度的学习控制算法及在非完整移动机器人中的应用

首先提出了一种基于非线性系统对度的迭代学习控制算法,并证明了其收敛性,该算法通过对系统以前的输入和输出跟踪误差信号进行学习来反复调整输入量,使得系统在经过一定次数的学习以后,其实际输出趋于期望输出且其内部状态也具有良好的收敛特性,其次将此算法应用于两轮驱动的移动机器人动力学系统,数值仿真结果表明了这种算法的有效性。     

Dynamics of Generalized PCA and MCA Learning Algorithms

Principal component analysis (PCA) and minor component analysis (MCA) are two important statistical tools which have many applications in the fields of signal processing and data analysis. PCA and MCA neural networks (NNs) can be used to online extract principal component and minor component from input data. It is interesting to develop generalized learning algorithms of PCA and MCA NNs. Some novel generalized PCA and MCA learning algorithms are proposed in this paper. Convergence of PCA and MCA learning algorithms is an essential issue in practical applications. Traditionally, the convergence is studied via deterministic continuous-time (DCT) method. The DCT method requires the learning rate of the algorithms to approach to zero, which is not realistic in many practical applications. In this paper, deterministic discrete-time (DDT) method is used to study the dynamical behaviors of the proposed algorithms. The DDT method is more reasonable for the convergence analysis since it does not require constraints as that of the DCT method. It is proven that under some mild conditions, the weight vector in these proposed algorithms will converge exponentially to principal or minor component. Simulation results are further used to illustrate the theoretical results.     

随机变批次长度的反馈辅助PD型量化迭代学习控制

针对离散线性系统,研究批次长度随机变化的反馈辅助PD型量化迭代学习控制问题.考虑系统信号经量化后传输到控制器或执行器的情况,给出两种量化方案:跟踪误差信号量化和控制输入信号量化.基于两种不同的量化信号,在批次长度和初始条件随机变化前提下设计反馈辅助PD型迭代学习控制算法.采用扇形界的处理方法和堆积系统框架,推导数学期望下的学习收敛条件:在误差信号量化情况下,所提出控制算法可以保证跟踪误差渐近收敛到零;在控制输入信号量化情况下,所提出控制算法能够保证跟踪误差有界收敛.仿真示例对比验证了两种量化方案下所提出方法的有效性和优越性.     

An incremental approach to genetic-algorithms-based classification.

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multiagent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an "integration" operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed.     

Fuzzy algorithms for learning vector quantization

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.     

Iterative learning control for constrained linear systems

This article considers iterative learning control (ILC) for linear systems with convex control input constraints. First, the constrained ILC problem is formulated in a novel successive projection framework. Then, based on this projection method, two algorithms are proposed to solve this constrained ILC problem. The results show that, when perfect tracking is possible, both algorithms can achieve perfect tracking. The two algorithms differ, however, in that one algorithm needs much less computation than the other. When perfect tracking is not possible, both algorithms can exhibit a form of practical convergence to a ‘best approximation’. The effect of weighting matrices on the performance of the algorithms is also discussed and finally, numerical simulations are given to demonstrate the effectiveness of the proposed methods.     

Genetic algorithms in norm-optimal linear and non-linear iterative learning control

In this paper it is analysed whether or not it is possible to apply the norm-optimal iterative learning control algorithm to non-linear plant models. As a new theoretical result it is shown that if the non-linear plant meets a certain technical invertibility condition, the sequence of tracking errors generated by the norm-optimal algorithm will converge geometrically to zero. However, due to the non-linear nature of the plant, it is typically impossible to calculate analytically the sequence of input functions produced by the norm-optimal algorithm. Therefore it is proposed that genetic algorithms can be used as a computational tool to calculate the sequence of norm-optimal inputs. The proposed approach benefits from the design of a low-pass FIR filter. This filter successfully removes unwanted high frequency components of the input signal, which are generated by the genetic algorithm method due to the random nature of the genetic algorithm search. Simulations are used to illustrate the performance of this new approach, and they demonstrate good results in terms of convergence speed and tracking of the reference signal regardless of the nature of the plant.     

具有时变输入输出函数的反馈过程神经网络及应用

针对输入和输出均为时变函数或过程的实际系统建模和仿真问题，提出一种输入和输出均为时变函数的反馈过程神经网络模型，该模型的第1隐层对来自输入层的时变信号进行空间加权聚合和激励运算，并在将其输出传送至第2隐层的同时反馈至输入层；第2隐层完成对其时变输入的空间加权聚合、时间累积聚合和激励运算，并将其输出传送至输出层．给出了相应的学习算法，并以实例验证了该模型及其学习算法的有效性．     

Iterative learning control of multi-agent systems with random noises and measurement range limitations

In this paper, the iterative learning control is introduced to solve the consensus tracking problem of a multi-agent system with random noises and measurement range limitation. A distributed learning control algorithm is proposed for all agents by utilising its nearest neighbour measured information from previous iterations. With the help of the stochastic approximation technique, we first establish the consensus convergence of the input sequences in almost sure sense for fixed topology as the iteration number increases. Then, we extend the results to switching topologies case which is dynamically changing along the time axis. Illustrative simulations verify the effectiveness of the proposed algorithms.     

Optimality of first-order ILC among higher order ILC

Higher order iterative learning control (HO-ILC) algorithms use past system control information from more than one past iterative cycle. This class of ILC algorithms have been proposed aiming at improving the learning efficiency and performance. This paper addresses the optimality of HO-ILC in the sense of minimizing the trace of the control error covariance matrix in the presence of a class of uncorrelated random disturbances. It is shown that the optimal weighting matrices corresponding to the control information associated with more than one cycle preceding the current cycle are zero. That is, an optimal HO-ILC does not add to the optimality of standard first-order ILC in the sense of minimizing the trace of the control error covariance matrix. The system under consideration is a linear discrete-time varying systems with different relative degree between the input and each output.     

结合状态预测的深度强化学习交通信号控制

深度强化学习（deep reinforcement learning,DRL）可广泛应用于城市交通信号控制领域,但在现有研究中,绝大多数的DRL智能体仅使用当前的交通状态进行决策,在交通流变化较大的情况下控制效果有限。提出一种结合状态预测的DRL信号控制算法。首先,利用独热编码设计简洁且高效的交通状态;然后,使用长短期记忆网络（long short-term memory,LSTM）预测未来的交通状态;最后,智能体根据当前状态和预测状态进行最优决策。在SUMO（simulation of urban mobility）仿真平台上的实验结果表明,在单交叉口、多交叉口的多种交通流量条件下,与三种典型的信号控制算法相比,所提算法在平均等待时间、行驶时间、燃油消耗、CO2排放等指标上都具有最好的性能。     

Networked iterative learning control design for discrete-time systems with stochastic communication delay in input and output channels

This paper develops two kinds of derivative-type networked iterative learning control (NILC) schemes for repetitive discrete-time systems with stochastic communication delay occurred in input and output channels and modelled as 0-1 Bernoulli-type stochastic variable. In the two schemes, the delayed signal of the current control input is replaced by the synchronous input utilised at the previous iteration, whilst for the delayed signal of the system output the one scheme substitutes it by the synchronous predetermined desired trajectory and the other takes it by the synchronous output at the previous operation, respectively. In virtue of the mathematical expectation, the tracking performance is analysed which exhibits that for both the linear time-invariant and nonlinear affine systems the two kinds of NILCs are convergent under the assumptions that the probabilities of communication delays are adequately constrained and the product of the input–output coupling matrices is full-column rank. Last, two illustrative examples are presented to demonstrate the effectiveness and validity of the proposed NILC schemes.