基于最小二乘的Q（λ）强化学习算法

通过分析经典的Q（λ）学习算法所存在的经验利用率低、收敛速度慢的问题,根据当前和多步的经验知识样本建立了状态-动作对值函数的最小二乘逼近模型,推导了该逼近函数在一组基底上的权向量所满足的一组线性方程,从而提出了快速而实用的最小二乘Q（λ）算法及改进的递推算法。倒立摆实验表明,该算法可以提高经验利用率,有效加快收敛速度。     

加权粗糙集的矩阵表示

在加权近似空间中提出了关系矩阵和布尔列向量加权α下乘法和加权α上乘法的概念。证明了加权α下乘法就是加权下近似,加权α上乘法就是加权上近似,从而为加权粗糙集（可变精度粗糙集、经典粗糙集）的计算和研究提供了一种新的矩阵方法。最后给出了计算加权上下近似的算法。     

半监督FCM聚类算法目标函数研究

分析了现有半监督FCM算法目标函数的物理意义和平衡系数α的选取,说明Stutz对Pedrycz目标函数的修改使半监督的物理意义更清楚,它在α=1,0时均退化为标准FCM算法,给出了修改后SS-FCM算法的交替求解过程。实验结果：（1）修改算法与Pedrycz算法有相同的半监督作用和清楚的物理解释;（2）对labeled样本采用FCM算法赋值比用随机数的收敛稳定性高;（3）优选的少量labeled样本,使用模糊协方差的SS-CFCM算法提高了聚类准确性和收敛速度。     

基于蕴涵算子族Lλ0λG的反向α-三I支持算法

提出了基于蕴涵算子族Lλ0λG的模糊推理的思想,这将有助于提高推理结果的可靠性。针对蕴涵算子族Lλ0λG给出了模糊推理的FMP模型及FMT模型的反向α-三I支持算法。     

基于蕴涵算子族Lλ0λG的反向α-三I约束算法

提出了基于蕴涵算子族Lλ0λG的模糊推理的思想,这将有助于提高推理结果的可靠性。针对蕴涵算子族Lλ0λG给出了模糊推理的FMP模型及FMT模型的反向α-三I约束算法。     

模糊λ商空间研究

把模糊商空间理论推广为模糊λ商空间理论,证明论域上的一个模糊等价关系一一对应于λ商空间上的一个归一化等腰距离函数;给定一个λ截关系序列,则唯一确定一个模糊等价关系。给出模糊λ商空间的定义,并讨论其性质,引入模糊λ商空间族来求解模糊问题。     

挖掘数据流近似频繁项的改进算法

数据流的无限性、连续性和速度快等特点,使得挖掘出所有准确的数据流频繁项通常是不可能的.算法的空间复杂度和时间复杂度通常是评价频繁项挖掘算法优劣的两个主要度量.通过引入局部性原理改进数据流近似频繁项的挖掘算法,该算法的空间复杂性为O（1/ε）,数据流每个数据项的最坏处理时间是O（1/ε）,其最好处理时间是O（1）,输出结果的频率值误差为∑_(i=2)^j（1-μ_i）×k_i。     

基于seeds集和频繁项集挖掘的半监督聚类算法

半监督聚类在无监督学习中通过对少量监督信息的有效利用提高聚类性能。提出一种基于seeds集的半监督聚类算法,它采用Apiori算法对初始seeds集和扩大规模后seeds集的数据进行频繁项集挖掘,使得数据中存在的噪音数据和误标记数据得到净化、修正,以改善seeds集质量,提高聚类性能。该算法使用带权χ²测试这一数学模型作为分类规则度量指标,以对无标记数据进行类标签值预测。实验结果显示,所提出的结合了频繁项集挖掘和带权χ²测试的基于seeds集的半监督聚类算法不仅改善了seeds集质量,也提高了预测结果的精确度,优化了聚类性能。     

未建模误差补偿Elman网络直接广义预测控制

针对一类非线性系统,提出了Elman网络直接广义预测控制算法。先将非线性系统等价转换成线性系统,然后对未建模动态误差进行估计,最后利用Elman网络进行预测控制器设计,并根据跟踪误差对控制器参数中的未知向量进行自适应调整,理论证明了误差收敛到原点一个小邻域内。该方法不需要求解Diophantine方程和矩阵求逆,只需要辨识一个参数θ（k）,因此减少了在线计算量,提高了实时性。仿真结果验证了该方法的有效性。     

不同蕴涵算子下三I方法的解

以R_M、R_Z、R₀三个蕴涵算子为基础,研究了11种形式的三I算法的解,并在此基础上给出了这11种解的同一形式：B*（y）=SUP{A*（x）∧φ（x,y）},其中φ：X×Y→[0,1]表示某一函数,而且φ（x,y）与E_Y的选取依赖蕴涵算子R_i的选取。     

Recursive least-squares temporal difference learning for adaptive traffic signal control at intersection

This paper presents a new method to solve the scheduling problem of adaptive traffic signal control at intersection. The method involves recursive least-squares temporal difference (RLS-TD(λ)) learning that is integrated into approximate dynamic programming. The learning mechanism of RLS-TD(λ) is to make an adaptation of linear function approximation by updating its parameters based on environmental feedback. This study investigates the method implementation after modeling a traffic dynamic system at intersection in discrete time. In the model, different traffic control schemes regarding signal phase sequence are considered, especially the defined adaptive phase sequence (APS). By simulating traffic scenarios, RLS-TD(λ) is superior to TD(λ) for updating functional parameters in the approximation, and APS outperforms other conventional control schemes on reducing traffic delay. By comparing with other traffic signal control algorithms, the proposed algorithm yields satisfying results in terms of traffic delay and computation time.

     

正则系统在Lebesgue-p范数意义下的快速迭代学习控制

针对一类线性正则系统,传统迭代学习控制算法收敛速度较低的问题,设计了一种快速迭代学习控制算法。该算法在传统P型迭代学习控制算法基础上,增加了由相邻两次迭代时跟踪误差构成的上一次差分信号和当前差分信号,并在Lebesgue-p范数度量意义下,利用卷积推广的Young不等式严格证明了,当迭代次数趋于无穷大时,系统的跟踪误差收敛于零,并给出了算法的收敛条件。该算法与传统P型迭代学习控制算法相比,不仅提高了收敛速度,而且还避免了采用λ范数度量跟踪误差的缺陷,最后通过仿真结果进一步验证了所提算法的有效性。     

基于二阶时序差分误差的双网络DQN算法

针对深度Q网络(DQN)算法因过估计导致收敛稳定性差的问题,在传统时序差分(TD)的基础上提出N阶TD误差的概念,设计基于二阶TD误差的双网络DQN算法。构造基于二阶TD误差的值函数更新公式,同时结合DQN算法建立双网络模型,得到两个同构的值函数网络分别用于表示先后两轮的值函数,协同更新网络参数,以提高DQN算法中值函数估计的稳定性。基于Open AI Gym平台的实验结果表明,在解决Mountain Car和Cart Pole问题方面,该算法较经典DQN算法具有更好的收敛稳定性。     

一种基于递归最小二乘法的强化学习算法及其应用研究

文章推导了递归最小二乘瞬时差分法,较通常的瞬时差分法有样本使用效率高,收敛速度快,计算量少等特点。并将基于递归最小二乘的强化学习应用于船舶航向控制,克服了通常智能算法的学习需要一定数量样本数据的缺陷,对控制器的参数进行在线学习与调整,可以在一定程度上解决船舶运动中的不确定性问题,仿真结果表明,在有各种分浪流干扰的条件下,船舶航向的控制仍能取得令人满意的效果,说明该算法是有效可行的。     

TD(λ) learning without eligibility traces: a theoretical analysis

A common approach to learning from delayed rewards is to use temporal difference (TD) methods for predicting future reinforcement values. They are parameterized by a recency factor λ which determines whether and how the outcomes from several consecutive time steps contribute to a single prediction update. TD(λ > 0) has been found to usually yield noticeably faster learning than TD(0), but its standard eligibility traces implementation is associated with some well known deficiencies, in particular significantly increased computation expense. This article investigates theoretically two possible ways of implementing TD(λ) without eligibility traces, both proposed by prior work. One is the TTD procedure, which efficiently approximates the effects of eligibility traces by the use of truncated TD(λ) returns. The other is experience replay, which relies on replaying TD prediction updates backwards in time. We provide novel theoretical results related to the former and present an original analysis of the effects of two variations of the latter. The ultimate effect of these investigations is a unified view of the apparently different computational techniques. This contributes to the TD(λ) research in general, by highlighting interesting relationships between several TD-based algorithms and facilitating their further analysis.     

一种面向自动驾驶推理任务的工作流调度策略

目前自动驾驶推理任务调度中要解决的关键问题是如何在不同的时间窗内,让实时推理任务满足可容忍时间约束的前提下,在相应的处理设备上被调度执行完成.在不同时间窗内,依据边缘节点的数量变化以及推理任务的不同,设计了一种边缘环境下基于强化学习算法的工作流调度策略.首先,利用推理任务工作流调度算法计算任务的完成时间;其次,采用基于模拟退火的Q学习算法(Q-learning based on simulated annealing,SA-QL)来优化推理任务的完成时间;最后,从可行性、收敛性、有效性和探索性四个角度来体现基于模拟退火的强化学习算法(Reinforement learning based on simulated annealing,SA-RL)和粒子群优化算法(Particle Swarm Optimization,PSO)的性能差异.实验结果表明,模拟退火的强化学习算法和粒子群优化算法都具有可行性和有效性,单步时序差分算法(TD(0))具有更强的探索性,多步时序差分算法(TD(λ))具有更强的收敛性.     

Optimal short-term hydro-thermal scheduling using quasi-oppositional teaching learning based optimization

This paper presents a new approach for solving short-term hydrothermal scheduling (HTS) using an integrated algorithm based on teaching learning based optimization (TLBO) and oppositional based learning (OBL). The practical hydrothermal system is highly complex and possesses nonlinear relationship of the problem variables, cascading nature of hydro reservoirs, water transport delay and scheduling time linkage that make the problem of optimization difficult using standard optimization methods. To overcome these problems, the proposed quasi-oppositional teaching learning based optimization (QOTLBO) is employed. To show its efficiency and robustness, the proposed QOTLBO algorithm is applied on two test systems. Numerical results of QOTLBO are compared with those obtained by two phase neural network, augmented Lagrange method, particle swarm optimization (PSO), improved self-adaptive PSO (ISAPSO), improved PSO (IPSO), differential evolution (DE), modified DE (MDE), fuzzy based evolutionary programming (Fuzzy EP), clonal selection algorithm (CSA) and TLBO approaches. The simulation results reveal that the proposed algorithm appears to be the best in terms of convergence speed, solution time and minimum cost when compared with other established methods. This method is considered to be a promising alternative approach for solving the short-term HTS problems in practical power system.     

An accelerated learning algorithm for multilayer perceptronnetworks

An accelerated learning algorithm (ABP-adaptive back propagation) is proposed for the supervised training of multilayer perceptron networks. The learning algorithm is inspired from the principle of "forced dynamics" for the total error functional. The algorithm updates the weights in the direction of steepest descent, but with a learning rate a specific function of the error and of the error gradient norm. This specific form of this function is chosen such as to accelerate convergence. Furthermore, ABP introduces no additional "tuning" parameters found in variants of the backpropagation algorithm. Simulation results indicate a superior convergence speed for analog problems only, as compared to other competing methods, as well as reduced sensitivity to algorithm step size parameter variations.     

蚁群算法的收敛速度分析

蚁群算法(ACO)作为一类新型的机器学习技术,已经广泛用于组合优化问题的求解,同时也应用于工业工程的优化设计.相对于遗传算法(GA),蚁群算法的理论研究在国内外均起步较晚,特别是收敛速度的分析理论是该领域急待解决的第一大公开问题.文中的研究内容主要是针对这一公开问题而开展的.根据蚁群算法的特性,该研究基于吸收态Markov过程的数学模型,提出了蚁群算法的收敛速度分析理论.作者给出了估算蚁群算法期望收敛时间的几个理论方法,以分析蚁群算法的收敛速度,并结合著名的ACS算法作了具体的案例研究.基于该文提出的收敛速度分析理论,作者还提出ACO-难和ACO-易两类问题的界定方法;最后,利用ACS算法求解TSP问题的实验数据,验证了文中提出的分析结论,得出了初步的算法设计指导原则.     

基于改进仿射投影算法的数字助听器自适应回声消除

提出了一种改进的仿射投影算法。该算法建立了步长因子与误差能量之间一种新的非线性函数关系,根据误差能量的变化自动调整步长因子,以达到加快滤波器收敛速度、降低稳态失调的目的;在对误差能量的估计中提出了遗忘因子选择规则,提高了误差能量估计的准确性。对提出的算法进行的数学分析,为其快速收 敛性提供了理论依据。实验仿真表明,与传统的自适应算法以及固定步长的仿射投影算法相比,提出的改进仿射算法在收敛速度、稳态失调等方面有明显改善。