Robust optimal tracking control for multiplayer systems by off‐policy Q‐learning approach

In this article, a novel off‐policy cooperative game Q‐learning algorithm is proposed for achieving optimal tracking control of linear discrete‐time multiplayer systems suffering from exogenous dynamic disturbance. The key strategy, for the first time, is to integrate reinforcement learning, cooperative games with output regulation under the discrete‐time sampling framework for achieving data‐driven optimal tracking control and disturbance rejection. Without the information of state and input matrices of multiplayer systems, as well as the dynamics of exogenous disturbance and command generator, the coordination equilibrium solution and the steady‐state control laws are learned using data by a novel off‐policy Q‐learning approach, such that multiplayer systems have the capability of tolerating disturbance and follow the reference signal via the optimal approach. Moreover, the rigorous theoretical proofs of unbiasedness of coordination equilibrium solution and convergence of the proposed algorithm are presented. Simulation results are given to show the efficacy of the developed approach.     

Sliding mode output‐feedback causal output tracking for a class of discrete‐time nonlinear systems

The output tracking (OT) of arbitrary references in discrete‐time (DT) nonlinear systems is addressed by designing an output‐feedback control. A set of difference‐algebraic equations is proposed as an exact solution of the problem. Using a novel technique of approximating DT functions, the system disturbance and steady states, characterized by tracking error identically zero, for both the system state and the control input, are represented as signals generated by a disturbed dynamic system. Using the mentioned dynamics, the control system is extended. Then, a state observer is proposed to estimate the resulting extended system state. Finally, a DT sliding mode controller is designed to achieve the approximate OT. Simulations show the effectiveness of the proposed control scheme.     

Dynamic average consensus with topology balancing under a directed graph

In this paper, the distributed average tracking problem is studied on the premise of a strongly connected directed graph. To this end, we propose a weight balance strategy that could potentially make the adjacency matrix doubly stochastic for any strongly connected directed graph. The proposed scheme is fully distributive with finite time convergence and we again prove that network connectivity (described by the first left eigenvector) is instrumental in networked control systems. Then, a discrete‐time average tracking observer is introduced to ensure that all networked systems can track the average of the reference signals with bounded error. Simulation results verify the effectiveness of the proposed methods.     

Novel MPC‐Based Fault Tolerant Tracking Control Against Sensor Faults

The problem of active fault‐tolerant tracking control with control input and system output constraints is studied for a class of discrete‐time systems subject to sensor faults. A time‐varying fault‐tolerant observer is first developed to estimate the real system state from the faulty sensor output and control input signals. Then by using the estimated state at each time step, a model predictive control (MPC)‐based fault‐tolerant tracking control scheme is presented to guarantee the desired tracking performance and the given input and output constraints on the faulty system. In comparison with many existing fault‐tolerant MPC methods, its main contribution is that the proposed state estimator is designed by the simple and online numerical computation to tolerate the possible sensor faults, so that the regular MPC algorithm without fault information can be adopted for the online calculation of fault‐tolerant control signal. The potential recursive infeasibility and computational complexity due to the faults are avoided in the scheme. Additionally, the closed‐loop stability of the post‐fault system is discussed. Simulative results of an electric throttle control system verify the effectiveness of the proposed method.     

SLIDING MODE PREDICTION TRACKING CONTROL DESIGN FOR UNCERTAIN SYSTEMS

In this paper, a tracking control algorithm based on sliding mode prediction for a class of discrete‐time uncertain systems is presented. By creating a special model to predict the future sliding mode function value and by combining feedback correction and receding horizon optimization approaches, which are extensively applied in predictive control strategy, a discrete‐time sliding mode control law for tracking problem is constructed. With the designed control law, closed‐loop systems have strong robustness to matched or unmatched uncertainties as they eliminate chattering. Besides, in the robustness analysis, the boundary condition for uncertainties, which is a universal presupposition in sliding mode control method, is not required. Numerical simulation and cart‐pendulum experiment results illustrate the validity of the proposed algorithm.     

A robust discrete‐time adaptive control approach for systems with almost periodic time‐varying parameters

A periodic adaptive control approach is proposed for a class of nonlinear discrete‐time systems with time‐varying parametric uncertainties which are almost periodic, and the only prior knowledge is the periodicity. The new adaptive controller updates the parameters and the control signal periodically in a pointwise manner over one entire period, in the sequel that achieves a bounded tracking convergence. The result is further extended to scenarios with unknown input gain, higher order dynamics, and tracking. Copyright © 2012 John Wiley & Sons, Ltd.     

H∞ output tracking control for uncertain networked control systems via a switched system approach

In this paper, the problem of H_∞ output tracking control for networked control systems with random time delays and system uncertainties is investigated. Effective sampling instant that is tightly related with transmission delay from sensor to actuator is proposed to ensure that the random variable time delay is always shorter than one effective sampling period. By using both active time‐varying sampling period strategy and hybrid node‐driven mechanism, the switching instant is coincided with the effective sampling instant. An augmented time‐varying networked tracking system model is provided by including the output tracking error as an additional state. However, random transmission delay causes indeterminate sampling period, which induces infinite subsystems. Gridding approach is introduced to transform the continuous time axis into discrete‐time sequences, which guarantees the finite number of switching rules. By employing multiple Lyapunov–Krasovskii functions, linear matrix inequality (LMI)‐based output tracking H_∞ performance analysis is presented, and robust switching H_∞ model reference tracking controller for networked control systems with communication constraints and system uncertainties is designed to guarantee asymptotic tracking of prescribed reference outputs while rejecting disturbances. Finally, simulation results illustrate the correctness and effectiveness of the proposed approaches. Copyright © 2015 John Wiley & Sons, Ltd.     

Neural‐network‐based predictive controller design: An application to temperature control of a plastic injection molding process

This paper presents a neural‐network‐based predictive control (NPC) method for a class of discrete‐time multi‐input multi‐output (MIMO) systems. A discrete‐time mathematical model using a recurrent neural network (RNN) is constructed and a learning algorithm adopting an adaptive learning rate (ALR) approach is employed to identify the unknown parameters in the recurrent neural network model (RNNM). The NPC controller is derived based on a modified predictive performance criterion, and its convergence is guaranteed by adopting an optimal algorithm with an adaptive optimal rate (AOR) approach. The stability analysis of the overall MIMO control system is well proven by the Lyapunov stability theory. A real‐time control algorithm is proposed which has been implemented using a digital signal processor, TMS320C31 from Texas Instruments. Two examples, including the control of a MIMO nonlinear system and the control of a plastic injection molding process, are used to demonstrate the effectiveness of the proposed strategy. Results from both numerical simulations and experiments show that the proposed method is capable of controlling MIMO systems with satisfactory tracking performance under setpoint and load changes. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Tracking control for sampled‐data systems with uncertain time‐varying sampling intervals and delays

In this paper, a solution to the approximate tracking problem of sampled‐data systems with uncertain, time‐varying sampling intervals and delays is presented. Such time‐varying sampling intervals and delays can typically occur in the field of networked control systems. The uncertain, time‐varying sampling and network delays cause inexact feedforward, which induces a perturbation on the tracking error dynamics, for which a model is presented in this paper. Sufficient conditions for the input‐to‐state stability (ISS) of the tracking error dynamics with respect to this perturbation are given. Hereto, two analysis approaches are developed: a discrete‐time approach and an approach in terms of delay impulsive differential equations. These ISS results provide bounds on the steady‐state tracking error as a function of the plant properties, the control design and the network properties. Moreover, it is shown that feedforward preview can significantly improve the tracking performance and an online extremum seeking (nonlinear programming) algorithm is proposed to online estimate the optimal preview time. The results are illustrated on a mechanical motion control example showing the effectiveness of the proposed strategy and providing insight into the differences and commonalities between the two analysis approaches. Copyright © 2009 John Wiley & Sons, Ltd.     

Robust adaptive output feedback control of a class of discrete‐time nonlinear systems with nonlinear uncertainties and unknown control directions

In this paper, robust adaptive output feedback control is studied for a class of discrete‐time nonlinear systems with functional nonlinear uncertainties of the Lipschitz type and unknown control directions. In order to construct an output feedback control, the system is transformed into the form of a nonlinear autoregressive moving average with eXogenous inputs (NARMAX) model. In order to avoid the noncausal problem in the control design, future output prediction laws and parameter update laws with the dead‐zone technique are constructed on the basis of the NARMAX model. With the employment of the predicted future outputs, a constructive output feedback adaptive control is proposed, where the discrete Nussbaum gain technique and the dead‐zone technique are used in parameter update laws. The effect of the functional nonlinear uncertainties is compensated for, such that an asymptotic tracking performance is achieved, whereas other signals in the closed‐loop systems are guaranteed to be bounded. Simulation studies are performed to demonstrate the effectiveness of the proposed approach. Copyright © 2012 John Wiley & Sons, Ltd.     

Quantized tracking control for a multi‐agent system with high‐order leader dynamics

This paper presents a study of a tracking control problem for a multi‐agent system with an active leader and quantized communication constraints. We first design a discrete‐time distributed estimator‐based tracking control for each follower‐agent and analyze the tracking convergence with the help of the Riccati equation and common Lyapunov function when the communication channel is perfect and the interconnection topology is time‐varying. Then a stochastic quantization strategy is applied to model the information communication in the agent coordination and the corresponding solution to the tracking control problem is also given. Finally, a numerical example is given to illustrate the tracking control algorithm. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

A STABLE ON‐LINE SELF‐TUNING OPTIMAL PID CONTROLLER FOR A CLASS OF UNKNOWN SYSTEMS

In this paper, a novel self‐tuning method of optimal PID control laws is proposed for both continuous‐time systems and discrete‐time systems. The controlled plant is assumed to be unknown except the system order (or system delay) and the direction of transmitting control input. Through the minimization of PID gains subject to the Lyapunov stability based reaching condition, the tuning of the three PID control gains is transformed to solve the inequality constraint optimization problem. An unknown SISO nonlinear system subject to a unit step input, and the tracking control problem of the piezoelectric actuator (PZA) with unknown dynamics are simulated. The simulation results show that the excellent tracking performance can be achieved.     

Supervised Multi‐Rejector of Periodic Disturbances for Nonlinear Systems Using Decoupled State Multimodel

In this paper, a multi‐rejector of periodic disturbances is proposed for discrete‐time nonlinear systems represented by a decoupled state multimodel. We report a decoupled state multimodel repetitive‐predictive control based on a supervised algorithm to ensure reference trajectory tracking and periodic disturbances rejection. Partial predictors associated to the local controllers make the best choice of the most valid partial controller that meets the desired closed loop performances. The effectiveness of the supervised multi‐rejector is shown via a simulation example. The obtained results are satisfactory and show a good rejection of periodic disturbances and reference trajectory tracking.     

Error‐constrained finite‐horizon tracking control with incomplete measurements and bounded noises

This paper is concerned with the finite‐horizon tracking control problem for discrete nonlinear time‐varying systems with state delays, bounded noises and incomplete measurement output. The exogenous bounded noises are unknown and confined to specified ellipsoidal sets. A deterministic measurement output model is proposed to account for the incomplete data transmission phenomenon caused by possible sensor aging or failures. The aim of the addressed tracking control problem is to develop an observer‐based control over a finite‐horizon such that, for the admissible time delays, nonlinearities and bounded noises, both the quadratic tracking error and the estimation error are not more than certain upper bounds that are minimized at every time step. A recursive linear matrix inequality approach is used to solve the problem addressed. The observer and controller parameters are characterized in terms of the solution to a convex optimization problem that can be easily solved by using the semi‐definite programme method. A simulation example is exploited to illustrate the effectiveness of the proposed design procedures. Copyright © 2011 John Wiley & Sons, Ltd.     

Stable Inversion Based Precise Tracking for Periodic Systems

Stable inversion based precise tracking for discrete‐time periodically time‐varying square systems is studied. By means of the lifting technique, the time‐varying system is reformulated equivalently to the time‐invariant lifted system that is analyzed for different cases such as the non‐singular case and the singular case. Combining the stable inversion method of these cases with the optimal state transition method for time‐varying systems, precise tracking is achieved from a limited initial time. The tracking performance of the proposed method is validated through simulations.     

Adaptive Active Fault Tolerant Control for Discrete‐Time Systems with Uncertainties

An active fault‐tolerant control scheme for discrete‐time systems is proposed to solve a difficult problem of fault‐tolerant controller design in the presence of partial loss of actuator effectiveness faults and structural parameter uncertainties assumed to be matched, using adaptive control techniques to help a faster and more accurate compensation of failure and uncertainty. An automated fault estimation scheme is developed together with an adaptive model parameter identification to obtain system parameter estimates. With these estimates fed back to the system, a model reference adaptive controller is constructed to achieve a desired tracking performance. Since parameters are obtained and updated online, the control system has an automatic failure compensation capability so as to recognize or reconfigure the control law in real time in response to failure indications. The stability and convergence follow from discrete‐time Lyapunov arguments. Simulation results from the linearized lateral dynamics model of the Boeing 747 airplane are presented to show the efficiency of proposed methods.     

Data‐driven high‐order terminal iterative learning control with a faster convergence speed

In this paper, a novel high‐order optimal terminal iterative learning control (high‐order OTILC) is proposed via a data‐driven approach for nonlinear discrete‐time systems with unknown orders in the input and output. The objective is to track the desired values at the endpoint of the operation cycle. The terminal tracking errors over more than one previous iterations are used to enhance the high‐order OTILC's performance with faster convergence. From rigor of the analysis, the monotonic convergence of the terminal tracking error is proved along the iteration direction. More importantly, the condition for a high‐order OTILC to outperform the low‐order ones is first established by this work. The learning gain is not fixed but iteratively updated by using the input and output (I/O) data, which enhances the flexibility of the proposed controller for modifications and expansions. The proposed method is data‐driven in which no explicit models are used except for the input and output data. The applications to a highly nonlinear continuous stirred tank reactor and a highly nonlinear fed‐batch fermentater demonstrate the effectiveness of the proposed high‐order OTILC design.     

Coordination of discrete‐time multi‐agent systems via relative output feedback

This paper investigates the joint effects of agent dynamic and network topology on the consensusability of linear discrete‐time multi‐agent systems via relative output feedback. An observer‐based distributed control protocol is proposed. A necessary and sufficient condition for consensusability under this control protocol is given, which explicitly reveals how the intrinsic entropy rate of the agent dynamic and the eigenratio of the undirected communication graph affect consensusability. As a special case, multi‐agent systems with discrete‐time double integrator dynamics are discussed where a simple control protocol directly using two‐step relative position feedback is provided to reach a consensus. Finally, the result is extended to solve the formation and formation‐based tracking problems. The theoretical results are illustrated by simulations. Copyright © 2010 John Wiley & Sons, Ltd.     

Adaptive quantized tracking control for uncertain switched nonstrict‐feedback nonlinear systems with discrete and distributed time‐varying delays

This paper is concerned with an adaptive tracking problem for a more general class of switched nonstrict‐feedback nonlinear time‐delay systems in the presence of quantized input. The system structure in a nonstrict‐feedback form, the discrete and distributed time‐varying delays, the sector‐bounded quantized input, and arbitrary switching behavior are involved in the considered systems. In particular, to overcome the difficulties from the distributed time‐varying delays and the sector‐bounded quantized input, the mean‐value theorem for integrals and some special techniques are exploited respectively. Moreover, by combining the Lyapunov‐Razumikhin method, dynamic surface control technique, fuzzy logic systems approximation, and variable separation technique, a quadratic common Lyapunov function is easily built for all subsystems and a common adaptive quantized control scheme containing only 1 adaptive parameter is proposed. It is shown that the tracking error converges to an adjustable neighborhood of the origin whereas all signals of the closed‐loop systems are semiglobally uniformly ultimately bounded. Finally, 2 simulation examples are provided to verify the feasibility and effectiveness of the proposed design methodology.     

Discrete‐time low‐gain control of linear systems with input/output nonlinearities

Discrete‐time low‐gain control strategies are presented for tracking of constant reference signals for finite‐dimensional, discrete‐time, power‐stable, single‐input, single‐output, linear systems subject to a globally Lipschitz, non‐decreasing input nonlinearity and a locally Lipschitz, non‐decreasing, affinely sector‐bounded output nonlinearity (the conditions on the output nonlinearities may be relaxed if the input nonlinearity is bounded). Both non‐adaptive and adaptive gain sequences are considered. In particular, it is shown that applying error feedback using a discrete‐time ‘integral’ controller ensures asymptotic tracking of constant reference signals, provided that (a) the steady‐state gain of the linear part of the plant is positive, (b) the positive gain sequence is ultimately sufficiently small and (c) the reference value is feasible in a very natural sense. The classes of input and output nonlinearities under consideration contain standard nonlinearities important in control engineering such as saturation and deadzone. The discrete‐time results are applied in the development of sampled‐data low‐gain control strategies for finite‐dimensional, continuous‐ time, exponentially stable, linear systems with input and output nonlinearities. Copyright © 2001 John Wiley & Sons, Ltd.