Ergodic impulsive control of Feller processes with costly information

In this paper we study the problem of ergodic impulsive control of Feller processes with costly information. We prove continuity of the value functions for optimal stopping and impulsive control with long run average cost. We characterize the value functions as generalized solutions of respective quasi-variational inequalities and describe optimal policies. We study also an equation associated to impulsive control with long run average cost.     

Optimal control of markov chains admitting strong and weak interactions

This paper is devoted to the study of an optimal control problem for a Markov chain with generator B + εA, where ε is a small parameter. It is shown that an approximate solution can be calculated by a policy improvement algorithm involving computations relative to an ‘aggregated’ problem (the dimension of which is given by N, the number of ergodic sets for the B matrix) together with a family of ‘decentralized’ problems (the dimensions of which are given by the number of elements in each ergodic set for the B matrix).     

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.     

Singular ergodic control for multidimensional Gaussian processes

A multidimensional Wiener process is controlled by an additive process of bounded variation. A convex nonnegative function measures the cost associated with the position of the state process, and the cost of controlling is proportional to the displacement induced. We minimize a limiting time-average expected (ergodic) criterion. Under reasonable assumptions, we prove that the optimal discounted cost converges to the optimal ergodic cost. Moreover, under some additional conditions there exists a convex Lipschitz continuous function solution to the corresponding Hamilton-Jacobi-Bellman equation which provides an optimal stationary feedback control.Research supported in part by NSF Grant DMS-8702236.Research supported in part by Grant AFOSR-88-D183.     

Large Deviations of Empirical Estimates in Stochastic Programming Problems

The authors analyze a stochastic programming problem where the random factor is a stationary ergodic sequence. The problem is approximated by minimizing an empirical function. It is proved that, under some conditions, the probability of large deviations of empirical estimates from the initial problem solution decreases exponentially.     

On adaptive boundary control for stochastic parabolic systems withunknown potential coefficient

An adaptive boundary control problem for a stochastic heat diffusion equation is studied. The considered system contains an unknown potential coefficient which is a function of the spatial variables. The estimation algorithm for the unknown potential coefficient is proposed by using the stochastic approximation technique. After showing the strong consistency of the estimated parameter, the cost for the adaptive control scheme presented here is shown to converge to the optimal ergodic cost. Finally some numerical examples are shown     

Asymptotic buffer overflow probabilities in multiclassmultiplexers: an optimal control approach

We consider a multiclass multiplexer with support for multiple service classes and dedicated buffers for each service class. Under specific scheduling policies for sharing bandwidth among these classes, we seek the asymptotic (as the buffer size goes to infinity) tail of the buffer overflow probability for each dedicated buffer. We assume dependent arrival and service processes as is usually the case in models of bursty traffic. In the standard large deviations methodology, we provide a lower and a matching (up to first degree in the exponent) upper bound on the buffer overflow probabilities. We introduce a novel optimal control approach to address these problems. In particular, we relate the lower bound derivation to a deterministic optimal control problem, which we explicitly solve. Optimal state trajectories of the control problem correspond to typical congestion scenarios. We explicitly and in detail characterize the most likely modes of overflow. We specialize our results to the generalized processor sharing policy (GPS) and the generalized longest queue first policy (GLQF). The performance of strict priority policies is obtained as a corollary. We compare the GPS and GLQF policies and conclude that GLQF achieves smaller overflow probabilities than GPS for all arrival and service processes for which our analysis holds. Our results have important implications for traffic management of high-speed networks and can be used as a basis for an admission control mechanism which guarantees a different loss probability for each class     

The equivalence between infinite-horizon optimal control ofstochastic systems with exponential-of-integral performance index andstochastic differential games

A new method, based on the theory of large deviations from the invariant measure, is introduced for the analysis of stochastic systems with an infinite-horizon exponential-of-integral performance index. It is shown that the infinite-horizon optimal exponential-of-integral stochastic control problem is equivalent to a stationary stochastic differential game for an auxiliary system. As an application of the developed technique, the infinite-horizon risk-sensitive LQG problem is analyzed for both the completely observed and partially observed case     

Pathwise optimality for benchmark tracking

We consider the problem of investing in a portfolio in order to track or "beat" a given benchmark. We study this problem from the point of view of almost sure/pathwise optimality. We first obtain a control that is optimal in the mean and this control is then shown to be also pathwise optimal. The standard Merton model leads to lognormality of the value process so that it does not possess the required ergodic properties. We obtain ergodicity by transforming the process so that it remains bounded thereby using a method that can be related to a random time change. We furthermore describe a general approach to solve the Hamilton-Jacobi-Bellman equation corresponding to the given problem setup.     

A non-equilibrium analysis and control framework for active queue management

We present a non-equilibrium analysis and control approach for the Active Queue Management (AQM) problem in communication networks. Using simplified fluid models, we carry out a bifurcation study of the complex dynamic queue behavior to show that non-equilibrium methods are essential for analysis and optimization in the AQM problem. We investigate an ergodic theoretic framework for stochastic modeling of the non-equilibrium behavior in deterministic models and use it to identify parameters of a fluid model from packet level simulations. For computational tractability, we use set-oriented numerical methods to construct finite-dimensional Markov models, including control Markov chains and hidden Markov models. Subsequently, we develop and analyze an example AQM algorithm using a Markov Decision Process (MDP) based control framework. The control scheme developed is optimal with respect to a reward function, defined over the queue size and aggregate flow rate. We implement and simulate our illustrative AQM algorithm in the ns-2 network simulator. The results obtained confirm the theoretical analysis and exhibit promising performance when compared with well-known alternative schemes under persistent non-equilibrium queue behavior.     

Customization strategies for the solution of linear programming problems arising from large scale model predictive control of a paper machine

This paper presents an application of a customized linear programming (LP) based model predictive control strategy to the paper machine cross direction (CD) control problem. The objective of CD control is to maintain flat profiles of variables of interest by minimizing worst case deviations from setpoints (defects). These control problems can have as many as 200 actuators (inputs) and 400 sensor measurements (outputs). This large size coupled with the stringent real-time requirement of computing a control move in a few seconds poses a very challenging control problem. Computational results that demonstrate the effectiveness of this strategy will be presented. For typical disturbances this algorithm can compute provably optimal control moves for a 400 input ×400 output control problem in approximately 5 s versus approximately 90 s for a generic LP algorithm on a HP 9000/770 workstation.     

Optimal control laws for lot-sizing and timing of jobs on a single production facility

The optimal control of a single machine processing a certain number of jobs and modeled as a discrete-event dynamic system is considered. The number of jobs and their sequence are fixed, whereas their timing and sizes represent the control variables of the system. The objective function to be optimized is a weighted sum of the quadratic earliness and tardiness of each job, and of the quadratic deviations of job lot sizes and actual machine service speeds from those specified by the production demand and by the regular machine speeds. An optimization problem with quadratic cost function and nonlinear constraints is stated and formalized as a multistage optimal control problem. Necessary conditions to be satisfied by an optimal control sequence are derived. A simpler model is also considered in which the machine speeds are fixed; in this case, the control problem is solved by a procedure making use of dynamic programming techniques. The optimal control laws at each stage are thus obtained.     

Stochastic minimax control for stabilizing uncertain quasi- integrable Hamiltonian systems

A procedure for designing feedback control to asymptotically stabilize, with probability one, quasi-integrable Hamiltonian systems with bounded uncertain parametric disturbances is proposed. First, the partially averaged Itô stochastic differential equations are derived from given system by using the stochastic averaging method for quasi-integrable Hamiltonian systems. Second, the Hamilton-Jacobi-Issacs (HJI) equation for the ergodic control problem of the averaged system and a performance index with undetermined cost function is established based on the principle of optimality. This equation is then solved to yield the worst disturbances and the associated optimal controls. Third, the asymptotic Lyapunov stability with probability one of the optimally controlled system with worst disturbances is analyzed by evaluating the maximal Lyapunov exponent of the fully averaged Itô equations. Finally, the cost function and feedback control are determined by the requirement of stabilizing the worst-disturbed system. A simple example is worked out to illustrate the application of the proposed procedure and the effects of optimal control on stabilizing the uncertain system.     

Adaptive control design under structured model information limitation: A cost-biased maximum-likelihood approach

Networked control strategies based on limited information about the plant model usually result in worse closed-loop performance than optimal centralized control with full plant model information. Recently, this fact has been established by utilizing the concept of competitive ratio, which is defined as the worst-case ratio of the cost of a control design with limited model information to the cost of the optimal control design with full model information. We show that an adaptive controller, inspired by a controller proposed by Campi and Kumar, with limited plant model information, asymptotically achieves the closed-loop performance of the optimal centralized controller with full model information for almost any plant. Therefore, there exists, at least, one adaptive control design strategy with limited plant model information that can achieve a competitive ratio equal to one. The plant model considered in the paper belongs to a compact set of stochastic linear time-invariant systems and the closed-loop performance measure is the ergodic mean of a quadratic function of the state and control input.     

Lossless convexification of a class of optimal control problems with non-convex control constraints

We consider a class of finite time horizon optimal control problems for continuous time linear systems with a convex cost, convex state constraints and non-convex control constraints. We propose a convex relaxation of the non-convex control constraints, and prove that the optimal solution of the relaxed problem is also an optimal solution for the original problem, which is referred to as the lossless convexification of the optimal control problem. The lossless convexification enables the use of interior point methods of convex optimization to obtain globally optimal solutions of the original non-convex optimal control problem. The solution approach is demonstrated on a number of planetary soft landing optimal control problems.     

Modeling and control of high-throughput screening systems in a max-plus algebraic setting

In this paper, we present a max-plus algebraic modeling and control approach for cyclically operated high-throughput screening plants. In previous work an algorithm has been developed to determine the globally optimal solution of the cyclic scheduling problem. The obtained optimal schedule is modeled in a max-plus algebraic framework. The max-plus algebraic model can then be used to generate appropriate control actions to handle unexpected deviations from the predetermined cyclic operation during runtime.     

A non-linear optimal control approach to central sewer network flow control

The problem of optimal water distribution to several retention reservoirs in an urban sewer network during rainfall is considered. The goal of the control actions is the minimization of overflows and eventually the reduction of their polluting impact on receiving waters. To this end, a non-linear optimal control approach is used and the numerical solution of the control problem is effectuated by use of a feasible direction algorithm. A detailed study of the central control problem for a particular large sewer network using this method is presented. Results demonstrate the efficiency and the real-time feasibility of the developed methodology.     

An optimal control approach to robust control design

We propose an optimal control approach to robust control design. Our goal is to design a state feedback to stabilize a system under uncertainty. We translate this robust control problem into an optimal control problem of minimizing a cost. Because the uncertainty bound is reflected in the cost, the solution to the optimal control problem is a solution to the robust control problem. Our approach can deal with both linear and non-linear systems. Furthermore it can handle both matched and unmatched uncertainties. It can also handle uncertainty in the control input matrix.     

A decomposition technique for an optimal control problem with "PQDZ" cost and bounded controls

In this note we present a solution method for a discrete-time linear optimal control problem where the controls are bounded and both the states and controls have asymmetric costs with dead zones containing the nominal values. This model generalizes the well-known optimal control model for a linear system with symmetric (quadratic) objective functional. We transform the problem into an equivalent large quadratic programming problem with equality and inequality, constraints and decompose it intoNsimpler subproblems, each with very easily computed optimal solutions. Using a two-level approach suggested by Lasdon and Schoeffler [11] the optimal solution to the overall problem is obtained.     

Sliding mode control based on a modified linear control input

In this study, we explain and demonstrate a design method of sliding mode control based on a modified linear control input. In the proposed method, the optimal gain matrix is derived such that it does not depend on the plant parameters. We confirmed the robustness of the proposed method by applying input-side disturbances and plant parameter deviations to plants and the effectiveness of the proposed method by performing a DC motor position control experiment