An improved sequential quadratic programming algorithm for solving general nonlinear programming problems

In this paper, a class of general nonlinear programming problems with inequality and equality constraints is discussed. Firstly, the original problem is transformed into an associated simpler equivalent problem with only inequality constraints. Then, inspired by the ideals of the sequential quadratic programming (SQP) method and the method of system of linear equations (SLE), a new type of SQP algorithm for solving the original problem is proposed. At each iteration, the search direction is generated by the combination of two directions, which are obtained by solving an always feasible quadratic programming (QP) subproblem and a SLE, respectively. Moreover, in order to overcome the Maratos effect, the higher-order correction direction is obtained by solving another SLE. The two SLEs have the same coefficient matrices, and we only need to solve the one of them after a finite number of iterations. By a new line search technique, the proposed algorithm possesses global and superlinear convergence under some suitable assumptions without the strict complementarity. Finally, some comparative numerical results are reported to show that the proposed algorithm is effective and promising.     

Computational schemes for large-scale problems in extended linear-quadratic programming

Numerical approaches are developed for solving large-scale problems of extended linear-quadratic programming that exhibit Lagrangian separability in both primal and dual variables simultaneously. Such problems are kin to large-scale linear complementarity models as derived from applications of variational inequalities, and they arise from general models in multistage stochastic programming and discrete-time optimal control. Because their objective functions are merely piecewise linear-quadratic, due to the presence of penalty terms, they do not fit a conventional quadratic programming framework. They have potentially advantageous features, however, which so far have not been exploited in solution procedures. These features are laid out and analyzed for their computational potential. In particular, a new class of algorithms, called finite-envelope methods, is described that does take advantage of the structure. Such methods reduce the solution of a high-dimensional extended linear-quadratic program to that of a sequence of low-dimensional ordinary quadratic programs.This work was supported in part by grants AFOSR 87-0821 and AFOSR 89-0081 from the Air Force Office of Scientific Research.     

A new descent algorithm for solving quadratic bilevel programming problems

1. IntroductionA bilevel programming problem (BLPP) involves two sequential optimization problems where the constraint region of the upper one is implicitly determined by the solutionof the lower. It is proved in [1] that even to find an approximate solution of a linearBLPP is strongly NP-hard. A number of algorithms have been proposed to solve BLPPs.Among them, the descent algorithms constitute an important class of algorithms for nonlinear BLPPs. However, it is assumed for almost all…     

Modified proximal point algorithm for extended linear-quadratic programming

Extended linear-quadratic programming arises as a flexible modeling scheme in dynamic and stochastic optimization, which allows for penalty terms and facilitates the use of duality. Computationally it raises new challenges as well as new possibilities in large-scale applications. Recent efforts have been focused on the fully quadratic case ([15] and [23]), while relying on the fundamental proximal point algorithm (PPA) as a shell of outer iterations when the problem is not fully quadratic. In this paper, we focus on the nonfully quadratic cases by proposing some new variants of the fundamental PPA. We first construct a continuously differentiable saddle function S(u, v) through infimal convolution in such a way that the optimal primal-dual pairs of the original problem are just the saddle points of S(u, v) on the whole space. Then the original extended linear-quadratic-programming problem reduces to solving the nonlinear equation S(u, v)=0. We then embed the fundamental PPA and some of its previous variants in the framework of a Newton-like iteration for this equation. After revealing the local quadratic structure of S near the solution, we derive new extensions of the fundamental PPA. In numerical tests, the modified iteration scheme based on the quasi-Newton update formula outperforms the fundamental PPA considerably.     

A trust region SQP algorithm for mixed-integer nonlinear programming

We propose a modified sequential quadratic programming method for solving mixed-integer nonlinear programming problems. Under the assumption that integer variables have a smooth influence on the model functions, i.e., that function values do not change drastically when in- or decrementing an integer value, successive quadratic approximations are applied. The algorithm is stabilized by a trust region method with Yuan’s second order corrections. It is not assumed that the mixed-integer program is relaxable or, in other words, function values are evaluated only at integer points. The Hessian of the Lagrangian function is approximated by a quasi-Newton update formula subject to the continuous and integer variables. Numerical results are presented for a set of 80 mixed-integer test problems taken from the literature. The surprising result is that the number of function evaluations, the most important performance criterion in practice, is less than the number of function calls needed for solving the corresponding relaxed problem without integer variables.     

非线性互补问题的SQP方法

In this paper, the nonlinear complementarity problem is transformed into the least squares problem with nonnegative constraints ,and a SQP algorithm for this reformulation based on a damped Gauss-Newton type method is presented. It is shown that the algorithm is globally and locally superlinearly (quadratically) convergent without the assumption of monotonicity.     

Stability results for stochastic programming problems

The paper deals with a statistical approach to stability analysis in nonlinear stochastic programming. Firstly the distribution function of the underlying random variable is estimated by the empirical distribution function, and secondly the problem of estimated parameters is considered. In both the cases the probability that the solution set of the approximate problem, is not contained in an l-neighbourhood of the solution set to the original problem is estimated, and under differentiability properties an asymptotic expansion for the density of the (unique) solution to the approximate problem is derived.     

等式约束优化一个新的SQP算法

提出了一个处理等式约束优化问题新的SQP算法,该算法通过求解一个增广Lagrange函数的拟Newton方法推导出一个等式约束二次规划子问题,从而获得下降方向．罚因子具有自动调节性,并能避免趋于无穷．为克服Maratos效应采用增广Lagrange函数作为效益函数并结合二阶步校正方法．在适当的条件下,证明算法是全局收敛的,并且具有超线性收敛速度．     

Convergence of a Dinkelbach-type algorithm in generalized fractional programming

The convergence of a Dinkelbach-type algorithm in generalized fractional programming is obtained by considering the sensitivity of a parametrized problem. We show that the rate of convergence is at least equal to (1+5)/2 when regularity conditions hold in a neighbourhood of the optimal solution. We give also a necessary and sufficient condition for the convergence to be quadratic (which will be verified in particular in the linear case) and an idea of its implementation in the convex case.

---

Zusammenfassung Die Konvergenz eines Verfahrens i. S. von Dinkelbach zur Lösung verallgemeinerter Quotientenprogramme wird durch Untersuchung der Sensitivität eines parametrisierten Problems abgeleitet. Es wird gezeigt, daß die Konvergenzrate durch (1+5)/2 nach unten beschränkt ist, falls gewisse Regularitätsbedingungen in einer Umgebung der Optimallösung erfüllt sind. Ferner wird eine notwendige und hinreichende Bedingung zur quadratischen Konvergenz hergeleitet. Es wird gezeigt, wie diese im Falle konvexer Probleme implementiert werden kann.

     

Parallel decomposition of multistage stochastic programming problems

A new decomposition method for multistage stochastic linear programming problems is proposed. A multistage stochastic problem is represented in a tree-like form and with each node of the decision tree a certain linear or quadratic subproblem is associated. The subproblems generate proposals for their successors and some backward information for their predecessors. The subproblems can be solved in parallel and exchange information in an asynchronous way through special buffers. After a finite time the method either finds an optimal solution to the problem or discovers its inconsistency. An analytical illustrative example shows that parallelization can speed up computation over every sequential method. Computational experiments indicate that for large problems we can obtain substantial gains in efficiency with moderate numbers of processors.This work was partly supported by the International Institute for Applied Systems Analysis, Laxenburg, Austria.     

ISTMO: An interval reference point-based method for stochastic multiobjective programming problems

In this paper, we present an interactive algorithm (ISTMO) for stochastic multiobjective problems with continuous random variables. This method combines the concept of probability efficiency for stochastic problems with the reference point philosophy for deterministic multiobjective problems. The decision maker expresses her/his references by dividing the variation range of each objective into intervals, and by setting the desired probability for each objective to achieve values belonging to each interval. These intervals may also be redefined during the process. This interactive procedure helps the decision maker to understand the stochastic nature of the problem, to discover the risk level (s)he is willing to assume for each objective, and to learn about the trade-offs among the objectives.     

An interior point method for general large-scale quadratic programming problems

In this paper, we present an interior point algorithm for solving both convex and nonconvex quadratic programs. The method, which is an extension of our interior point work on linear programming problems efficiently solves a wide class of largescale problems and forms the basis for a sequential quadratic programming (SQP) solver for general large scale nonlinear programs. The key to the algorithm is a three-dimensional cost improvement subproblem, which is solved at every interation. We have developed an approximate recentering procedure and a novel, adaptive big-M Phase I procedure that are essential to the sucess of the algorithm. We describe the basic method along with the recentering and big-M Phase I procedures. Details of the implementation and computational results are also presented.Contribution of the National Institute of Standards and Tedchnology and not subject to copyright in the United States. This research was supported in part by ONR Contract N-0014-87-F0053.     

Analysis and comparisons of some solution concepts for stochastic programming problems

The aim of this study is to analyse the resolution of Stochastic Programming Problems in which the objective function depends on parameters which are continuous random variables with a known distribution probability. In the literature on these questions different solution concepts have been defined for problems of these characteristics. These concepts are obtained by applying a transformation criterion to the stochastic objective which contains a statistical feature of the objective, implying that for the same stochastic problem there are different optimal solutions available which, in principle, are not comparable. Our study analyses and establishes some relations between these solution concepts. The work of these authors was supported byMinisterio de Ciencia y Tecnología andConsejería de Educación y Ciencia, Junta de Andalucía.     

Dynamic programming for multidimensional stochastic control problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem, singular control problem and impulse control problem as special cases. Using a unified treatment of dynamic programming, we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasivariational inequality. The uniqueness of such a quasi-variational inequality is proved. Supported in part by USA Office of Naval Research grant #N00014-96-1-0262. Supported in part by the NSFC Grant #79790130, the National Distinguished Youth Science Foundation of China Grant #19725106 and the Chinese Education Ministry Science Foundation.     

Relaxations of linear programming problems with first order stochastic dominance constraints

Linear stochastic programming problems with first order stochastic dominance (FSD) constraints are non-convex. For their mixed 0-1 linear programming formulation we present two convex relaxations based on second order stochastic dominance (SSD). We develop necessary and sufficient conditions for FSD, used to obtain a disjunctive programming formulation and to strengthen one of the SSD-based relaxations.     

A potential reduction algorithm for an extended SDP problem

An extended semi-definite programming, the SDP with an additional quadratic term in the objective function, is studied. Our generalization is similar to the generalization from linear programming to quadratic programming. Optimal conditions for this new class of problems are discussed and a potential reduction algorithm for solving QSDP problems is presented. The convergence properties of this algorithm are also given.     

An infeasible-interior-point algorithm for linear complementarity problems

We modify the algorithm of Zhang to obtain anO(n²L) infeasible-interior-point algorithm for monotone linear complementarity problems that has an asymptoticQ-subquadratic convergence rate. The algorithm requires the solution of at most two linear systems with the same coefficient matrix at each iteration.This research was supported by the Office of Scientific Computing, U.S. Department of Energy, under Contract W-31-109-Eng-38.     

Valid inequalities and restrictions for stochastic programming problems with first order stochastic dominance constraints

Stochastic dominance relations are well studied in statistics, decision theory and economics. Recently, there has been significant interest in introducing dominance relations into stochastic optimization problems as constraints. In the discrete case, stochastic optimization models involving second order stochastic dominance constraints can be solved by linear programming. However, problems involving first order stochastic dominance constraints are potentially hard due to the non-convexity of the associated feasible regions. In this paper we consider a mixed 0–1 linear programming formulation of a discrete first order constrained optimization model and present a relaxation based on second order constraints. We derive some valid inequalities and restrictions by employing the probabilistic structure of the problem. We also generate cuts that are valid inequalities for the disjunctive relaxations arising from the underlying combinatorial structure of the problem by applying the lift-and-project procedure. We describe three heuristic algorithms to construct feasible solutions, based on conditional second order constraints, variable fixing, and conditional value at risk. Finally, we present numerical results for several instances of a real world portfolio optimization problem. This research was supported by the NSF awards DMS-0603728 and DMI-0354678.     

A new technique for inconsistent QP problems in the SQP method

Successful treatment of inconsistent QP problems is of major importance in the SQP method, since such occur quite often even for well behaved nonlinear programming problems. This paper presents a new technique for regularizing inconsistent QP problems, which compromises in its properties between the simple technique of Pantoja and Mayne [36] and the highly successful, but expensive one of Tone [47]. Global convergence of a corresponding algorithm is shown under reasonable weak conditions. Numerical results are reported which show that this technique, combined with a special method for the case of regular subproblems, is quite competitive to highly appreciated established ones.     

A modified SQP algorithm for minimax problems

In this paper, a modified nonmonotone line search SQP algorithm for nonlinear minimax problems is presented. During each iteration of the proposed algorithm, a main search direction is obtained by solving a reduced quadratic program (QP). In order to avoid the Maratos effect, a correction direction is generated by solving the reduced system of linear equations. Under mild conditions, the global and superlinear convergence can be achieved. Finally, some preliminary numerical results are reported.