Automatic Differentiation for MATLAB Programs

Derivative information is required in numerous applications, including sensitivity analysis and numerical optimization. For simple functions, symbolic differentiation–done either manually or with a computer algebra system–can provide the derivatives, whereas divided differences (DD) have been used traditionally for functions defined by (potentially very complex) computer programs, even if only approximate values can be obtained this way. An alternative approach for such functions is automatic differentiation (AD), yielding exact derivatives at often lower cost than DD, and without restrictions on the program complexity. In this paper we compare the functionality and describe the use of ADMIT/ADMAT and ADiMat. These two AD tools provide derivatives for programs written in the MATLAB language, which is widely used for prototype and production software in scientific and engineering applications. While ADMIT/ADMAT implements a pure operator overloading approach of AD, ADiMat also employes source transformation techniques.     

A Note on the Recursive and Parallel Structure of the Birge and Qi Factorization for Tree Structured Linear Programs

In this note, we derive the complete recursive structure of the Birge and Qi factorization for interior point methods (IPM) for tree structured linear programs as they appear in multistage stochastic programs. This recursive structure allows for an elegant implementation on parallel hardware, since multiple versions of the same program may be run on on different processors. Our preliminary computational experiment, conducted on a Beowulf cluster, demonstrates the scalability of this approach.     

Dynamic Cost-Benefit Analysis of Drug Substitution Programs

This paper studies optimal spending for drug substitution programs in the context of a dynamic epidemic model of both drug use and drug use-related infections. Two types of costs are considered in addition to control costs: social costs resulting from individuals being dependent on drugs; additional costs due to drug users being vulnerable to infections like hepatitis C or HIV. Analysis of the model demonstrates that the long-run equilibrium is not necessarily unique. Instead, there may be multiple equilibria. Which of these equilibria is optimal depends on the initial conditions for the number of drug addicts and the number of those who are infected. So, for a given set of epidemic parameters, it may be optimal to spend a lot on substitution programs that reduce the number of drug addicts or to spend little and to accept a high level of drug use.     

On Distributionally Robust Chance-Constrained Linear Programs

In this paper, we discuss linear programs in which the data that specify the constraints are subject to random uncertainty. A usual approach in this setting is to enforce the constraints up to a given level of probability. We show that, for a wide class of probability distributions (namely, radial distributions) on the data, the probability constraints can be converted explicitly into convex second-order cone constraints; hence, the probability-constrained linear program can be solved exactly with great efficiency. Next, we analyze the situation where the probability distribution of the data is not completely specified, but is only known to belong to a given class of distributions. In this case, we provide explicit convex conditions that guarantee the satisfaction of the probability constraints for any possible distribution belonging to the given class.Communicated by B. T. PolyakThis work was supported by FIRB funds from the Italian Ministry of University and Research.     

Die Nutzung von Programmierwerkzeugen für das Lösen offener Aufgaben im Mathematikunterricht der Sekundarstufe I

Due to the fact that the computer tools available at present in mathematics teaching offer only inferior capabilities for solving (open) problems, their capabilities being restricted to sequential algorithms, and also because the of mathematics—informaties paradigm is inadequately formulated, standardized programming tools are needed in order to handle certain types of problems. The article presents examples of arithmetic problems and their solution using standardized programming tools, so as to illustrate their applications in: developing programs for the derivation of multi-element solution sets (as well as for strengthening the inductive basis required in order to find propositions); modifying available programs in order to be able to deal with diversified or extended open problems; using standardized programs as a ?black-box” software for checking or elaborating strategies for open problem solving.     

Decomposition Methods for Solving Nonconvex Quadratic Programs via Branch and Bound*

The aim of this paper is to suggest branch and bound schemes, based on a relaxation of the objective function, to solve nonconvex quadratic programs over a compact polyhedral feasible region. The various schemes are based on different d.c. decomposition methods applied to the quadratic objective function. To improve the tightness of the relaxations, we also suggest solving the relaxed problems with an algorithm based on the so called “optimal level solutions” parametrical approach. *This paper has been partially supported by M.I.U.R. and C.N.R.     

一种求解线性二层规划的割平面方法

以下层问题的K-T最优性条件代替下层问题,将线性二层规划转化为相应的单层规划问题,通过分析单层规划可行解集合的结构特征,设计了一种求解线性二层规划全局最优解的割平面算法.数值结果表明所设计的割平面算法是可行、有效的.     

Generalized Derivatives of Lexicographic Linear Programs

Lexicographic linear programs are fixed-priority multiobjective linear programs that are a useful model of biological systems using flux balance analysis and for goal-programming problems. The objective function values of a lexicographic linear program as a function of its right-hand side are nonsmooth. This work derives generalized derivative information for lexicographic linear programs using lexicographic directional derivatives to obtain elements of the Bouligand subdifferential (limiting Jacobian). It is shown that elements of the limiting Jacobian can be obtained by solving related linear programs. A nonsmooth equation-solving problem is solved to illustrate the benefits of using elements of the limiting Jacobian of lexicographic linear programs.     

Some Lessons in Solving Practical Integer Programs

Integer programs are harder to solve than linear programs of similar size. Even those of modest size may prove sufficiently difficult to deter practitioners from using them. But, formulated with care and solved with an appropriate branching strategy, they may be solved quickly. This paper discusses the elements of good formulation, high level branching constructs and effective branching strategies. These methods are applied to four practical case studies which are explored in depth.     

A Polynomial-Time Solution Scheme for Quadratic Stochastic Programs

We consider quadratic stochastic programs with random recourse—a class of problems which is perceived to be computationally demanding. Instead of using mainstream scenario tree-based techniques, we reduce computational complexity by restricting the space of recourse decisions to those linear and quadratic in the observations, thereby obtaining an upper bound on the original problem. To estimate the loss of accuracy of this approach, we further derive a lower bound by dualizing the original problem and solving it in linear and quadratic recourse decisions. By employing robust optimization techniques, we show that both bounding problems may be approximated by tractable conic programs.     

A Fritz John Approach to First Order Optimality Conditions for Mathematical Programs with Equilibrium Constraints

Mathematical programs with equilibrium constraints (MPECs) are nonlinear programs which do not satisfy any of the common constraint qualifications. In order to obtain first order optimality conditions, constraint qualifications tailored to MPECs have been developed and researched in the past. This has been done by falling back on technical proofs or results from nonsmooth analysis. In this article, we use a completely different approach and show how the standard Fritz John conditions may be used in order to obtain short and elementary proofs for the most important optimality conditions for MPECs. As a by-product, we obtain a new stationarity concept.     

DAG reversal is NP-complete

Runs of numerical computer programs can be visualized as directed acyclic graphs (DAGs). We consider the problem of restoring the intermediate values computed by such a program (the vertices in the DAG) in reverse order for a given upper bound on the available memory. The minimization of the associated computational cost in terms of the number of performed arithmetic operations is shown to be NP-complete. The reversal of the data-flow finds application, for example, in the efficient evaluation of adjoint numerical programs. We derive special cases of numerical programs that require the intermediate values exactly in reverse order, thus establishing the NP-completeness of the optimal adjoint computation problem. Last but not least we review some state-of-the-art approaches to efficient data-flow reversal taken by existing software tools for automatic differentiation.     

TASS: The Toolkit for Accurate Scientific Software

The Toolkit for Accurate Scientific Software (TASS) is a suite of integrated tools for the formal verification of programs used in computational science, including numerically-intensive message-passing-based parallel programs. While TASS can verify a number of standard safety properties (such as absence of deadlocks and out-of-bound array indexing), its most powerful feature is the ability to establish that two programs are functionally equivalent. These properties are verified by performing an explicit state enumeration of a model of the program(s). In this model, symbolic expressions are used to represent the inputs and the values of variables. TASS uses novel techniques to simplify the symbolic representation of the state and to reduce the number of states explored and saved. The TASS front-end supports a large subset of C, including (multi-dimensional) arrays, structs, dynamically allocated data, pointers and pointer arithmetic, functions and recursion, and other commonly used language constructs. A number of experiments on small but realistic numerical programs show that TASS can scale to reasonably large configurations and process counts. TASS is open source software distributed under the GNU Public License. The Java source code, examples, experimental results, and reference materials are all available at .     

Program equilibrium—a program reasoning approach

The concept of program equilibrium, introduced by Howard (Theory and Decision 24(3):203–213, 1988) and further formalised by Tennenholtz (Game Econ Behav 49:363–373, 2004), represents one of the most ingenious and potentially far-reaching applications of ideas from computer science in game theory to date. The basic idea is that a player in a game selects a strategy by entering a program, whose behaviour may be conditioned on the programs submitted by other players. Thus, for example, in the prisoner’s dilemma, a player can enter a program that says “If his program is the same as mine, then I cooperate, otherwise I defect”. It can easily be shown that if such programs are permitted, then rational cooperation is possible even in the one-shot prisoner’s dilemma. In the original proposal of Tennenholtz, comparison between programs was limited to syntactic comparison of program texts. While this approach has some considerable advantages (not the least being computational and semantic simplicity), it also has some important limitations. In this paper, we investigate an approach to program equilibrium in which richer conditions are allowed, based on model checking—one of the most successful approaches to reasoning about programs. We introduce a decision-tree model of strategies, which may be conditioned on strategies of others. We then formulate and investigate a notion of “outcome” for our setting, and investigate the complexity of reasoning about outcomes. We focus on coherent outcomes: outcomes in which every decision by every player is justified by the conditions in his program. We identify a condition under which there exist a unique coherent outcome. We also compare our notion of (coherent) outcome with that of (supported) semantics known from logic programming. We illustrate our approach with many examples.     

The semantics and complexity of parallel programs for vector computations. Part I: A case study using ADA

Recent research in parallel numerical computation has tended to focus on the algorithmic level. Less attention has been given to the programming level where algorithm is matched, to some extent, to computer architecture. This two-part paper presents a three-level approach to parallel programming which distinguishes between mathematical algorithm, program and computer architecture. In part I, we motivate our approach by a case study using the Ada language. In part II, a mathematical concept of parallel algorithm is introduced in terms of partial orders. This serves as the basis of a theory of parallel computation which makes possible a precise semantics and a precise criterion of complexity of parallel programs. It also suggests some notation for specifying parallel numerical algorithms. To illustrate the ideas presented in part II, we concentrate here on parallel numerical computations which have vector spaces as their central data type and which are intended to be executed on a multi-processor system. The Ada language, with its task constructs, allows one to program computer algorithms to be executed on multi-processor systems, rather than on vector (pipelined) architectures. To provide a concrete example of the general problem of programming parallel numerical algorithms for multi-processor computers, we do a case study of how Ada can be used to program the solution of a system of linear equations on such computers. The case study includes an analysis of complexity which addresses the cost of data movement and process control/synchronization as well as the usual arithmetic complexity.Dedicated to Peter Naur on the occasion of his 60th birthdayThis research was partially supported by NSF Grants DCR-8406290 & CCR-8712192.     

A note on reduction of quadratic and bilinear programs with equality constraints

Reduction of some classes of global optimization programs to bilinear programs may be done in various ways, and the choice of method clearly influences the ease of solution of the resulting problem. In this note we show how linear equality constraints may be used together with graph theoretic tools to reduce a bilinear program, i.e., eliminate variables from quadratic terms to minimize the number of complicating variables. The method is illustrated on an example. Computer results are reported on known test problems.     

On a class of bilevel linear mixed-integer programs in adversarial settings

We consider a class of bilevel linear mixed-integer programs (BMIPs), where the follower’s optimization problem is a linear program. A typical assumption in the literature for BMIPs is that the follower responds to the leader optimally, i.e., the lower-level problem is solved to optimality for a given leader’s decision. However, this assumption may be violated in adversarial settings, where the follower may be willing to give up a portion of his/her optimal objective function value, and thus select a suboptimal solution, in order to inflict more damage to the leader. To handle such adversarial settings we consider a modeling approach referred to as \(\alpha \)-pessimistic BMIPs. The proposed method naturally encompasses as its special classes pessimistic BMIPs and max–min (or min–max) problems. Furthermore, we extend this new modeling approach by considering strong-weak bilevel programs, where the leader is not certain if the follower is collaborative or adversarial, and thus attempts to make a decision by taking into account both cases via a convex combination of the corresponding objective function values. We study basic properties of the proposed models and provide numerical examples with a class of the defender–attacker problems to illustrate the derived results. We also consider some related computational complexity issues, in particular, with respect to optimistic and pessimistic bilevel linear programs.     

回报计划对重复购买行为模式的影响研究

客户回报计划已成为一种重要的关系营销手段。本文在讨论回报计划如何对稳定市场结构下的重复购买行为产生影响的基础上,通过建立NBD-DM随机模型,提供了一种研究消费者重复购买行为的模型方法,并利用一组护肤品品类销售的固定样本组数据(panel data)对该方法进行了实证分析。结果表明NBD-DM模型是研究消费者重复购买行为的有效模型方法,并且证实回报计划在改变客户重复购买行为上的有效性,其是企业建立长期客户关系的有效手段。最后讨论了结论对战略及营销管理实践的意义。     

Scenario optimization

Uncertainty in the parameters of a mathematical program may present a modeller with considerable difficulties. Most approaches in the stochastic programming literature place an apparent heavy data and computational burden on the user and as such are often intractable. Moreover, the models themselves are difficult to understand. This probably explains why one seldom sees a fundamentally stochastic model being solved using stochastic programming techniques. Instead, it is common practice to solve a deterministic model with different assumed scenarios for the random coefficients. In this paper we present a simple approach to solving a stochastic model, based on a particular method for combining such scenario solutions into a single, feasible policy. The approach is computationally simple and easy to understand. Because of its generality, it can handle multiple competing objectives, complex stochastic constraints and may be applied in contexts other than optimization. To illustrate our model, we consider two distinct, important applications: the optimal management of a hydro-thermal generating system and an application taken from portfolio optimization.     

Interval Methods for Semi-Infinite Programs

A new approach for the numerical solution of smooth, nonlinear semi-infinite programs whose feasible set contains a nonempty interior is presented. Interval analysis methods are used to construct finite nonlinear, or mixed-integer nonlinear, reformulations of the original semi-infinite program under relatively mild assumptions on the problem structure. In certain cases the finite reformulation is exact and can be solved directly for the global minimum of the semi-infinite program (SIP). In the general case, this reformulation is over-constrained relative to the SIP, such that solving it yields a guaranteed feasible upper bound to the SIP solution. This upper bound can then be refined using a subdivision procedure which is shown to converge to the true SIP solution with finite -optimality. In particular, the method is shown to converge for SIPs which do not satisfy regularity assumptions required by reduction-based methods, and for which certain points in the feasible set are subject to an infinite number of active constraints. Numerical results are presented for a number of problems in the SIP literature. The solutions obtained are compared to those identified by reduction-based methods, the relative performances of the nonlinear and mixed-integer nonlinear formulations are studied, and the use of different inclusion functions in the finite reformulation is investigated.