A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.     

A solution process for simulation-based multiobjective design optimization with an application in the paper industry

In this paper, we address some computational challenges arising in complex simulation-based design optimization problems. High computational cost, black-box formulation and stochasticity are some of the challenges related to optimization of design problems involving the simulation of complex mathematical models. Solving becomes even more challenging in case of multiple conflicting objectives that must be optimized simultaneously. In such cases, application of multiobjective optimization methods is necessary in order to gain an understanding of which design offers the best possible trade-off. We apply a three-stage solution process to meet the challenges mentioned above. As our case study, we consider the integrated design and control problem in paper mill design where the aim is to decrease the investment cost and enhance the quality of paper on the design level and, at the same time, guarantee the smooth performance of the production system on the operational level. In the first stage of the three-stage solution process, a set of solutions involving different trade-offs is generated with a method suited for computationally expensive multiobjective optimization problems using parallel computing. Then, based on the generated solutions an approximation method is applied to create a computationally inexpensive surrogate problem for the design problem and the surrogate problem is solved in the second stage with an interactive multiobjective optimization method. This stage involves a decision maker and her/his preferences to find the most preferred solution to the surrogate problem. In the third stage, the solution best corresponding that of stage two is found for the original problem.     

An economical cognitive approach for bi-objective optimization using bliss points, visualization, and interaction

When an optimization problem encompasses multiple objectives, it is usually difficult to define a single optimal solution. The decision maker plays an important role when choosing the final single decision. Pareto-based evolutionary multiobjective optimization (EMO) methods are very informative for the decision making process since they provide the decision maker with a set of efficient solutions to choose from. Despite that the set of efficient solutions may not be the global efficient set, we show in this paper that the set can still be informative when used in an interactive session with the decision maker. We use a combination of EMO and single objective optimization methods to guide the decision maker in interactive sessions.     

Tuning of PID controller based on a multiobjective genetic algorithm applied to a robotic manipulator

Most controllers optimization and design problems are multiobjective in nature, since they normally have several (possibly conflicting) objectives that must be satisfied at the same time. Instead of aiming at finding a single solution, the multiobjective optimization methods try to produce a set of good trade-off solutions from which the decision maker may select one. Several methods have been devised for solving multiobjective optimization problems in control systems field. Traditionally, classical optimization algorithms based on nonlinear programming or optimal control theories are applied to obtain the solution of such problems. The presence of multiple objectives in a problem usually gives rise to a set of optimal solutions, largely known as Pareto-optimal solutions. Recently, Multiobjective Evolutionary Algorithms (MOEAs) have been applied to control systems problems. Compared with mathematical programming, MOEAs are very suitable to solve multiobjective optimization problems, because they deal simultaneously with a set of solutions and find a number of Pareto optimal solutions in a single run of algorithm. Starting from a set of initial solutions, MOEAs use iteratively improving optimization techniques to find the optimal solutions. In every iterative progress, MOEAs favor population-based Pareto dominance as a measure of fitness. In the MOEAs context, the Non-dominated Sorting Genetic Algorithm (NSGA-II) has been successfully applied to solving many multiobjective problems. This paper presents the design and the tuning of two PID (Proportional–Integral–Derivative) controllers through the NSGA-II approach. Simulation numerical results of multivariable PID control and convergence of the NSGA-II is presented and discussed with application in a robotic manipulator of two-degree-of-freedom. The proposed optimization method based on NSGA-II offers an effective way to implement simple but robust solutions providing a good reference tracking performance in closed loop.     

Issues related to the computer realization of a multidisciplinary and multiobjective optimization system

Issues and novel ideas to be considered when developing computer realizations of complex multidisciplinary and multiobjective optimization systems are introduced. The aim is to discuss computer realizations that make possible both computationally efficient multidisciplinary analysis and multiobjective optimization of real world problems. We introduce software tools that make typically very time-consuming simulation processes more effective and, thus, enable even interactive multiobjective optimization with a real decision maker. In this paper, we first define a multidisciplinary and multiobjective optimization system and after that present an implementation overview of such problems including basic components participating in the solution process. Furthermore, interfaces and data flows between the components are described. A couple of important features related to the implementation are discussed in detail, for example, the usage of automatic differentiation. Finally, the ideas presented are illustrated with an industrial multiobjective optimization problem, when we describe numerical experiments related to quality properties in paper making.     

Scalable Pareto set generation for multiobjective co-design problems in water distribution networks: a continuous relaxation approach

In this paper, we study the multiobjective co-design problem of optimal valve placement and operation in water distribution networks, addressing the minimization of average pressure and pressure variability indices. The presented formulation considers nodal pressures, pipe flows and valve locations as decision variables, where binary variables are used to model the placement of control valves. The resulting optimization problem is a multiobjective mixed integer nonlinear optimization problem. As conflicting objectives, average zone pressure and pressure variability can not be simultaneously optimized. Therefore, we present the concept of Pareto optima sets to investigate the trade-offs between the two conflicting objectives and evaluate the best compromise. We focus on the approximation of the Pareto front, the image of the Pareto optima set through the objective functions, using the weighted sum, normal boundary intersection and normalized normal constraint scalarization techniques. Each of the three methods relies on the solution of a series of single-objective optimization problems, which are mixed integer nonlinear programs (MINLPs) in our case. For the solution of each single-objective optimization problem, we implement a relaxation method that solves a sequence of nonlinear programs (NLPs) whose stationary points converge to a stationary point of the original MINLP. The relaxed NLPs have a sparse structure that come from the sparse water network graph constraints. In solving the large number of relaxed NLPs, sparsity is exploited by tailored techniques to improve the performance of the algorithms further and render the approaches scalable for large scale networks. The features of the proposed scalarization approaches are evaluated using a published benchmarking network model.     

Parameter Settings and Representations in Pareto-Based Optimization for Job Shop Scheduling

A multiobjective optimization problem involves multiple objectives, often conflicting, to be met or optimized. A Pareto front provides a set of best solutions to determine the trade-offs between the various objectives. New evolutionary approaches demonstrated its ability to build well-delineated Pareto fronts in diverse multiobjective optimization problems, including multicriteria optimization in job shop scheduling with regular and nonregular objective functions. Good parameter settings and appropriate representations can enhance the behavior of an evolutionary algorithm. The present article shows a study of the influence of distinct parameter combinations as well as different chromosome representations. Details of implementation and results are discussed.     

Design of sound phase diffusers by means of multiobjective optimization approach using ev-MOGA evolutionary algorithm

In this paper a new approach to design sound phase diffusers is presented. The acoustic properties of such diffusers are usually increased by using single objective optimization methods. Here we propose the use of a multiobjective (MO) approach to design them in order to take into account several conflicting characteristic simultaneously. Three different MO problems are posed to consider various scenarios where fundamentally the objective is to maximize the normalized diffusion coefficient (following the corresponding Audio Engineering Society standard) for the so-called medium frequencies. This single objective could be divided into other several objectives to adjust performances to designer preferences. A multi-objective evolutionary algorithm (called ev-MOGA) is used to characterize the Pareto front in a smart way. ev-MOGA is modified, by using integer codification and tuning some of its genetic operators, to adapt it to the new requirements. Special interest is posed in selecting the diffusers codification properly to eliminate duplicities that would produce a multimodal problem. Precision in the manufacturing process is taking into account in the diffuser codification causing, that the number of different diffusers are quantified. Robust considerations related with the precision manufacturing process are considered in the decision making process. Finally, an optimal diffuser is selected considering designer preferences.     

Multiobjective Particle Swarm Algorithm With Fuzzy Clustering for Electrical Power Dispatch

Economic dispatch is a highly constrained optimization problem encompassing interaction among decision variables. Environmental concerns that arise due to the operation of fossil fuel fired electric generators, transforms the classical problem into multiobjective environmental/economic dispatch (EED). In this paper, a fuzzy clustering-based particle swarm (FCPSO) algorithm has been proposed to solve the highly constrained EED problem involving conflicting objectives. FCPSO uses an external repository to preserve nondominated particles found along the search process. The proposed fuzzy clustering technique, manages the size of the repository within limits without destroying the characteristics of the Pareto front. Niching mechanism has been incorporated to direct the particles towards lesser explored regions of the Pareto front. To avoid entrapment into local optima and enhance the exploratory capability of the particles, a self-adaptive mutation operator has been proposed. In addition, the algorithm incorporates a fuzzy-based feedback mechanism and iteratively uses the information to determine the compromise solution. The algorithm's performance has been examined over the standard IEEE 30 bus six-generator test system, whereby it generated a uniformly distributed Pareto front whose optimality has been authenticated by benchmarking against the epsiv -constraint method. Results also revealed that the proposed approach obtained high-quality solutions and was able to provide a satisfactory compromise solution in almost all the trials, thereby validating the efficacy and applicability of the proposed approach over the real-world multiobjective optimization problems.     

Pareto-optimal solutions for multicriteria optimization of a chemical engineering process using a diploid genetic algorithm

In many, if not most, optimization problems, industrialists are often confronted with multi‐objective decision problems. For example, in manufacturing processes, it may be necessary to optimize several criteria to take into account all the market constraints. Hence, the purpose is to choose the best trade‐offs among all the defined and conflicting objectives. This paper presents a multi‐objective optimization procedure based on a diploid genetic algorithm, which yields an optimal zone containing the solution under the concept of Pareto dominance. Pair‐wise points are compared, and non‐dominated points are collected in the Pareto region. Then a ranking is established, and the decision maker selects the first‐best solution. Finally, the procedure is applied to the chemical engineering process of cattle feed manufacture.     

Improving convergence of evolutionary multi-objective optimization with local search: a concurrent-hybrid algorithm

A local search method is often introduced in an evolutionary optimization algorithm, to enhance its speed and accuracy of convergence to optimal solutions. In multi-objective optimization problems, the implementation of local search is a non-trivial task, as determining a goal for local search in presence of multiple conflicting objectives becomes a difficult task. In this paper, we borrow a multiple criteria decision making concept of employing a reference point based approach of minimizing an achievement scalarizing function and integrate it as a search operator with a concurrent approach in an evolutionary multi-objective algorithm. Simulation results of the new concurrent-hybrid algorithm on several two to four-objective problems compared to a serial approach, clearly show the importance of local search in aiding a computationally faster and accurate convergence to the Pareto optimal front.     

Multi-criteria human resource allocation for solving multistage combinatorial optimization problems using multiobjective hybrid genetic algorithm

Multi-criteria human resource allocation involves deciding how to divide human resource of limited availability among multiple demands in a way that optimizes current objectives. In this paper, we focus on multi-criteria human resource allocation for solving multistage combinatorial optimization problem. Hence we tackle this problem via a multistage decision-making model. A multistage decision-making model is similar to a complex problem solving, in which a suitable sequence of decisions is to be found. The task can be interpreted as a series of interactions between a decision maker and an outside world, at each stage of which some decisions are available and their immediate effect can be easily computed. Eventually, goals would be reached due to the found of optimized variables. In order to obtain a set of Pareto solutions efficiently, we propose a multiobjective hybrid genetic algorithm (mohGA) approach based on the multistage decision-making model for solving combinatorial optimization problems. According to the proposed method, we apply the mohGA to seek feasible solutions for all stages. The effectiveness of the proposed algorithm was validated by its application to an illustrative example dealing with multiobjective resource allocation problem.     

MOSubdue: a Pareto dominance-based multiobjective Subdue algorithm for frequent subgraph mining

Graph-based data mining approaches have been mainly proposed to the task popularly known as frequent subgraph mining subject to a single user preference, like frequency, size, etc. In this work, we propose to deal with the frequent subgraph mining problem from multiobjective optimization viewpoint, where a subgraph (or solution) is defined by several user-defined preferences (or objectives), which are conflicting in nature. For example, mined subgraphs with high frequency are often of small size, and vice-versa. Use of such objectives in the multiobjective subgraph mining process generates Pareto-optimal subgraphs, where no subgraph is better than another subgraph in all objectives. We have applied a Pareto dominance approach for the evaluation and search subgraphs regarding to both proximity and diversity in multiobjective sense, which has incorporated in the framework of Subdue algorithm for subgraph mining. The method is called multiobjective subgraph mining by Subdue (MOSubdue) and has several advantages: (i) generation of Pareto-optimal subgraphs in a single run (ii) selection of subgraph-seeds from the candidate subgraphs based on all objectives (iii) search in the multiobjective subgraphs lattice space, and (iv) capability to deal with different multiobjective frequent subgraph mining tasks by customizing the tackled objectives. The good performance of MOSubdue is shown by performing multiobjective subgraph mining defined by two and three objectives on two real-life datasets.     

Random fuzzy multi-objective linear programming: Optimization of possibilistic value at risk (pVaR)

This paper considers multiobjective linear programming problems (MOLPP) where random fuzzy variables are contained in objective functions and constraints. A new decision making model optimizing possibilistic value at risk (pVaR) is proposed by incorporating the concept of value at risk (VaR) into possibility theory. It is shown that the original MOLPPs involving random fuzzy variables are transformed into deterministic problems. An interactive algorithm is presented to derive a satisficing solution for a decision maker (DM) from among a set of Pareto optimal solutions. Each Pareto optimal solution that is a candidate of the satisficing solution is exactly obtained by using convex programming techniques. A simple numerical example is provided to show the applicability of the proposed methodology to real-world problems with multiple objectives in uncertain environments.     

Multi-objective reliability-redundancy allocation problem using particle swarm optimization

This paper considers the multi-objective reliability redundancy allocation problem of a series system where the reliability of the system and the corresponding designing cost are considered as two different objectives. Due to non-stochastic uncertain and conflicting factors it is difficult to reduce the cost of the system and improve the reliability of the system simultaneously. In such situations, the decision making is difficult, and the presence of multi-objectives gives rise to multi-objective optimization problem (MOOP), which leads to Pareto optimal solutions instead of a single optimal solution. However in order to make the model more flexible and adaptable to human decision process, the optimization model can be expressed as fuzzy nonlinear programming problems with fuzzy numbers. Thus in a fuzzy environment, a fuzzy multi-objective optimization problem (FMOOP) is formulated from the original crisp optimization problem. In order to solve the resultant problem, a crisp optimization problem is reformulated from FMOOP by taking into account the preference of decision maker regarding cost and reliability goals and then particle swarm optimization is applied to solve the resulting fuzzified MOOP under a number of constraints. The approach has been demonstrated through the case study of a pharmaceutical plant situated in the northern part of India.     

A simulation-based evolutionary multiobjective approach to manufacturing cell formation

The purpose of this paper is to propose a multiobjective optimization approach for solving the manufacturing cell formation problem, explicitly considering the performance of this said manufacturing system. Cells are formed so as to simultaneously minimize three conflicting objectives, namely, the level of the work-in-process, the intercell moves and the total machinery investment. A genetic algorithm performs a search in the design space, in order to approximate to the Pareto optimal set. The values of the objectives for each candidate solution in a population are assigned by running a discrete-event simulation, in which the model is automatically generated according to the number of machines and their distribution among cells implied by a particular solution. The potential of this approach is evaluated via its application to an illustrative example, and a case from the relevant literature. The obtained results are analyzed and reviewed. Therefore, it is concluded that this approach is capable of generating a set of alternative manufacturing cell configurations considering the optimization of multiple performance measures, greatly improving the decision making process involved in planning and designing cellular systems.     

Multiobjective model predictive control

This paper proposes a novel model predictive control (MPC) scheme based on multiobjective optimization. At each sampling time, the MPC control action is chosen among the set of Pareto optimal solutions based on a time-varying, state-dependent decision criterion. Compared to standard single-objective MPC formulations, such a criterion allows one to take into account several, often irreconcilable, control specifications, such as high bandwidth (closed-loop promptness) when the state vector is far away from the equilibrium and low bandwidth (good noise rejection properties) near the equilibrium. After recasting the optimization problem associated with the multiobjective MPC controller as a multiparametric multiobjective linear or quadratic program, we show that it is possible to compute each Pareto optimal solution as an explicit piecewise affine function of the state vector and of the vector of weights to be assigned to the different objectives in order to get that particular Pareto optimal solution. Furthermore, we provide conditions for selecting Pareto optimal solutions so that the MPC control loop is asymptotically stable, and show the effectiveness of the approach in simulation examples.     

Coupling dynamic simulation and interactive multiobjective optimization for complex problems: An APROS-NIMBUS case study

Dynamic process simulators for plant-wide process simulation and multiobjective optimization tools can be used by industries as a means to cut costs and enhance profitability. Specifically, dynamic process simulators are useful in the process plant design phase, as they provide several benefits such as savings in time and costs. On the other hand, multiobjective optimization tools are useful in obtaining the best possible process designs when multiple conflicting objectives are to be optimized simultaneously. Here we concentrate on interactive multiobjective optimization. When multiobjective optimization methods are used in process design, they need an access to dynamic process simulators, hence it is desirable for them to coexist on the same software platform. However, such a co-existence is not common. Hence, users need to couple multiobjective optimization software and simulators, which may not be trivial. In this paper, we consider APROS, a dynamic process simulator and couple it with IND-NIMBUS, an interactive multiobjective optimization software. Specifically, we: (a) study the coupling of interactive multiobjective optimization with a dynamic process simulator; (b) bring out the importance of utilizing interactive multiobjective optimization; (c) propose an augmented interactive multiobjective optimization algorithm; and (d) apply an APROS-NIMBUS coupling for solving a dynamic optimization problem in a two-stage separation process.     

An extended study on multi-objective security games

The burgeoning area of security games has focused on real-world domains where security agencies protect critical infrastructure from a diverse set of adaptive adversaries. In such domains, decision makers have multiple competing objectives they must consider which may take different forms that are not readily comparable including safety, cost, and public perception. Thus, it can be difficult to know how to weigh the different objectives when deciding on a security strategy. To address the challenges of these domains, we propose a fundamentally different solution concept, multi-objective security games (MOSGs). Instead of a single optimal solution, MOSGs have a set of Pareto optimal (non-dominated) solutions referred to as the Pareto frontier, which can be generated by solving a sequence of constrained single-objective optimization problems (CSOPs). The Pareto frontier allows the decision maker to analyze the tradeoffs that exist between the multiple objectives. Our contributions include: (i) an algorithm, Iterative-ε-Constraints,, for generating the sequence of CSOPs; (ii) an exact approach for solving an mixed-integer linear program (MILP) formulation of a CSOP; (iii) heuristics that achieve speed up by exploiting the structure of security games to further constrain the MILP; (iv) an approximate approach for solving a CSOP built off those same heuristics, increasing the scalability of our approach with quality guarantees. Additional contributions of this paper include proofs on the level of approximation, detailed experimental evaluation of the proposed approaches and heuristics, as well as a discussion on techniques for visualizing the Pareto frontier.