Multiobjective global surrogate modeling, dealing with the 5-percent problem

When dealing with computationally expensive simulation codes or process measurement data, surrogate modeling methods are firmly established as facilitators for design space exploration, sensitivity analysis, visualization, prototyping and optimization. Typically the model parameter (=hyperparameter) optimization problem as part of global surrogate modeling is formulated in a single objective way. Models are generated according to a single objective (accuracy). However, this requires an engineer to determine a single accuracy target and measure upfront, which is hard to do if the behavior of the response is unknown. Likewise, the different outputs of a multi-output system are typically modeled separately by independent models. Again, a multiobjective approach would benefit the domain expert by giving information about output correlation and enabling automatic model type selection for each output dynamically. With this paper the authors attempt to increase awareness of the subtleties involved and discuss a number of solutions and applications. In particular, we present a multiobjective framework for global surrogate model generation to help tackle both problems and that is applicable in both the static and sequential design (adaptive sampling) case.     

A hybrid differential evolution approach based on surrogate modelling for scheduling bottleneck stages

Surrogate modelling based optimization has attracted much attention due to its ability of solving expensive-to-evaluate optimization problems, and a large majority of successful applications from various fields have been reported in literature. However, little effort has been devoted to solve scheduling problems through surrogate modelling, since evaluation for a given complete schedule of these complex problems is computationally cheap in most cases. In this paper, we develop a hybrid approach for solving the bottleneck stage scheduling problem (BSP) using the surrogate modelling technique. In our approach, we firstly transform the original problem into an expensive-to-evaluate optimization problem by cutting the original schedule into two partial schedules using decomposition, then a surrogate model is introduced to, quickly but crudely, evaluate a given partial schedule. Based on the surrogate model, we propose a differential evolution (DE) algorithm for solving BSPs in which a novel mechanism is developed to efficiently utilize the advantage of the surrogate model to enhance the performance of DE. Also, an improved adaptive proximity-based method is introduced to balance the exploration and exploitation during the evolutionary process of DE. Considering that data for training the surrogate model is generated at different iteration of DE, we adopt an incremental extreme learning machine as the surrogate model to reduce the computational cost while preserving good generalization performance. Extensive computational experiments demonstrate that significant improvements have been obtained by the proposed surrogate-modelling based approach.     

Efficient use of iterative solvers in nested topology optimization

In the nested approach to structural optimization, most of the computational effort is invested in the solution of the analysis equations. In this study, it is suggested to reduce this computational cost by using an approximation to the solution of the analysis problem, generated by a Krylov subspace iterative solver. By choosing convergence criteria for the iterative solver that are strongly related to the optimization objective and to the design sensitivities, it is possible to terminate the iterative solution of the nested equations earlier compared to traditional convergence measures. The approximation is computationally shown to be sufficiently accurate for the purpose of optimization though the nested equation system is not necessarily solved accurately. The approach is tested on several large-scale topology optimization problems, including minimum compliance problems and compliant mechanism design problems. The optimized designs are practically identical while the time spent on the analysis is reduced significantly.     

An algorithm for the biobjective integer minimum cost flow problem

In this paper, we study the single commodity flow problems, optimizing two objectives simultaneously, where the flow values must be integer values. We propose a method that finds all the efficient integer points in the objective space. Our algorithm performs two phases. In the first phase, all integer points on the efficient boundary are found and in the second phase, the efficient integer points that do not lie on the efficient boundary are calculated. In addition, we carry out a computational experiment showing that the number of efficient integer solutions that do not lie on the efficient boundary is greater than the number of integer solutions on the efficient boundary.Scope and purposeIn many combinatorial optimization problems, the selection of the optimum solution takes into account more than one criterion. For example, in transportation problems or in network flows problems, the criteria that can be considered are the minimization of the cost for selected routes, the minimization of arrival times at the destinations, the minimization of the deterioration of goods, the minimization of the load capacity that would not be used in the selected vehicles, the maximization of safety, reliability, etc. Often, these criteria are in conflict and for this reason, a multiobjective network flow formulation of the problem is necessary. The solution to this problem is searched for among the set of efficient points. Although multiobjective network flow problems can be solved using the techniques available for the multiobjective linear programming problem, network-based methods are computationally better. The multicriteria minimum cost flow problem has already merited the attention of several authors and the case which has been considered in literature is that which has two objectives, where the continuous flow values are permissible. However, the integer case of the biobjective minimum cost flow problem has scarcely been studied. Whereas, in many real network flow problems, integer values on flow values are required. In this paper, we propose an approach to solve the biobjective integer minimum cost flow problem. An algorithm to obtain all efficient integer solutions of this problem is introduced. This method is characterized by the use of the classic resolution tools of network flow problems, such as the network simplex method. It does not utilize the biobjective integer linear programming methodology. Furthermore, the method does not calculate dominated solutions, so it is not necessary to incorporate tools to eliminate dominated solutions.     

Modified particle swarm optimization for a multimodal mixed-variable laser peening process

Optimization problems that result in shock, impact, and explosion type disciplines typically have mixed design variables, multiple optimal solutions, and high computational cost of an analysis. In the optimization literature, many researchers have solved problems involving mixed variables or multiple optima, but it is difficult to find multiple optima of a mixed-variable and high computation cost problem using an particle swarm optimization (PSO). To solve such problems, a mixed-variable niching PSO (MNPSO) is developed. The four modifications introduced to the PSO are: Latin Hypercube sampling-based particle generation, a mixed-variable handling technique, a niching technique, and surrogate model-based design space localization. The proposed method is demonstrated on the laser peening (LP) problem. The LP process induces favorable residual stress on the peened surface to improve the fatigue and fretting properties of the material. In many applications of LP, geometric configurations and dimensional integrity requirements of the component can constrain implementation of an optimal solution. In such cases, it is necessary to provide multiple alternatives to the designer so that a suitable one can be selected according to the requirements. It takes 24–72 CPU hours to perform an LP finite element analysis.     

Reliability-based robust assessment for multiobjective optimization design of improving occupant restraint system performance

Optimal performance of vehicle occupant restraint system (ORS) requires an accurate assessment of occupant injury values including head, neck and chest responses, etc. To provide a feasible framework for incorporating occupant injury characteristics into the ORS design schemes, this paper presents a reliability-based robust approach for the development of the ORS. The uncertainties of design variables are addressed and the general formulations of reliable and robust design are given in the optimization process. The ORS optimization is a highly nonlinear and large scale problem. In order to save the computational cost, an optimal sampling strategy is applied to generate sample points at the stage of design of experiment (DOE). Further, to efficiently obtain a robust approximation, the support vector regression (SVR) is suggested to construct the surrogate model in the vehicle ORS design process. The multiobjective particle swarm optimization (MPSO) algorithm is used for obtaining the Pareto optimal set with emphasis on resolving conflicting requirements from some of the objectives and the Monte Carlo simulation (MCS) method is applied to perform the reliability and robustness analysis. The differences of three different Pareto fronts of the deterministic, reliable and robust multiobjective optimization designs are compared and analyzed in this study. Finally, the reliability-based robust optimization result is verified by using sled system test. The result shows that the proposed reliability-based robust optimization design is efficient in solving ORS design optimization problems.     

混合个体选择机制的多目标进化算法

在多目标进化算法中,如何从后代候选集中选择最优解,显著地影响优化过程.当前,最优解的选择方式主要是基于实际目标值或者代理模型估计目标值.然而,这些选择方式往往是非常耗时或者存在精度差等问题,特别是对于一些实际的复杂优化问题.最近,一些研究人员开始利用有监督分类辅助后代选择,但是这些工作难以准备准确的正例和负例样本,或者存在耗时的参数调整等问题.为了解决这些问题,提出了一种新颖的融合分类与代理的混合个体选择机制,用于从后代候选集中选择最优解.在每一代优化中,首先利用分类器选择优良解;然后设计了一个轻量级的代理模型用于估计优良解的目标值;最后利用这些目标值对优良解进行排序,并选择最优解作为后代解.基于典型的多目标进化算法MOEA/D,利用混合个体选择机制设计了新的算法框架MOEA/D-CS.与当前流行的基于分解多目标进化算法比较,实验结果表明,所提出的算法取得了最好的性能.     

Weld sequence optimization: The use of surrogate models for solving sequential combinatorial problems

The solution of combinatorial optimization problems usually involves the consideration of many possible design configurations. This often makes such approaches computationally expensive, especially when dealing with complex finite element models. Here a surrogate model is proposed that can be used to reduce substantially the computational expense of sequential combinatorial finite element problems. The model is illustrated by application to a weld path planning problem.     

Branch and bound method for multiobjective pairing selection

Most of the available methods for selection of input-output pairings for decentralized control require evaluation of all alternatives to find the optimal pairings. As the number of alternatives grows rapidly with process dimensions, pairing selection through an exhaustive search can be computationally forbidding for large-scale processes. Furthermore, the different criteria can be conflicting necessitating pairing selection in a multiobjective optimization framework. In this paper, an efficient branch and bound (BAB) method for multiobjective pairing selection is proposed. The proposed BAB method is illustrated through a biobjective pairing problem using selection criteria involving the relative gain array and the μ-interaction measure. The computational efficiency of the proposed method is demonstrated by using randomly generated matrices and the large-scale case study of cross-direction control.     

A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.     

一种基于自适应代理模型的并行贝叶斯优化方法

提出一种基于自适应代理模型的并行贝叶斯优化方法,用于求解计算成本高的复杂优化问题.该方法基于多点期望改进判据,通过批次采样实现并行优化.针对并行优化产生的大量历史数据会导致全局代理模型建模成本高的问题,提出一种改进的基于数据并行的高斯过程建模方法,在线构造局部代理模型.此外,针对多点期望改进判据计算成本高的问题,提出一种启发式的分层优化策略,通过序贯优化基于自适应代理模型的单点期望改进判据,近似计算多点期望改进判据.最后通过5个测试问题验证所提出方法的有效性.     

Accelerated multiobjective design of miniaturized microwave components by means of nested kriging surrogates

Design of microwave components is an inherently multiobjective task. Often, the objectives are at least partially conflicting and the designer has to work out a suitable compromise. In practice, generating the best possible trade‐off designs requires multiobjective optimization, which is a computationally demanding task. If the structure of interest is evaluated through full‐wave electromagnetic (EM) analysis, the employment of widely used population‐based metaheuristics algorithms may become prohibitive in computational terms. This is a common situation for miniaturized components, where considerable cross‐coupling effects make traditional representations (eg, network equivalents) grossly inaccurate. This article presents a framework for accelerated EM‐driven multiobjective design of compact microwave devices. It adopts a recently reported nested kriging methodology to identify the parameter space region containing the Pareto front and to render a fast surrogate, subsequently used to find the first approximation of the Pareto set. The final trade‐off designs are produced in a separate, surrogate‐assisted refinement process. Our approach is demonstrated using a three‐section impedance matching transformer designed for the best matching and the minimum footprint area. The Pareto set is generated at the cost of only a few hundred of high‐fidelity EM simulations of the transformer circuit despite a large number of geometry parameters involved.     

Ant-Tree: an ant colony optimization approach to the generalized minimum spanning tree problem

The ant colony optimization is a meta-heuristic inspired by knowledge sharing amongst ants using pheromone, which serves as a kind of collective memory. Since the past few years, there have been several successful applications of this new approach for finding approximate solutions for computationally difficult problems in reasonable times. In this paper, we study the generalized minimum spanning tree problem that involves the design of a minimum weight connected network spanning at least one node out of every disjoint subset of the nodes in a graph. This problem has a wealth of pertinence to a wide range of applications in different areas. As the problem is known as computationally challenging, we adopt the ant colony optimization strategy and present a new solution method, called Ant-Tree, to develop approximate solutions. As an initial attempt, our study aims to provide an investigation of the ant colony optimization approach for coping with tree optimization problems. Through computational experiments, we compare the performances of our approach and the method available in the literature. Numerical results indicate that the proposed method is effective in producing quality approximate solutions.     

An Evolutionary Algorithm-Based Approach to Automated Design of Analog and RF Circuits Using Adaptive Normalized Cost Functions

Typical analog and radio frequency (RF) circuit sizing optimization problems are computationally hard and require the handling of several conflicting cost criteria. Many researchers have used sequential stochastic refinement methods to solve them, where the different cost criteria can either be combined into a single-objective function to find a unique solution, or they can be handled by multiobjective optimization methods to produce tradeoff solutions on the Pareto front. This paper presents a method for solving the problem by the former approach. We propose a systematic method for incorporating the tradeoff wisdom inspired by the circuit domain knowledge in the formulation of the composite cost function. Key issues have been identified and the problem has been divided into two parts: a) normalization of objective functions and b) assignment of weights to objectives in the cost function. A nonlinear, parameterized normalization strategy has been proposed and has been shown to be better than traditional linear normalization functions. Further, the designers' problem specific knowledge is assembled in the form of a partially ordered set, which is used to construct a hierarchical cost graph for the problem. The scalar cost function is calculated based on this graph. Adaptive mechanisms have been introduced to dynamically change the structure of the graph to improve the chances of reaching the near-optimal solution. A correlated double sampling offset-compensated switched capacitor analog integrator circuit and an RF low-noise amplifier in an industry-standard 0.18mum CMOS technology have been chosen for experimental study. Optimization results have been shown for both the traditional and the proposed methods. The results show significant improvement in both the chosen design problems     

Performance assessment of a multi-objective parametric optimization algorithm with application to a multi-physical engineering system

The original problem of reliability-based design optimization (RBDO) is mathematically a nested two-level structure that is computationally time consuming for real engineering problems. In order to overcome the computational difficulties, many formulations have been proposed in the literature. These include SORA (sequential optimization and reliability assessment) that decouples the nested problems. SLA (single loop approach) further improves efficiency in that reliability analysis becomes an integrated part of the optimization problem. However, even SLA method can become computationally challenging for real engineering problems involving many reliability constraints. This paper presents an enhanced version of SLA where the first phase is based on approximation at nominal design point. After convergence of first iterative phase is reached the process transitions to a second phase where approximations of reliability constraints are carried out at their respective minimum performance target point (MPTP). The solution is implemented in Altair OptiStruct, where adaptive approximation and constraint screening strategies are utilized in the RBDO process. Examples show that the proposed two-phase approach leads to reduction in finite element analyses while preserving equal solution quality.     

A problem space genetic algorithm in multiobjective optimization

In this study, a problem space genetic algorithm (PSGA) is used to solve bicriteria tool management and scheduling problems simultaneously in flexible manufacturing systems. The PSGA is used to generate approximately efficient solutions minimizing both the manufacturing cost and total weighted tardiness. This is the first implementation of PSGA to solve a multiobjective optimization problem (MOP). In multiobjective search, the key issues are guiding the search towards the global Pareto-optimal set and maintaining diversity. A new fitness assignment method, which is used in PSGA, is proposed to find a well-diversified, uniformly distributed set of solutions that are close to the global Pareto set. The proposed fitness assignment method is a combination of a nondominated sorting based method which is most commonly used in multiobjective optimization literature and aggregation of objectives method which is popular in the operations research literature. The quality of the Pareto-optimal set is evaluated by using the performance measures developed for multiobjective optimization problems.     

A simulated annealing algorithm for prioritized multiobjective optimization--implementation in an adaptive model predictive control configuration.

This paper presents a new stochastic algorithm for solving hierarchical multiobjective optimization problems. The algorithm is based on the simulated annealing concept and returns a single solution that corresponds to the lexicographic ordering approach. The algorithm optimizes simultaneously the multiple objectives by assigning a different initial temperature to each one, according to its position in the hierarchy. A major advantage of the proposed method is its low computational cost. This is very critical, particularly, for online applications, where the time that is available for decision making is limited. The method is tested in a number of benchmark problems, which illustrate its ability to find near-optimal solutions even in nonconvex multiobjective optimization problems. The results are comparable with those that are produced by state-of-the-art multiobjective evolutionary algorithms, such as the Nondominated Sorting Genetic Algorithm II. The algorithm is further applied to the solution of a large-scale problem that is formulated online, when a multiobjective adaptive model predictive control (MPC) configuration is adopted. This particular control scheme involves an adaptive discrete-time model of the system, which is developed using the radial-basis-function neural-network architecture. A key issue in the success of the adaptation strategy is the introduction of a persistent excitation constraint, which is transformed to a top-priority objective. The overall methodology is applied to the control problem of a pH reactor and proves to be superior to conventional MPC configurations.     

A Simulated Annealing Algorithm for Prioritized Multiobjective Optimization—Implementation in an Adaptive Model Predictive Control Configuration

This paper presents a new stochastic algorithm for solving hierarchical multiobjective optimization problems. The algorithm is based on the simulated annealing concept and returns a single solution that corresponds to the lexicographic ordering approach. The algorithm optimizes simultaneously the multiple objectives by assigning a different initial temperature to each one, according to its position in the hierarchy. A major advantage of the proposed method is its low computational cost. This is very critical, particularly, for online applications, where the time that is available for decision making is limited. The method is tested in a number of benchmark problems, which illustrate its ability to find near-optimal solutions even in nonconvex multiobjective optimization problems. The results are comparable with those that are produced by state-of-the-art multiobjective evolutionary algorithms, such as the nondominated sorting genetic algorithm II. The algorithm is further applied to the solution of a large-scale problem that is formulated online, when a multiobjective adaptive model predictive control (MPC) configuration is adopted. This particular control scheme involves an adaptive discrete-time model of the system, which is developed using the radial-basis-function neural-network architecture. A key issue in the success of the adaptation strategy is the introduction of a persistent excitation constraint, which is transformed to a top-priority objective. The overall methodology is applied to the control problem of a pH reactor and proves to be superior to conventional MPC configurations.     

DS-DPSO: A dual surrogate approach for intelligent watermarking of bi-tonal document image streams

Intelligent watermarking (IW) techniques employ population-based evolutionary computing in order to optimize embedding parameters that trade-off between watermark robustness and image quality for digital watermarking systems. Recent advances indicate that it is possible to decrease the computational burden of IW techniques in scenarios involving long heterogeneous streams of bi-tonal document images by recalling embedding parameters (solutions) from a memory based on Gaussian Mixture Model (GMM) representation of optimization problems. This representation can provide ready-to-use solutions for similar optimization problem instances, avoiding the need for a costly re-optimization process. In this paper, a dual surrogate dynamic Particle Swarm Optimization (DS-DPSO) approach is proposed which employs a memory of GMMs in regression mode in order to decrease the cost of re-optimization for heterogeneous bi-tonal image streams. This approach is applied within a four level search for near-optimal solutions, with increasing computational burden and precision. Following previous research, the first two levels use GMM re-sampling to recall solutions for recurring problems, allowing to manage streams of heterogeneous images. Then, if embedding parameters of an image require a significant adaptation, the third level is activated. This optimization level relies on an off-line surrogate, using Gaussian Mixture Regression (GMR), in order to replace costly fitness evaluations during optimization. The final level also performs optimization, but GMR is employed as a costlier on-line surrogate in a worst-case scenario and provides a safeguard to the IW system. Experimental validation were performed on the OULU image data set, featuring heterogeneous image streams with a varying levels of attacks. In this scenario, the DS-DPSO approach has been shown to provide comparable level of watermarking performance with a 93% decline in computational cost compared to full re-optimization. Indeed, when significant parameter adaptation is required, fitness evaluations may be replaced with GMR.     

Ant colony approaches for multiobjective structural optimization problems with a cardinality constraint

Two Ant Colony Optimization algorithms are proposed to tackle multiobjective structural optimization problems with an additional constraint. A cardinality constraint is introduced in order to limit the number of distinct values of the design variables appearing in any candidate solution. Such constraint is directly enforced when an ant builds a candidate solution, while the other mechanical constraints are handled by means of an adaptive penalty method (APM). The test-problems are composed by structural optimization problems with discrete design variables, and the objectives are to minimize both the structure’s weight and its maximum nodal displacement. The Pareto sets generated in the computational experiments are evaluated by means of performance metrics, and the obtained designs are also compared with solutions available from single-objective studies in the literature.