一种分布式动态负载平衡算法

在多处理机系统中,负载平衡是提高并行处理效率的一条重要途径。基于分布存贮的TRANSCUBE多处理机环境,本文提出一种分布式动态负载平衡算法。算法采用接收者开始的异步调度策略,通过“握手”协议在空载和重载处理机间建立联系,并自动实现任务(或进程)从重载处理机到空载处理机的迁移,该算法适于并行解具有动态特性的应用问题,而且在问题规模较大和处理机负载变化较慢时,性能较好。     

A decomposition algorithm for optimal static load balancing in treehierarchy network configurations

We study the static load balancing problem in a distributed computer system with the tree hierarchy configuration. It is formulated as a nonlinear optimization problem. After studying the conditions that the solution to the optimization problem of the tree hierarchy network satisfies, we demonstrate that the special structure of the optimization problem leads to an interesting decomposition technique. A new effective decomposition algorithm to solve the optimization problem is presented. The proposed algorithm Is compared with two other well known algorithms: the Flow Deviation (FD) algorithm and the Dafermos-Sparrow (D-S) algorithm. It is shown that the amounts of the storage required for the proposed algorithm and the FD algorithm are O(n) for load balancing of an n-node system. However, the amount of the storage required for the D-S algorithm is O(n log(n)). By using numerical experiments, we show that both the proposed algorithm and the D-S algorithm have much faster convergence in terms of central processing unit (CPU) time than the FD algorithm     

基于消息传递的数据交错重分布负载平衡技术

数据重分布是实现消息传递环境下负载平衡的重要手段,提出了数据交错分布的模型问题及模型问题的并行计算模型,分析了模型问题在消息传递环境下的实现,讨论了性能和适用条件,给出了分析结果,讨论了通信与计算的时间重叠问题,将数据交错重分布负载平衡技术应用到非平衡刚性动力学方程组的并行计算中,获得了很好的负载平衡效果。     

The Double Edge Sword Based Distributed Executor Service

Scalability is one of the most important quality attribute of software-intensive systems, because it maintains an effective performance parallel to the large fluctuating and sometimes unpredictable workload. In order to achieve scalability, thread pool system (TPS) (which is also known as executor service) has been used extensively as a middleware service in software-intensive systems. TPS optimization is a challenging problem that determines the optimal size of thread pool dynamically on runtime. In case of distributed-TPS (DTPS), another issue is the load balancing b/w available set of TPSs running at backend servers. Existing DTPSs are overloaded either due to an inappropriate TPS optimization strategy at backend servers or improper load balancing scheme that cannot quickly recover an overload. Consequently, the performance of software-intensive system is suffered. Thus, in this paper, we propose a new DTPS that follows the collaborative round robin load balancing that has the effect of a double-edge sword. On the one hand, it effectively performs the load balancing (in case of overload situation) among available TPSs by a fast overload recovery procedure that decelerates the load on the overloaded TPSs up to their capacities and shifts the remaining load towards other gracefully running TPSs. And on the other hand, its robust load deceleration technique which is applied to an overloaded TPS sets an appropriate upper bound of thread pool size, because the pool size in each TPS is kept equal to the request rate on it, hence dynamically optimizes TPS. We evaluated the results of the proposed system against state of the art DTPSs by a client-server based simulator and found that our system outperformed by sustaining smaller response times.     

Considering data skew factor in multi-way join query optimization for parallel execution

A consensus on parallel architecture for very large database management has emerged. This architecture is based on a shared-nothing hardware organization. The computation model is very sensitive to skew in tuple distribution, however. Recently, several parallel join algorithms with dynamic load balancing capabilities have been proposed to address this issue, but none of them consider multi-way join problems. In this article we propose a dynamic load balancing technique for multi-way joins, and investigate the effect of load balancing on query optimization. In particular, we present a join-ordering strategy that takes load-balancing issues into consideration. Our performance study indicates that the proposed query optimization technique can provide very impressive performance improvement over conventional approaches.An earlier version of this article was presented at the 1993 International Conference on Parallel and Distributed Information Systems in San Diego, California, U.S.A.     

Heterogeneous pipelined square-root Kalman Filter algorithm for the MMSE-OSIC problem

This paper describes a pipelined parallel algorithm for the MMSE-OSIC decoding procedure proposed in V-BLAST wireless MIMO systems, for heterogeneous networks of processors. It is based on a block version of the square-root Kalman Filter algorithm that was initially devised to solve the RLS problem. It has been parallelized in a pipelined way obtaining a good efficiency and scalability. The optimum load balancing for this parallel algorithm is dynamic, but we derive a static load balancing scheme with good performance.     

基于禁忌搜索的流式计算平台负载均衡策略

针对大数据流式计算平台原生的调度机制存在计算负载分配不均衡、资源利用率低的问题,提出异构环境下基于禁忌搜索算法的负载均衡策略,并将其应用于Apache Flink平台。首先,通过构建作业拓扑模型将流式计算作业的拓扑结构抽象为有向无环图（directed acyclic graph,DAG）,并将每个任务槽（task slot）抽象为节点,为计算节点的性能评估奠定基础;其次,通过建立性能评估模型将有向无环图中带性能权值的节点导入性能评估模型,进行归一化处理得到节点性能的优劣;再将评估参数传入禁忌调度算法（tabu search for schedule,TBS）进行作业路径优化,从而得出最优作业路径;最后,使用Flink平台提供的CustomPatitionerWrapper接口将数据分配到最优作业路径包含的节点中,完成计算负载的均衡分配,从而提升Flink平台的整体性能。实验结果表明：通过禁忌调度算法优化后的负载均衡策略与原生的Flink平台相比,平均计算延迟降低了10~20 ms,资源利用率显著提高,平均吞吐量提升约15%,有效证明了负载均衡策略的有效性和优化效果。     

Load Balancing Parallel Explicit State Model Checking

This paper first identifies some of the key concerns about the techniques and algorithms developed for parallel model checking; specifically, the inherent problem with load balancing and large queue sizes resultant in a static partition algorithm. This paper then presents a load balancing algorithm to improve the run time performance in distributed model checking, reduce maximum queue size, and reduce the number of states expanded before error discovery. The load balancing algorithm is based on generalized dimension exchange (GDE). This paper presents an empirical analysis of the GDE based load balancing algorithm on three different supercomputing architectures—distributed memory clusters, Networks of Workstations (NOW) and shared memory machines. The analysis shows increased speedup, lower maximum queue sizes and fewer total states explored before error discovery on each of the architectures. Finally, this paper presents a study of the communication overhead incurred by using the load balancing algorithm, which although significant, does not offset performance gains.     

基于渗透式人工蜂群与蚁群优化混合的负载平衡算法

针对当前云计算负载平衡调度过程中出现的虚拟机迁移效率低和能耗高问题,提出了一种基于渗透式人工蜂群与蚁群混合优化负载平衡算法,该算法将化学渗透行为与生物启发的负载平衡算法相结合,在充分利用人工蜂群和蚁群两种优化算法优点的同时,将渗透技术应用于负载均衡。由于渗透技术支持通过云基础设施迁移的虚拟机的自动部署,从而克服了现有仿生算法在实现物理机之间负载平衡方面的缺点,提高了迁移效率。实验结果表明,以现有负载平衡算法相比,提出的算法在迁移性能上提升明显。     

Distributing and Load Balancing Sparse Fluid Simulations

This paper describes a general algorithm and a system for load balancing sparse fluid simulations. Automatically distributing sparse fluid simulations efficiently is challenging because the computational load varies across the simulation domain and time. A key challenge with load balancing is that optimal decision making requires knowing the fluid distribution across partitions for future time steps, but computing this state for an arbitrary simulation requires running the simulation itself. The key insight of this paper is that it is possible to predict future load by running a speculative low resolution simulation in parallel. We mathematically formulate the problem of load balancing over multiple time steps and present a polynomial time algorithm to compute an approximate solution to it. Our experimental results show that distributing and speculatively load balancing sparse FLIP simulations over 8 nodes speeds them up by 5.3× to 7.9×, and that speculative load balancing generates assignments that perform within 20% of optimal.     

并行演化算法在MEMS继电器参数优化中的应用

使用演化算法求解MEMS继电器参数优化主要瓶颈在于算法运行时间过长,而算法运行时间过长主要由于电磁仿真软件进行建模和分析需要耗费大量的计算时间。针对该问题,采用主从并行模式,对演化算法个体适应值计算阶段并行化处理。在充分考虑计算机资源的使用效率与负载均衡等因素下,使服务器尽量少地参与任务计算及减少与客户机的通信以增强并行模式的分布能力,并且增加了客户端掉线处理,任务重分配等操作以增强并行模式的容错能力。经过测试,该并行演化算法在MEMS微波继电器参数优化上加速比接近线速,具有良好的并行效率且容错性较高。     

网络环境中FFT算法的优化设计

以并行FFT算法为例,研究微机构成的网络计算平台上的并行程序设计与优化问题。由于整个系统的性能取决于网络消息通讯,为减少实施负载平衡策略本身所带来的通信开销,在进行数据分配时尽可能使计算数据局部化。由于异构性,数据分配还应考虑节点机的性能。     

面向并行负载平衡的数据剖分技术*

对传统的数据剖分技术和负载平衡对大规模并行计算性能的影响进行了综述,介绍了目前典型的几何剖分方法和图剖分方法的特点,并分析比较各种剖分算法及常用剖分软件包(ParMETIS、Zoltan、JOSTLE等)在实际应用中的优缺点,深入探讨了数据剖分技术是如何对超大规模数值模拟计算任务进行高效划分以解决负载平衡问题的,以期为开展并行计算研究和并行性能优化的研究人员提供参考。     

Parallel extremal optimization in processor load balancing for distributed applications

The paper concerns parallel methods for extremal optimization (EO) applied in processor load balancing in execution of distributed programs. In these methods EO algorithms detect an optimized strategy of tasks migration leading to reduction of program execution time. We use an improved EO algorithm with guided state changes (EO-GS) that provides parallel search for next solution state during solution improvement based on some knowledge of the problem. The search is based on two-step stochastic selection using two fitness functions which account for computation and communication assessment of migration targets. Based on the improved EO-GS approach we propose and evaluate several versions of the parallelization methods of EO algorithms in the context of processor load balancing. Some of them use the crossover operation known in genetic algorithms. The quality of the proposed algorithms is evaluated by experiments with simulated load balancing in execution of distributed programs represented as macro data flow graphs. Load balancing based on so parallelized improved EO provides better convergence of the algorithm, smaller number of task migrations to be done and reduced execution time of applications.     

Agent scheduling model for adaptive dynamic load balancing in agent-based distributed simulations

Agent-based distributed simulations are confronted with load imbalance problem, which significantly affects simulation performance. Dynamic load balancing can be effective in decreasing simulation execution time and improving simulation performance. The characteristics of multi-agent systems and time synchronization mechanisms make the traditional dynamic load balancing approaches not suitable for dynamic load balancing in agent-based distributed simulations. In this paper, an adaptive dynamic load balancing model in agent-based distributed simulations is proposed. Due to the complexity and huge time consuming for solving the model, a distributed approximate optimized scheduling algorithm with partial information (DAOSAPI) is proposed. It integrates the distributed mode, approximate optimization and agent set scheduling approach. Finally, experiments are conducted to verify the efficiency of the proposed algorithm and the simulation performance under dynamic agent scheduling. The experiments indicate that DAOSPI has the advantage of short execution time in large-scale agent scheduling, and the distributed simulation performance under this dynamic agent scheduling outperforms that under static random agent distribution.     

基于任务的Cholesky分解多核并行化研究

针对Cholesky分解算法采用OpenMP并行程序设计时的并行性开销增大和线程负载不平衡的问题,利用并行性能分析工具对串行程序进行热点分析,提出了一种基于任务的Cholesky分解多核并行算法。该算法将大循环问题划分成各个相互独立的小任务,并运用任务窃取技术和动态负载均衡算法使多个任务能够并行完成。采用ParallelAmplifier对并行程序进行调试和优化,实验结果表明,其性能得到较大幅度的提升。     

数据驱动的多星任务网络预测调度算法

多星任务调度是具有NP-hard特性的优化问题,随着卫星资源规模和任务需求规模的双重增长,传统调度方法求解效率不高.在轨卫星在常年运行过程中积累了丰富的调度数据.针对大规模多星任务调度场景,建立多星多波束任务调度模型,并提出数据驱动的多星任务网络预测调度算法对其求解.以分割的思想,实现多星场景下任务可调度性预测.从历史调度数据中,提取设定的3个静态特征和5个动态特征,构建并训练预测网络,预测任务被不同卫星完成的概率,并以冲突避免、负载均衡等为原则,得到初始任务和资源卫星的分配方案.进一步设计双链结构的进化算法,以双链编码形式表征上述关系,配合设计的交叉、修复等进化算子,优化初始方案中的任务序列与资源分配关系,输出最终任务调度方案.仿真结果表明,与改进蚁群算法、混合遗传算法和数据驱动并行调度算法相比,所提出算法在运行时间、方案收益和卫星负载均衡3方面均有较好的表现.     

Scalable distributed Louvain algorithm for community detection in large graphs

Community detection (or clustering) in large-scale graphs is an important problem in graph mining. Communities reveal interesting organizational and functional characteristics of a network. Louvain algorithm is an efficient sequential algorithm for community detection. However, such sequential algorithms fail to scale for emerging large-scale data. Scalable parallel algorithms are necessary to process large graph datasets. In this work, we show a comparative analysis of our different parallel implementations of Louvain algorithm. We design parallel algorithms for Louvain method in shared memory and distributed memory settings. Developing distributed memory parallel algorithms is challenging because of inter-process communication and load balancing issues. We incorporate dynamic load balancing in our final algorithm DPLAL (Distributed Parallel Louvain Algorithm with Load-balancing). DPLAL overcomes the performance bottleneck of the previous algorithms and shows around 12-fold speedup scaling to a larger number of processors. We also compare the performance of our algorithm with some other prominent algorithms in the literature and get better or comparable performance . We identify the challenges in developing distributed memory algorithm and provide an optimized solution DPLAL showing performance analysis of the algorithm on large-scale real-world networks from different domains.

     

LU分解在Godson-Tvl众核体系结构上的半行化研究

随着集成电路工艺的发展,众核体系结构成为人们日益关注的计算平台.LU分解是科学和工程计算中被广泛使用的核心算法之一,尽管在传统的并行体系结构上已有大量的并行化研究工作,但是结合新犁众核体系结构特征的工作还不多.文章从负载均衡、延迟容忍和性能分析模型3个方面系统研究了LU分解在众核体系结构上的并行化问题.该文的贡献在于:首先,针对二维卷帘负载分配方案难以达到良好负载均衡的缺点,提出一种新的"之"字形分配方案,实验表明不经任何优化的情况下性能比前者提高20%,优化后达到了40%;其次,提出了一个性能加速比的分析模型,并用实验定量研究了实测性能加速比和理论值之间的差距,发现在合理利用片上存储优化访存延迟,并恰当选择矩阵分块参数的情况下,实测加速效果能比较接近理论值;通过实验还证明实测性能难以达到理论预测值的两个主要原因:访存带宽有限和片上网络的资源竞争.     

Optimal Static Load Balancing in Star Network Configurations with Two-Way Traffic

We study the static load balancing problem in a distributed computer system that consists of a set of heterogeneous computer systems interconnected by a star network with two-way traffic. We formulate the static load balancing problem as a nonlinear optimization problem which minimizes the mean response time. We prove that in the optimal solution the satellite nodes in the star network can be divided into four different types: the idle source nodes, the active source nodes, the neutral nodes, and the sink nodes. The necessary and sufficient conditions for optimal solution are studied. An efficient algorithm of complexity O(n) is proposed for the static load balancing of an n-satellite system. The effects of link communication time on optimal load balancing in a star network are also studied by parametric analysis. By employing the proposed algorithm, a significant system performance improvement over that without load balancing is illustrated in a numerical example. The numerical example also shows that the effects of the link communication time in a star network are large.