基于资源域的分级调度模型

作业调度是要实现作业和资源的最佳匹配,协调对资源的征用矛盾.软件环境和硬件资源对作业运行都有影响,实验证明硬件资源对作业的影响很大,据此提出了基于资源域的分级调度模型,提供全局和局部两种调度方式.节点根据配置划分成不同的资源域,调度时根据作业类型匹配最佳的资源域,该模型有效利用了集群资源,减轻了Server的负担,并能提高系统的可扩展性.在全局调度中,还提供了预约机制调度高优先级作业.     

分布式并行服务器透明性及任务调度研究

针对当前多服务器系统透明性和任务调度研究中存在的问题，提出一种分布式并行服务器的网络服务透明性实现机制和相应的任务调度算法。该透明性机制修改服务器结点的ARP地址解析协议以及客户端到服务器端的连接和数据请求处理，使得整个服务器系统对外界表现为惟一的VIP地址和VMAC地址；相应的任务调度算法则根据负载和阈值设置将服务器结点分成两个链：有效服务器结点链和过载服务器结点链，然后由量值循环法对有效服务器结点链进行任务调度，在修改Linux内核网卡驱动程序和部分底层网络协议的基础上进行了实现，测试结果表明其具有良好的响应特性和较强的服务能力。     

Performance of gang scheduling strategies in a parallel system

In this paper we study job scheduling performance in a partitionable parallel system. Jobs consist of parallel tasks scheduled to execute concurrently on processor partitions, where each task starts at the same time and computes at the same pace. The performance of different scheduling schemes is compared over various workloads. The impact of the variability of tasks service time is also studied. Various performance metrics are examined. The objective is to achieve good overall performance and also small scheduling overhead. Simulated results reveal that periodic job scheduling and also scheduling which depends on the number of job insertions in the queue can succeed these goals.     

Computation and communication schedule optimization for data-sharing tasks on uniprocessor

Almost every computation task requires input data in order to find a solution. This is not a problem for a centralized system because data is usually available locally. However, in a parallel and distributed system, e.g., computation grids, the data may be in remote sites and must be transferred to the local site before the computation can proceed. As a result, the interleaved sequence of data transfer and job execution has a significant impact on the overall computational efficiency. In this paper, we analyze the computational complexity of the shared-data job scheduling problem on uniprocessor, with and without consideration of the storage capacity constraint on the local site.We show that if there is an upper bound on the server capacity, the problem is NP-complete, even when each job depends on at most two data items. For the case where there is no upper bound on the server capacity, we show that there exists an efficient algorithm that can provide an optimal job schedule when each job depends on at most two data items. We also propose an efficient heuristic algorithm that can determine good schedules for cases where there is no limit on the amount of data a job may access. The reported experiment results demonstrate that this heuristic algorithm performs very well, and derives near optimal solutions.     

Task Scheduling on the PASM Parallel Processing System

PASM is a proposed large-scale distributed/parallel processing system which can be partitioned into independent SIMD/MIMD machines of various sizes. One design problem for systems such as PASM is task scheduling. The use of multiple FIFO queues for nonpreemptive task scheduling is described. Four multiple-queue scheduling algorithms with different placement policies are presented and applied to the PASM parallel processing system. Simulation of a queueing network model is used to compare the performance of the algorithms. Their performance is also considered in the case where there are faulty control units and processors. The multiple-queue scheduling algorithms can be adapted for inclusion in other multiple-SIMD and partitionable SIMD/MIMD systems that use similar types of interconnection networks to those being considered for PASM.     

A trace-driven simulation study of dynamic load balancing

A trace-driven simulation study of dynamic load balancing in homogeneous distributed systems supporting broadcasting is presented. Information about job CPU and input/output (I/O) demands collected from production systems is used as input to a simulation model that includes a representative CPU scheduling policy and considers the message exchange and job transfer cost explicitly. Seven load-balancing algorithms are simulated and their performances compared. Load balancing is capable of significantly reducing the mean and standard deviation of job response times, especially under heavy load, and for jobs with high resource demands. Algorithms based on periodic or nonperiodic load information exchange provide similar performance, and, among the periodic policies, the algorithms that use a distinguished agent to collect and distribute load information cut down the overhead and scale better. With initial job placements only, source initiative algorithms were found to perform better than server initiative algorithms. The performances of all hosts, even those originally with light loads, are generally improved by load balancing     

支持SRP协议的实时调度技术

实时操作系统对多任务资源访问控制提出了较高的要求.资源竞争常会引起优先级翻转问题导致任务阻塞,增加资源等待延迟时间.传统资源访问控制协议可以部分解决此问题,但是存在上下文切换次数高,任务调度效率低等不足.在结合SRP协议的提前阻塞特性和MiniCore实时调度器设计后,分析了协议中任务调度规则,指出了调度器支持SRP协议时效率低下的原因,并引入胜者树结构改进就绪队列,给出了新的任务搜索算法.理论分析与实验结果表明改进的调度器在任务集规模较大时,提高了调度效率,较好的支持了SRP协议.     

Load Balancing in a Cluster-Based Web Server for Multimedia Applications

We consider a cluster-based multimedia Web server that dynamically generates video units to satisfy the bit rate and bandwidth requirements of a variety of clients. The media server partitions the job into several tasks and schedules them on the backend computing nodes for processing. For stream-based applications, the main design criteria of the scheduling are to minimize the total processing time and maintain the order of media units for each outgoing stream. In this paper, we first design, implement, and evaluate three scheduling algorithms, first fit (FF), stream-based mapping (SM), and adaptive load sharing (ALS), for multimedia transcoding in a cluster environment. We determined that it is necessary to predict the CPU load for each multimedia task and schedule them accordingly due to the variability of the individual jobs/tasks. We, therefore, propose an online prediction algorithm that can dynamically predict the processing time per individual task (media unit). We then propose two new load scheduling algorithms, namely, prediction-based least load first (P-LLF) and prediction-based adaptive partitioning (P-AP), which can use prediction to improve the performance. The performance of the system is evaluated in terms of system throughput, out-of-order rate of outgoing media streams, and load balancing overhead through real measurements using a cluster of computers. The performance of the new load balancing algorithms is compared with all other load balancing schemes to show that P-AP greatly reduces the delay jitter and achieves high throughput for a variety of workloads in a heterogeneous cluster. It strikes a good balance between the throughput and output order of the processed media units     

耦合分布式仿真中任务调度的研究与实现

在耦合分布式仿真系统中,为解决任务对资源合理利用的问题,采用一种First—Available调度策略,即在对资源和任务有效管理的基础上,根据工况计算任务的资源请求,在资源表中选取第一个符合要求的计算节点,指定此节点运行某项工况任务。测试表明,该调度策略能够快速有效匹配任务的资源请求,有非常强的实用价值和实效性,能够有效合理地利用资源,满足了大规模多工况多任务耦合仿真计算的要求。     

Assessing situation awareness in multitasking supervisory control using success rate of self-terminating search

The purpose of this study is to investigate measuring situation awareness (SA) by using the success rate of self-terminating visual search (STVS) in a multitasking supervisory control environment. Currently, many researchers have focused on developing unobtrusive SA metrics in a multitasking environment. Although some of the advanced eye-tracking metrics can estimate the operator's SA during a task, certain methodological issues remain for measuring SA in a supervisory control and multitasking environment. In this study, participants performed a distributed control task with the multi-attribute task battery. To measure the participant's SA, situation awareness global assessment technique (SAGAT) was used during the experiment. SA performance was compared to the success rate of STVS. An eye-head integrated tracking system was used to measure the rate of STVS based on the participants' eye movements and head positions in different monitoring conditions. The results showed that task-switching activity was influenced by salient features in a display. It causes a significant difference on SA between different monitoring conditions, and the difference is also monitored in the success rate of STVS. Furthermore, the study found that the attention level in visual searching influences situational awareness in a supervisory control environment. The findings from this study suggest that the success rate of STVS could be used as a new metric for measuring SA related to the visual perception of the elements in a multitasking supervisory control environment.     

基于关联数据本地化的多密码作业流调度算法

针对云密码服务系统中服务请求多样、数据依赖性作业流与非数据依赖性作业流随机交叉并发等问题,为了避免处理节点之间关联数据的交互而带来的系统通信性能开销和数据安全性威胁,设计一种基于关联数据本地化的云密码作业流调度算法。首先通过任务请求密码功能的映射,保障多作业流请求密码功能的正确实现;然后对于具有相同请求密码功能的各任务中不同工作模式交叉问题,在提出任务优先级计算方法以促进多作业流调度公平性的基础上,采用分类调度的方法,在实现关联数据本地化的同时,保障了调度系统的整体性能。仿真结果表明,该算法不仅可以有效减少系统任务完成时间,提高资源利用率和公平性,并且具有良好的稳定性。     

Performance bounds for concurrent software with rendezvous

Synchronous message-passing communication, or rendezvous, occurring between software tasks can have a significant effect on system performance. The rendezvous style of communication is coming into wider use in programming languages and operating systems for parallel and distributed environments. Understanding the performance implications of this style of inter-task communication is becoming a matter of practical importance. The dual nature of a task which acts both like a customer as well as a server, makes the performance analysis of rendezvous-based multitasking systems quite different from the analysis of the other queueing systems with known results. This research focuses on rendezvous-based systems in which the execution behavior of the software has a nondeterministic component of a very general nature which may for example be the manifestation of a data dependent behavior. Based on a model called the Stochastic Rendezvous Network the computation of bounds on task throughputs for multitasking systems characterized by rendezvous style communication is presented. Although the behavior of tasks is called stochastic, it is very general and the results are valid for general distributions of computation times and the number of messages generated by tasks. The inter-relationship among task throughputs, however, makes it difficult to extract the bounds in closed analytic form. The notion of a feasible throughput region which encloses the set of feasible tasks throughputs and captures the inter-relationship among the behavior of tasks is introduced. Variations of this basic bounding approach that are useful in the context of different types of multitasking systems are considered in the article. For example, a novel technique based on interval arithmetic is proposed for the computation of numerical values for the bounds. The applicability of the bounds and their tightness are analyzed through case studies. Issues such as the inter-relationship between the software architecture and system performance, and the effect of processor contention on the performance bounds are discussed.     

一种异构环境下的基于MapReduce任务调度改进机制

针对在异构环境下采用现有MapReduce任务调度机制可能出现各计算节点间数据迁移和系统资源分配难以管理的问题, 提出一种动态的任务调度机制来改善这些问题。该机制先根据节点的计算能力按比例放置数据, 然后通过资源预测方法估计异构环境下MapReduce任务的完成时间, 并根据完成时间计算任务所需的资源。实验结果表明, 该机制提高了异构环境下任务的数据本地性比例, 且能动态地调整资源分配, 以保证任务在规定时间内完成, 是一种有效可行的任务调度机制。     

M/M/1-PS queue and size-aware task assignment

We consider a distributed server system in which heterogeneous servers operate under the processor sharing (PS) discipline. Exponentially distributed jobs arrive to a dispatcher, which assigns each task to one of the servers. In the so-called size-aware system, the dispatcher is assumed to know the remaining service requirements of some or all of the existing jobs in each server. The aim is to minimize the mean sojourn time, i.e., the mean response time. To this end, we first analyze an M/M/1-PS queue in the framework of Markov decision processes, and derive the so-called size-aware relative value of state, which sums up the deviation from the average rate at which sojourn times are accumulated in the infinite time horizon. This task turns out to be non-trivial. The exact analysis yields an infinite system of first order differential equations, for which an explicit solution is derived. The relative values are then utilized to develop efficient dispatching policies by means of the first policy iteration (FPI). Numerically, we show that for the exponentially distributed job sizes the myopic approach, ignoring the future arrivals, yields an efficient and robust policy when compared to other heuristics. However, in the case of highly asymmetric service rates, an FPI based policy outperforms it. Additionally, the size-aware relative value of an M/G/1-PS queue is shown to be sensitive with respect to the form of job size distribution, and indeed, the numerical experiments with constant job sizes confirm that the optimal decision depends on the job size distribution.     

分布式环境中的多作业执行调度策略与优化

分布式大数据计算引擎是科研机构、互联网企业和政府部门处理大规模数据必不可少的工具,它们的使用和推广促进了各个领域的快速发展,为社会进步做出了巨大贡献.但是,在多作业处理的情况下,目前主流的大数据计算引擎在资源分配和作业调度方面仍有许多不足之处,它们通常对多作业平均划分内存资源并以先进先出FIFO的方式调度作业,这样简单...     

DAGMap: efficient and dependable scheduling of DAG workflow job in Grid

DAG has been extensively used in Grid workflow modeling. Since Grid resources tend to be heterogeneous and dynamic, efficient and dependable workflow job scheduling becomes essential. It poses great challenges to achieve minimum job accomplishing time and high resource utilization efficiency, while providing fault tolerance. Based on list scheduling and group scheduling, in this paper, we propose a novel scheduling heuristic called DAGMap. DAGMap consists of two phases, namely Static Mapping and Dependable Execution. Four salient features of DAGMap are: (1) Task grouping is based on dependency relationships and task upward priority; (2) Critical tasks are scheduled first; (3) Min-Min and Max-Min selective scheduling are used for independent tasks; and (4) Checkpoint server with cooperative checkpointing is designed for dependable execution. The experimental results show that DAGMap can achieve better performance than other previous algorithms in terms of speedup, efficiency, and dependability.     

Scheduling multiple task graphs with end-to-end deadlines in distributed real-time systems utilizing imprecise computations

In order to meet the inherent need of real-time applications for high quality results within strict timing constraints, the employment of effective scheduling techniques is crucial in distributed real-time systems. In this paper, we evaluate by simulation the performance of strategies for the dynamic scheduling of composite jobs in a homogeneous distributed real-time system. Each job that arrives in the system is a directed acyclic graph of component tasks and has an end-to-end deadline. For each scheduling policy, we provide an alternative version which allows imprecise computations, taking into account the effects of input error on the processing time of the component tasks of a job. The simulation results show that the alternative versions of the algorithms outperform their respective counterparts. To our knowledge, an imprecise computations approach for the dynamic scheduling of multiple task graphs with end-to-end deadlines and input error has never been discussed in the literature before.     

Predictive resource management for meta-applications

This paper defines meta-applications as large, related collections of computational tasks, designed to achieve a specific overall result, running on a (possibly geographically) distributed, non-dedicated meta-computing platform. To carry out such applications in an industrial context, one requires resource management and job scheduling facilities (including capacity planning), to ensure that the application is feasible using the available resources, that each component job will be sent to an appropriate resource, and that everything will finish before the computing resources are needed for other purposes.

This requirement has been addressed by the PAC in three major European collaborative projects: PROMENVIR, TOOLSHED and HPC-VAO, leading to the creation of job scheduling software, in which scheduling is brought together with performance modelling of applications and systems, to provide meta-applications management facilities. This software is described, focusing on the performance modelling approach which was needed to support it.

Early results from this approach are discussed, raising some new issues in performance modelling and software deployment for meta-applications. An indication is given about ongoing work at the PAC designed to overcome current limitations and address these outstanding issues.     

抢占阈值调度的功耗优化

DVS(Dynamic Voltage Scaling)技术的应用使得任务执行时间延长进而使得处理器的静态功耗(由CMOS电路的泄露电流引起)迅速增加.延迟调度(Procrastination Scheduling)算法是近年提出用于减少静态功耗的有效方法,它通过推迟任务的正常执行来尽可能长时间地让处理器处于睡眠或关闭状态,从而避免过多的静态功耗泄露.文中针对可变电压处理器上运用抢占阈值调度策略的周期性任务集合,将节能调度和延迟调度结合起来,提出一种两阶段节能调度算法,先使用离线算法来计算每个任务的最优处理器执行速度,而后使用在线模拟调度算法来计算每个任务的延迟时间,从而动态判定处理器开启/关闭时刻.实例研究和仿真实验表明,作者的方法能够进一步降低抢占阈值任务调度算法的功耗.     

Analysis of fork-join program response times on multiprocessors

Models for two processor sharing policies called task scheduling processor sharing and job scheduling processor sharing are developed and analyzed. The first policy schedules each task independently and allows parallel execution of an individual program, whereas the second policy schedules each job as a unit, thereby not allowing parallel execution of an individual program. It is found that task scheduling performs better than job scheduling for most system parameter values. The performance of the task scheduling processor sharing is compared to a first come first serve policy. First come first serve performs better than processor sharing over a wide range of system parameters. Processor sharing performs best when the task service time variability is high. The performance of processor sharing and first come first serve is studied with two classes of jobs, and for when a specific number of processors is statically assigned to each of the classes