面向分布对象存储结构的高性能计算系统资源管理方法

当前的高性能计算系统的资源管理和调度关注的焦点是计算资源,然而随着高性能计算系统的规模增大和计算能力增强,其I/O瓶颈问题日益突出.由于高性能计算系统的存储结构多样性带来了存储资源管理分配的难题,在目前主流的资源管理系统中尚未有针对I/O存储资源的调度和管理.随着对象存储结构的发展和广泛使用,大多数主流高性能系统采用分布对象存储系统,研究对分布对象存储系统的管理并结合资源管理系统,实现面向存储的作业优化调度,对提升高性能计算系统的实际性能有重要意义.针对具有分布对象存储结构的高性能计算系统,研究面向分布存储的资源管理方法,在作业调度和资源分配时考虑不同应用的I/O需求,通过建立分布对象存储资源模型和应用程序I/O能力需求模型,并在资源调度和分配上根据不同的I/O应用级别,为作业分配合适的存储资源,设计并实现基于I/O能力分级的作业调度和资源分配算法.系统测试表明:该方法可以显著提高多作业环境下应用的性能,保证应用程序的性能稳定性,提高系统的吞吐率.

Resource Management Method for High Performance Computing System Based on Distributed Objective Storage Architecture

Most resource management and scheduling systems foeus on the computing resources now.But the I/O bottleneck is more and more important with the enlarging systems both on scale and performance.In this paper,a resource management method is introduced for HPC based on distributed objective storage architecture,which considers not only the computing resources,but also the storage resources and I/O needs of applications.The model of storage resources and classification for I/O performance needs of applications are built.An algorithm of the job scheduling and resource allocating is designed and implemented based on the classification for I/O performance needs.Some experiments show the effective results of the proposed method.