Ripple-RAID:一种面向连续数据存储的高效能盘阵

视频监控、备份、归档等应用具有独特的负载特性和I/O访问模式,需研究特定的存储节能方法.磁盘阵列的局部并行策略有利于实现该类存储系统的节能,但通常会导致RAID执行小写操作而严重影响性能.为此,提出一种面向该类存储系统的高效能盘阵——Ripple-RAID,采用新的局部并行数据布局,通过综合运用地址转换、异地更新、基于流水技术渐进生成校验、分段数据恢复等策略,在单盘容错条件下,保持了局部并行的节能性,又有效解决了局部并行带来的小写问题.Ripple-RAID具有突出的性能和节能效率,在80%顺序写负载情况下,请求长度为512KB时,写性能为S-RAID 5的3.9倍,Hibernator、MAID写性能的1.9倍,PARAID、eRAID 5写性能的0.49倍;而比S-RAID 5节能20%,比Hibernator、MAID节能33%,比eRAID 5节能70%,比PARAID节能72%.     

A reliable and energy-efficient storage system with erasure coding cache

In modern energy-saving replication storage systems, a primary group of disks is always powered up to serve incoming requests while other disks are often spun down to save energy during slack periods. However, since new writes cannot be immediately synchronized into all disks, system reliability is degraded. In this paper, we develop a high-reliability and energy-efficient replication storage system, named RERAID, based on RAID10. RERAID employs part of the free space in the primary disk group and uses erasure coding to construct a code cache at the front end to absorb new writes. Since code cache supports failure recovery of two or more disks by using erasure coding, RERAID guarantees a reliability comparable with that of the RAID10 storage system. In addition, we develop an algorithm, called erasure coding write (ECW), to buffer many small random writes into a few large writes, which are then written to the code cache in a parallel fashion sequentially to improve the write performance. Experimental results show that RERAID significantly improves write performance and saves more energy than existing solutions.     

基于对象存储的负载均衡存储策略

对象存储文件系统中将大数据文件分片存储到多个存储节点上,以获取更好的并行I/O性能,提高系统吞吐率.现有对象存储文件系统的存储策略并未充分考虑存储对象本身负载的动态变化,不利于提高系统资源利用率.针对此问题,考虑存储对象的空间及I/O等负载实时变化,提出了一种简单、灵活、高效的负载均衡存储策略,并对该策略进行了实现.实验结果表明,该策略能有效提高对象存储系统资源的利用率和吞吐率,保证对象存储文件系统高效的读写性能.     

Prober: exploiting sequential characteristics in buffer for improving SSDs write performance

Solid state disks (SSDs) are becoming one of the mainstream storage devices due to their salient features, such as high read performance and low power consumption. In order to obtain high write performance and extend flash lifespan, SSDs leverage an internal DRAM to buffer frequently rewritten data to reduce the number of program operations upon the flash. However, existing buffer management algorithms demonstrate their blank in leveraging data access features to predict data attributes. In various real-world workloads, most of large sequential write requests are rarely rewritten in near future. Once these write requests occur, many hot data will be evicted from DRAM into flash memory, thus jeopardizing the overall system performance. In order to address this problem, we propose a novel large write data identification scheme, called Prober. This scheme probes large sequential write sequences among the write streams at early stage to prevent them from residing in the buffer. In the meantime, to further release space and reduce waiting time for handling the incoming requests, we temporarily buffer the large data into DRAM when the buffer has free space, and leverage an actively write-back scheme for large sequential write data when the flash array turns into idle state. Experimental results demonstrate that our schemes improve hit ratio of write requests by up to 10%, decrease the average response time by up to 42% and reduce the number of erase operations by up to 11%, compared with the state-of-the-art buffer replacement algorithms.     

高分辨宽带雷达高速数据大容量存储系统设计

随着现代高分辨雷达带宽的不断提高,雷达系统对大批量数据传输和存储速度有了更高的要求。采用光纤通道磁盘阵列实现大容量雷达数据的存储,通过设计高速CompactPCI总线接口实现雷达数据到主机的数据传输,利用CompactPCI平台实现了大容量数据的高速存储。设计的存储系统实际测试存储速度达到155.6 MB/s,已能满足目前大部分高分辨雷达宽带数据去斜采集的存储要求。     

基于外存性能模型的多通道并行I／O

I／O部分一直是制约计算机系统整体性能提升的瓶颈。本文提出外存性能模型，用于定量分析外存的I／O性能并帮助克服I／O瓶颈，并在此基础上，提出用多通道I／O克服PCI总线瓶颈。采用多钱程控制和异步I／O技术，使所有通道的磁盘并行工作。对比实．验表明，最大顺序读性能提升了46％，顺序写提升48％，随机读提升4％，随机写提升57％。     

海量视频数据分布式存储性能优化方法研究

针对文件系统存储效率低、多余副本导致空间浪费以及磁盘能源损耗严重的问题,提出一种新型分布式优化存储策略(distributed optimized storage strategy,DOSS).首先,引入Bcache混合存储技术在磁盘阵列前增设固态硬盘,作为高速缓冲区对多路视频流进行临时数据组织,变多线程并发写任务为单线程顺序写入任务,规避磁盘内部碎片产生,有效提高系统写入效率.其次,提出改进的liberation码对视频数据进行压缩存储,在保证系统可靠性的同时提高磁盘空间利用率.最后,基于ioctl系统调用编写盘片转速控制函数,实现磁盘多级休眠和低延迟唤醒,减低磁盘能耗,提高使用寿命.结果表明单台存储服务器在500路4 Mbps并发视频流下存储效率提高约36％,存储空间节省约40％,系统应对12000路并发视频流时仍存在休眠磁盘约10％.     

使用固态硬盘管理主存KV数据库的虚拟内存

主存键值(key-value,KV)数据库具有高效性、易用性和可扩展性.由于主存容量有限,一些数据量较大的应用必须使用磁盘进行数据交换.而固态硬盘(solid state disk,SSD)有高速的随机读特点,使用固态硬盘作为主存KV数据库的虚拟内存会提高对不在主存中的数据的读性能.但是固态硬盘的随机写性能较差,于是提...     

云计算环境下软硬件节能和负载均衡策略

针对云计算服务环境下软硬件节能和负载均衡优化问题,提出一种自适应的云计算环境下虚拟机(VM)动态迁移软节能策略。该策略采用常用的硬件能耗感知技术——动态电压频率调节(DVFS)来实现分段优化的系统部件静态节能,又通过VM在线迁移技术实现云平台的动态自适应软件节能。在CloudSim云仿真平台下对比实现DVFS静态节能和自适应负载均衡的软节能策略,经PlanetLab云平台监测数据验证,结果表明：软硬结合的自适应能耗感知策略能够高效节能96%; DVFS+MAD_MMT节能策略(采用平均绝对偏差算法判定主机是否超载,基于最短迁移时间(MMT)原则选择VM移出)     

Graph-Based Replication and Two Factor Authentication in Cloud Computing

Many cutting-edge methods are now possible in real-time commercial settings and are growing in popularity on cloud platforms. By incorporating new, cutting-edge technologies to a larger extent without using more infrastructures, the information technology platform is anticipating a completely new level of development. The following concepts are proposed in this research paper: 1) A reliable authentication method Data replication that is optimised; graph-based data encryption and packing colouring in Redundant Array of Independent Disks (RAID) storage. At the data centre, data is encrypted using crypto keys called Key Streams. These keys are produced using the packing colouring method in the web graph’s jump graph. In order to achieve space efficiency, the replication is carried out on optimised many servers employing packing colours. It would be thought that more connections would provide better authentication. This study provides an innovative architecture with robust security, enhanced authentication, and low cost.     

基于嵌入式Linux的磁盘阵列控制软件设计

磁盘阵列是解决I／O瓶颈的一条有效途径。本文介绍了一种采用嵌入式Linux操作系统为软件平台、使用通用计算机部件集成的EIDE磁盘阵列。该阵列使用SCSI接口与主机连接，是一种具有良好性能和低廉价格的异构磁盘阵列。     

Modeling the aging process of flash storage by leveraging semantic I/O

The major advantages of flash memory such as small physical size, no mechanical components, low power consumption, and high performance have made it likely to replace the magnetic disk drives in more and more systems. Many research efforts have been invested in employing flash memory to build high performance and large-scale storage systems for data-intensive applications. However, the endurance cycle of flash memory has become one of the most important challenges in further facilitating the flash memory based systems. This paper proposes to model the aging process of flash memory based storage systems constructed as a Redundant Array of Independent Disks (RAID) by leveraging the semantic I/O. The model attempts to strike a balance between the program/erase cycles and the rebuilding process of RAID. The analysis results demonstrate that a highly skewed data access pattern ages the flash memory based RAID with an arbitrary aging rate, and a properly chosen threshold of aging rate can prevent the system from aging with a uniform data access pattern. The analysis results in this paper provide useful insights for understanding and designing effective flash memory based storage systems.     

Efficient journaling writeback schemes for reliable and high-performance storage systems

We propose a efficient writeback scheme that enables guaranteeing throughput in high-performance storage systems. The proposed scheme, called de-fragmented writeback (DFW), reduces positioning time of storage devices in writing workloads, and thus enables fast writeback in storage systems. We consider both of storage media in designing DFW scheme; traditional rotating disk and emerging solid-state disks. First, sorting and filling holes methods are used for rotating disk media for the higher throughput. The scheme converts fragmented data blocks into sequential ones so that it reduces the number of write requests and unnecessary disk-head movements. Second, flash block aware clustering-based writeback scheme is used for solid-state disks considering the characteristics of flash memory. The experimental results show that our schemes guarantee system’s high throughput while guaranteeing data reliability.     

基于模糊动态阈值机制的能效虚拟机合并决策方法

动态虚拟机合并是云数据中心改善功耗和资源利用率的有效方法,但负载变化使数据中心较难维持服务等级协议SLA和最优能效.针对该问题,提出一种模糊动态阈值方法对虚拟机合并过程进行决策,在动态负载环境下实现最小化的虚拟机迁移量.该算法利用模糊推理系统动态调整主机资源利用阈值,使得超载主机上的虚拟机迁移大幅降低,并可以满足服务等...     

基于SSD SMR混合存储的LSM树键值存储系统的性能优化

大数据对存储系统的可扩展性、性能和成本等方面提出了更高的要求。瓦记录(Shingled Magnetic Recor-ding,SMR)硬盘由于存储密度高、价格便宜,正逐步被广泛应用于大数据存储系统。但是,SMR硬盘的随机写性能较差,与快速的基于闪存的固态硬盘(Solid State Drive,SSD)一起构成混合存储时可以显著提升性能。同时,基于写优化的日志结构合并(Log-Structured Merge,LSM)树的键值存储已被广泛应用于许多NoSQL系统,如BigTable,Cassandra和HBase等。因此,如何基于新型的SSD-SMR混合存储构建出高性能的LSM树键值存储系统是一个具有很大研究价值的问题。首先建立基于SSD-SMR混合存储的LSM树键值系统的性能模型,然后针对SSD和SMR的硬件特征以及LSM树键值存储的软件特点,设计了一套面向SSD-SMR混合存储进行性能优化的LSM树键值存储系统,并基于LevelDB实现了该系统。在仅仅使用0.4%～2%空间的SSD的情况下,所提方法可以使SSD-SMR混合存储方案比普通磁盘方案的随机写性能提高20%,随机读性能提高5倍。     

科学计算浮点数据的高性能无损压缩

科学计算在科学界及工业界发挥着越来越重要的作用,所随之产生的科学数据也越来越多.因二级存储(如硬盘)的读写速度通常较慢,庞大的数据量除了占据存储空间之外,还影响着系统性能.文中通过系统研究浮点数据的特性,建立预测精度和压缩比之间的关系的理论体系.通过利用科学数据之间的相关性,采用多种预测器以及高效熵编码方法,提出一种科学数据高性能无损压缩方法.该方法既不需要使用者有关于原始数据的先验知识,也不需要使用者自行设计预测器.通过与其他压缩方法进行比较,结果表明文中方法的压缩比远高于其他方法,并同时具有恒定的海量压缩吞吐量.该方法已被应用于大规模颗粒动力学仿真系统的数据压缩.     

并行动态位向量频繁闭合序列模式挖掘算法

针对在时间和空间上都具有高计算成本的长序列数据库,一个更有效和更紧凑且可以完全提取信息的挖掘模式是当前的研究热点。提出一种并行动态位向量频繁闭合序列模式的挖掘算法（PDBV FCSP）,该算法采用多核处理器架构和DBV数据结构相结合的方式,有效加快了序列数据库的处理速度,并对搜索空间进行划分,尽早执行预处理序列的闭合检查,减少了所需的存储空间和挖掘频繁闭合序列模式的执行时间,克服了现有并行挖掘算法通信开销、同步和数据复制等问题。利用重新分配工作的动态负载平衡机制,解决处理器之间的负载均衡问题,最大限度地减少了CPU空闲时间。对DBV VDF算法和PDBV FCSP（2 4核）算法进行仿真比较,结果表明,PDBV FCSP算法在运行时间、内存使用和可伸缩性等方面都有较优的性能提升,且当内核数增加时,性能更优。     

MapReduce并行加速数据流多模式相似性搜索

设计时间序列数据在Hadoop分布式文件系统（HDFS）中的有效存储方式,利用分布式缓存工具Distributed Cache将各子序列分发到Hadoop集群的计算节点上,将动态时间弯曲距离矩阵划分成多个子矩阵,采取并行迭代计算每条反对角线上子矩阵的方法,基于MapReduce编程模型,实现高效并行计算时间序列动态弯曲距离,通过改进剪裁冗余计算方法,设计实现一种数据流多模式相似性搜索并行算法。中国雪深长时间序列数据集的实验结果表明,当每条时间序列的长度达到5000以上时,并行计算动态弯曲距离所需时间少于串行计算所需时间,当每条时间序列的长度达到9000以上时,参与计算的集群节点越多,并行计算所需时间越少;当模式长度达到4000、参与计算的集群节点数达5个以上时,从数据流中并行搜索出与模式匹配的相似子序列所需时间约为串行搜索所需时间的20%。     

An energy-efficient storage for video surveillance

With the rapid growth of the video surveillance applications, the storage energy consumption of video surveillance is more noticeable, but existed energy-saving methods for massive storage system most concentrate on the data centers mainly with random accesses. The storage of video surveillance has inherent access pattern, and requires special energy-saving approach to save more energy. An energy-efficient data layout for video surveillance, Semi-RAID is proposed. It adopts partial-parallelism strategy, which partitions disk data into different groups, and implements parallel accesses in each group. Grouping benefits to realize only partial disks working and the rest ones idle, and inner-group parallelism provides the performance guarantee. In addition, greedy strategy for address allocation is adopted to effectively prolong the idle period of the disks; particular Cache strategies are used to filter the small amount of random accesses. The energy-saving efficiency of Semi-RAID is verified by a simulated video surveillance consisting of 32 cameras with D1 resolution. The experiment shows: Semi-RAID can save 45 % energy than Hibernator; 80 % energy than PARAID; 33 % energy than MAID; 79 % energy than eRAID-5, while providing single disk fault tolerance and meeting the performance requirement, such as throughput.     

多媒体应用中磁盘阵列读预取Petri网模型及结果分析

多媒体应用需要大容量、高速磁盘阵列的支持。为了分析采用读预取策略后磁盘阵列的性能,本文建立了读预取Petri网[1]模型。此模型可用以判断系统各环节对读速率的影响和求取读平均数传率。实验表明,采用顺序读预取策略的实际读平均数传率提高了60％,可以满足数字化系统对I/O性能的高要求。