一种新型的能够防止两块磁盘失败的技术

海量存储系统的建设是目前计算机系统最热门和发展最快的领域,存储系统的主要部分是在线存储系统。RAID(磁盘阵列)对于提升存储系统的效率、数据的高可靠性、防止数据破坏和业务停顿具有重大意义。目前实际应用中的RAID 1,RAID 0+1,RAID 4,RAID 5都只能防止单块磁盘的损坏,实际生产中已经出现了很多由于双盘损坏造成业务长时间停顿的事故。在介绍了通用的RAID级别的基础上,介绍了一种新型的对角线奇偶校验方法,结合水平奇偶校验,可以防止两块磁盘损坏。通过可靠的数学分析,可以看到该方法可以极大提高磁     

一种面向数据可用性和存储可靠性动态要求的自适应纠删码存储策略设计

为了满足指数级增长的大数据存储需求,现代的分布式存储系统需要提供大容量的存储空间以及快速的存储服务.因此在主流的分布式存储系统中,均应用了纠删码技术以节约数据中心的磁盘成本,保证数据的可靠性,并且满足应用程序和客户端的快速存储需求.在实际应用中数据往往重要程度并不相同,对数据可用性要求不一,且不同磁盘的故障率和可靠性动态不一的特点,对于传统RAID存储方式包括基于纠删码的存储系统提出了新的挑战.本文提出了一种面向数据可用性和磁盘可靠性动态要求的灵活自适应纠删码存储设计On-demand ARECS(On-demand Availability and Reliability Oriented Adaptive Erasure Coded Storage System),根据存储后端数据可用性和磁盘可靠性的多个维度进行设计,综合确定纠删码编码策略和存储节点选择,从而减少存储冗余度和存储延迟,同时提高数据可用性和存储可靠性.我们在Tahoe-LAFS开源分布式文件系统中进行了实验,实验结果验证了我们的理论分析,在保证具有多样性要求的数据可用性和磁盘可靠性的前提下,明显减少了数据冗余度和存储延迟.     

基于智能预警的自恢复存储系统研究

为提高磁盘阵列存储的可靠性,提出一种基于智能预警的自恢复存储系统。在对磁盘S.M.A.R.T信息实时监测采集的基础上,利用机器学习方法对磁盘故障进行预测,对于预警磁盘采取以数据为中心的自修复迁移保护策略。分析结果表明,该系统能提高存储可靠性,数据迁移过程对用户透明,可用性较强。     

使用备份加速RAID5数据重构

对于大部分中小企业,为了保证生产系统所存放数据的安全,通常使用RAID5和备份两种技术.提出了一种新型的RAID5加速重构模型,该模型具有以下优点:1)能够灵活整合生产系统和备份系统;2)充分利用备份系统的资源,加速生产系统内RAID5数据重构;3)改善生产系统服务质量;4)提高生产系统的可靠性和可用性.     

RAID-VCR:一种能够承受三个磁盘故障的RAID结构

提出了一种新RAID结构——RAID-VCR.这种结构仅需要3个额外的磁盘来保存校验信息,但是却能够承受任意模式的3个成员磁盘故障.与现有的其它RAID结构相比,RAID-VCR的容灾能力大幅提高,但是对磁盘空间利用率和系统吞吐量的影响却非常小.RAID-VCR的编码和解码过程都是基于简单的XOR操作,并且以明文方式保存了用户数据,从而可以高效地执行读操作.仿真实验结果表明,RAID-VCR的编码和解码性能较好,具有很好的应用前景.     

A multiple disk failure recovery scheme in RAID systems

In this paper, we propose a practical disk error recovery scheme tolerating multiple simultaneous disk failures in a typical RAID system, resulting in improvement in availability and reliability. The scheme is composed of the encoding and the decoding processes. The encoding process is defined by making one horizontal parity and a number of vertical parities. The decoding process is defined by a data recovering method for multiple disk failures including the parity disks. The proposed error recovery scheme is proven to correctly recover the original data for multiple simultaneous disk failures regardless of the positions of the failed disks. The proposed error recovery scheme only uses exclusive OR operations and simple arithmetic operations, which can be easily implemented on current RAID systems without hardware changes.     

子阵列数据布局方法在网络存储中的应用

RAID的高性能源于无差异磁盘组的并行工作。但受网络等因素的影响,网络存储阵列系统中各个磁盘的性能是有差异的,这直接导致系统并行度的降低。现有的研究大都是通过改变RAID中数据单元和校验单元的布局方式来平衡磁盘间的差异。该文根据不同存储应用的特点,提出了子阵列数据布局方法来解决这一问题,显著提高了网络存储系统的性能。     

A reliable and energy-efficient storage system with erasure coding cache

In modern energy-saving replication storage systems, a primary group of disks is always powered up to serve incoming requests while other disks are often spun down to save energy during slack periods. However, since new writes cannot be immediately synchronized into all disks, system reliability is degraded. In this paper, we develop a high-reliability and energy-efficient replication storage system, named RERAID, based on RAID10. RERAID employs part of the free space in the primary disk group and uses erasure coding to construct a code cache at the front end to absorb new writes. Since code cache supports failure recovery of two or more disks by using erasure coding, RERAID guarantees a reliability comparable with that of the RAID10 storage system. In addition, we develop an algorithm, called erasure coding write (ECW), to buffer many small random writes into a few large writes, which are then written to the code cache in a parallel fashion sequentially to improve the write performance. Experimental results show that RERAID significantly improves write performance and saves more energy than existing solutions.     

一种高性能阵列架构研究

现今的应用程序需要更可靠的数据存储.到目前为止,数据存储的可靠性都是依靠不同的RAID级别来保证数据的可靠性,一般采用5个RAID级别[1,2]中的一种.存储方面的数据已经很明显地增长了,但是磁盘的可靠性并没有多大的改善.所以,为了控制存储的成本,有必要提供多元化的存储.在现有系统中加入新的RAID代码需要大量的开发、测试和调试工作,从成本上来讲是不现实的.因此,提出了一种新的通用RAID架构,此架构是基于异或的纠删码的,并将任意扇区和磁盘故障的组合作为基础,因而具有通用性.     

Stripped mirroring RAID architecture

Redundant arrays of independent disks (RAID) provide an efficient stable storage system for parallel access and fault tolerance. The most common fault tolerant RAID architecture is RAID-1 or RAID-5. The disadvantage of RAID-1 lies in excessive redundancy, while the write performance of RAID-5 is only 1/4 of that of RAID-0. In this paper, we propose a high performance and highly reliable disk array architecture, called stripped mirroring disk array (SMDA). It is a new solution to the small-write problem for disk array. SMDA stores the original data in two ways, one on a single disk and the other on a plurality of disks in RAID-0 by stripping. The reliability of the system is as good as RAID-1, but with a high throughput approaching that of RAID-0. Because SMDA omits the parity generation procedure when writing new data, it avoids the write performance loss often experienced in RAID-5.     

A Case for Continuous Data Protection at Block Level in Disk Array Storages

This paper presents a study of data storages for continuous data protection (CDP). After analyzing the existing data protection technologies, we propose a new disk array architecture that provides Timely Recovery to Any Point-in-time, referred to as TRAP. TRAP stores not only the data stripe upon a write to the array but also the time-stamped Exclusive ors (xors) of successive writes to each data block. By leveraging the xor operations that are performed upon each block write in today's RAID4/5 controllers, TRAP does not incur noticeable performance overhead. More importantly, TRAP is able to recover data very quickly to any point-in-time upon data damage by tracing back the sequence and history of xors resulting from writes. What is interesting is that the TRAP architecture is very space efficient. We have implemented a prototype of the new TRAP architecture using software at the block level and carried out extensive performance measurements using TPC-C benchmarks running on Oracle and Postgres databases, TPC-W running on a MySQL database, and file system benchmarks running on Linux and Windows systems. Our experiments demonstrated that TRAP not only is able to recover data to any point-in-time very quickly upon a failure but also uses less storage space than traditional daily incremental backup/snapshot. Compared to the state-of-the-art CDP technologies, TRAP saves disk storage space by one to two orders of magnitude with a simple and a fast encoding algorithm. In addition, TRAP can provide two-way data recovery with the availability of only one reference image in contrast to the one-way recovery of snapshot and incremental backup technologies.     

The distributed data center: front-end solutions

The need to provide reliable data availability has always driven data center design. Redundant storage systems and backup mechanisms are essential for enterprises to provide 24/7 data access. Using a redundant array of independent disks (RAID) and data server clusters helps prevent catastrophic data loss. However, redundancy within the data center itself represents only a partial solution. Although backup mechanisms preserve data, you have a high-availability solution only if redundancy also protects against a network failure between the center and the outside world. A distributed data center provides such a solution.     

FTDSS：高容错分布式共享存储机制

Computerized data has become critical to the survival of an enterprise. Company must have a strategy for recovering their data should data lose. RAID is a popular mechanism to offer good fault-tolerance. But RAID can notwork well when one more disks fail. In this paper, we present an efficient Network-based high High-Tolerance Dis-tributed Shared Storage mechanism called FTDSS. FTDSS makes use of disk space of node in Network to build alarge pubhc shared storage space. Users can read/write their file from/to the public storage space from node of net-work. Physically File is stored in each node in form of data fragment or XOR verify fragment. Because of redundant XOR fragments, file is available even when two more nodes fail. FTDSS realize distant redundant srorage. At last,this paper use experiment to prove that FTDSS can offer high. fault-tolerance and advanced performance.     

冗余磁盘阵列RAID5性能损失研究

冗余磁盘阵列虽然引入了容错机制使得磁盘阵列的数据可靠性得到了很大的提高，但同时也引起性能不降。而且随着磁盘数量的增加，磁盘失效的概率将明显增大，当单个磁盘失效后，虽然此时磁盘阵列数据并未矢失，且仍能服务于系统的请量此时磁盘阵列是带“病”工作，处于一种降级模式，本文对冗余磁盘阵列ＲＡＩＤ５进行了队列建模和仿真计算，提出了性能损失率的概念，并作为评价磁盘阵列性能损失的衡量指标。计算结果分析表明，ＲＡＩ     

Massive Storage Systems

To accommodate the explosively increasing amount of data in many areas such as scientific computing and e-Business, physical storage devices and control components have been separated from traditional computing systems to become a scalable, intelligent storage subsystem that, when appropriately designed, should provide transparent storage interface, effective data allocation, flexible and efficient storage management, and other impressive features. The design goals and desirable features of such a storage subsystem include high performance, high scalability, high availability, high reliability and high security. Extensive research has been conducted in this field by researchers all over the world, yet many issues still remain open and challenging. This paper studies five different online massive storage systems and one offline storage system that we have developed with the research grant support from China. The storage pool with multiple network-attached RAIDs avoids expensive store-and-forward data copying between the server and storage system, improving data transfer rate by a factor of 2-3 over a traditional disk array. Two types of high performance distributed storage systems for local-area network storage are introduced in the paper. One of them is the Virtual Interface Storage Architecture (VISA) where VI as a communication protocol replaces the TCP/IP protocol in the system. VISA's performance is shown to achieve better than that of IP SAN by designing and implementing the vSCSI (Vl-attached SCSI) protocol to support SCSI commands in the VI network. The other is a fault-tolerant parallel virtual file system that is designed and implemented to provide high I/O performance and high reliability. A global distributed storage system for wide-area network storage is discussed in detail in the paper, where a Storage Service Provider is added to provide storage service and plays the role of user agent for the storage system. Object based Storage Systems not only store data but also adopt the attributes and methods of objects that encapsulate the data. The adaptive policy triggering mechanism (APTM), which borrows proven machine learning techniques to improve the scalability of object storage systems, is the embodiment of the idea about smart storage device and facilitates the self-management of massive storage systems. A typical offline massive storage system is used to backup data or store documents, for which the tape virtualization technology is discussed. Finally, a domain-based storage management framework for different types of storage systems is presented in the paper.     

主动容错副本存储系统的可靠性分析模型

主动容错机制通过预先发现即将故障的硬盘来提醒系统提前迁移备份危险数据,从而显著提高存储系统的可靠性。针对现有研究无法准确评价主动容错副本存储系统可靠性的问题,提出几种副本存储系统的状态转换模型,然后利用蒙特卡洛仿真算法实现了该模型,从而模拟主动容错副本存储系统的运行,最后统计系统在某个运行时期内发生数据丢失事件的期望次数。采用韦布分布函数模拟设备故障和故障修复事件的时间分布,并定量评价了主动容错机制、节点故障、节点故障修复、硬盘故障以及硬盘故障修复事件对存储系统可靠性的影响。实验结果表明,当预测模型的准确率达到50%时,系统的可靠性可以提高1~3倍;与二副本系统相比,三副本系统对系统参数更敏感。所提模型可以帮助系统管理者比较权衡不同的容错方式以及系统参数下的系统可靠性水平,从而搭建高可靠和高可用的存储系统。     

自修复磁盘阵列技术研究

以数据为中心的计算模式对数据的可靠性提出新的需求,高可靠存储设备成为高性能计算的基础.自从Xiotech公司提出ISE结构并取得成功后,关注磁盘失效之前的征兆而不是失效后的具体处理的"自修复"概念已成为磁盘阵列设计新的指导思想之一.提出存储系统可靠性系数的概念,分析单个磁盘可靠性与磁盘阵列可靠性关系,选择合适的磁盘S.M.A.R.T属性结合磁盘当前利用率和历史信息建立评定磁盘状态的T~2US算法,接着介绍RAID 5结合T2US监测的高可靠、自修复的RAID 5T结构,说明RAID 5T结构的运行方式和磁盘不符合T~2US监测标准时数据迁移策略,最后对结构的可靠性利用存储系统可靠性系数和MTTF分别进行了分析.     

阵列矩阵RAID50研究——容量聚合与全并发策略

提出了一种跨多阵列通道的海量存储RAID50模型,通过采取多阵列卡的RAID0分条和阵列卡上多磁盘RAID5分条和校验的二级并发的数据组织与分块方式,以扩展块（大小等于阵列卡上的一个RAID5校验组）作为Cache和阵列之间数据交换的单位,实现了将阵列矩阵中所有磁盘的容量聚合及全并发访问。设计了该模型逻辑卷管理的最佳适配算法及二级地址映射算法。理论分析与实验结果表明：该策略将I/O响应时间降到了最低,且获得了与阵列通道数线性相关的逻辑卷容量和I/O性能。     

Reconstruct versus read-modify writes in RAID

RAID5 (Redundant Arrays of Independent Disk level 5) is a popular paradigm, which uses parity to protect against single disk failures. A major shortcoming of RAID5 is the small write penalty, i.e., the cost of updating parity when a data block is modified. Read-modify writes and reconstruct writes are alternative methods for updating small data and parity blocks. We use a queuing formulation to determine conditions under which one method outperforms the other. Our analysis shows that in the case of RAID6 and more generally disk arrays with k check disks tolerating k disk failures, RCW outperforms RMW for higher values of N and G. We note that clustered RAID and variable scope of parity protection methods favor reconstruct writes. A dynamic scheme to determine the more desirable policy based on the availability of appropriate cached blocks is proposed.     

MT2RAID:一种高可靠大规模磁盘阵列结构

传统的磁盘阵列一般采用集中式控制结构,其连接的底层磁盘数受系统总线的制约,容易出现性能瓶颈,且不能容两个以上磁盘出错。从模块化系统的组织方法出发,提出一种采用标准模块化存储单元组成的通过胖树结构互连的大规模磁盘阵列结构MT2RAID,分别就其各种数据分布的性能和可靠性进行了分析和讨论。原型系统测试结果表明,相比集中式磁盘阵列结构,MT2RAID也具有较高的性能。