FeGC: An efficient garbage collection scheme for flash memory based storage systems

NAND flash memory is a promising storage media that provides low-power consumption, high density, high performance, and shock resistance. Due to these versatile features, NAND flash memory is anticipated to be used as storage in enterprise-scale systems as well as small embedded devices. However, unlike traditional hard disks, flash memory should perform garbage collection that consists of a series of erase operations. The erase operation is time-consuming and it usually degrades the performance of storage systems seriously. Moreover, the number of erase operations allowed to each flash memory block is limited. This paper presents a new garbage collection scheme for flash memory based storage systems that focuses on reducing garbage collection overhead, and improving the endurance of flash memory. The scheme also reduces the energy consumption of storage systems significantly. Trace-driven simulations show that the proposed scheme performs better than various existing garbage collection schemes in terms of the garbage collection time, the number of erase operations, the energy consumption, and the endurance of flash memory.     

Improving multimedia systems performance using constant-density recording disks

Multimedia systems store and retrieve large amounts of data which require extremely high disk bandwidth and their performance critically depends on the efficiency of disk storage. However, existing magnetic disks are designed for small amounts of data retrievals geared to traditional operations; with speed improvements mainly focused on how to reduce seek time and rotational latency. When the same mechanism is applied to multimedia systems, overheads in disk I/O can result in dramatic deterioration in system performance. In this paper, we present a mathematical model to evaluate the performance of constant-density recording disks, and use this model to analyze quantitatively the performance of multimedia data request streams. We show that high disk throughput may be achieved by suitably adjusting the relevant parameters. In addition to demonstrating quantitatively that constant-density recording disks perform significantly better than traditional disks for multimedia data storage, a novel disk-partitioning scheme which places data according to their bandwidths is presented.     

An Efficient Data Layout Scheme for Multi-Disks Continuous Media Servers

Recently, zoning technique has been widely applied to disks to increase their capacities. Under the technique, an interesting feature of a disk is that there are a number of various bandwidths on it. Herein, a novel data layout scheme called cluster-pairing on efficiently exploiting this feature for continuous media (CM) servers is proposed. We first applied track-pairing method between a pair of homogeneous disks, and then partitioned each disk into a same number of clusters to facilitate the retrieval of region-based data placement. The proposed method can take the advantages of track-pairing and region-based data placement schemes to fully utilize the various bandwidths from zoned-disks and meanwhile reduce the seek time overhead. According to the simulation results, the disk throughput after applying our approach can be promoted by 35% to 65% than that of the traditional data striping strategies, 10% to 30% than that of region-based data placement method, and about 10% than that of track-pairing scheme. And the wasted storage space is less than 1% which is negligible in terms of the improvement of the disk throughput.     

RAID0.5: design and implementation of a low cost disk array data protection method

RAID has long been established as an effective way to provide highly reliable as well as high-performance disk subsystems. However, reliability in RAID systems comes at the cost of extra disks. In this paper, we describe a mechanism that we have termed RAID0.5 that enables striped disks with very high data reliability but low disk cost. We take advantage of the fact that most disk systems use offline backup systems for disaster recovery. With the use of these offline backup systems, the disk system needs to only replicate data since the last backup, thus drastically reducing the storage space requirement. Though RAID0.5 has the same data loss characteristics of traditional mirroring, the lower storage space comes at the cost of lower availability. Thus, RAID0.5 is a tradeoff between lower disk cost and lower availability while still preserving very high data reliability. We present analytical reliability models and experimental results that demonstrate the enhanced reliability and performance of the proposed RAID0.5 system.     

Highly available and heterogeneous continuous media storage systems

A number of recent technological trends have made data intensive applications such as continuous media (audio and video) servers a reality. These servers store and retrieve large volumes of data using magnetic disks. Servers consisting of multiple nodes and large arrays of heterogeneous disk drives have become a fact of life for several reasons. First, magnetic disks might fail. Failed disks are almost always replaced with newer disk models because the current technological trend for these devices is one of annual increase in both performance and storage capacity. Second, storage requirements are ever increasing, forcing servers to be scaled up progressively. In this study, we present a framework to enable parity-based data protection for heterogeneous storage systems and to compute their mean lifetime. We describe the tradeoffs associated with three alternative techniques: independent subservers, dependent subservers, and disk merging. The disk merging approach provides a solution for systems that require highly available secondary storage in environments that also necessitate maximum flexibility.     

A scalable bandwidth guaranteed distributed continuous media file system using network attached autonomous disks

Repository for continuous media data differs from that of the traditional text-based data both in storage space and streaming bandwidth requirements. The file systems used for continuous media streams need to support large volumes and high bandwidth. We propose a scalable distributed continuous media file system built using autonomous disks. Autonomous disks are attached directly to the network and are able to perform lightweight processing. We discuss different ways to realize the autonomous disk, and describe a prototype implementation on a Linux platform using PC-based hardware. We present the basic requirements of the continuous media file system and present the design methodology and a prototype Linux-based implementation of the distributed file system that supports the requirements. We present experimental results on the performance of the proposed file system prototyped using autonomous disks. We show that the performance of the file system scales linearly with the number of disks and the number of clients. The file system performs much superior to NFS running on the same hardware platform and can deliver higher raw disk bandwidth to the applications. We also present bandwidth and time sensitive read/write procedures for the file system and show that the file system can provide strict bandwidth guarantees for continuous media streams.     

基于CH375的单片机U盘控制器的设计与实现

介绍了一种以USB接口芯片CH375和单片机为核心,U盘为存储介质的数据存储方案,给出了系统硬件实现和单片机读写U盘的应用程序。实验结果表明,该方案具有成本低、通用性强、可靠性高等特点,可方便地集成到各种测控系统中。     

RAID-VCR:一种能够承受三个磁盘故障的RAID结构

提出了一种新RAID结构——RAID-VCR.这种结构仅需要3个额外的磁盘来保存校验信息,但是却能够承受任意模式的3个成员磁盘故障.与现有的其它RAID结构相比,RAID-VCR的容灾能力大幅提高,但是对磁盘空间利用率和系统吞吐量的影响却非常小.RAID-VCR的编码和解码过程都是基于简单的XOR操作,并且以明文方式保存了用户数据,从而可以高效地执行读操作.仿真实验结果表明,RAID-VCR的编码和解码性能较好,具有很好的应用前景.     

Issues in the design of a storage server for video-on-demand

We examine issues related to the design of a storage server for video-on-demand (VOD) applications. The storage medium considered is magnetic disks or arrays of disks. We investigate disk scheduling policies, buffer management policies and I/O bus protocol issues. We derive the number of sessions that can be supported from a single disk or an array of disks and determine the amount of buffering required to support a given number of users. Furthermore, we propose a scheduling mechanism for disk accesses that significantly lowers the buffer-size requirements in the case of disk arrays. The buffer size required under the proposed scheme is independent of the number of disks in the array. This property allows for striping video content over a large number of disks to achieve higher concurrency in access to a particular video object. This enables the server to satisfy hundreds of independent requests to the same video object or to hundreds of different objects while storing only one copy of each video object. The reliability implications of striping content over a large number of disks are addressed and two solutions are proposed. Finally, we examine various policies for dealing with disk thermal calibration and the placement of videos on disks and disk arrays.     

连续媒体服务器存储系统中的容错算法

连续媒体服务器(如VOD服务器)要对大量连续媒体数据(如声频、视频)进行管理,按一定速率为用户提供连续的媒体服务。因此,在这样的系统中,作为存储设备的磁盘阵列要具有高可靠性和一定的容错能力。文章提出一种基于奇偶检验的数据重构恢复算法,以保证系统中只有一个盘出现故障时,能使服务器及时重构出故障盘上的数据,并且算法充分利用了媒体流内在特性———回放时数据的连续性,与目前使用的标准故障恢复算法相比,大大减少了磁盘在线故障后数据重构过程的系统开销。最后通过分析、比较证明了算法的有效性。     

一种面向数据可用性和存储可靠性动态要求的自适应纠删码存储策略设计

为了满足指数级增长的大数据存储需求,现代的分布式存储系统需要提供大容量的存储空间以及快速的存储服务.因此在主流的分布式存储系统中,均应用了纠删码技术以节约数据中心的磁盘成本,保证数据的可靠性,并且满足应用程序和客户端的快速存储需求.在实际应用中数据往往重要程度并不相同,对数据可用性要求不一,且不同磁盘的故障率和可靠性动态不一的特点,对于传统RAID存储方式包括基于纠删码的存储系统提出了新的挑战.本文提出了一种面向数据可用性和磁盘可靠性动态要求的灵活自适应纠删码存储设计On-demand ARECS(On-demand Availability and Reliability Oriented Adaptive Erasure Coded Storage System),根据存储后端数据可用性和磁盘可靠性的多个维度进行设计,综合确定纠删码编码策略和存储节点选择,从而减少存储冗余度和存储延迟,同时提高数据可用性和存储可靠性.我们在Tahoe-LAFS开源分布式文件系统中进行了实验,实验结果验证了我们的理论分析,在保证具有多样性要求的数据可用性和磁盘可靠性的前提下,明显减少了数据冗余度和存储延迟.     

Disk placement for arbitrary-rate playback in an interactive video server

Multimedia data, especially continuous media including video and audio objects, represent a rich and natural stimulus for humans, but require large amount of storage capacity and real-time processing. In this paper, we describe how to organize video data efficiently on multiple disks in order to support arbitrary-rate playback requested by different users independently. Our approach is to segment and decluster video objects and to place the segments in multiple disks using a restricted round-robin scheme, called prime round-robin (PRR). Its placement scheme provides uniform load balance of disks for arbitrary retrieval rate as well as normal playback, since it eliminates hot spots. Moreover, it does not require any additional disk bandwidth to support VCR-like operations such as fast-forward and rewind. We have studied the various effects of placement and retrieval schemes in a storage server by simulation. The results show that PRR offers even disk accesses, and the failure in reading segment by deadline occurs only at the beginning of new operations. In addition, the number of users admitted is not decreased, regardless of arbitrary-rate playback requests.     

基于QoS的磁盘调度策略

随着视频点播、视频会议、视频监控、数字图书馆等流媒体应用的普及，流媒体服务器存储资源管理成为制约服务质量的瓶颈之一。根据多媒体服务器的性能要求，提出了一种支持QoS的磁盘调度策略。它由三个主要部分组成：探测模块、负载监测模块和自适应管理模块。探测模块，负责判断当前的资源情况能否满足服务请求；自适应模块，根据负载监删模块检测到的负载变化情况，动态调整服务周期在实时请求和尽力服务请求之间的分配。实验表明此磁盘调度策略能在保证实时请求无抖动执行的同时，明显减少了非实时请求的响应时间。     

Performance and reliability concern scheme for efficient garbage collection and wear leveling on flash memory-based solid state disk

In the near future, data is expected to double in every two years, which translates to more than one terabyte of data for every person on earth. The difficulty of storing and fetching required data from data centers and servers will consequently increase. As a result, significant attention has been given to the flash memory-based solid state drive (SSD) from consumer electronics companies, which has begun to replace the existing hard disk drive and is highly likely to be used as a storage unit for most of consumer electronics to achieve low power consumption. In contrast to traditional disk, SSD uses semiconductor chips to store data. This structure enjoys original technical characteristics, including low power consumption, shock resistance, and high performance in random access. Such features can overcome the shortcomings of magnetic disks. However, flash memory, the basic unit of SSD, has many distinctive characteristics that cause various challenges. Flash memory does not support updating in the place method. A write operation can be performed only on an empty or erased unit, making the process more time-consuming. Moreover, each storage unit has a limited number of erase cycles, after which the block becomes invalid. This research proposes a new scheme called performance and reliability concern to increase the reliability and performance of SSDs. The eligibility of the proposed scheme is proven through EagleTree simulator. The scheme is also compared with other state-of-the-art techniques in terms of effectiveness and efficiency. Experimental results show that the proposed scheme provides sufficient performance and reliability for SSD.

     

关系型数据库内存化存储模型研究

大数据环境下,磁盘数据库存在高并发I/O瓶颈,磁盘数据内存化是解决传统关系型磁盘数据库I/O瓶颈的有效方案。已有的内存化技术存在数据丢失、配置复杂等问题。基于内存数据库Redis,以典型开源关系型数据库MySQL为例,提出一种解决关系型数据库磁盘I/O瓶颈的轻量级内存化解决方案,实现MySQL的内存化存储。同时构建行式键值（RB-KVM）和分段列式键值交叉（PCB-KVCM）存储转换模型,实现异构数据库存储模型的转换和自动化数据迁移。通过分析对比,RB-KVM数据存取效率更高,而PCB-KVCM具有更高的内存利用率且在时间开销上优于RB-KVM。通过实际运行分析,该技术不仅提高了数据库在高并发场景下的吞吐能力,实现了海量热点数据的高效存取,在实际应用中也更加快捷,实施成本更低。     

A high performance NAND array file system based on multiple NAND flash memories

The existing NAND flash memory file systems have not taken into account multiple NAND flash memories for large-capacity storage. In addition, since large-capacity NAND flash memory is much more expensive than the same capacity hard disk drive, it is cost wise infeasible to build large-capacity flash drives. To resolve these problems, this paper suggests a new file system called NAFS for large-capacity storage with multiple small-capacity and low-cost NAND flash memories. It adopts a new cache policy, mount scheme, and garbage collection scheme in order to improve read and write performance, to reduce the mount time, and to improve the wear-leveling effectiveness. Our performance results show that NAFS is more suitable for large-capacity storage than conventional NAND file systems such as YAFFS2 and JFFS2 and a disk-based file system for Linux such as HDD-RAID5-EXT3 in terms of the read and write transfer rate using a double cache policy and the mount time using metadata stored on a separate partition. We also demonstrate that the wear-leveling effectiveness of NAFS can be improved by our adaptive garbage collection scheme.     

外存储设备建模技术研究

外存储设备已经成为计算机系统性能的瓶颈,用于提升外存储设备速度的技术层出不穷,而建模是研究这些技术效能的主要手段.分析了外存储设备建模技术的研究现状,设计了一种固态盘存储系统模型,并应用该模型定量分析了提升固态盘性能的几项主要技术.结果表明:利用建模技术能分析出各种技术的效能,为外存储设备的设计提供有效的指导.     

Low-Cost Rotating Memories: Status and Future

The first six years of the seventies have been a period of rapid developments in the field of rotating memory. Innovation has dominated the gamut of disk drive applications. For example, substantial inroads have been made by flexible disks into the low-entry-cost area where rotating memory was previously not competitive. Cartridge disk drives have gained dramatically in capacity and performance while prices have remained relatively steady. The announcement of the IBM 3340, Winchester, brought to the marketplace the concept of a pack with its own captive heads in order to obtain large capacity at modest cost by increased surface storage density. The IBM System 32 brought the concept of the "disk enclosure" which treats a substantial part of a disk system as a field replaceable component. Independent disk manufacturers have responded to the latter two IBM developments with their versions of modest-cost, high-capacity, replaceable media drives and with very low cost, fixed-disk systems.     

Profit data caching and hybrid disk‐aware Completely Fair Queuing scheduling algorithms for hybrid disks

Recently, a hybrid disk drive that integrates a small amount of flash memory within a mechanical drive has received significant attention. The hybrid drive extends the storage hierarchy by using flash memory to cache data from the mechanical disk. Unfortunately, current caching architectures fail to fully exploit the potential of the hybrid drive. Furthermore, current disk input/output (I/O) schedulers are optimized for rotational mechanical disk drives and thus must be re‐targeted for the hybrid disk drive. In this paper, we propose a new data caching scheme, called Profit Caching, for hybrid drives. Profit Caching is a self‐optimizing caching algorithm. It considers and seamlessly integrates all possible data characteristics that impact the performance of hybrid drives, including read count, write count, sequentiality, randomness, and recency, to determine the caching policy. Moreover, we propose a hybrid disk‐aware Completely Fair Queuing (HA‐CFQ) scheduler to avoid unnecessary I/O anticipations of the CFQ scheduler. We have implemented Profit Caching and HA‐CFQ scheduler in the Linux kernel. Coupled with a trace‐driven simulator, we have also conducted detailed experiments under a variety of workloads. Experimental results show that Profit Caching provides significantly improved performance compared with the previous schemes. In particular, the throughput of Profit Caching outperforms previous Random Access First and FlashCache caching schemes by factors of up to 1.8 and 7.6, respectively. In addition, the HA‐CFQ scheduler reduces the total execution time of the CFQ scheduler by up to 1.74%. Finally, the experimental results show that the runtime overhead of Profit Caching is extremely insignificant and can be ignored. Copyright © 2014 John Wiley & Sons, Ltd.     

Secure storage—Confidentiality and authentication

Secure disk storage is a rich and complex topic and its study is challenging in theory as well as in practice. In case of loss or theft of mobile devices (such as laptops and smartphones), the threat of data exposure is important and a natural security objective is to guarantee the confidentiality of the data-at-rest stored in such devices (e.g. on disks or solid-state drives). Classical approaches to encrypt data may have a severe impact on performance if the underlying architectural specificities are not considered. In particular, it is usually assumed that an encryption scheme suitable for the application of disk encryption must be length preserving. This so-called “full disk encryption” method provides confidentiality but does not provide cryptographic data integrity protection. It indeed rules out the use of authenticated encryption where an authentication tag is concatenated to the ciphertext. Moreover, authenticated encryption requires storing tags, and latency is added due to extra read/write accesses and tag computations. We present a comprehensive study of full disk encryption solutions and compare their features from a security perspective. We additionally present threat models for authenticated disk encryption and present a systematized analysis of the techniques usable in these settings (which has, up to now, received little attention from the research community). We finally review the current state-of-the-art of incremental cryptography and provide new insights for its use in secure disk storage contexts.