云存储系统中基于分簇的数据复制策略

云存储技术已经成为当前互联网中共享存储和数据服务的基础技术,云存储系统普遍利用数据复制来提高数据可用性,增强系统容错能力和改善系统性能。提出了一种云存储系统中基于分簇的数据复制策略,该策略包括产生数据复制的时机判断、复制副本数量的决定以及如何放置复制所产生的数据副本。在放置数据副本时,设计了一种基于分簇的负载均衡副本放置方法。相关的仿真实验表明,提出的基于分簇的负载均衡副本放置方法是可行的,并且具有良好的性能。     

Consolidated cluster systems for data centers in the cloud age: a survey and analysis

In the cloud age, heterogeneous application modes on large-scale infrastructures bring about the challenges on resource utilization and manageability to data centers. Many resource and runtime management systems are developed or evolved to address these challenges and relevant problems from different perspectives. This paper tries to identify the main motivations, key concerns, common features, and representative solutions of such systems through a survey and analysis. A typical kind of these systems is generalized as the consolidated cluster system, whose design goal is identified as reducing the overall costs under the quality of service premise. A survey on this kind of systems is given, and the critical issues concerned by such systems are summarized as resource consolidation and runtime coordination. These two issues are analyzed and classified according to the design styles and external characteristics abstracted from the surveyed work. Five representative consolidated cluster systems from both academia and industry are illustrated and compared in detail based on the analysis and classifications. We hope this survey and analysis to be conducive to both design implementation and technology selection of this kind of systems, in response to the constantly emerging challenges on infrastructure and application management in data centers.     

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

Cloud computing systems are splitting compute- and data-intensive jobs into smaller tasks to execute them in a parallel manner using clusters to improve execution time. However, such systems at increasing scale are exposed to stragglers, whereby abnormally slow running tasks executing within a job substantially affect job performance completion. Such stragglers are a direct threat towards attaining fast execution of data-intensive jobs within cloud computing. Researchers have proposed an assortment of different mechanisms, frameworks, and management techniques to detect and mitigate stragglers both proactively and reactively. In this paper, we present a comprehensive review of straggler management techniques within large-scale cloud data centres. We provide a detailed taxonomy of straggler causes, as well as proposed management and mitigation techniques based on straggler characteristics and properties. From this systematic review, we outline several outstanding challenges and potential directions of possible future work for straggler research.

     

Performance issues and performance analysis tools for HPC cloud applications: a survey

Cloud Computing is an eminent emerging technology that surpasses Grids from their IT resource administrations and arduous Grid middleware solutions. At present, users could access an abundant number of pre-defined cloud services or run their programs on demand as a pay-as-you-go processing model without much distribution problems. In addition, the IT business market has pumped enough revenue for establishing salient common-use cloud solutions. Despite adequate researchers have been involved in the cloud development, scientific application developers are still reluctant to execute their applications in the cloud due to the performance concerns, such as, scalability, availability, and service level agreement violations of the cloud providers. In this paper, a survey of various High Performance Computing (HPC) applications and possible performance concerns while executing applications in cloud is presented. Pointing out the need for Performance Analysis (PA) tools, this paper focuses on the study of cloud-based PA tools in detail. This paper could leverage HPC application developers to cope with the performance issues and to best utilize the available performance analysis tools of clouds.     

SWORD: workload-aware data placement and replica selection for cloud data management systems

Cloud computing is increasingly being seen as a way to reduce infrastructure costs and add elasticity, and is being used by a wide range of organizations. Cloud data management systems today need to serve a range of different workloads, from analytical read-heavy workloads to transactional (OLTP) workloads. For both the service providers and the users, it is critical to minimize the consumption of resources like CPU, memory, communication bandwidth, and energy, without compromising on service-level agreements if any. In this article, we develop a workload-aware data placement and replication approach, called SWORD, for minimizing resource consumption in such an environment. Specifically, we monitor and model the expected workload as a hypergraph and develop partitioning techniques that minimize the average query span, i.e., the average number of machines involved in the execution of a query or a transaction. We empirically justify the use of query span as the metric to optimize, for both analytical and transactional workloads, and develop a series of replication and data placement algorithms by drawing connections to several well-studied graph theoretic concepts. We introduce a suite of novel techniques to achieve high scalability by reducing the overhead of partitioning and query routing. To deal with workload changes, we propose an incremental repartitioning technique that modifies data placement in small steps without resorting to complete repartitioning. We propose the use of fine-grained quorums defined at the level of groups of data items to control the cost of distributed updates, improve throughput, and adapt to different workloads. We empirically illustrate the benefits of our approach through a comprehensive experimental evaluation for two classes of workloads. For analytical read-only workloads, we show that our techniques result in significant reduction in total resource consumption. For OLTP workloads, we show that our approach improves transaction latencies and overall throughput by minimizing the number of distributed transactions.     

Towards a cloud storage data management model based on RNPT network

Multimedia Tools and Applications - Data management is the core module of cloud storage system. Constructed network topology of data center, the data management model of Recursion-based N-regular...     

A survey of RDF data management systems

RDF is increasingly being used to encode data for the semantic web and data exchange. There have been a large number of works that address RDF data management following different approaches. In this paper we provide an overview of these works. This review considers centralized solutions (what are referred to as warehousing approaches), distributed solutions, and the techniques that have been developed for querying linked data. In each category, further classifications are provided that would assist readers in understanding the identifying characteristics of different approaches.     

City digital pulse: a cloud based heterogeneous data analysis platform

In recent years, increasing attention has been paid to developing exceptional technologies for efficiently processing massive collection of data. This is essential in the research on smart city, which involves various types of data generated by different kinds of sensors (hard and soft). In this paper, we propose a cloud-based platform named City Digital Pulse (CDP), where a unified mechanism and extensible architecture are provided to facilitate the various aspects in big data analysis, ranging from data acquisition to data visualization. We instantiate the proposed system using multi-model data collected from two social networks, namely Twitter and Instagram, which can provide instant geo-tagged data. Data analysis is performed to detect human affections from user uploaded content. The information revealed from the collected social data can be visualized at multiple dimensions through a well-designed Web application. This allows users to easily sense changes in human affective status and identify the underlying reasons. This offers priceless opportunities to improve the decision making in many critical tasks using the detected attitudes in the social messages, such as promotion strategy for companies or new policy making for the government. Our experiment results confirm the effectiveness of the proposed architecture and algorithms.

     

Incomplete data management: a survey

Incomplete data accompanies our life processes and covers almost all fields of scientific studies, as a result of delivery failure, no power of battery, accidental loss, etc. However, how to model, index, and query incomplete data incurs big challenges. For example, the queries struggling with incomplete data usually have dissatisfying query results due to the improper incompleteness handling methods. In this paper, we systematically review the management of incomplete data, including modelling, indexing, querying, and handling methods in terms of incomplete data. We also overview several application scenarios of incomplete data, and summarize the existing systems related to incomplete data. It is our hope that this survey could provide insights to the database community on how incomplete data is managed, and inspire database researchers to develop more advanced processing techniques and tools to cope with the issues resulting from incomplete data in the real world.     

基于迭代MapReduce的混合云大数据分析

现有混合云平台运行大数据分析等数据密集型工作负载存在数据迁移开销大、执行时间长等问题,提出基于迭代MapReduce的大数据分析方法.提出一种数据存储和数据迁移机制,迭代时将初始不变量从私有云迁移至公有云,无需修改MapReduce框架或底层存储层;利用随机森林估计所提迭代MapReduce应用程序所需的计算时间.Op...     

Oblivious access control policies for cloud based data sharing systems

Conventional procedures to ensure authorized data access by using access control policies are not suitable for cloud storage systems as these procedures can reveal valid access parameters to a cloud service provider. In this paper, we have proposed oblivious access control policy evaluation (O-ACE); a data sharing system, which obliviously evaluates access control policy on a cloud server and provisions access to the outsourced data. O-ACE reveals no useful information about the access control policy neither to the cloud service provider nor to the unauthorized users. Through the security analysis of O-ACE it has been observed that computational complexity to compromise privacy of the outsourced data is same as reverting asymmetric encryption without valid key pair. We have realized O-ACE for Google Cloud. Our evaluation results show the fact that O-ACE CPU utilization cost is 0.01–0.30 dollar per 1,000 requests.     

Dynamic power management in energy-aware computer networks and data intensive computing systems

Energy awareness is an important aspect of modern network and computing system design and management, especially in the case of internet-scale networks and data intensive large scale distributed computing systems. The main challenge is to design and develop novel technologies, architectures and methods that allow us to reduce energy consumption in such infrastructures, which is also the main reason for reducing the total cost of running a network. Energy-aware network components as well as new control and optimization strategies may save the energy utilized by the whole system through adaptation of network capacity and resources to the actual traffic load and demands, while ensuring end-to-end quality of service. In this paper, we have designed and developed a two-level control framework for reducing power consumption in computer networks. The implementation of this framework provides the local control mechanisms that are implemented at the network device level and network-wide control strategies implemented at the central control level. We also developed network-wide optimization algorithms for calculating the power setting of energy consuming network components and energy-aware routing for the recommended network configuration. The utility and efficiency of our framework have been verified by simulation and by laboratory tests. The test cases were carried out on a number of synthetic as well as on real network topologies, giving encouraging results. Thus, we come up with well justified recommendations for energy-aware computer network design, to conclude the paper.     

基于云存储技术的手术室数据管理系统

针对我国医疗信息化进程中,数字化手术室的建设需求,基于云存储技术,研究并设计了手术室的数据管理系统。该系统在设计过程中详细梳理了手术室的工作流程,对手术的排班、患者体征数据采集、麻醉记录、手术数据查询等功能进行了重点设计。系统使用了HDFS分布式文件管理系统,保证了系统的可靠性和可扩展性,实现了临床数据的实时传输与共享。     

基于多级索引的云平台镜像管理系统Milance的设计与实现

OpenStack云平台的开源掀起了对于基础设施即服务的云平台的研究热潮,但基于本地存储的镜像管理存在资源利用率不足的问题。为此,开发了新型的多级索引的镜像管理系统Milance替换现有的镜像管理系统,旨在解决现有系统存在的虚拟机实例启动延时大、做快照时间长、镜像池占用空间较大的问题。通过测试,相对于现有的系统,Milance系统在虚拟机启动、虚拟机做快照花费的时间较少。同时,Milance系统也相对于现有系统更加节省镜像空间。     

Energy efficiency on location based applications in mobile cloud computing: a survey

The constrained battery power of mobile devices poses a serious impact on user experience. As an increasingly prevalent type of applications in mobile cloud environments, location-based applications (LBAs) present some inherent limitations concerning energy. For example, the Global Positioning System based positioning mechanism is well-known for its extremely power-hungry attribute. Due to the severity of the issue, considerable researches have focused on energy-efficient locating sensing mechanism in the last a few years. In this paper, we provide a comprehensive survey of recent work on low-power design of LBAs. An overview of LBAs and different locating sensing technologies used today are introduced. Methods for energy saving with existing locating technologies are investigated. Reductions of location updating queries and simplifications of trajectory data are also mentioned. Moreover, we discuss cloud-based schemes in detail which try to develop new energy-efficient locating technologies by leveraging the cloud capabilities of storage, computation and sharing. Finally, we conclude the survey and discuss the future research directions.