数据复制技术在城市轨道交通系统中的应用

文章介绍了数据复制技术的概念和方法以及Sybase复制服务器的组成和工作原理,给出了sybase复制技术在城市轨道交通系统中的应用实例,基于数据复制技术的容灾体系提高了系统的健壮性和可用性.     

一种Hadoop数据复制优化方法

针对当前Hadoop采用固定个数的数据复制来提高数据可用性方法的不足,建立了数据复制的数学模型,该模型根据数据节点失效率、数据访问延迟、数据节点的网络带宽、期望的数据可用性计算优化的数据复制个数,在Hadoop上实现了提出的数据复制优化方法,进行性能测试实验,实验结果表明该模型不仅可以改进数据可用性,而且提高了系统存储空间的利用率。     

基于模糊预测的数据复制优化模型的研究

云数据处理系统中广泛采用了多数据副本复制技术,以防止数据丢失,如果数据复制的份数或位置不当,就会引起数据的可用性小于用户期望的数据可用性或存储空间的浪费（如复制份数过多）。针对该问题,经研究提出了一种基于模糊预测的数据复制优化模型,该模型由模糊预测模块和复制优化模块组成。模糊预测模块以节点信息（CPU信息、节点带宽信息、内存信息和硬盘信息）作为输入,预测出节点的可用性;复制优化模块把节点的可用性和用户期望的数据可用性作为输入,计算出在满足用户期望情况下数据复制的份数和位置。提出的复制优化模型能根据云数据存储系统中数据节点可用性实现动态的优化数据复制,能获得较高的存储性价比。模拟实验中基于模糊预测的数据复制优化模型策略需要的存储空间分别是Hadoop策略的42．62％,42．84％,但文件的平均可用性可达到88．69％,90．54％,表明提出的基于模糊预测的复制模型实现了在节省存储空间的同时保证了文件可用性。     

基于模糊预测的数据复制优化模型的研究

云数据处理系统中广泛采用了多数据副本复制技术,以防止数据丢失,如果数据复制的份数或位置不当,就会引起数据的可用性小于用户期望的数据可用性或存储空间的浪费（如复制份数过多）。针对该问题,经研究提出了一种基于模糊预测的数据复制优化模型,该模型由模糊预测模块和复制优化模块组成。模糊预测模块以节点信息（CPU信息、节点带宽信息、内存信息和硬盘信息）作为输入,预测出节点的可用性;复制优化模块把节点的可用性和用户期望的数据可用性作为输入,计算出在满足用户期望情况下数据复制的份数和位置。提出的复制优化模型能根据云数据存储系统中数据节点可用性实现动态的优化数据复制,能获得较高的存储性价比。模拟实验中基于模糊预测的数据复制优化模型策略需要的存储空间分别是Hadoop策略的42.62%,42.84%,但文件的平均可用性可达到88.69%,90.54%,表明提出的基于模糊预测的复制模型实现了在节省存储空间的同时保证了文件可用性。     

乐观复制方法研究

在分布式数据库和数据仓库中,普遍采用数据复制的方法来提高数据的可用性和系统的性能。传统的同步复制方法由于高通信开销和操作延迟,会引起事务的大量阻塞和死锁,已很少采用。取而代之的是乐观复制方法,本文对数据冲突的处理采取乐现方式。本文对乐观复制的各种协议进行了分类和研究,并讨论了乐现复制的延迟与一致性问题。     

清远供电局数据存储优化的思考

数据库系统的性能决定数据库的可用性和生命力。通常数据库系统在业务数据增加后都会存在性能瓶颈,主要涉及数据库服务器、应用程序、操作系统、数据库参数等方面。其中数据库性能调整与优化涉及到多个层面,通过统一规划、系统性调整,可以提高数据库的稳定性和可用性,保障系统高效地运行,解决系统瓶颈。通过建立并实施数据库后台存储过程不断改进的管理机制和设计合理的数据存储过程工作指引,来不断提升整个系统性能,实现对现有信息系统的优化,达到提升业务运营和经营决策效率的目标。     

容错网络服务器系统可用性

通过分析服务器群集系统的结构和工作原理,采用随机Petri网分别建立基本处理单元的软硬件可用性模型、共享数据磁盘可用性模型和服务器可用性模型。综合各个模型,得到服务器集群系统的可用性模型,通过仿真分析可知,采用集群技术可以有效提高服务器的可靠性。     

Goldengate实现异构数据库间的数据同步

随着大数据时代的到来,数据之间的关系越来越复杂;人们需要一种强大的数据拓扑结构来提高数据的可用性;Goldengate是一种基于数据库日志的数据复制产品,可以利用极少的系统资源,实时复制数据,提高数据可用性。     

高效同步复制模型的研究

数据复制是分布式数据库中常用的方法,以提高数据的可用性和系统的性能。文章引入若干技术解决同步复制方法的缺陷,提出了一个高效同步复制的模型,给出了事务的具体处理过程。     

基于MooseFS的纠错码存储方法设计和实现

在云存储技术中,云存储系统的数据容错十分重要,直接关系到整个系统的可用性。当前多数分布式存储系统通过多副本来保证数据的可用性,然而,多副本存储方式也使得数据存储空间翻倍增加,为了降低存储空间,提高数据可用性,有些分布式存储系统开始采用纠错码技术来提高数据可用性和降低数据存储空间占用。通过对MooseFS分布式文件系统进行分析,提出了一种基于MooseFS的纠错码实现方法。通过数据存储效率测试,该方法能够保证常用的“热数据”按照多副本存储,不常用的“冷数据”按照纠错码方式存储,在保证可靠性的同时极大地降低了多副本方式空间占用量。     

Fault detection for Byzantine quorum systems

In this paper, we explore techniques to detect Byzantine server failures in asynchronous replicated data services. Our goal is to detect arbitrary failures of data servers in a system where each client accesses the replicated data at only a subset (quorum) of servers in each operation. In such a system, some correct servers can be out-of-date after a write and can therefore, return values other than the most up-to-date value in response to a client's read request, thus complicating the task of determining the number of faulty servers in the system at any point in time. We initiate the study of detecting server failures in this context, and propose two statistical approaches for estimating the risk posed by faulty servers based on responses to read requests     

A paracasting model for concurrent access to replicated Internet content

In this paper, we develop a model to study how to effectively download a document from a set of replicated servers. We propose a generalized application-layer anycasting protocol, known as paracasting, to advocate concurrent access of a subset of replicated servers to cooperatively satisfy a client's request. Each participating server satisfies the request in part by transmitting a subset of the requested file to the client. The client can recover the complete file when different parts of the file sent from the participating servers are received. This model allows us to estimate the average time to download a file from the set of homogeneous replicated servers, and the request blocking probability when each server can accept and serve a finite number of concurrent requests. Our results show that the file download time drops when a request is served concurrently by a larger number of homogeneous replicated servers, although the performance improvement quickly saturates when the number of servers increases. If the total number of requests that a server can handle simultaneously is finite, the request blocking probability increases with the number of replicated servers used to serve a request concurrently. Therefore, paracasting is effective when a small number of servers, say, up to four, are used to serve a request concurrently.     

Retrieval scheduling for collaborative multimedia presentations

The single-system approach is no longer sufficient to handle the load on popular Internet servers, especially for those offering extensive multimedia content. Such services have to be replicated to enhance their availability, performance, and reliability. In a highly replicated and available environment, server selection is an important issue. In this paper, we propose an application-layer broker (ALB) for this purpose. ALB employs a content-based, client-centric approach to negotiate with the servers and to identify the best server for the requested objects. ALB aims to maximize client buffer utilization in order to efficiently handle dynamic user interactions such as skip, reverse presentation, go back in time. We also present details of a collaborative multimedia presentation platform that we have developed based on ALB.     

WWW集群服务器的数据副本分布方式研究

为了有效地提高WWW服务器的吞吐能力、反应速度和可扩展性,国际上许多著名站点纷纷转向采用WWW集群服务器来替代原有的单一主机服务器.采用不同副本分布方式的WWW集群服务器,其数据可靠性也有所不同.对不同数据副本分布方式进行探讨,同时,论证了最优副本分布方案.     

Probabilistic Quorum Systems

We initiate the study of probabilistic quorum systems, a technique for providing consistency of replicated data with high levels of assurance despite the failure of data servers. We show that this technique offers effective load reduction on servers and high availability. We explore probabilistic quorum systems both for services tolerant of benign server failures and for services tolerant of arbitrary (Byzantine) ones. We also prove bounds on the server load that can be achieved with these techniques.     

Distributed Downloads of Bulk,Replicated Grid Data

Data-sharing scientific communities use storage systems as distributed data stores by replicating content. In such highly replicated environments, a particular dataset can reside at multiple locations and can thus be downloaded from any one of them. Since datasets of interest are significantly large in size, improving download speeds either by server selection or by co-allocation can offer substantial benefits. In this paper, we present an architecture for co-allocating Grid data transfers across multiple connections, enabling the parallel download of datasets from multiple servers. We have developed several co-allocation strategies comprising of simple brute-force, predictive and dynamic load balancing techniques as a means both to exploit rate differences among the various client–server links and to address dynamic rate fluctuations. We evaluate our approaches using the GridFTP data movement protocol in a wide-area testbed and present our results.     

一个网络文件存储系统TNS关键技术

基于多服务器架构、为多用户服务的网络文件存储系统普遍存在资源分配不均,重复文件多,存储空间浪费严重的问题。设计并实现了TNS网络文件存储系统,该系统基于多服务器存储架构,分别由用户服务器、索引服务器、数据服务器、共享服务器、管理服务器和登录服务器组成,为多用户服务,采用一致性Hash实现负载均衡,支持在客户端进行文件粒度的重复数据删除。经过实际生产环境运行测试,具有良好的负载均衡能力和重复数据删除功能,可以有效节省存储空间,提高存储设备利用率。     

Replicated Server Placement with QoS Constraints

The network planning problem of placing replicated servers with QoS constraints is considered. Each server site may consist of multiple server types with varying capacities and each site can be placed in any location among those belonging to a given set. Each client can be served by more than one location as long as the round-trip delay of data requests satisfies predetermined upper bounds. Our main focus is to minimize the cost of using the servers and utilizing the link bandwidth, while serving requests according to their delay constraint. This is an NP-hard problem. A pseudopolynomial and a polynomial algorithm that provide guaranteed approximation factors with respect to the optimal for the problem at hand are presented.     

The Raid distributed database system

Raid, a robust and adaptable distributed database system for transaction processing, is described. Raid is a message-passing system, with server processes on each site. The servers manage concurrent processing, consistent replicated copies during site failures and atomic distributed commitment. A high-level, layered communications package provides a clean, location-independent interface between servers. The latest design of the communications package delivers messages via shared memory in a high-performance configuration in which several servers are linked into a single process. Raid provides the infrastructure to experimentally investigate various methods for supporting reliable distributed transaction processing. Measurements on transaction processing time and server CPU time are presented. Data and conclusions of experiments in three categories are also presented: communications software, consistent replicated copy control during site failures, and concurrent distributed checkpointing. A software tool for the evaluation of transaction processing algorithms in an operating system kernel is proposed     

USING AUTONOMOUS SYSTEM TOPOLOGICAL INFORMATION IN A WEB SERVER PERFORMANCE PREDICTION

The main problem we explore in this paper involves predicting the performance of Web-resource downloads from unknown Web servers, based on knowledge about client-to-unknown-server network paths and performance measurements carried out on the set of known Web servers. We propose unknown-server-to-known-server topology-aware distance metrics based on the knowledge of network paths to both unknown and known servers at the autonomous systems level of Internet organization. The throughput value we want to predict for an unknown-server is approximated by the value achievable for the known-server—called the best one—with the least value of unknown-server-to-known-server distance metrics. The best server is selected using the nearest neighbor algorithm. The usefulness of this method for Web-performance prediction has been confirmed in real-life experiments. The results of the work allowed us to formulate positive recommendations for applying this approach to efficient gaining of Web resources in replicated content systems, file mirrors, content delivery networks, and digital libraries.