分布式系统中数据复制的研究与应用

维持数据的一致性是复制数据库并发执行的关键。介绍分析了MS SQL Server 2000中内置的数据复制构件以及MS SQL Server 2000的事务模型,最后结合实际的应用软件开发对数据复制的并发控制原理与措施进行了相关实现。     

Scalable and adaptive log manager in distributed systems

On-line transaction processing (OLTP) systems rely on transaction logging and quorum-based consensus protocol to guarantee durability, high availability and strong consistency. This makes the log manager a key component of distributed database management systems (DDBMSs). The leader of DDBMSs commonly adopts a centralized logging method to writing log entries into a stable storage device and uses a constant log replication strategy to periodically synchronize its state to followers. With the advent of new hardware and high parallelism of transaction processing, the traditional centralized design of logging limits scalability, and the constant trigger condition of replication can not always maintain optimal performance under dynamic workloads. In this paper, we propose a new log manager named Salmo with scalable logging and adaptive replication for distributed database systems. The scalable logging eliminates centralized contention by utilizing a highly concurrent data structure and speedy log hole tracking. The kernel of adaptive replication is an adaptive log shipping method, which dynamically adjusts the number of log entries transmitted between leader and followers based on the real-time workload. We implemented and evaluated Salmo in the open-sourced transaction processing systems Cedar and DBx1000. Experimental results show that Salmo scales well by increasing the number of working threads, improves peak throughput by 1.56\times and reduces latency by more than 4\times over log replication of Raft, and maintains efficient and stable performance under dynamic workloads all the time.     

基于HBase的地理分布副本管理机制

针对分布式存储系统中数据通常在多个数据中心有冗余的副本进行备份,需要健壮的机制维护各个副本的一致性,对分布式系统的副本复制理论作了深入研究后,提出了一套管理地理分布副本的算法.微软研究院提出服务等级协议,把用户对一致性的要求分成若干级别,每个级别与用户可容忍的延迟有关.系统保证在可容忍的延迟范围内,用户能拥有较高的服务等级.Tuba系统拓展了Pileus,允许系统根据所有用户发送的统计信息动态地改变主从副本存放的位置,以提高系统的平均性能,但Tuba系统的复制只是基于单个目标单位进行.对Tuba系统中的方法作出改进,提出了一套改变主从副本存放位置的算法,并在HBase分布式系统的副本复制中实现了该机制.系统完成后,通过实验验证了在改变主从副本存放位置时综合考虑两个region的相关性可以提高系统整体的效用.     

移动分布式实时事务实时原子提交

形式地给出了移动分布式实时事务实时原子提交协议的定义,在此基础上提出了适合于移动分布式实时事务的实时原子提交协议：一阶段实时原子提交协议(1PRACP)。1PRACP通过参与者与协调者的一次消息交换,在一个阶段完成移动分布式实时事务提交活动;结合超时恢复处理协议,1PRACP能避免由于站点故障或网络通信链路故障而导致的阻塞。对1PRACP进行了性能比较和评测,显示了它在各方面的优越性。     

A Transactional Asynchronous Replication Scheme for Mobile Database Systems

In mobile database systems,mobility of users has a significant impact on data replication.As a result,the various replica control protocols that exist today in traditional distributed and multidatabase environments are no longer suitable To solve this problem,a new mobile database replication scheme,the Transaction-Level Result-Set Propagation(TLRSP)model,is put forward in this paper,The conflict dectction and resolution strategy based on TLRSP is discussed in detail,and the implementation algorithm is proposed,In order to compare the performance of the TLRSP model with that of other mobile replication schemes,we have developed a detailde simulation model.Experimantal results show that the TLRSP model provides an effcient support for replicated mobile database systems by reducing reprocessing overhead and maintaining database consistency.     

乐观复制方法研究

在分布式数据库和数据仓库中,普遍采用数据复制的方法来提高数据的可用性和系统的性能。传统的同步复制方法由于高通信开销和操作延迟,会引起事务的大量阻塞和死锁,已很少采用。取而代之的是乐观复制方法,本文对数据冲突的处理采取乐现方式。本文对乐观复制的各种协议进行了分类和研究,并讨论了乐现复制的延迟与一致性问题。     

多副本分布式事务处理关键技术及典型数据库系统综述

数据复制是分布式数据库提高可用性的重要手段,通过在不同区域放置数据库的部分副本,还可以提高本地读写操作的响应速度,增加副本数量也会提升读负载的线性扩展能力.考虑到这些优良特性,近年来国内外都出现了众多多副本分布式数据库系统,包括Google Spanner、CockroachDB、TiDB、OceanBase等一系列主流的工业界系统,也出现了包括Calvin、Aria、Berkeley Anna等一系列优秀的学术界系统.然而,多副本数据库带来诸多收益的同时,也带来了一致性维护、跨节点事务、事务隔离等一系列挑战.总结分析现有的复制架构、一致性维护策略、跨节点事务并发控制等技术,对比几个代表性多副本数据库系统之间在分布式事务处理方面上的差异与共同点,并在阿里云环境下搭建跨区域的分布式集群环境,对几个代表性系统的分布式事务处理能力进行了实验测试分析.     

分布式仿真系统中的全局一致性问题研究

系统一致是分布仿真应用DSA的基础和前提。为提高DSA的全局一致性程度,获得真实、可信的仿真结果,首先阐述了DSA全局一致性问题;然后以某分布仿真系统为例,系统阐述了其全局不一致现象;在此基础上,分析了DSA系统的全局一致性需求;最后,重点给出了模型域、服务域和认知域内的一致性控制策略与建议。本研究可为DSA系统设计与开发人员提供技术与方法指导。     

A performance evaluation of distributed database architectures

The globally integrated contemporary business environment has prompted new challenges to database architectures in order to enable organizations to improve database applications performance, scalability, reliability and data privacy in adapting to the evolving nature of business. Although a number of distributed database architectures are available for choice, there is a lack of an in‐depth understanding of the performance characteristics of these database architectures in a comparison way. In this paper, we report a performance study of three typical (centralized, partitioned and replicated) database architectures. We used the TPC‐C as the evaluation benchmark to simulate a contemporary business environment, and a commercially available database management system that supports the three architectures. We compared the performance of the partitioned and replicated architectures against the centralized database, which results in some interesting observations and practical experience. The findings and the practice presented in this paper provide useful information and experience for the enterprise architects and database administrators in determining the appropriate database architecture in moving from centralized to distributed environments. Copyright © 2012 John Wiley & Sons, Ltd.     

Divergence control for distributed database systems

This paper presents distributed divergence control algorithms for epsilon serializability for both homogeneous and heterogeneous distributed databases. Epsilon serializability allows for more concurrency by permitting non-serializable interleavings of database operations among epsilon transactions. We first present a strict 2-phase locking divergence control algorithm and an optimistic divergence control algorithm for a homogeneous distributed database system, where the local orderings of all the sub-transactions of a distributed epsilon transaction are the same. In such an environment, the total inconsistency of a distributed epsilon transaction is simply the sum of those of all its sub-transactions. We then describe a divergence control algorithm for a heterogeneous distributed database system, where the local orderings of all the sub-transactions of a distributed epsilon transaction may not be the same and the total inconsistency of a distributed epsilon transaction may be greater than the sum of those of all its sub-transactions. As a result, in addition to executing a local divergence control algorithm in each site to maintain the local inconsistency, a global mechanism is needed to take into account the additional inconsistency Recommended by: Meichum Hsu     

Processing real-time transactions in a replicated database system

A database system supporting a real-time application has to provide real-time information to the executing transactions. Each real-time transaction is associated with a timing constraint, typically in the form of a deadline. It is difficult to satisfy all timing constraints due to the consistency requirements of the underlying database. In scheduling the transactions it is aimed to process as many transactions as possible within their deadlines. Replicated database systems possess desirable features for real-time applications, such as a high level of data availability, and potentially improved response time for queries. On the other hand, multiple copy updates lead to a considerable overhead due to the communication required among the data sites holding the copies. In this paper, we investigate the impact of storing multiple copies of data on satisfying the timing constraints of real-time transactions. A detailed performance model of a distributed database system is employed in evaluating the effects of various workload parameters and design alternatives on the system performance. The performance is expressed in terms of the fraction of satisfied transaction deadlines. A comparison of several real-time concurrency control protocols, which are based on different approaches in involving timing constraints of transactions in scheduling, is also provided in performance experiments. Recommended by: A. ElmagarmidThis work was initiated while the author was at the Computer Science Department, University of Illinois at Urbana-Champaign.     

数据复制与一致性

数据复制是分布式数据库和数据仓库中常用的方法,以提高数据的可用性和系统的性能。文中介绍了在数据库系统中广泛使用的数据复制方法,分析了其优缺点和适用的范围,重点讨论了异步复制方法中的一致性恢复、一致性偏离的度量和数据存取的限制等问题,给出了数据复制策略的评价目标。     

一种海量的人口信息存储与复制策略

在分析公安信息系统环境和持征的基础上，提出了一种适合公安信息系统的海量人口信息存储与复制策略，该策略采用多数据库结构，实现了数据访问的负载分担，大大提高PSMIS的访问性能，可用性和可靠性，同时，该策略支持可接受的数据弱一致性，能够使多数据库系统在有限的时间内收敛，达到各个数据节点数据完全一致，因而该策略能够很好的应用于公安信息系统的海量人口信息管理系统。     

实时数据库系统的正确性

实时数据库系统的正确性就是与定时限制紧密相联的数据库一致性和事务正确性，中对此作了详细讨论，指出ＲＴＯＢ的一致性包括内部一致、外部一致和相互一致；事务正确性包括结果正确性，行为正确性，结构正确性和时间正确性，进一步的关键问题是判断正确性的准则和表示方法，对此中提出了一个经历模型，它可方便地且来说明各种恰当的正确性。     

分布式数据库中一致性与可用性的关系

随着各类应用在数据量和业务量上的扩展,单机数据库系统越发难以应对现实需求。分布式数据库能够根据业务的需求动态地扩容,因此逐步开始受到应用的青睐。近年来,分布式数据库产品层出不穷,并在互联网应用中被大量投入使用。然而,分布式数据库的系统复杂度前所未有。为了让系统可用,设计者需要在多种属性中作合理选择和折中。这造成现有的数据库产品形态各异、优缺点对比分明。至今为止,尚未有人对分布式数据库的设计空间和折中方案进行过深入分析和整理。本文作者在对多个分布式数据库产品进行深入理解之后认识到：分布式数据库系统的设计方案可以通过三个属性进行基本刻画–操作一致性、事务一致性和系统可用性。虽然这三个属性并不新颖,但它们在数据库语境下的含义在文献中尚未得到充分澄清。本文对这三个属性进行澄清,并通过它们对典型数据库产品的格局进行概括、对现有的分布式数据库技术进行综述。此外,本文还对这三个属性之间的相互关系进行深入分析,以期帮助未来的开发者在分布式数据库的设计过程中作出合理选择。     

混合的实时嵌套事务并发控制协议

嵌套事务作为扩展事务模型的一种,提供了事务内部的并行性和更好的失败恢复选择,具有较为广泛的应用,然而却给事务并发控制带来了更高的复杂性,尤其在实时数据库中事务具有时间限制.针对实时嵌套事务模型,提出了一种混合并发控制协议,对同一事务树内的子事务采用锁协议,对不同事务树之间的事务采用基于动态调整串行化顺序的乐观并发控制协议,提高了事务的成功率,最后证明了协议的正确性.     

A Commit Strategy for Distributed Real-Time Transaction

Ramamritham gives three common types of constraints for the execution his-tory of concurrent transactions. This paper extends the constraints and gives the fourth type of constraint. Then the weak commit dependency and abort dependency between transactions, be-cause of data access conflicts, axe analyzed. Based on the analysis, an optimistic commit protocol 2LC (two-Level Commit) is proposed, which is specially designed for the distributed real-time do-main. It allows transactions to optimistically access the locked data in a controlled manner, which reduces the data inaccessibility and priority inversion inherent and undesirable in distributed real-time database systems. Furthermore, if the prepared transaction is aborted, the transactions in its weak commit dependency set will execute as normal according to 2LC. Extensive simulation ex-periments have been performed to compare the performance of 2LC with that of the base protocol,the permits reading of modified prepared-data for timeliness (PROMPT) and the deadline-driven conflict resolution (DDCR). The simulation results show that 2LC is effective in reducing the num-ber of missed transaction deadlines. Furthermore, it is easy to be incorporated with the existing concurrency control protocols.     

分布式数据库的复制和分片

通过对集中式数据库和分布式数据库进行比较,指出集中式数据库在存储能力、访问速度等方面存在的问题;对分布式数据库中的数据复制和分片的概念、原则、方法进行了详细的论述;最后对数据复制和分片要解决的关键问题进行了分析。     

基于组通信技术的同步复制方法的研究

针对同步复制方法本身的缺点提出了一种新的研究思想，即引入组通信技术。利用组通信技术中关于转发信息的定序性质，借助天类队列的并发控制思想，很好地解决了复制领域中数据的一致性及通信量的问题。     

基于网关模式的数据同步复制的研究与应用

为了满足分布式实时数据库中对关键数据的强一致性和系统中各个节点数据副本的完整性的要求 ,提出了基于网关式的主副本复制技术的方案 ,很好地实现了分布式实时数据库的异地镜像功能。从成功用于电力系统的配调综合自动化管理 (SCADA)系统的实效来看 ,文中所述思想可行 ,整个系统对在传输系统关键数据时能迅速准确同步更新到网络中的各个节点 ,能稳定可靠地长时间运行