支持分布式合作实时事务处理的协同检验点方法

在实时事务执行时，事务故障或数据竞争会导致事务重启，为减少事务重启损失的工作量，可以采用检验点技术保证事务的时间正确性．在一类分布式实时数据库应用中，不同结点的事务通过消息交换形成合作关系，为保证合作事务间的全局一致性，当某一事务记检验点时，相关事务也要记检验点．传统协同检验点方法没有考虑应用的定时约束，不能很好地支持分布式合作实时事务处理．该文提出了一种基于图论的协同检验点方法，利用在每个计算结点上为每个合作事务集维护的局部有向图，使用一个基于图论的计算过程标识出应记检验点的事务，该方法既具有最小协同检验点特性，又使全局检验点的时延最小．实验表明该算法减少了全局检验点时延，有利于实时事务截止期的满足．     

Distributed optimistic concurrency control methods forhigh-performance transaction processing

There is an ever-increasing demand for more complex transactions and higher throughputs in transaction processing systems leading to higher degrees of transaction concurrency and, hence, higher data contention. The conventional two-phase locking (2PL) Concurrency Control (CC) method may, therefore, restrict system throughput to levels inconsistent with the available processing capacity. This is especially a concern in shared-nothing or data-partitioned systems due to the extra latencies for internode communication and a reliable commit protocol. The optimistic CC (OCC) is a possible solution, but currently proposed methods have the disadvantage of repeated transaction restarts. We present a distributed OCC method followed by locking, such that locking is an integral part of distributed validation and two-phase commit. This method ensures at most one re-execution, if the validation for the optimistic phase fails. Deadlocks, which are possible with 2PL, are prevented by preclaiming locks for the second execution phase. This is done in the same order at all nodes. We outline implementation details and compare the performance of the new OCC method with distributed 2PL through a detailed simulation that incorporates queueing effects at the devices of the computer systems, buffer management, concurrency control, and commit processing. It is shown that for higher data contention levels, the hybrid OCC method allows a much higher maximum transaction throughput than distributed 2PL in systems with high processing capacities. In addition to the comparison of CC methods, the simulation study is used to study the effect of varying the number of computer systems with a fixed total processing capacity and the effect of locality of access in each case. We also describe several interesting variants of the proposed OCC method, including methods for handling access variance, i.e., when rerunning a transaction results in accesses to a different set of objects     

Distributed concurrency control based on limited wait-depth

The performance of high-volume transaction processing systems for business applications is determined by the degree of contention for hardware resources as well as for data. Hardware resource requirements may be met cost-effectively with a data-partitioned or shared-nothing architecture. However, the two-phase locking (2PL) concurrency control method may restrict the performance of a shared-nothing system more severely than that of a centralized system due to increased lock holding times. Deadlock detection and resolution are an added complicating factor in shared-nothing systems. The authors describe distributed Wait-Depth Limited (WDL) concurrency control (CC), a locking-based distributed CC method that limits the wait-depth of blocked transactions to one, thus preventing the occurrence of deadlocks. Several implementations of distributed WDL which vary in the number of messages and the amount of information available for decision making are discussed. The performance of a generic implementation of distributed WDL is compared with distributed 2PL (with general-waiting policy) and the Wound-Wait CC method through a detailed simulation. It is shown that distributed WDL behaves similarly to 2PL for low lock contention levels, but for substantial lock contention levels (caused by higher degrees of transaction concurrency), distributed WDL outperforms the other methods to a significant degree     

Hardware transactional memory architecture with adaptive version management for multi-processor FPGA platforms

Multiprocessor embedded systems integrates diverse dedicated processing units to handle high performance applications such as in multimedia and network processing. However, lock-based synchronization limits the efficiency of such heterogeneous concurrent systems. Hardware Transactional Memory (HTM) is a promising approach in creating an abstraction layer for multi-threaded programming. However, HTM performance is application-specific and determined by version and conflict management configurations. Most previous HTM implementations for embedded system in literature were built on fixed version management that result in significant performance loss when transaction behaviour changes. In this paper, we propose a HTM targeted for embedded applications which is able to adapt its version management based on application behaviour at runtime. It is prototyped and analysed on Altera Cyclone IV platform. Random requests at different contention levels and different transaction sizes are used to verify the performance of the proposed HTM. Based on our experiments, lazy version management is able to obtain up to 12.82% speed-up compared to eager version management at high contention level. Meanwhile, eager version management obtains up to 37.84% speed-up compared to lazy version management at low contention. The adaptive mechanism is able to switch configuration at runtime based on applications behaviour for maximum performance.     

面向更新密集型应用的内存数据库高效检查点技术

面向更新密集型应用的内存数据库系统,其检查点技术应符合几个关键的要求,包括检查点操作对正常事务处理的干扰尽可能小、能够处理存取倾斜状况、支持数据库系统的快速恢复、提供恢复过程中的系统可用性等.该文提出一种事务一致的分区检查点技术,采用基于元组的动态多版本并发控制机制,避免了读写事务的加锁冲突,提高系统吞吐能力;检查点操作以只读事务形式实现,存多版本并发控制下,避免检查点操作对正常事务处理的堵塞;由于检查点文件是事务一致的,只需要记录事务的Redo 日志信息,在系统恢复过程中,只需要对日志文件进行一遍扫描处理,加快恢复过程;基于优先级的数据分区装载和恢复,使得恢复过程中新事务的数据存取请求迅速得到满足,保证了恢复过程中的系统可用性.由于采用两级版本管理机制以及动态版本共享技术,多版本管理的空间开销降低到可以接受的水平.实验结果表明,文中提出的检查点技术方案获得比模糊检查点技术高27%的系统吞吐量,同时版本管理的空间开销在可接受的范围之内,满足高性能应用的要求.     

Performance analysis of concurrency control using locking withdeferred blocking

The concurrency control (CC) method employed can be critical to the performance of transaction processing systems. Conventional locking suffers from the blocking phenomenon, where waiting transactions continue to hold locks and block other transactions from progressing. In a high data contention environment, as an increasing number of transactions wait, a larger number of lock requests get blocked and fewer lock requests can get through. The proposed scheme reduces the blocking probability by deferring the blocking behavior of transactions to the later stages of their execution. By properly balancing the blocking and abort effects, the proposed scheme can lead to better performance than either the conventional locking or the optimistic concurrency control (OCC) schemes at all data and resource contention levels. We consider both static and dynamic approaches to determine when to switch from the nonblocking phase to the blocking phase. An analytical model is developed to estimate the performance of this scheme and determine the optimal operating or switching point. The accuracy of the analytic model is validated through a detailed simulation     

A Low-Cost Checkpointing Technique for Distributed Databases

For distributed databases, checkpointing is used to ensure an efficient way to perform global reconstruction. However, the need for global reconstruction is infrequent. Most current checkpointing approaches for distributed databases are too expensive during run time. Some of them allow the checkpointing process to run in parallel with normal transactions at the cost of more data and resource contention, which in turn causes longer response time for normal transactions. Thus, an efficient way to checkpoint distributed databases is needed to avoid degrading the system performance. This paper presents a low-cost solution, called Loosely Synchronized Local Fuzzy Checkpointing (LSLFC), to these problems. LSLFC supports global reconstruction, and our performance study shows that LSLFC has little overhead during run time.     

一种高效的合作实时事务并行检验点算法

许多数据和活动上都有很强时间性的应用在地理上同时具有分布性,这种应用需求使得分布式实时数据库的研完成为数据库研究领域的热点。在实时事务执行时,事务故障或数据竞争会导致事务重启,为了减少因重启而损失的工作量,可以采用检验点技术以利于事务时间正确性的满足。在一些分布式实时数据库应用中,不同结点的事务通过消息交换形成合作关系,当某一事务记检验点时,为保证合作事务间的全局一致性,相关事务也要相应地记检验点。传统的协同检验点方法没有考虑应用的定时约束,不能很好地支持分布式实时事务处理。本文提出了一种高效的并行协同检验点方法,该算法既具有最小协同检验点特性又使全局检验点过程延时最小。实验表明该算法减少了全局检验点阻塞时间,有利于分布式实时事务截止期的满足。     

A quasi-synchronous checkpointing algorithm that prevents contention for stable storage

Checkpointing and rollback recovery are established techniques for handling failures in distributed systems. Under synchronous checkpointing, each process involved in the distributed computation takes checkpoint almost simultaneously. This causes contention for network stable storage and hence degrades performance as processes may have to wait for long time for the checkpointing operation to complete. In this paper, we propose a staggered quasi-synchronous checkpointing algorithm which reduces contention for network stable storage without any synchronization overhead.     

Optimizing checkpoint-based fault-tolerance in distributed stream processing systems: Theory to practice

Fault-tolerance is an essential part of a stream processing system that guarantees data analysis could continue even after failures. State-of-the-art distributed stream processing systems use checkpointing to support fault-tolerance for stateful computations where the state of the computations is periodically persisted. However, the frequency of performing checkpoints impacts the performance (utilization, latency, and throughput) of the system as the checkpointing process consumes resources and time that can be used for actual computations. In practice, systems are often configured to perform checkpoints based on crude values ignoring factors such as checkpoint and restart costs, leading to suboptimal performance. In our previous work, we proposed a theoretical optimal checkpoint interval that maximizes the system utilization for stream processing systems to minimize the impact of checkpointing on system performance. In this article, we investigate the practical benefits of our proposed theoretical optimal by conducting experiments in a real-world cloud setting using different streaming applications; we use Apache Flink, a well-known stream processing system for our experiments. The experiment results demonstrate that an optimal interval can achieve better utilization, confirming the practicality of the theoretical model when applied to real-world applications. We observed utilization improvements from 10% to 200% for a range of failure rates from 0.3 failures per hour to 0.075 failures per minute. Moreover, we explore how performance measures: latency and throughput are affected by the optimal interval. Our observations demonstrate that significant improvements can be achieved using the optimal interval for both latency and throughput.     

Divergence control algorithms for epsilon serializability

The paper presents divergence control methods for epsilon serializability (ESR) in centralized databases. ESR alleviates the strictness of serializability (SR) in transaction processing by allowing for limited inconsistency. The bounded inconsistency is automatically maintained by divergence control (DC) methods in a way similar to SR is maintained by concurrency control (CC) mechanisms. However, DC for ESR allows more concurrency than CC for SR. The authors first demonstrate the feasibility of ESR by showing the design of three representative DC methods: two-phase locking, timestamp ordering and optimistic approaches. DC methods are designed by systematically enhancing CC algorithms in two stages: extension and relaxation. In the extension stage, a CC algorithm is analyzed to locate the places where it identifies non-SR conflicts of database operations. In the relaxation stage, the non-SR conflicts are relaxed to allow for controlled inconsistency. They then demonstrate the applicability Of ESR by presenting the design of DC methods using other most known inconsistency specifications, such as absolute value, age and total number of nonserializably read data items. In addition, they present a performance study using an optimistic divergence control algorithm as an example to show that a substantial improvement in concurrency can be achieved in ESR by allowing for a small amount of inconsistency     

Buffer analysis for a data sharing environment with skewed dataaccess

Examines the effect of skewed database access on the transaction response time in a multisystem data sharing environment, where each computing node has access to shared data on disks, and has a local buffer of recently accessed granules. Skewness in data access can increase data contention since most accesses go to few data items. For the same reason, it can also increase the buffer hit probability. We quantify the resultant effect on the transaction response time, which depends not only on the various system parameters but also on the concurrency control (CC) protocol. Furthermore, the CC protocol can give rise to rerun transactions that have different buffer hit probabilities. In a multisystem environment, when a data block gets updated by a system, any copies of that block in other systems' local buffers are invalidated. Combining these effects, we find that higher skew does not necessarily lead to worse performance, and that with skewed access, optimistic CC is more robust than pessimistic CC. Examining the buffer hit probability as a function of the buffer size, we find that the effectiveness of additional buffer allocation can be broken down into multiple regions that depend on the access frequency distribution     

Nonblocking checkpointing for optimistic parallel simulation: description and an implementation

Describes a nonblocking checkpointing mode in support of optimistic parallel discrete event simulation. This mode allows real concurrency in the execution of state saving and other simulation specific operations (e.g, event list update, event execution) with the aim of removing the cost of recording state information from the completion time of the parallel simulation application. We present an implementation of a C library supporting nonblocking checkpointing on a myrinet based cluster, which demonstrates the practical viability of this checkpointing mode on standard off-the-shelf hardware. By the results of an empirical study on classical parameterized synthetic benchmarks, we show that, except for the case of minimal state granularity applications, nonblocking checkpointing allows improvement of the speed of the parallel execution, as compared to commonly adopted, optimized checkpointing methods based on the classical blocking mode. A performance study for the case of a personal communication system (PCS) simulation is additionally reported to point out the benefits from nonblocking checkpointing for a real world application.     

Analyzing Conflicts in Hardware-Supported Memory Transactions

In order to exploit parallel resources, most transactional memory (TM) systems execute atomic blocks concurrently and must thus be prepared for data conflicts. In the event of a conflict, a TM system must choose a policy to decide when and how to manage the resulting contention. In this paper, we analyze the interplay between conflict resolution time and contention management policy in the context of hardware-supported TM systems, highlighting the performance and implementation implications of the various points in the design space. We show that both policy decisions have a significant impact on the ability to exploit available parallelism and ensure progress of individual transactions. Our analysis corroborates previous research findings that stalling (especially at access time and retrying the problematic access) helps side-step conflicts and avoid wasted work. We demonstrate that conflict resolution time has the dominant effect on performance: Lazy (which delays resolution to commit time) uncovers more parallelism than Eager (which resolves conflicts at access time). With Lazy, in spite of an aborted transaction tending to waste more work (31% more than Eager), the cumulative amount of wasted work is lower since fewer transactions are aborted (1.6 × less than Eager). Lazy’s delayed conflict resolution also ensures progress and decreases the likelihood of pathologies (like livelock) while Eager needs sophisticated priority mechanisms in the contention management to avoid pathologies. Finally, we evaluate a mixed conflict resolution that detects write–write conflicts eagerly while detecting read–write conflicts lazily, and show that it provides a good compromise between implementation complexity and exploiting concurrency for performance.     

A study of concurrency control in real-time, active databasesystems

Real-time active database systems (RTADBSs) have attracted a considerable amount of research attention in the past and a number of important applications have been identified for such systems, such as telecommunications network management, automated air traffic control, automated financial trading, process control and military command and control systems. In spite of the recognized importance of this area, very little research has been devoted to exploring the dynamics of transaction processing in RTADBSs. Concurrency control (CC) constitutes an integral part of any transaction processing strategy and, thus, deserves special attention. We study CC strategies in RTADBSs and postulate a number of CC algorithms. These algorithms exploit the special needs and features of RTADBSs and are shown to deliver substantially superior performance to conventional real-time CC algorithms     

Priority Scheduling of Transactions in Distributed Real-Time Databases

One of the most important issues in the design of distributed real-time database system (DRTDBS) is transaction scheduling which consists of two parts: priority scheduling and real-time concurrency control. In the past studies, mostly, these issues are studied separately although they have a very close interaction with each other. In this paper, we propose new priority assignment policies for DRTDBS and study their impact on two typical real-time concurrency control protocols (RT-CCPs), High Priority Two Phase Locking (HP-2PL) and Optimistic Concurrency Control with Broadcast Commit (OCC-BC). Our performance results show that many factors, such as data conflict resolution, degree of data contention and transaction restarts, that are unique to database systems, have significant impact on the performance of the policies which in turn affect the performance of the real-time concurrency control protocols. OCC-BC is more affected by the priority assignment policies than HP-2PL owing to the late detection of conflict. In the design of priority assignment policies, we have found that neither the purely deadline driven policies nor data contention driven policies are suitable for DRTDBS. Our proposed policy, the Mixed Method (MM), which considers both transaction timeliness and data contention, outperforms other policies over a wide range of system parameter settings.     

Performance analysis of dynamic locking with the no-waiting policy

A transaction processing system with two-phase dynamic locking with the no waiting policy (DLNW) for concurrency control is considered. In this method, transactions making conflicting lock requests are aborted and restarted rather than blocked, thereby eliminating blocking delays (and deadlocks), but making it susceptible to cyclic restarts. Cyclic restarts are dealt with by delaying the restart of a transaction encountering a lock conflict or replacing it with a new transaction. Analytic solution methods for evaluating the performance of the variants of the DLNW method are described. The analytic methods, validated against simulation and shown to be acceptably accurate, are used to study the effect of the following parameters on system performance: transaction size and its distribution, degree of concurrency, the throughput characteristic of the computer system, and the mixture of read-only query and update transactions. A comparison of the DLNW and dynamic locking with waiting (DLW) methods shows that DLW provides higher throughput than DLNW, except when there is no hardware resource contention and conflicted transactions can be replaced by new transactions. The DLNW method outperforms the time-stamp ordering method, as observed from simulation results as well as case by case analyses of possible scenarios     

Incremental recovery in main memory database systems

Recovery activities, like checkpointing and restart, in traditional database management systems are performed in a quiescent state where no transactions are active. This approach impairs the performance of online transaction processing systems, especially when a large volatile memory is used. An incremental scheme for performing recovery in main memory database systems (MMDBs), in parallel with transaction execution, is presented. A page-based incremental restart algorithm that enables the resumption of transaction processing as soon as the system is up is proposed. Pages are recovered individually and according to the demands of the post-crash transactions. A method for propagating updates from main memory to the backup database on disk is also provided. The emphasis is on decoupling the I/O activities related to the propagation to disk from the forward transaction execution in memory. The authors also construct a high-level recovery manager based on operation logging on top of the page-based algorithms. The proposed algorithms are motivated by the characteristics of large MMDBs, and exploit the technology of nonvolatile RAM     

Real-time transaction scheduling in database systems

A database system supporting a real-time application, which can be called “a real-time database system (RTDBS)”, has to provide real-time information to the executing transactions. Each RTDB transaction is associated with a timing constraint, usually in the form of a deadline. Efficient resource scheduling algorithms and concurrency control protocols are required to schedule the transactions so as to satisfy both timing constraints and data consistency requirements. In this paper,^† we concentrate on the concurrency control problem in RTDBSs. Our work has two basic goals: real-time performance evaluation of existing concurrency control approaches in RTDBSs, and proposing new concurrency control protocols with improved performance. One of the new protocols is locking-based, and it prevents the priority inversion problem^{, †} by scheduling the data lock requests based on prioritizing data items. The second new protocol extends the basic timestamp-ordering method by involving real-time priorities of transactions in the timestamp assignment procedure. Performance of the protocols is evaluated through simulations by using a detailed model of a single-site RTDBS. The relative performance of the protocols is examined as a function of transaction load, data contention (which is determined by a number of system parameters) and resource contention. The protocols are also tested under various real-time transaction processing environments. The performance of the proposed protocols appears to be good, especially under conditions of high transaction load and high data contention.     

The Raid distributed database system

Raid, a robust and adaptable distributed database system for transaction processing, is described. Raid is a message-passing system, with server processes on each site. The servers manage concurrent processing, consistent replicated copies during site failures and atomic distributed commitment. A high-level, layered communications package provides a clean, location-independent interface between servers. The latest design of the communications package delivers messages via shared memory in a high-performance configuration in which several servers are linked into a single process. Raid provides the infrastructure to experimentally investigate various methods for supporting reliable distributed transaction processing. Measurements on transaction processing time and server CPU time are presented. Data and conclusions of experiments in three categories are also presented: communications software, consistent replicated copy control during site failures, and concurrent distributed checkpointing. A software tool for the evaluation of transaction processing algorithms in an operating system kernel is proposed