PheonixLSM:高性能低空间开销的分布式键值存储本地引擎 |
| |
引用本文: | 李润辉,古亮,喻之斌.PheonixLSM:高性能低空间开销的分布式键值存储本地引擎[J].集成技术,2022,11(2):28-40. |
| |
作者姓名: | 李润辉 古亮 喻之斌 |
| |
作者单位: | 中国科学院深圳先进技术研究院 深圳 518055;深信服科技股份有限公司 深圳 518071,深信服科技股份有限公司 深圳 518071,中国科学院深圳先进技术研究院 深圳 518055 |
| |
摘 要: | 分布式键值存储将数据复制到多个存储服务器的本地引擎中,并通过一致性协议保证各副本数据的一致性。其中,以日志结构合并树为核心数据结构的实现方式最为常见。然而,面向通用业务模式设计的日志结构合并树,并不适合一致性逻辑的特殊业务模式,会引发增删改性能的降低,并在全量修复过程中造成空间放大。针对上述问题,该文提出了一种新型本地引擎 PheonixLSM,通过增加增删改操作和回刷操作的约束,消除了分布式键值存储增删改流程中的双写问题,提升了引擎性能。通过重构日志结构合并树底层的 SST 文件布局,支持删除实时回收空间,消除了全量修复时的额外空间放大。实验结果显示,与原生本地引擎相比,使用 PheonixLSM 的分布式键值存储系统,增删改性能提升 90.7%,全量修复的空间放大从 65.6% 降至 6.4%,并减少了 72.3% 的修复时间。
|
关 键 词: | 分布式系统 键值存储 一致性协议 空间放大 |
PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption |
| |
Authors: | LI Runhui GU Liang YU Zhibin |
| |
Abstract: | Distributed key-value storage is a key component in the distributed storage system, which replicates the key-value pairs to local engines in different storage servers and uses the consensus algorithm to keep replicas consistent. The log-structured merge tree based local key-value engine is the most popular storage algorithm that designed for general purpose. However, the original LSM-tree structure is not suitable for the specific workload of the upper-layer consensus logic. It usually causes performance loss in the write operations and extra spatial amplification during full-node repair. To solve this problem, a local engine named PheonixLSM is designed for distributed key-value storage. PheonixLSM boosts performance by eliminating the double-sync problem for write operations. It also reorganizes the SST file layout to eliminate extra write amplification during full-node repair. Experimental results showed that, compared with distributed key-value storage using default local engine, that PheonixLSM can achieve up to 90.7% write performance gain and reduce the extra write amplification from 65.6% to 6.4%, and the repair time also can be reduced by 72.3%. |
| |
Keywords: | distributed system key-value storage consensus algorithm spatial amplificationF |
本文献已被 万方数据 等数据库收录! |
| 点击此处可从《集成技术》浏览原始摘要信息 |
|
点击此处可从《集成技术》下载全文 |