首页 | 官方网站   微博 | 高级检索  
     

PheonixLSM:高性能低空间开销的分布式键值存储本地引擎
引用本文:李润辉,古亮,喻之斌.PheonixLSM:高性能低空间开销的分布式键值存储本地引擎[J].集成技术,2022,11(2):28-40.
作者姓名:李润辉  古亮  喻之斌
作者单位:中国科学院深圳先进技术研究院 深圳 518055;深信服科技股份有限公司 深圳 518071,深信服科技股份有限公司 深圳 518071,中国科学院深圳先进技术研究院 深圳 518055
摘    要:分布式键值存储将数据复制到多个存储服务器的本地引擎中,并通过一致性协议保证各副本数据的一致性。其中,以日志结构合并树为核心数据结构的实现方式最为常见。然而,面向通用业务模式设计的日志结构合并树,并不适合一致性逻辑的特殊业务模式,会引发增删改性能的降低,并在全量修复过程中造成空间放大。针对上述问题,该文提出了一种新型本地引擎 PheonixLSM,通过增加增删改操作和回刷操作的约束,消除了分布式键值存储增删改流程中的双写问题,提升了引擎性能。通过重构日志结构合并树底层的 SST 文件布局,支持删除实时回收空间,消除了全量修复时的额外空间放大。实验结果显示,与原生本地引擎相比,使用 PheonixLSM 的分布式键值存储系统,增删改性能提升 90.7%,全量修复的空间放大从 65.6% 降至 6.4%,并减少了 72.3% 的修复时间。

关 键 词:分布式系统  键值存储  一致性协议  空间放大

PheonixLSM: Local Key-Value Engine for Distributed Key-Value Storage with High Performance and Low Spatial Consumption
Authors:LI Runhui  GU Liang  YU Zhibin
Abstract:Distributed key-value storage is a key component in the distributed storage system, which replicates the key-value pairs to local engines in different storage servers and uses the consensus algorithm to keep replicas consistent. The log-structured merge tree based local key-value engine is the most popular storage algorithm that designed for general purpose. However, the original LSM-tree structure is not suitable for the specific workload of the upper-layer consensus logic. It usually causes performance loss in the write operations and extra spatial amplification during full-node repair. To solve this problem, a local engine named PheonixLSM is designed for distributed key-value storage. PheonixLSM boosts performance by eliminating the double-sync problem for write operations. It also reorganizes the SST file layout to eliminate extra write amplification during full-node repair. Experimental results showed that, compared with distributed key-value storage using default local engine, that PheonixLSM can achieve up to 90.7% write performance gain and reduce the extra write amplification from 65.6% to 6.4%, and the repair time also can be reduced by 72.3%.
Keywords:distributed system  key-value storage  consensus algorithm  spatial amplificationF
本文献已被 万方数据 等数据库收录!
点击此处可从《集成技术》浏览原始摘要信息
点击此处可从《集成技术》下载全文
设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司    京ICP备09084417号-23

京公网安备 11010802026262号