Long-haul secure data transfer using hardware-assisted GridFTP

Extreme-scale scientific collaborations require high-performance wide-area end-to-end data transports to enable fast and secure transfer of high data volumes among collaborating institutions. GridFTP is the de facto protocol for large-scale data transfer in science environments. Existing predominant network transport protocols such as TCP have serious limitations that consume significant CPU power and prevent GridFTP from achieving high throughput on long-haul networks with high latency and potential packet loss, reordering and jitter. On the other hand, protocols such as UDT that address some of the TCP shortcomings demand high computing resources on data transfer nodes. These limitations have caused underutilization of existing high-bandwidth links in scientific and collaborative grids. To address this situation, we have enhanced Globus GridFTP, the most widely used GridFTP implementation, by developing transport offload engines such as UDT and iWARP on SmartNIC, a programmable 10GbE network interface card (NIC). Our results show significant reduction in server utilization and full line-rate sustained bandwidth in high-latency networks, as measured for up to 100 ms of network latency. In our work, we also offload OpenSSL on SmartNIC to reduce host utilization for secure file transfers. The offload engine can provide line-rate data channel encryption/decryption on top of UDT offload without consuming additional host CPU resources. Lower CPU utilization leads to increased server capacity, which allows data transfer nodes to support higher network and data-processing rates. Alternatively, smaller or fewer DTNs can be used for a particular data rate requirement.     

InfiniBand结构中SDP协议分析

InfiniBand是一种新型宽带互联结构，可以提高CPU之间、服务器各设备之间、网络域之间的通信速率。SDP是InfiniBand结构中一种轻量级通信协议，保持了TCP/IP上的SOCK_STREAM套接字语法，通过旁路操作系统内核，实现了零拷贝，从而大大提高了传输速率。文章介绍了InfiniBand的互联结构和通信模式、分析了SDP协议在系统中的结构模型。最后从缓冲模型、连接建立过程和数据传输机制等方面分析了SDP的具体执行过程。     

用户级通信中基于网络接口的虚实地址变换技术

用户级通信允许应用程序直接访问网络接口,减小了通信操作的软件层开销。为了支持用户级通信,高效的虚拟地址到物理地址的变换起到关键作用。本文提出了基于地址变换表的地址变换机制,虚实地址变换都在网络接口控制器上完成,变换过程不需要操作系统的参与,并且无需失效处理。采用这种机制,我们实现了基于PCI-X面向集群系统的互连通信子系统CNI。实际测试获得了2．4μs的最小单边延迟和850MB／s的峰值带宽。     

基于SRP的IB SAN设计与实现*

介绍了基于SRP的IB-SAN的整体结构,分析了SCSI read/write命令处理过程中的两个关键问题,即数据缓冲区的内存注册/注销方法和RDMA操作的执行者获取对方数据缓冲区信息的方式。实验结果表明,SRP能够使InfiniBand网络性能得到充分发挥,系统的总体I/O性能明显优于基于iSCSI协议的IB-SAN。     

当前网络存储中零拷贝技术浅析

零拷贝技术能有效消除端系统中非必要的拷贝过程，大大提高网络吞吐率，降低网络存储中的通讯延迟，提高存储效率。介绍了几种网络存储中的零拷贝技术，特别是RDMA技术，并对其特点进行了比较分析，展望了未来零拷贝技术的发展前景。     

基于InfiniBand网络的MPI实现与分析

近年来,利用普通PC和高速网络搭建集群已经成为构建高性能计算环境的一种趋势。虽然InfiniBand在高性能计算领域还是相对比较新的技术,但是它的丰富特性使得它在高性能领域所占份额日趋扩大。InfiniBand Architecture(IBA)这种工业标准的主要设计思想是采用支持多并发链接的“转换线缆”技术提供高性能和高可靠性.这篇文章介绍了InfiniBand的体系结构和在InfiniBand上实现MPI的有关技术。最后在集群环境中,用MPI对InfiniBand网络的各项性能指标进行了测试。     

High Performance RDMA-Based MPI Implementation over InfiniBand

Although InfiniBand Architecture is relatively new in the high performance computing area, it offers many features which help us to improve the performance of communication subsystems. One of these features is Remote Direct Memory Access (RDMA) operations. In this paper, we propose a new design of MPI over InfiniBand which brings the benefit of RDMA to not only large messages, but also small and control messages. We also achieve better scalability by exploiting application communication pattern and combining send/receive operations with RDMA operations. Our RDMA-based MPI implementation achieves a latency of 6.8 sec for small messages and a peak bandwidth of 871 million bytes/sec. Performance evaluation shows that for small messages, our RDMA-based design can reduce the latency by 24%, increase the bandwidth by over 104%, and reduce the host overhead by up to 22% compared with the original design. For large data transfers, we improve performance by reducing the time for transferring control messages. We have also shown that our new design is beneficial to MPI collective communication and NAS Parallel Benchmarks.     

支持共享存储的千兆机群网络的实现

在共享虚拟存储机群系统中,互连网络已经成为提高并行性能的瓶颈.本文设计并实现了一种机群互连网络—FRAMP,它不但提供高性能的消息传递,而且对共享虚拟存储提供一些低层硬件支持,包括基于虚地址的RDMA读写操作和TLB自动更新.设计了一种定制的用户级通信协议,大幅度降低系统和协议开销.设计并实现了基于输入缓冲输出分布式调度的交叉开关的原型交换机和支持四种数据传输方式的网络接口,改进了虫洞交换并在一个时钟周期内完成带初始值的轮循调度.测试表明,FRAMP网络在各个层次获得良好的性能,带宽利用率在同类系统中最高,应用程序并行性能与与Myrinet和SCI相当,在某些情况下比它们更好.     

基于动态连接的RDMA可靠传输协议设计

未来100P/E级高性能计算机系统对网络的传输可靠性、性能均衡性、可扩展性方面有更高的需求。本文提出的RDMA传输模型,采取配置少量资源,动态连接使用的策略实现端到端的数据可靠传输。与传统的可靠通信协议如Infiniband相比,本方案的优势为:(1)支持自动重路由,可绕过网络故障区域保证消息的可靠传输;(2)支持报文乱序到达,支持源和目的间的多路径传输,提供消息的流控机制,能较好地均衡网络整体性能,减少网络热点和缓解网络拥塞;(3)基于通信接口硬件实现可靠性数据结构,不需要消耗主存为通信建立连接,具有极高的系统可扩展性。初步测试结果表明,采取了优化措施后,该协议不会增加小于4K字节消息的传输延迟。     

基于RDMA的区块传输机制设计与实现

随着区块链技术的不断发展,区块的传输延迟成为区块链系统可扩展性的性能瓶颈。远程直接内存访问（RDMA）技术能够支持高带宽和低时延的数据传输,为低延迟区块传输提供了新的思路。因此,结合RDMA原语的特性,设计了用于区块信息共享的区块目录结构,并在此基础上设计并实现了区块传输的基本工作过程。实验结果表明,相较于基于TCP的方案,在1 MB大小的区块上基于RDMA的区块传输机制将节点间的区块传输延迟降低了44%,全网络的区块传输延迟降低了24.4%,在10 000节点规模的区块链上,区块链发生临时分叉的数量降低了22.6%。可见,基于RDMA的区块传输机制充分发挥了高速网络的性能优势,降低了区块传输延迟及临时分叉的数量,提高了现有区块链系统的可扩展性。