对等网络副本最佳占有率缓存管理策略

随着对等网络应用的不断深入,如何减少时间延迟,减轻集中性带宽负载,提高服务质量,已经成为研究的一个重点.提出了CORPC缓存管理方案.该方案通过使用流媒体片段的流行度来定义媒体片段副本数可占用的最佳系统缓存容量,综合考虑流媒体片段已有的副本容量、流媒体片段的热度、系统节点存储容量,使用启发式贪婪算法来实现缓存准入和缓存替换机制.该方案兼顾了不同热度的媒体片段的服务质量.模拟环境的测试结果表明,随着节点缓存空间的增加,系统服务质量得到改善.     

A Statistical Admission Control Scheme for Continuous Media Servers Using Caching

In continuous media servers, disk load can be reduced by using buffer cache. In order to utilize the saved disk bandwidth by caching, a continuous media server must employ an admission control scheme to decide whether a new client can be admitted for service without violating the requirements of clients already being serviced. A scheme providing deterministic QoS guarantees in servers using caching has already been proposed. Since, however, deterministic admission control is based on the worst case assumption, it causes the wastage of the system resources. If we can exactly predict the future available disk bandwidth, both high disk utilization and hiccup-free service are achievable. However, as the caching effect is not analytically determined, it is difficult to predict the disk load without substantial computation overhead. In this paper, we propose a statistical admission control scheme for continuous media servers where caching is used to reduce disk load. This scheme improves disk utilization and allows more streams to be serviced while maintaining near-deterministic service. The scheme, called Shortsighted Prediction Admission Control (SPAC), combines exact prediction through on-line simulation and statistical estimation using a probabilistic model of future disk load in order to reduce computation overhead. It thereby exploits the variation in disk load induced by VBR-encoded objects and the decrease in client load by caching. Through trace-driven simulations, it is demonstrated that the scheme provides near-deterministic QoS and keeps disk utilization high.     

Delay Management in Mesh-Based P2P Live Streaming Using a Three-Stage Peer Selection Strategy

Peer-to-peer (P2P) live streaming systems have gained popularity due to the self-scalability property of the P2P overlay networks. In P2P live streaming, peers retrieve stream content from other peers in the system. Therefore, peer selection strategy is a fundamental element to build an overlay which manages the playback delay and startup delay experienced by the peers. In this paper, we propose a peer selection strategy which manages to build a minimum delay overlay using three different stages of overlay construction. In the first stage, the tracker suggests some peers as prospective partners to a new peer. In the second stage, the peer selects its partners out of these peers such that delay is minimized. The third stage is the topology adaptation phase of peers, where peers reposition themselves in the overlay to maintain minimum delay during peer churn. In the proposed peer selection strategy, peers are selected in all the stages based on parameters such as propagation delay, upload capacity, buffering duration and buffering level. The proposed strategy is compared with two existing strategies in the literature: Fast-Mesh (Ren et al. in IEEE Trans Multimed 11: 1446, 2009) and Hybrid live p2p streaming protocol (Hammami et al., 2014) using simulations. Our results show that playback delay and startup delay are reduced significantly with the help of proposed strategy. We demonstrate that the stability of the system also improves during peer churn.     

SmartPeerCast: a Smart QoS driven P2P live streaming framework

The P2P swarm technologies have been shown to be very efficient for medium scale content distribution systems in the last few years, such as the file sharing and video-on-demand (VOD) applications. However it is still an open topic about how to deploy the P2P paradigm for the real time video broadcasting (RTVB) applications. The P2P RTVB application is different from the cache based P2P system because it has more stringent restrictions for startup time and packet loss ratio. In this paper, an adaptive media broadcasting P2P framework named SmartPeerCast which employs the media transrating service to control the quality of service (QoS), is proposed. SmartPeerCast achieves a network awareness, codec awareness, and high performance RTVB service with four key designs: (1) It groups the newly joined peers into different quality clusters by their uploading capability. This clustering mechanism avoids the bandwidth bottleneck between the heterogeneous peers of the overall P2P overlay by only forwarding the same quality stream over the peers in the same cluster. (2) The streaming quality is adjusted adaptively between the sending and the receiving peers by a Smart QoS algorithm to compensate for the network jitters to reduce the receiving peer’s playback jitter. (3) The receiving peer monitors the data forwarding QoS of the sending peer to select the best suitable parent node dynamically. The SmartPeerCast uses this Smart QoS framework to implement an incentive mechanism to award the peers with high uploading contributions by migrating them to a higher quality cluster. (4) A transrating engine is used at the leaf nodes of the high quality cluster to forward the stream with suitable bits rate to the nodes of the low quality cluster; this transrating service not only can fully utilize the uploading bandwidth of the peers in the higher quality cluster but also avoids the bandwidth bottleneck of stream forwarding between the heterogeneous peers. Our experiment results and the real deployment show that SmartPeerCast can eliminate the bandwidth bottleneck and content bottleneck between the heterogeneous peers with a smaller startup time and packet loss and it is a high performance and medium scale P2P RTVB framework.     

一种机群文件系统的缓存模型

提出了一种新的机群文件系统缓存模型，它充分利用机群系统累积的系统资源和高速的互联网络，将文件系统元数据和内容数据分离，分别使用分布式元数据缓存和统一缓存模型进行管理。元数据缓存使用改进的广播一致性协议和LRU替换算法。内容数据统一缓存则将磁盘缓存整个文件和内存缓存文件块相结合组成一个单一映像的多层次分布协作缓存，并使用单拷贝优先LRU和向前传递调度缓存块替换算法以及一种贪心的数据预取方法。实验结果表明，这两种缓存机制结合使用能极大地提高机群文件系统的性能。     

An Alliance Based Peering Scheme for P2P Live Media Streaming

While recent measurement studies have shown the effectiveness of P2P network in media streaming, there have been questions raised about the quality of service (QoS), reliability of streaming services and sub optimal uplink utilization in particular. P2P streaming systems are inherently less reliable because of churn, internet dynamics, node heterogeneity and randomness in the swarm. We present a new model for P2P media streaming based on clustering of peers, called alliances. We show that alliance formation is a loosely coupled and an effective way to organize the peers. We show that our model maps to a ldquosmall-worldrdquo network, which form efficient overlay structures and are robust to network perturbations such as churn. We present a comparative simulation based study of our model with CoolStreaming/DONet and present a quantitative performance evaluation. Simulation results are promising and show that our model scales well under varying workloads and conditions, delivers near optimal levels of QoS, and for most cases, performs at par or even better than CoolStreaming/DONet.     

Fast proxy delivery of multiple streaming sessions in shared running buffers

With the falling price of memory, an increasing number of multimedia servers and proxies are now equipped with a large memory space. Caching media objects in the memory of a proxy helps to reduce the network traffic, the disk I/O bandwidth requirement, and the data delivery latency. The running buffer approach and its alternatives are representative techniques to caching streaming data in the memory. There are two limits in the existing techniques. First, although multiple running buffers for the same media object co-exist in a given processing period, data sharing among multiple buffers is not considered. Second, user access patterns are not insightfully considered in the buffer management. In this paper, we propose two techniques based on shared running buffers in the proxy to address these limits. Considering user access patterns and characteristics of the requested media objects, our techniques adaptively allocate memory buffers to fully utilize the currently buffered data of streaming sessions, with the aim to reduce both the server load and the network traffic. Experimentally comparing with several existing techniques, we show that the proposed techniques achieve significant performance improvement by effectively using the shared running buffers.     

P2P系统中基于Web的索引信息扩展算法的研究

基于环球网（Web）的特点和用户在点对点（P2P）系统中搜索的习惯,提出了一个在P2P系统中对媒体文件自动生成索引的方法。该方法有效地解决了媒体文件描述符不足所带来的查询精度低的问题。同时,提出了一个在P2P系统中节点信息的更新策略。实验表明,描述符扩展后,媒体文件查询结果的准确率得到了显著的提高。     

Hybrid storage-based caching strategy for content delivery network services

This study proposes a novel caching scheme for content delivery network services. In general, video content users often watch the first part of video clips and then switch to other content. Therefore, a caching scheme is proposed, in which the first part of the frequently referenced content is stored on a solid state drive (SSD) while the remaining video content is stored on a hard disk drive (HDD),. The proposed hybrid (SSD/HDD) caching scheme offers several benefits, such as an improved average data output capacity due to the high average data rate and average hit capacity of the SSD. That is, performance can be significantly improved at a low extra cost with the cache server of a content delivery network (CDN).     

基于P2P和CDN的混合流媒体分发模型及分析*

在CDN和P2P两种主流的流媒体分发技术的基础上,提出了基于系统流量的混合流媒体分发模型,根据系统中节点数量和媒体流量之间的关系,在CDN自治域内实现CDN和P2P的混合式服务,并对上述理论进行了仿真。实验结果表明,与传统的CDN和P2P相比,新的混合模型既可以减少主干网络的流量,又能在不降低流媒体质量的情况下,有效降低CDN服务器压力,减少系统的响应时间,提高网络利用率。     

HAMS: Scalable peer-to-peer multimedia streaming model in heterogeneous networks

In a peer-to-peer (P2P) overlay network, a large number and various types of peer processes are interconnected in networks and are cooperating by using multimedia contents like movies and music. Here, multimedia contents are in nature distributed to peers in various ways like downloading and caching to the peers. Multimedia streaming is a key technology to realize multimedia applications in networks. In multimedia streaming applications, multimedia contents are required to be reliable and continuously delivered to processes in a real-time manner. Some contents peer may not send packets of a content at a required rate due to limited computation resource and a communication channel may not support enough Quality of Service (QoS) due to congestions and faults. Thus, P2P overlay networks are in nature heterogeneous. In this paper, we newly discuss a heterogeneous asynchronous multi-source streaming (HAMS) model where multiple contents peers transmit packets of a multimedia content to a requesting leaf peer to increase the throughput, reliability, and scalability in P2P overlay networks. Here, some pair of channels between contents and leaf peers may support different QoS. Peers may be faulty and some pair of contents peers may have different transmission rates. Finally, we show the HAMS model can support higher throughput and shorter transmission time than the other models in the evaluation.     

A unified benchmarking and model-based framework for building QoS-aware streaming media services

A number of technology and workload trends motivate us to consider the appropriate resource allocation mechanisms and policies for streaming media services in shared cluster environments. We present MediaGuard – a model-based infrastructure for building streaming media services – that can efficiently determine the fraction of server resources required to support a particular client request over its expected lifetime. The proposed solution is based on a unified cost function that uses a single value to reflect overall resource requirements such as the CPU, disk, memory, and bandwidth necessary to support a particular media stream based on its bit rate and whether it is likely to be served from memory or disk. We design a novel, time-segment-based memory model of a media server to efficiently determine in linear time whether a request will incur memory or disk access when given the history of previous accesses and the behavior of the server's main memory file buffer cache. Using the MediaGuard framework, we design two media services: (1) an efficient and accurate admission control service for streaming media servers that accounts for the impact of the server's main memory file buffer cache, and (2) a shared streaming media hosting service that can efficiently allocate the predefined shares of server resources to the hosted media services, while providing performance isolation and QoS guarantees among the hosted services. Our evaluation shows that, relative to a pessimistic admission control policy that assumes that all content must be served from disk, MediaGuard (as well as services that are built using it) deliver a factor of two improvement in server throughput.     

一种基于衍生树的交互式P2P 流媒体系统

提出了一种基于衍生树的P2P 系统框架,以支持交互式流媒体应用.该系统利用分布式发现服务来进 行资源定位,并通过基于衍生树的缓存结构来维护数据传输拓扑.使用基于衍生树的系统管理策略可以显著地降低 节点动态加入和退出等交互操作的开销.另外,通过使用分布式散列表(distributed hash table,简称DHT)来维护会话, 可以较低的代价实现资源查找、服务重构和拓扑维护等任务.仿真实验结果表明,与现有的P2P 流媒体系统相比,该 系统具有良好的性能,其用户交互操作的开销可以降低超过50%.     

基于时间间隔的P2P流媒体直播系统缓存算法

针对基于分片传输机制的P2P流媒体直播系统客户端缓存问题,为提高节点间请求数据分片的命中率和避免数据分片大量冗余,提出一种依赖邻居节点请求量的节点缓存替换算法.该算法将分片在节点的缓存时间进行等间隔划分,利用马尔可夫链转移概率矩阵计算理论,预测分片在下一时刻的缓存价值.并将该算法和传统算法FIFO及LRU算法进行了对比实验,实验结果表明,在同等条件下,该算法相较于传统算法能更好地提高数据的命中率.     

Tulip: A New Hash Based Cooperative Web Caching Architecture

With the exponential growth of WWW traffic, web proxy caching becomes a critical technique for Internet web services. Well-organized proxy caching systems with multiple servers can greatly reduce the user perceived latency and decrease the network bandwidth consumption. Thus, many research papers focused on improving web caching performance with the efficient coordination algorithms among multiple servers. Hash based algorithm is the most widely used server coordination mechanism, however, there's still a lot of technical issues need to be addressed. In this paper, we propose a new hash based web caching architecture, Tulip. Tulip aggregates web objects that are likely to be accessed together into object clusters and uses object clusters as the primary access units. Tulip extends the locality-based algorithm in UCFS to hash based web proxy systems and proposes a simple algorithm to reduce the data grouping overhead. It takes into consideration the access speed dispatch between memory and disk and replaces expensive small disk I/O with less large ones. In case a client request cannot be fulfilled by the server in the memory, the system fetches the whole cluster which contains the required object into memory, the future requests for other objects in the same cluster can be satisfied directly from memory and slow disk I/Os are avoided. It also introduces a simple and efficient data dupllication algorithm, few maintenance work need to be done in case of server join/leave or server failure. Along with the local caching strategy, Tulip achieves better fault tolerance and load balance capability with the minimal cost. Our simulation results show Tulip has better performance than previous approaches.     

Flash as cache extension for online transactional workloads

Considering the current price gap between hard disk and flash memory SSD storages, for applications dealing with large-scale data, it will be economically more sensible to use flash memory drives to supplement disk drives rather than to replace them. This paper presents FaCE, which is a new low-overhead caching strategy that uses flash memory as an extension to the RAM buffer of database systems. FaCE aims at improving the transaction throughput as well as shortening the recovery time from a system failure. To achieve the goals, we propose two novel algorithms for flash cache management, namely multi-version FIFO replacement and group second chance. This was possible due to flash write optimization as well as disk access reduction obtained by the FaCE caching methods. In addition, FaCE takes advantage of the nonvolatility of flash memory to fully support database recovery by extending the scope of a persistent database to include the data pages stored in the flash cache. We have implemented FaCE in the PostgreSQL open-source database server and demonstrated its effectiveness for TPC-C benchmarks in comparison with existing caching methods such as Lazy Cleaning and Linux Bcache.     

基于拥塞控制的非结构化P2P网络的路由负载均衡策略

非结构化P2P(unstructured peer-to-peer network)对等网络中的节点资源定位的路由查询是对等网络研究中的一个主要难题,特别是当网络中客户端节点由于其频繁加入、离开导致网络结构动态变化所带来的资源查询难题.提出了一种新的基于拥塞控制的路由查询方法来实现动态网络下的资源查询.该方法分两部分实现:首先是网络资源的分组与节点重连策略.该策略使得具有同等资源的节点相互连接,并周期性地调整节点上的节点连接数量以减少同组资源节点上的负载.通过以上策略,使得网络的拓扑结构自动地从随机网络结构进化到以资源组为单位的聚类网络,从而使得网络中形成网络资源组间的查询负载均衡.另一方面,组内的节点之间的路由负载均衡是通过节点间协同学习实现的.采用协同Q-学习方法,所研究的方法不仅从节点上学习其处理能力、连接数和资源的个数等参数,还将节点的拥塞状态作为协同Q-学习的重要参数,并建立模型.通过这种技术,同一组节点上的资源查询被有目的地引导,以避开那些组内拥塞的节点,从而最终实现资源组内节点之间的查询均衡.仿真实验结果表明,相比常用的random walk资源查找方法,该研究所实现的资源定位方法能够更迅速地实现网络的资源查询.仿真结果还表明,相比random walk方法,所提出的方法在网络高强度查询和网络节点动态加入和退出的情况下进行查询具有更高的鲁棒性和适应性.     

P2P流媒体的数据调度算法

数据驱动型覆盖网络中的数据调度算法是影响P2P流媒体系统性能的重要因素,为了解决当前算法未能有效利用数据块和节点的特性导致流媒体服务质量差的问题,提出了一种基于数据块优先级和节点能力度的数据调度算法。该算法能够根据数据块的稀缺性、紧迫性得到块的优先级大小,根据节点的上行带宽、在线时间、相对距离得到节点能力度的大小,使优先级高的数据块和能力度大的节点优先被请求,减少了数据块的播放等待时间。在OPNET网络仿真实验表明该算法能够有效降低流媒体播放系统的启动延时和服务器的负载。     

基于协作缓存的VOD服务器端Cache设计

近年来,随着计算机网络技术和多媒体技术的发展,视频点播服务已逐渐成为现实。分布式VOD(VideoOnDemand)服务器系统的提出是为了支持更多的大量并发数据流,和单一服务器相比,这样的结构拥有更好的使用效率、可靠性和可扩展性。协作缓存CC(cooperativecache)技术将各服务器的内存协调工作,形成全局的cache。这样的结构不仅充分发挥了分布式VOD服务器结构的特点,同时也增大cache容量,提高系统全局命中率,从而提高了系统效率。该文在协作缓存技术基础上,针对流媒体和VOD系统的特点,提出了GBBcache替换算法。该算法以数据块的生命周期作为出发点,充分考虑了现有用户和请求接入用户的服务需求,提高了内存使用效率。笔者对该算法进行了理论分析,并证明了它在性能上与传统的cache替换算法相比的优越性。     

基于遗传算法的P2P流媒体多播模型

为解决P2P流媒体通信中的流量控制问题，给出一个P2P社区覆盖多播通信优化模型及其建立覆盖核心多播树OCMT的QoS约束条件，并采用一个面向目标的遗传算法解决建立OCMT中的路径优化问题。仿真实验结果表明该模型能为建立具有低延迟和强扩展能力的核心多播树获得较优化的方案，并且有效地降低不同P2P社区间的数据流量，实现控制Internet流量的目的。