基于图结构索引的分布式OLAP加速方法

随着业务数据的规模增大,一些重要的应用场景需要使用分布式在线分析处理(OLAP)支持大规模数据的分析,例如商务智能(BI),企业资源计划(ERP),用户行为分析等.同时,分布式OLAP打破单机存储的限制,可以将数据放在内存中以提升OLAP的处理性能.然而,基于内存的分布式OLAP在消除磁盘I/O后,性能瓶颈转移到了连接操作.连接操作是OLAP中的一种常用操作,会进行大量的数据读取与计算操作.通过对现有的几种连接操作方式进行分析,提出了一种能够加速连接操作的图结构索引以及基于图结构索引的连接操作方式LinkJoin.图结构索引通过用户所指定的连接关系,将数据在内存中的位置以图结构的形式进行存储.基于图结构索引的连接方式,不仅能够有等同于哈希连接的较低复杂度,而且在执行过程中能减少数据读取与计算操作次数.将目前先进的开源内存OLAP系统MonetDB从单机系统扩展成分布式系统,并且在该系统上设计与实现了基于图结构索引的连接操作方式.针对该系统的图索引结构,列式存储以及分布式执行引擎这3个重要方面,进行一系列设计与优化,以提升系统的分布式OLAP处理性能.测试结果表明,在TPC-H标准测试中...     

基于消息传递机制的MapReduce图算法研究

单机运行环境难以满足基于海量数据的大图算法对时空开销的需求,如何设计高效的面向云计算环境的分布式大图算法越来越受到人们的关注,MapReduce作为云计算的核心计算模式受限于易并行(EP)计算模型的制约不易表达图算法.文中突破了MapReduce基于易并行计算的假设,增强了MapReduce既有的编程规范,新的大同步(...     

RGraph:基于RDMA的高效分布式图数据处理系统

图是描述实体间关系的重要数据结构,被广泛地应用于信息科学、物理学、生物学、环境生态学等重要的科学领域.现如今,随着图数据规模的不断增大,利用分布式系统来处理大图数据已经成为主流,出现了形如Pregel、GraphX、PowerGraph和Gemini等经典的分布式大图数据处理系统.然而,与当前先进的基于单机的图处理系统...     

基于可扩展计算平台的大图数据管理

随着数据采集和存储技术的发展,社交网络、生物信息科学、交通导航等领域中出现了规模庞大、内部结构复杂、查询需求多样的大图数据。传统基于单机内存的图处理方法无法满足大图数据管理需求。可扩展计算平台的发展为大图数据管理提供了可行的技术方案。本文首先分析了大图数据之上的不同类型查询,重点探讨了基于关系数据库、基于MapReduce计算框架、基于BSP(Bulk Synchronous Parallel)计算模型和基于第三方外包服务器的大图数据管理方法,并分析了未来可能的研究路线。     

面向流式数据处理系统的高效故障恢复方法

针对流式数据处理系统Flink无法高效处理单点故障的问题,提出了一种基于增量状态和备份的故障容错系统Flink+。首先,提前建立备份算子和数据通路;然后,对数据流图中的输出数据进行缓存,必要时使用磁盘;其次,在系统快照时进行任务状态同步;最后,在系统故障时使用备份任务和缓存的数据恢复计算。在系统实验测试中,Flink+在无故障运行时没有显著增加额外容错开销;而在单机和分布式环境下处理单点故障时,与Flink系统相比,所提系统在单机8任务并行度下故障恢复时间减少了96.98%,在分布式16任务并行度下故障恢复时间减少了88.75%。实验结果表明,增量状态和备份方法一起使用可以有效减少流式系统单点故障的恢复时间,增强系统的鲁棒性。     

图嵌入算法的分布式优化与实现

随着人工智能时代的到来,图嵌入技术被越来越多的用来挖掘图中的信息.然而,现实生活中的图通常很大,因此分布式图嵌入技术得到了广泛的关注,分布式图嵌入算法面临着两大难点：（1）图嵌入算法多种多样,没有一个通用的框架能够描述大部分的算法;（2）现在的分布式图嵌入算法扩展性不足,当处理大图时性能较低,针对以上两个挑战,本文首先提出一个通用的分布式图嵌入框架,具体地,本文将图嵌入算法中的采样流程和训练流程进行解耦,使得框架能够较好的表达多种不同的算法;其次,本文提出了一种基于参数服务器的模型切分嵌入策略,具体地,本文将模型分别切分到计算节点和参数服务器上,同时使用数据洗牌的操作保证计算节点之间没有模型交互,从而大大减少了分布式计算中的通信开销,笔者基于参数服务器实现了一个原型系统,并且用充分的实验证明了在不损失精度的前提下,基于模型切分的策略能够比基线系统取得更好的性能.     

YARN平台上的并行主题标引算法

文档主题标引是当前个性化智能检索的重要前提,但面对大规模海量数据资源时,主题标引也成为性能瓶颈。当前在MapReduce框架上设计实现的主题标引算法,通常存在启动任务耗时长,中间数据过多地进行磁盘IO等缺陷。为了解决此类问题,采用YARN（yet another resource negotiator）作为底层分布式资源管理平台,选择更加合适的计算框架来改善计算性能。针对文档主题标引算法计算步骤多、阶段性强的特点,选择有向无环图（directed acyclic graph, DAG）计算模型进行算法实现,避免不必要的作业拆分,从而减少中间结果的磁盘IO。另外,考虑到MapReduce的排序策略耗时较多,而有些计算无需对结果排序,故可以改用基于Hash的数据归约策略来提高计算性能,但这又会带来随机读的问题。利用固态硬盘高速随机读的特性,设计相应的优化计算策略来解决随机读的问题。通过实验对比发现,以YARN为底层管理平台,在此基础上选择合适的计算框架并加以优化,可以有效改善分布式计算的性能。     

流式大数据数据清洗系统设计与实现

为了解决传统数据清洗工具面对海量数据时复杂度高、效率低的问题,设计实现了流式大数据数据清洗系统.利用分布式计算技术清洗数据,以解决性能低的问题.该系统由统一接入模块、计算集群和调度中心三部分组成,实现了多种数据源的统一接入,分布式处理,并通过Web界面进行清洗流程的交互式配置.实验结果表明,面对海量数据的时候,流式大数据数据清洗系统的性能强于传统的单机数据清洗,提高了清洗效率.     

一种分布式Twitter数据处理方案及应用

针对社交媒体数据的特点及其分析的挑战性,提出了一种基于实时计算框架Storm、批处理框架Hadoop和高效可水平扩展的NoSQL数据库MongoDB的分布式社交媒体数据处理方案,并依此指导实现基于Twitter流式数据的流感疫情可视化分析系统.实验证明,该分布式方案能较好支持Twitter流式数据的高效处理和储存,使之满足系统的性能需求.     

基于Flink的分布式在线集成学习框架研究

在大数据环境背景下,传统机器学习算法多采用单机离线训练的方式,显然已经无法适应持续增长的大规模流式数据的变化。针对该问题,提出一种基于Flink平台的分布式在线集成学习算法。该方法基于Flink分布式计算框架,首先通过数据并行的方式对在线学习算法进行分布式在线训练;然后将训练出的多个子模型通过随机梯度下降算法进行模型的动态权重分配,实现对多个子模型的结果聚合;与此同时,对于训练效果不好的模型利用其样本进行在线更新;最后通过单机与集群环境在不同数据集上做实验对比分析。实验结果表明,在线学习算法结合Flink框架的分布式集成训练,能达到集中训练方式下的性能,同时大大提高了训练的时间效率。     

Partitioning dynamic graph asynchronously with distributed FENNEL

Graph partitioning is important in distributed graph processing. Classical method such as METIS works well on relatively small graphs, but hard to scale for huge, dynamic graphs. Streaming graph partitioning algorithms overcome this issue by processing those graphs as streams. Among these algorithms, FENNEL achieves better edge cut ratio, even close to METIS, but consumes less memory and is significantly faster. On the other hand, graph partitioning may also benefit from distributed graph processing. However, to deploy FENNEL on a cluster, it is important to avoid quality loss and keep efficiency high. The direct implementation of this idea yields a synchronous model and a star-shaped network, which limits both scalability and efficiency. Targeting these two problems, we propose an asynchronous model, combined with a dedicated tree-shaped map-reduce network which is prevail in systems such as Apache Hadoop and Spark, to form AsyncFENNEL (asynchronous FENNEL). We theoretically prove that, the impact on partition quality brought by asynchronous model can be kept as minimal. We test AsyncFENNEL with various synthetic and real-world graphs, the comparison between synchronous and asynchronous models shows that, for streamed natural graphs, AsyncFENNEL can improve performance significantly (above 300% with 8 workers/partitions) with negligible loss on edge cut ratio. However, more worker nodes will introduce a heavier network traffic and reduce efficiency. The proposed tree-shaped map-reduce network can mitigate that impact and increase the performance in that case.     

一种基于GPU集群的深度优先并行算法设计与实现

深度优先搜索算法在GPU集群中大型图上的简单执行,会导致线程间的负载不平衡和无法合并内存访问的情况,这使得算法的性能较低.为了明显提高算法在单个GPU和多个GPU环境下的性能,在处理数据之前通过采取一系列有效的操作来进行重新编排.提出了构造线程和数据之间映射的新技术,通过利用前缀求和及二分查找操作来达到完美的负载平衡.为了降低通信开销,对DFS各分支中需要进行交换的边集执行修剪操作.实验结果表明,算法在单个GPU上可以尽可能地实现最佳的并行性,在多GPU环境下可以最小化通信开销.在一个GPU集群中,它可以对合有数十亿节点的图有效地执行分布式DFS.     

Para-G: Path pattern query processing on large graphs

There are plentiful and diverse applications of graph data management and mining techniques in the real-world scientific research and business activities. As one of the most basic operations, uniform path pattern query processing on graph data faces three big challenges. In this paper, we deal with these challenges by the following points. Firstly, a new query language on graph, called G-Path, is presented, which focuses on complex path pattern query processing on a very large graph. Also, the design of a system called Para-G is proposed, which is based on a BSP-like model as well as MapReduce model, and can effectively handle distributed graph data operations and queries. Secondly, the implementation of Para-G on the de facto cloud platform — Hadoop — is brought forward. Based on the concept of distributed path finite state automaton, the query processing of a G-Path statement in Para-G is detailed. In addition, as the query optimization of G-Path queries, several tricks are utilized to dramatically improve the performance of query execution. Finally, extensive experiments on several graph data sets are conducted to show the usability of the G-Path query language and the effectiveness of Para-G.     

iMapReduce: A Distributed Computing Framework for Iterative Computation

Iterative computation is pervasive in many applications such as data mining, web ranking, graph analysis, online social network analysis, and so on. These iterative applications typically involve massive data sets containing millions or billions of data records. This poses demand of distributed computing frameworks for processing massive data sets on a cluster of machines. MapReduce is an example of such a framework. However, MapReduce lacks built-in support for iterative process that requires to parse data sets iteratively. Besides specifying MapReduce jobs, users have to write a driver program that submits a series of jobs and performs convergence testing at the client. This paper presents iMapReduce, a distributed framework that supports iterative processing. iMapReduce allows users to specify the iterative computation with the separated map and reduce functions, and provides the support of automatic iterative processing within a single job. More importantly, iMapReduce significantly improves the performance of iterative implementations by (1) reducing the overhead of creating new MapReduce jobs repeatedly, (2) eliminating the shuffling of static data, and (3) allowing asynchronous execution of map tasks. We implement an iMapReduce prototype based on Apache Hadoop, and show that iMapReduce can achieve up to 5 times speedup over Hadoop for implementing iterative algorithms.     

Achieving Optimal Inter-Node Communication in Graph Partitioning Using Random Selection and Breadth-First Search

Processing large graph datasets represents an increasingly important area in computing research and applications. The size of many graph datasets has increased well beyond the processing capacity of a single computing node, thereby necessitating distributed approaches. As these datasets are processed over a distributed system of nodes, this leads to an inter-node communication cost problem that negatively affects system performance. Previously proposed algorithms implemented breadth-first search (BFS) for graph searching and focused on the execution, parallel performance and not the communication. In this paper a new methodology is proposed that combines BFS with random selection in order to partition large graph datasets and effectively minimize inter-node communication. The new method is discussed and applied to the single-source shortest path and PageRank algorithms using three graphs that are representative of real-world scenarios. Experimental results show that graph inter-node communication for canonical graphs representative of real-world data is improved up to 42 % in case of Powerlaw graph, up to 27 % in case of Random near K-regular graph (with low degree), and up to 7 % in case of Random near K-regular graph (with high degree).     

Magas: matrix-based asynchronous graph analytics on shared memory systems

Graph analytics plays an important role in many areas such as big data and artificial intelligence. The vertex-centric programming model provides friendly interfaces to programmers and is extensively used in graph processing frameworks. However, it is prone to generate many irregular memory accesses and scheduling overhead due to vertex-based execution and scheduling of programs in the backend. Instead, the matrix-based model provides a different approach by using high-performance matrix operations in the backend to improve the efficiency of graph processing. Unfortunately, current matrix-based frameworks only support the synchronous parallel model, which constrains its application to various graph algorithms. To address these problems, this paper proposes a graph processing framework, which combines matrix operations with the asynchronous model while providing friendly programming interfaces similar to vertex-centric programming model. Firstly, we propose an approach to map the vertex-based graph processing to matrix operations in the asynchronous model. Then, we propose two asynchronous scheduling policies, Gauss–Seidel policy and relaxed Gauss–Seidel policy, for different graph algorithms. After that, our framework applies the batch scheduling and optimized in-memory data structure to reduce the scheduling overhead introduced by the asynchronous model. Experimental results show that our framework performs better than the popular vertex programming frameworks such as GraphLab and GRACE in both performance and speedup and achieves similar performance compared to the BSP-based matrix framework such as GraphMat.

     

基于VB的石油馏程仪智能监控系统的通信与数据处理

本文介绍了以多台石油馏程仪为下位机、单台PC机为上位机组成的分布式智能监控系统的设计方案,说明了数据通信的方法,讨论了基于动态数据库的实时数据存储的具体实现过程.     

An asynchronous traversal engine for graph-based rich metadata management

Rich metadata in high-performance computing (HPC) systems contains extended information about users, jobs, data files, and their relationships. Property graphs are a promising data model to represent heterogeneous rich metadata flexibly. Specifically, a property graph can use vertices to represent different entities and edges to record the relationships between vertices with unique annotations. The high-volume HPC use case, with millions of entities and relationships, naturally requires an out-of-core distributed property graph database, which must support live updates (to ingest production information in real time), low-latency point queries (for frequent metadata operations such as permission checking), and large-scale traversals (for provenance data mining).Among these needs, large-scale property graph traversals are particularly challenging for distributed graph storage systems. Most existing graph systems implement a “level-synchronous” breadth-first search algorithm that relies on global synchronization in each traversal step. This performs well in many problem domains; but a rich metadata management system is characterized by imbalanced graphs, long traversal lengths, and concurrent workloads, each of which has the potential to introduce or exacerbate stragglers (i.e., abnormally slow steps or servers in a graph traversal) that lead to low overall throughput for synchronous traversal algorithms. Previous research indicated that the straggler problem can be mitigated by using asynchronous traversal algorithms, and many graph-processing frameworks have successfully demonstrated this approach. Such systems require the graph to be loaded into a separate batch-processing framework instead of being iteratively accessed, however.In this work, we investigate a general asynchronous graph traversal engine that can operate atop a rich metadata graph in its native format. We outline a traversal-aware query language and key optimizations (traversal-affiliate caching and execution merging) necessary for efficient performance. We further explore the effect of different graph partitioning strategies on the traversal performance for both synchronous and asynchronous traversal engines. Our experiments show that the asynchronous graph traversal engine is more efficient than its synchronous counterpart in the case of HPC rich metadata processing, where more servers are involved and larger traversals are needed. Moreover, the asynchronous traversal engine is more adaptive to different graph partitioning strategies.     

Secured Information Flow for Asynchronous Sequential Processes

We present in this article a precise security model for data confidentiality in the framework of ASP (Asynchronous Sequential Processes). ASP is based on active objects, asynchronous communications, and data-flow synchronizations. We extend it with security levels attached to activities (active objects) and transmitted data.We design a security model that guarantees data confidentiality within an application; this security model takes advantages of both mandatory and discretionary access models. We extend the semantics of ASP with predicate conditions that provide a formal security framework, dynamically checking for unauthorized information flows. As a final result, all authorized communication paths are secure: no disclosure of information can happen. This theoretically-founded contribution may have a strong impact on distributed object-based applications, that are more and more present and confidentiality-demanding on the Internet, it also arises a new issue in data confidentiality: authorization of secured information flow transiting (by the mean of futures) through an unsecured component.     

Tuning the granularity of parallelism for distributed graph processing

Popular distributed graph processing frameworks, such as Pregel and GraphLab, are based on the vertex-centric computation model, where users write their customized Compute function for each vertex to process the data iteratively. Vertices are evenly partitioned among the compute nodes, and the granularity of parallelism of the graph algorithm is normally tuned by adjusting the number of compute nodes. Vertex-centric model splits the computation into phases. Inside one specific phase, the computation proceeds as an embarrassingly parallel process, because no communication between compute nodes incurs. By default, current graph engine only handles one iteration of the algorithm in a phase. However, in this paper, we find that we can also tune the granularity of parallelism, by aggregating the computation of multiple iterations into one phase, which has a significant impact on the performance of the graph algorithm. In the ideal case, if all computations are handled in one phase, the whole algorithm turns into an embarrassingly parallel algorithm and the benefit of parallelism is maximized. Based on this observation, we propose two approaches, a function-based approach and a parameter-based approach, to automatically transform a Pregel algorithm into a new one with tunable granularity of parallelism. We study the cost of such transformation and the trade-off between the granularity of parallelism and the performance. We provide a new direction to tune the performance of parallel algorithms. Finally, the approaches are implemented in our graph processing system, N2, and we illustrate their performance using popular graph algorithms.