| 基于Hadoop的分布式ETL系统研究 |
| |
| 引用本文： | 罗鹏,袁兵,梁耿,梁宏谦.基于Hadoop的分布式ETL系统研究[J].广西通信技术,2014(4):17-20. |
| |
| 作者姓名： | 罗鹏 袁兵 梁耿 梁宏谦 |
| |
| 作者单位： | 中国移动通信集团广西有限公司,广西南宁530022 |
| |
| 摘 要： | 针对传统数据抽取、转换及加载(ETL)工具集中式执行的不足,设计实现了一种基于Hadoop平台的分布式ETL系统。该系统采用分布式文件系统存储和映射-规约并行处理海量数据架构,实现了ETL作业的分布式执行,提高了ETL效率,为大数据环境下的基础数据加工提供了解决方案。
|
| 关 键 词： | ETL Hadoop 大数据 |
Research on Distributed ETL System Based on Hadoop |
| |
| Luo Peng,Yuan Bing,Liang Geng,Liang Hongqian.Research on Distributed ETL System Based on Hadoop[J].Guangxi Communication Technology,2014(4):17-20. |
| |
| Authors: | Luo Peng Yuan Bing Liang Geng Liang Hongqian |
| |
| Affiliation: | (China Mobile Group Guangxi Company Limited, Guangxi Nanning 530022) |
| |
| Abstract: | Aiming at deficiency of centralized execution mode of traditional extraction-transformation-loading （ETL） tools, this paper give a distributed ETL system based on Hadoop. This architecture uses a parallel programming model for massive data processing with MapReduce, achieves distributed data storage with HDFS. It raises the ETL performance efficiency, gives the solution for data processing in big data environment. |
| |
| Keywords: | ETL Hadoop Big Data |
| 本文献已被 CNKI 维普 等数据库收录！ |
|
