一种轻量级架构的ETL系统设计与实现

ETL系统是构建数据仓库的关键组件。通过数据抽取、转换和装载工作，完成处理海量数据和构建数据仓库的任务。如何提高数据处理效率和增强系统稳定性是值得研究的问题。根据电信资源数据省级集中分析系统下的实际应用背景，提出了一种轻量级架构的ETL系统，该架构采取的是“化整为零”的思想，把原先核心的抽取和转换的任务部分划分到各个相应的子系统中做初步的处理，这样处理的优点在于应用中减轻了核心ETL引擎的承载负担，各个抽取任务单独进行互不依赖，提高了抽取效率。通过在具体应用环境下系统运行表明采用该架构ETL系统具良好的使用效果。

Design and Implementation of Lightweight Architecture of ETL System

ETL system is a key component responsible for constructing data warehouse.It processes large volumes of data and builds data warehouse through data extraction,transformation and loading.How to efficiently shorten the execution time and enhance system stability is a big challenge.According the application of practical background of telecommunication resources analysis system,gives a lightweight architecture of ETL system.The framework adopted the "piecemeal" thinking,the original core of the extraction tasks assign to the various parts of the subsystem to do preliminary processing,with the advantages of this application is to reduce the core ETL engine burden,separate tasks from each other,and improve extraction efficiency.The specific operation proves it to be efficient.