首页 | 官方网站   微博 | 高级检索  
     

基于Apache Spark的MODIS海表温度反演方法
引用本文:刘欢,陈能成,陈泽强. 基于Apache Spark的MODIS海表温度反演方法[J]. 计算机系统应用, 2018, 27(9): 112-117
作者姓名:刘欢  陈能成  陈泽强
作者单位:武汉大学 测绘遥感信息工程国家重点实验室, 武汉 430079,武汉大学 测绘遥感信息工程国家重点实验室, 武汉 430079,武汉大学 测绘遥感信息工程国家重点实验室, 武汉 430079
基金项目:国家自然科学基金(41771422);湖北省自然科学基金(2017CFB616)
摘    要:为应对海量遥感影像快速计算的需求,通过对影像获取、算法和计算过程优化和改进,提出了一种基于Apache Spark并行计算框架的MODIS海表温度反演方法,实现了海量MODIS遥感影像的海表温度快速反演.应用四轮网络查询请求获取特定的时空范围影像数据,提高影像获取阶段的效率;应用简化算法参数、拟合过程变量改进海表温度劈窗算法,使之适合快速并行计算;应用弹性分布式数据集(RDD)窄依赖关系的优点,避免并行计算中的数据交换延迟.通过单机模式与集群模式对比实验,发现集成了并行计算框架的集群模式影像处理效率约为单机模式的10倍.研究结果表明了融合集群计算技术的海表温度反演过程有效提高了传统单机应用程序的处理效率.

关 键 词:Apache Spark  MODIS  海表温度
收稿时间:2018-01-29
修稿时间:2018-02-27

Retrieving Method for MODIS Sea Surface Temperature with Apache Spark
LIU Huan,CHEN Neng-Cheng and CHEN Ze-Qiang. Retrieving Method for MODIS Sea Surface Temperature with Apache Spark[J]. Computer Systems& Applications, 2018, 27(9): 112-117
Authors:LIU Huan  CHEN Neng-Cheng  CHEN Ze-Qiang
Affiliation:State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China,State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China and State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
Abstract:In response to computing problems of massive remote sensing images, a method based on Apache Spark is proposed and implemented in retrieving MODIS Sea Surface Temperature (SST) by optimizing and improving the image acquisition, algorithm, and computing process. It applied four bouts of network requests to acquire user-defined data of specific time and zones to improve the efficiency of image acquisition. For a parallelizable algorithm, improvements that reduce parameters and simplify intermediate models are added to the split window algorithm, thus to adapt to fast parallelized computing. Taking advantage of narrow dependence between Resilient Distributed Datasets (RDD), delays for partitions'' interactions are evaded. With comparison between single mode and cluster mode, the latter incorporated with Apache Spark has an efficiency of ten times to the former. This study proves that, comparing with a single machine''s, programs that retrieving MODIS SST with cluster computing techniques has a higher efficiency.
Keywords:Apache Spark  MODIS  Sea Surface Temperature (SST)
点击此处可从《计算机系统应用》浏览原始摘要信息
点击此处可从《计算机系统应用》下载全文
设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司    京ICP备09084417号-23

京公网安备 11010802026262号