基于流式计算的遥感卫星数据快视处理方法

随着高分辨率遥感卫星数据获取能力和地面数传接收能力的提高,现有遥感卫星快视处理系统的处理负载增大,实时性要求越来越难以满足。针对这些问题,采用流式计算思想提出了一种新的遥感卫星数据快视处理系统设计方法。在分析遥感卫星数据快视处理数据流特点的基础上,应用Storm框架对现有系统进行并行优化,设计遥感数据流处理任务拓扑结构,同时利用消息队列中间件Kafka改进处理单元间数据交换和数据缓存方式。实验表明,该系统在数据吞吐率和可靠性方面测试效果良好。

Quick-View Processing Method for Remote Sensing Data Based on Stream Computing

With the improvement of data acquiring ability of high resolution remote sensing satellites and data receiving ability of ground receiving stations, the processing load of existing system grows increasingly heavier and the real-time processing demand becomes more difficult to meet. Focusing on these issues, this paper proposes a new system design method for quick-view processing of remote sensing satellite data by using the thought of stream computing. After analyzing the characteristics of quick-view processing data streams, this paper applies the Storm framework to the parallel optimization of the existing system, designs the topology of stream computing tasks for the remote sensing satellite data processing, and uses Kafka message oriented middleware to improve the mechanism of data exchanging and data buffering in processing units. In experiments the improved system shows good results in throughput and reliability.