矢量瓦片并行构建与分布式存储模型研究

矢量瓦片体积小、生成效率高、支持动态交互,较传统栅格瓦片有诸多优势,是下一代互联网地图服务研究的重点。为了解决当前矢量瓦片研究中处理速度慢,扩展性差等问题,本文利用并行计算框架Spark进行矢量瓦片快速构建,通过自定义转换函数,将原始矢量数据GeoJson转换成mvt瓦片集;对于生成的矢量瓦片集,本文基于分布式内存文件系统Alluxio设计一个瓦片存储模型-VectorTileStore,模型以键值对进行数据存储,瓦片元数据占据前八个键值对,单个瓦片占据一个键值对,在数据写入的同时,基于键构建一个哈希索引,用于快速访问,模型兼容海量瓦片的组织存储,具有很强的扩展性。通过实验结果表明,本文提出的矢量瓦片并行构建算法较单机构建算法运行时间平均减少49.6%,分布式存储模型VectorTileStore较传统方案更适合海量矢量瓦片存储,存取时间效率更高。

Parallel Construction and Distributed Storage for Vector Tile

With the deepening of the information technology, Internet maps containing multi-source geospatial information are widely used in many fields such as forestry, ocean, land, transportation, and military. At the same time, due to the advancement of Earth observation, surveying, and mapping technology, spatial data with high precision and wide coverage has grown rapidly, leading to an era of geospatial big data. Under this background, how to quickly and efficiently construct Internet map services becomes the current research priorities and challenges. Grid tiles has been used to construct Internet maps at the beginning, and played an important role in the fast-growing popularity of Internet maps. However, with the mobilization of maps and the gradual deepening of applications, the disadvantages of large size and low efficiency of applying grid tiles are becoming more and more obvious, which is difficult to meet the needs of applications. Vector tiles have many advantages over traditional grid tiles, such as small in size, high in generation efficiency, and support dynamic interaction, are becoming the focus of next generation Internet map service research. In order to further accelerate the processing speed and enhance the scalability in current vector tile application, this study uses big data technology for vector tile processing. Firstly, we uses the parallel computing framework-Spark, to build the vector tile pyramid model. Specifically, through customizing the Spark conversion function, the steps of tile generation are parallelized, and the original vector data GeoJson is converted into a vector tile set-MapBox Vector Tile (Mvt). Then we designs a tile storage model-VectorTile Store, to store the generated Mvt based on the distributed memory filesystem-Alluxio. The VectorTile Store model stores data with key-value pairs, with the tile metadata occupying the first eight key-value pairs, and each single tile occupying a key-value pair. When the data is being written, a hash index is built based on the key for fast access. This model efficiently stores massive tiles and is highly scalable. The experimental results show that the vector tile parallel construction algorithm and distributed storage model proposed in this paper are more efficient than traditional schemes, and are more suitable for massive vector tile data processing.