MongoDB数据库在网络行为分析与控制系统中的应用

在网络行为分析与控制系统中,传统的关系数据库在异构数据、海量日志的管理上难以胜任。NoSQL非关系型数据库的出现,对于解决面向文档的超大规模和高并发的问题提供了卓有成效的方案。本文在研究非关系数据库的基础上,着重分析MongoDB的特点和优势,通过性能测试,提出将MongoDB数据库应用于网络行为分析与控制系统,有效提高了大规模日志数据在网络行为分析与控制系统中的存储效率。     

一种基于数据热点的MongoDB负载均衡策略

MongoDB作为一种新兴的NoSQL数据库,以其模式自由、文档式存储、故障自动恢复、良好的水平扩展、自动负载均衡等特点深受国内外市场的青睐.MongoDB自带的负载均衡策略能使各个节点数据量达到平衡.但是在实际的生产环境中,节点之间数据访问热度不同也会导致负载失衡,特别是出现节点过热的情况.针对这一问题,引入Markov随机过程,提出一种基于Markov预测模型的负载均衡策略,根据Markov模型的稳态概率向量预测各个分片的负载并进行数据迁移.通过实验,验证了当各个分片间出现节点过热时,所提出的负载均衡策略能够很好地使分片间的负载达到基于访问热点的均衡.     

NoSQL在构建业务信令监测系统中的应用

国内电信运营商当前大多数的信令监测系统都是采用SQL数据库作为存储解决方案,但日志产生的速度往往超出存储的处理速度。在这种情况下,提出使用NoSQL数据库作为另一种存储的解决方案,探讨其在监测系统中的应用。通过比较Oracle和MongoDB的读写性能表现从而分析NoSQL用于处理海量数据的适用性和有效性。以实际使用效果为案例,说明该方案的成效,为众多类似业务的开发提供一种新的技术路线。     

面向海量数据的分布式用户态文件系统研究与设计

云计算、物联网、互联网技术的迅猛发展,使社会进入了大数据时代。随着数据量的指数增长,面对海量数据的高效存储与访问成为目前人们面临的主要问题。使用分布式存储作为底层数据存储系统,基于用户空间文件系统fuse,结合MongoDB数据库管理元数据,研究设计出一种面向分布式存储的用户态文件系统。相对于传统分布式对象存储系统的数据访问,设计的文件系统具有很高的灵活性和可扩展性,同时对于用户有着可操作性强、易于管理等优点,使其更适用于现代海量数据的访问与管理。     

A flexible data schema and system architecture for the virtualization of manufacturing machines (VMM)

Future factories will feature strong integration of physical machines and cyber-enabled software, working seamlessly to improve manufacturing production efficiency. In these digitally enabled and network connected factories, each physical machine on the shop floor can have its ‘virtual twin’ available in cyberspace. This ‘virtual twin’ is populated with data streaming in from the physical machines to represent a near real-time as-is state of the machine in cyberspace. This results in the virtualization of a machine resource to external factory manufacturing systems. This paper describes how streaming data can be stored in a scalable and flexible document schema based database such as MongoDB, a data store that makes up the virtual twin system. We present an architecture, which allows third-party integration of software apps to interface with the virtual manufacturing machines. We evaluate our database schema against query statements and provide examples of how third-party apps can interface with manufacturing machines using the VMM middleware. Finally, we discuss an operating system architecture for VMMs across the manufacturing cyberspace, which necessitates command and control of various virtualized manufacturing machines, opening new possibilities in cyber-physical systems in manufacturing.     

基于Flex、Red5和MongoDB的视频直播、录制及存储系统设计

为改善以往网络视频直播或点播过程中播放不流畅的情况和寻找大量视频数据的存储策略,提出了一种实时的视频直播、录制及存储系统的整体设计方案。利用开源的流媒体服务器Red5结合富互联网技术Flex实现直播和录制;通过开源NoSQL数据库MongoDB将录制的视频数据进行存储。实际测试结果表明,该平台能满足系统在多用户访问和数据存储方面的要求。     

基于MongoDB的HIAF Archive Engine设计与实现

强流重离子加速器装置（HIAF）的控制系统采用实验物理和工业控制系统（EPICS）架构,运行数据存储系统为HIAF运行中的运行数据、历史数据和静态参数提供稳定可靠的存储。针对当前EPICS架构下的数据存储工具仅支持关系型数据库的不足,开发非关系型数据库MongoDB接口,并利用MongoDB嵌套对象和多层级存储结构的特点优化存储表结构,实现波形、图像等非关系型数据的高效存储;为调束人员提供基于标志位PV（Flag PV）方式的加速器运行数据手动存储方案以提升历史数据检索效率及数据可用性;同时针对海量历史数据的存储需要,利用MongoDB Sharding搭建MongoDB分布式集群。     

NoSQL real-time database performance comparison

The amount of data being produced is increasing constantly, as the number and variety of connected devices are growing and the advances in data storage and mining are supporting this evolution. However, storing and handling high quantities of data is challenging the current Relational Database Management Systems. Big Data and its related products came to help in this matter, and the NoSQL databases arise with the purpose to offer better solutions and features to handle massive amounts of data with higher performance, sometimes near real-time. The present study presents the NoSQL databases scenario and background, and elaborates a detailed study with the characteristics, a features comparison and a performance evaluation of three different NoSQL databases extensively used in the market nowadays: Couchbase, MongoDB and RethinkDB. Tests were performed in two different scenarios: single thread and multiple threads. The results reveal that Couchbase had a better performance at most of the operations, except for retrieving multiple documents and inserting documents with multiple threads, operations in which MongoDB scored better.     

基于消息中间件及MongoDB的物联网应用服务平台

物联网正广泛应用于各行各业.将现实世界参数化并结合感知设备、网络通信技术、数据技术等诸多手段实现物联网数据对于用户的直接可用便是物联网技术的中心思想.本文依托于智能燃气数据管理系统,针对其物联网平台架构中应用服务平台在大规模设备接入场景下的性能瓶颈,提出一种结合消息中间件Kafka与NoSQL数据库MongoDB的混合方案.根据燃气公司和设备厂商的应用背景实现该方案,完成应用服务平台并发性能和数据持久化效率的提高.     

MongoDB在城市排水中的应用研究

In the construction of urban drainage information platform, the amount of rainfall data and water point data is huge. At the same time, it is necessary to combine multimedia data to monitor the drainage points. Based on this, the author takes advantage of MongoDB and the GridFS multimedia module to propose a solution to the storage and management of large amounts of data and multimedia data generated by drainage facilities.