基于Spark的油田应用日志行为分析系统

随着油田信息化建设的不断发展，越来越多的IT业务系统在油田各级单位普及应用.由于油田应用数量庞大、种类复杂，如何快速评估各类系统的运行情况和安全状况成为油田关注的重要问题.在使用这些应用系统的同时，一些访问信息会以日志的形式储存下来，因此通过分析日志数据可以挖掘出用户访问喜好，发觉业务系统潜在的安全问题，进而为油田应用评估提供决策依据.然而随着IT业务访问量剧增，应用日志的数量、容量也随之增加，仅依靠单机环境对海量数据进行分析已经无法满足油田业务需求.针对这个问题本文提出了基于Spark计算框架的应用日志行为分析方法，同时设计了可视化平台完成对整个分析系统的管理.

Oil Fild Application Log Behavior Analysis System Based on Spark

With the rapid development of Internet information construction, more and more IT systems are widely used. Due to the huge amount and complexity of oilfield applications, how to quickly evaluate the operation and safety status of various systems has become an important issue in oilfield. When using the business system, some access information was recorded in the form of logs at the same time. By analyzing the log data, the user''s access preferences can be excavated and the potential network security problems of the business system can be found, thus providing a decision basis for the evaluation of oil field applications. However, with the rapid increase of business access, the amount and storage capacity of logs also increase. Relying on single computer environment, analyzing massive log data has been unable to meet the needs of applications. In view of this problem, this study proposes a log behavior analysis method based on Spark calculation framework and designs a service platform for visual management based on Web.