Detecting anomalous access patterns in relational databases

A considerable effort has been recently devoted to the development of Database Management Systems (DBMS) which guarantee high assurance and security. An important component of any strong security solution is represented by Intrusion Detection (ID) techniques, able to detect anomalous behavior of applications and users. To date, however, there have been few ID mechanisms proposed which are specifically tailored to function within the DBMS. In this paper, we propose such a mechanism. Our approach is based on mining SQL queries stored in database audit log files. The result of the mining process is used to form profiles that can model normal database access behavior and identify intruders. We consider two different scenarios while addressing the problem. In the first case, we assume that the database has a Role Based Access Control (RBAC) model in place. Under a RBAC system permissions are associated with roles, grouping several users, rather than with single users. Our ID system is able to determine role intruders, that is, individuals while holding a specific role, behave differently than expected. An important advantage of providing an ID technique specifically tailored to RBAC databases is that it can help in protecting against insider threats. Furthermore, the existence of roles makes our approach usable even for databases with large user population. In the second scenario, we assume that there are no roles associated with users of the database. In this case, we look directly at the behavior of the users. We employ clustering algorithms to form concise profiles representing normal user behavior. For detection, we either use these clustered profiles as the roles or employ outlier detection techniques to identify behavior that deviates from the profiles. Our preliminary experimental evaluation on both real and synthetic database traces shows that our methods work well in practical situations. This material is based upon work supported by the National Science Foundation under Grant No. 0430274 and the sponsors of CERIAS.     

Geometric operations using sparse interpolation matrices

Based on the computation of a superset of the implicit support, implicitization of a parametrically given hypersurface is reduced to computing the nullspace of a numeric matrix. Our approach predicts the Newton polytope of the implicit equation by exploiting the sparseness of the given parametric equations and of the implicit polynomial, without being affected by the presence of any base points. In this work, we study how this interpolation matrix expresses the implicit equation as a matrix determinant, which is useful for certain operations such as ray shooting, and how it can be used to reduce some key geometric predicates on the hypersurface, namely membership and sidedness for given query points, to simple numerical operations on the matrix, without need to develop the implicit equation. We illustrate our results with examples based on our Maple implementation.     

Detecting patterns in process data with fractal dimension

In quality control discipline, pattern classification is focused on the detection of unnatural patterns in process data. In this paper, fractal dimension is proposed as a new classifier for pattern classification. Fractal dimension is an index for measuring the complexity of an object. Its applications were found in such diverse fields as manufacturing, material science, medical, and image processing. A method for detecting patterns in process data using the fractal dimension is proposed in this paper. A Monte Carlo study was carried out to study the fractal dimension (D) and the Y-intercept (Y_int) values of process data with patterns of interest. The patterns included in the study are natural pattern, upward linear trend, downward linear trend, cycle, systematic variable, stratification, mixture, upward sudden shift, and downward sudden shift. Based on the results, the approach is effective in detecting such non-periodic patterns as the natural patterns, linear trends (at slope ≥0.2), systematic variable, stratification, mixture, and sudden shifts. For the cyclical pattern, although the D and Y_int-values are not stable, the approach can provide useful information when the period of the cycle is greater than 2 and is less than or equal to half the window size (2N/2). The minor drawbacks of this approach are that it is not sensitive for detecting linear trends with small slope and the slope of the original data is needed to detect the difference between upward and downward linear trends and the difference between upward and downward sudden shifts.     

Monitoring fall foliage coloration dynamics using time-series satellite data

Fall foliage coloration is a phenomenon that occurs in many deciduous trees and shrubs worldwide. Measuring the phenology of fall foliage development is of great interest for climate change, the carbon cycle, ecology, and the tourist industry; but little effort has been devoted to monitoring the regional fall foliage status using remotely-sensed data. This study developed an innovative approach to monitoring fall foliage status by means of temporally-normalized brownness derived from MODIS (Moderate Resolution Imaging Spectroradiometer) data. Specifically, the time series of the MODIS Normalized Difference Vegetation Index (NDVI) was smoothed and functionalized using a sigmoidal model to depict the continuous dynamics of vegetation growth. The modeled temporal NDVI trajectory during the senescent phase was further combined with the mixture modeling to deduce the temporally-normalized brownness index which was independent of the surface background, vegetation abundance, and species composition. This brownness index was quantitatively linked with the fraction of colored and fallen leaves in order to model the fall foliage coloration status. This algorithm was tested by monitoring the fall foliage coloration phase using MODIS data in northeastern North America from 2001 to 2004. The MODIS-derived timing of foliage coloration phases was compared with in-situ measurements, which showed an overall absolute mean difference of less than 5 days for all foliage coloration phases and about 3 days for near peak coloration and peak coloration. This suggested that the fall foliage coloration phase retrieved from the temporally-normalized brownness index was qualitatively realistic and repeatable.     

Mapping cotton fields using data mining and MODIS time-series

ABSTRACT

Cotton is the most important fibre culture in the world. In Brazil, cotton cultivation is concentrated in the Cerrado biome, the Brazilian savanna, and is one of the most important commodities in the country. As an annual crop, the updating frequency of the spatial distribution data of cotton fields is extremely important for crop monitoring systems. In order to provide fast and accurate information for crop monitoring, time series of remote- sensing data has been used in the development of several applications in agriculture, since the high temporal resolution of some orbital sensor allows monitoring targets with high spectral-temporal variations in the land surface. However, there are still some challenges to systematize the processing of such a large amount of data available by long time series of remote-sensing imagery. Thus, this study contributes to the construction of models to identify and separate specific crop types with similar spectral behaviour to other crops practised in the same period. The objective of this study was to develop a systematic methodology based on data mining of time series of vegetation indices (VI) to map cotton fields at the regional scale. Field reference data and time series of NDVI and EVI images, obtained from MODIS sensor products during four cropping seasons (from 2012–2013 to 2015–2016), were used to construct mapping models based on decision tree algorithms. Phenological metrics were calculated from the VI time series and used to build classification rules for mapping cotton fields. Our results demonstrate that the proposed method to map cotton fields achieve high accuracy when field data and visual interpretation of NDVI temporal profiles were used for validation (accuracy higher than 95% and 93%, respectively). Comparisons with the official statistics indicated an optimal fit, with linear correlation (r) and coefficient of determination (R²) above 0.93. Therefore, the proposed method was efficient to distinguish cotton fields from other crop types with similar spectral behaviour. In addition, this method can also be applied to other cotton-producing regions and other production seasons, by reusing the models generated through machine learning approaches.     

A scalable sparse Cholesky based approach for learning high-dimensional covariance matrices in ordered data

Machine Learning - Covariance estimation for high-dimensional datasets is a fundamental problem in machine learning, and has numerous applications. In these high-dimensional settings the number of...     

Uncertainty propagation analysis using sparse grid technique and saddlepoint approximation based on parameterized p-box representation

Uncertainty propagation analysis, which assesses the impact of the uncertainty of input variables on responses, is an important component in risk assessment or reliability analysis of structures. This paper proposes an uncertainty propagation analysis method for structures with parameterized probability-box (p-box) representation, which could efficiently compute both the bounds on statistical moments and also the complete probability bounds of the response function. Firstly, based on the sparse grid numerical integration (SGNI) method, an optimized SGNI (OSGNI) is presented to calculate the bounds on the statistical moments of the response function and the cumulants of the cumulant generating function (CGF), respectively. Then, using the bounds on the first four cumulants, an optimization procedure based on the saddlepoint approximation is proposed to obtain the whole range of probability bounds of the response function. Through using the saddlepoint approximation, the present approach can achieve a good accuracy in estimating the tail probability bounds of a response function. Finally, two numerical examples and an engineering application are investigated to demonstrate the effectiveness of the proposed method.

     

Fast inference in SAM fuzzy systems using transition matrices

Fast inference using transition matrices (FITM) is a new fast algorithm for performing inferences in fuzzy systems. It is based on the assumption that fuzzy inputs can be expressed as a linear composition of the fuzzy sets used in the rule base. This representation let us interpret a fuzzy set as a vector, so we can just work with the coordinates of it instead of working with the whole set. The inference is made using transition matrices. The key of the method is the fact that a lot of operations can be precomputed offline to obtain the transition matrices, so actual inferences are reduced to a few online matrix additions and multiplications. The algorithm is designed for the standard additive model using the sum-product inference composition.     

图像重建中的非常稀疏循环矩阵

测量矩阵是压缩传感理论的关键要素之一。针对目前大部分工作中所用的高斯等随机测量矩阵独立随机变元过多,不利于物理实现的问题,引入稀疏带状和稀疏列的概念,形成稀疏带状随机、托普利兹和循环矩阵以及稀疏列随机、循环矩阵,随机变元个数减少约三分之一。采用通用的模拟实验方法,验证此类稀疏矩阵对于真实图像的重建效果及对0-1信号的成功重建概率均与随机高斯矩阵相当。     

Hierarchical visualization of time-series data using switching linear dynamical systems

We propose a novel visualization algorithm for high-dimensional time-series data. In contrast to most visualization techniques, we do not assume consecutive data points to be independent. The basic model is a linear dynamical system which can be seen as a dynamic extension of a probabilistic principal component model. A further extension to a particular switching linear dynamical system allows a representation of complex data onto multiple and even a hierarchy of plots. Using sensible approximations based on expectation propagation, the projections can be performed in essentially the same order of complexity as their static counterpart. We apply our method on a real-world data set with sensor readings from a paper machine.     

Mining closed patterns in multi-sequence time-series databases

In this paper, we propose an efficient algorithm, called CMP-Miner, to mine closed patterns in a time-series database where each record in the database, also called a transaction, contains multiple time-series sequences. Our proposed algorithm consists of three phases. First, we transform each time-series sequence in a transaction into a symbolic sequence. Second, we scan the transformed database to find frequent patterns of length one. Third, for each frequent pattern found in the second phase, we recursively enumerate frequent patterns by a frequent pattern tree in a depth-first search manner. During the process of enumeration, we apply several efficient pruning strategies to remove frequent but non-closed patterns. Thus, the CMP-Miner algorithm can efficiently mine the closed patterns from a time-series database. The experimental results show that our proposed algorithm outperforms the modified Apriori and BIDE algorithms.     

Task reduction using regression-based missing data imputation in sparse mobile crowdsensing

The Journal of Supercomputing - Mobile Crowd Sensing (MCS) involves allocation of sensing tasks associated with an area of interest to a crowd of participants over time. Consequently, the...     

Statistical downscaling of rainfall data using sparse variable selection methods

In many statistical downscaling methods, atmospheric variables are chosen by using a combination of expert knowledge with empirical measures such as correlations and partial correlations. In this short communication, we describe the use of a fast, sparse variable selection method, known as RaVE, for selecting atmospheric predictors, and illustrate its use on rainfall occurrence at stations in South Australia. We show that RaVE generates parsimonious models that are both sensible and interpretable, and whose results compare favourably to those obtained by a non-homogeneous hidden Markov model (Hughes et al., 1999).     

二次筛选法中大型稀疏矩阵规模缩减算法

利用二次筛选法分解RSA的模数时,矩阵规模对算法性能有着重要的影响,缩减矩阵的规模可以有效地缩短算法的运行时间。根据二次筛选法的原理,给出了3种缩减矩阵规模的方法,结合二次筛选中的稀疏矩阵的存储结构,提出了相应的3种缩减算法。最后实现了这3种缩减算法,并在二次筛选法分解70位十进制大数程序中进行了成功的应用,给出了实验的结果。     

Detecting and repairing anomalous evolutions in noisy environments

In systems where agents are required to interact with a partially known and dynamic world, sensors can be used to obtain knowledge about the environment. However, sensors may be unreliable, that is, they may deliver wrong information (due, e.g., to hardware or software malfunctioning) and, consequently, they may cause agents to take wrong decisions, which is a scenario that should be avoided. The paper considers the problem of reasoning in noisy environments in a setting where no (either certain or probabilistic) data is available in advance about the reliability of sensors. Therefore, assuming that each agent is equipped with a background theory encoding its general knowledge about the world, a concept of detecting an anomaly perceived in sensor data and the related concept of agent recovering to a coherent status of information are defined. In this context, the complexities of various anomaly detection and anomaly recovery problems are studied. Finally, rewriting algorithms are proposed that transform recovery problems into equivalent inference problems under answer set semantics, thereby making them effectively realizable on top of available answer set solvers.     

A formal semantics of extended hierarchical state transition matrices using CSP#

The extended hierarchical state transition matrices (EHSTMs) are a table-based modelling language frequently used in industry for specifying behaviours of systems. However, assuring correctness, i.e., having a design satisfy certain desired properties, is a non-trivial task. To address this problem, a model checker dedicated to EHSTMs called Garakabu2 has been developed. However, there is no formal justification for Garakabu2, since its semantics has never been fully formalised. In this paper, we give a formal semantics to EHSTMs by translating them into CSP, Communicating Sequential Processes. Among the variants of CSP, we use CSP#, which is the modelling language used by PAT model checker, as a target of translation. Our semantics covers most of the features supported by Garakabu2. We manually translate the small examples of EHSTMs to CSP#, and verify them by PAT. We also verify the examples directly using Garakabu2 and show that the results are same. The experiments also indicate that verification using our translation and PAT is much faster than that of Garakabu2 in some cases.     

Detecting nonlinear interrelation patterns among process variables using genetic programming

Detecting non-linear interaction patterns among process variables is an important task for fault detection and propagation analysis. There are many statistical and evolutionary techniques being developed in the literature for prediction of interaction strengths but their accuracy is generally unsatisfactory. This study demonstrates an evolutionary programming approach to uncover non-linear relations among process variables. In this study, we make an attempt to use genetic programming (GP) based approach for this purpose. GP overcomes many shortcomings faced by other statistical or evolutionary techniques in this context. The effectiveness, feasibility, and robustness of the proposed method are demonstrated on simulated data emanating from a well-known Tennessee Eastman process. The proposed method has successfully achieved reasonable detection and prediction of non-linear interaction patterns among process variables.