基于信息熵的不确定性数据清理方法

针对不确定性数据中往往包含一些异常数据而导致相应的查询结果出现错误的问题,提出了一种基于信息熵的不确定性数据清理方法以减少异常数据并提高不确定性数据的质量。首先使用信息熵来度量数据的不确定度,然后结合统计学方法计算出不确定性数据的可信区间,最后去除那些不在可信区间内的数据。实验结果验证了该方法的高效性和有效性。

Approach for cleaning uncertain data based on information entropy theory

In response to the issue that data anomalies in the uncertain databases often hamper the efficient and effective use of data, an uncertain data cleaning method was proposed to reduce abnormal data based on the information entropy theory. First, the uncertainty degree of uncertain data was defined by using information entropy. Then, the confidence interval of uncertain data was obtained based on statistical method with the degree of uncertainty. By means of the confidence interval, the uncertain databases were cleaned. The experimental results show the effectiveness and efficiency of the proposed method.