| 摘 要: 在机器学习和数据挖掘过程中，数据缺失现象经常发生。对缺失值的有效补全是数据预处理的重要组成部 分，也是后续分析挖掘工作的基础。最近邻填充算法(kNNI)因其易于实现、计算方便和局部填充效果好等特性而被广 泛应用。但是，它并不涉及全局信息，因而当大段缺失值发生时，补全效果会有所降低，而对于具有周期成分的时序 数据，其效果更是急剧下降。幸运的是，傅里叶变换能够解析出周期数据中的不同周期成分，并能在此基础上通过逆变 换基本实现数据复原，只不过其局部复原能力较弱。因此，本文结合傅里叶变换对周期性数据的全局复原能力和kNNI 对局部数据的补全能力，提出了基于傅里叶变换的kNNI缺失值补全算法(FkNNI)。通过对大量模拟数据的测试结果表 明，该算法比单纯的kNNI算法的缺失值补全准确性有很大提升。 |
| 关键词: 缺失值补全；最近邻填充算法；周期数据；傅里叶变换 |
|
中图分类号: TP391.4
文献标识码: A
|
| 基金项目: 学生创新计划(S201610022096). |
|
| A Missing Value Imputation Algorithm for Periodic Time Series Data Based on kNNI and Fourier Transform |
|
JIA Zijian,SONG Tengwei,WANG Jianxin
|
( School of Information, Beijing Forestry University, Beijing 100083, China)
|
| Abstract: Data missing often occurs during the process of machine learning and data mining.Missing value imputation is an important part of data preprocessing and is also a basis for subsequent work of analysis and mining.The algorithm of k-Nearest Neighbor Imputation (kNNI) is a popular method frequently employed for missing value imputation because it is easy to implement,easy to calculate and effective for local data completion. However,it does not involve global information, and as a result,its effect decreases somewhat when large fragments of missing values occur,especially when there are periodic components in the time series data.Fourier transform, however,is able to analyze the different periodic components in the periodic data,and to roughly restore the data by inverse transform, with its local recovery ability weak only.Therefore,this paper proposes akNNI algorithm based on Fourier transform (FkNNI),combining the global recovery ability of Fourier transform and the local recovery ability of kNNI.Experimental testing results on a large amount of data indicate that the new algorithm is far more accurate than kNNI only. |
| Keywords: missing value imputation;kNNI;cyclical data;Fourier transform |
