A multiple-kernel support vector regression approach for stock market price forecasting

Support vector regression has been applied to stock market forecasting problems. However, it is usually needed to tune manually the hyperparameters of the kernel functions. Multiple-kernel learning was developed to deal with this problem, by which the kernel matrix weights and Lagrange multipliers can be simultaneously derived through semidefinite programming. However, the amount of time and space required is very demanding. We develop a two-stage multiple-kernel learning algorithm by incorporating sequential minimal optimization and the gradient projection method. By this algorithm, advantages from different hyperparameter settings can be combined and overall system performance can be improved. Besides, the user need not specify the hyperparameter settings in advance, and trial-and-error for determining appropriate hyperparameter settings can then be avoided. Experimental results, obtained by running on datasets taken from Taiwan Capitalization Weighted Stock Index, show that our method performs better than other methods.     

Combining support vector regression with feature selection for multivariate calibration

Multivariate calibration is a classic problem in the analytical chemistry field and frequently solved by partial least squares (PLS) and artificial neural networks (ANNs) in the previous works. The spaciality of multivariate calibration is high dimensionality with small sample. Here, we apply support vector regression (SVR) as well as ANNs, and PLS to the multivariate calibration problem in the determination of the three aromatic amino acids (phenylalanine, tyrosine and tryptophan) in their mixtures by fluorescence spectroscopy. The results of the leave-one-out method show that SVR performs better than other methods, and appear to be one good method for this task. Furthermore, feature selection is performed for SVR to remove redundant features and a novel algorithm named Prediction RIsk based FEature selection for support vector Regression (PRIFER) is proposed. Results on the above multivariate calibration data set show that PRIFER is a powerful tool for solving the multivariate calibration problems.     

Combining wavelet-based feature extractions with relevance vector machines for stock index forecasting

Abstract: The relevance vector machine (RVM) is a Bayesian version of the support vector machine, which with a sparse model representation has appeared to be a powerful tool for time-series forecasting. The RVM has demonstrated better performance over other methods such as neural networks or autoregressive integrated moving average based models. This study proposes a hybrid model that combines wavelet-based feature extractions with RVM models to forecast stock indices. The time series of explanatory variables are decomposed using some wavelet bases and the extracted time-scale features serve as inputs of an RVM to perform the non-parametric regression and forecasting. Compared with traditional forecasting models, our proposed method performs best. The root-mean-squared forecasting errors are significantly reduced.     

Hybridizing nonlinear independent component analysis and support vector regression with particle swarm optimization for stock index forecasting

One of the major activities of financial firms and private investors is to predict future prices of stocks. However, stock index prediction is regarded as a challenging task of the prediction problem since the stock market is a complex, chaotic and nonlinear dynamic system. As stock markets are highly dynamic and exhibit wide variation, it may be more realistic and practical that assumed the stock index data are a nonlinear mixture data. In this study, a hybrid stock index prediction model by utilizing nonlinear independent component analysis (NLICA), support vector regression (SVR) and particle swarm optimization (PSO) is proposed. In the proposed model, first, the NLICA is used to deal with the nonlinearity property of the stock index data. The proposed model utilizes NLICA to extract features from the observed stock index data. The features which can be used to represent underlying/hidden information of the data are then served as the inputs of SVR to build the stock index prediction model. Finally, PSO is applied to optimize the parameters of the SVR prediction model since the parameters of SVR must be carefully selected in establishing an effective and efficient SVR model. In order to evaluate the performance of the proposed approach, the closing indexes of the Taiwan stock exchange capitalization weighted stock index, Shanghai stock exchange composite index and Bombay stock exchange index are used as illustrative examples. Experimental results showed that the proposed hybrid stock index prediction method significantly outperforms the other six comparison models. It is an efficient and effective alternative for stock index forecasting.     

Using support vector machine with a hybrid feature selection method to the stock trend prediction

In this paper, we developed a prediction model based on support vector machine (SVM) with a hybrid feature selection method to predict the trend of stock markets. This proposed hybrid feature selection method, named F-score and Supported Sequential Forward Search (F_SSFS), combines the advantages of filter methods and wrapper methods to select the optimal feature subset from original feature set. To evaluate the prediction accuracy of this SVM-based model combined with F_SSFS, we compare its performance with back-propagation neural network (BPNN) along with three commonly used feature selection methods including Information gain, Symmetrical uncertainty, and Correlation-based feature selection via paired t-test. The grid-search technique using 5-fold cross-validation is used to find out the best parameter value of kernel function of SVM. In this study, we show that SVM outperforms BPN to the problem of stock trend prediction. In addition, our experimental results show that the proposed SVM-based model combined with F_SSFS has the highest level of accuracies and generalization performance in comparison with the other three feature selection methods. With these results, we claim that SVM combined with F_SSFS can serve as a promising addition to the existing stock trend prediction methods.     

Incorporating prior knowledge in support vector regression

This paper explores the incorporation of prior knowledge in support vector regresion by the addition of constraints. Equality and inequality constraints are studied with the corresponding types of prior knowledge that can be considered for the method. These include particular points with known values, prior knowledge on any derivative of the function either provided by a prior model or available only at some specific points and bounds on the function or any derivative in a given domain. Moreover, a new method for the simultaneous approximation of multiple outputs linked by some prior knowledge is proposed. This method also allows consideration of different types of prior knowledge on single outputs while training on multiple outputs. Synthetic examples show that incorporating a wide variety of prior knowledge becomes easy, as it leads to linear programs, and helps to improve the approximation in difficult cases. The benefits of the method are finally shown on a real-life application, the estimation of in-cylinder residual gas fraction in spark ignition engines, which is representative of numerous situations met in engineering. Editor: Dale Schuurmans.     

A two-stage architecture for stock price forecasting by integrating self-organizing map and support vector regression

Stock price prediction has attracted much attention from both practitioners and researchers. However, most studies in this area ignored the non-stationary nature of stock price series. That is, stock price series do not exhibit identical statistical properties at each point of time. As a result, the relationships between stock price series and their predictors are quite dynamic. It is challenging for any single artificial technique to effectively address this problematic characteristics in stock price series. One potential solution is to hybridize different artificial techniques. Towards this end, this study employs a two-stage architecture for better stock price prediction. Specifically, the self-organizing map (SOM) is first used to decompose the whole input space into regions where data points with similar statistical distributions are grouped together, so as to contain and capture the non-stationary property of financial series. After decomposing heterogeneous data points into several homogenous regions, support vector regression (SVR) is applied to forecast financial indices. The proposed technique is empirically tested using stock price series from seven major financial markets. The results show that the performance of stock price prediction can be significantly enhanced by using the two-stage architecture in comparison with a single SVR model.     

A hybrid stock selection model using genetic algorithms and support vector regression

In the areas of investment research and applications, feasible quantitative models include methodologies stemming from soft computing for prediction of financial time series, multi-objective optimization of investment return and risk reduction, as well as selection of investment instruments for portfolio management based on asset ranking using a variety of input variables and historical data, etc. Among all these, stock selection has long been identified as a challenging and important task. This line of research is highly contingent upon reliable stock ranking for successful portfolio construction. Recent advances in machine learning and data mining are leading to significant opportunities to solve these problems more effectively. In this study, we aim at developing a methodology for effective stock selection using support vector regression (SVR) as well as genetic algorithms (GAs). We first employ the SVR method to generate surrogates for actual stock returns that in turn serve to provide reliable rankings of stocks. Top-ranked stocks can thus be selected to form a portfolio. On top of this model, the GA is employed for the optimization of model parameters, and feature selection to acquire optimal subsets of input variables to the SVR model. We will show that the investment returns provided by our proposed methodology significantly outperform the benchmark. Based upon these promising results, we expect this hybrid GA-SVR methodology to advance the research in soft computing for finance and provide an effective solution to stock selection in practice.     

Optimal feature selection for support vector machines

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.     

FS_SFS: A novel feature selection method for support vector machines

In many pattern recognition applications, high-dimensional feature vectors impose a high computational cost as well as the risk of “overfitting”. Feature Selection addresses the dimensionality reduction problem by determining a subset of available features which is most essential for classification. This paper presents a novel feature selection method named filtered and supported sequential forward search (FS_SFS) in the context of support vector machines (SVM). In comparison with conventional wrapper methods that employ the SFS strategy, FS_SFS has two important properties to reduce the time of computation. First, it dynamically maintains a subset of samples for the training of SVM. Because not all the available samples participate in the training process, the computational cost to obtain a single SVM classifier is decreased. Secondly, a new criterion, which takes into consideration both the discriminant ability of individual features and the correlation between them, is proposed to effectively filter out nonessential features. As a result, the total number of training is significantly reduced and the overfitting problem is alleviated. The proposed approach is tested on both synthetic and real data to demonstrate its effectiveness and efficiency.     

Covariance matrix forecasting using support vector regression

Applied Intelligence - Support vector regression is a promising method for time-series prediction, as it has good generalisability and an overall stable behaviour. Recent studies have shown that it...     

组合粗糙集和支持向量回归的船舶交通流预测

影响交通流变化的因素众多,为改进传统的船舶交通流预测精度不高,一种结合粗糙集和支持向量回归智能算法的交通流预测模型提出,通过ROSETTA软件进行属性约简预处理,筛选出影响交通流变化的关键影响因素,剔除冗余信息。筛选结果显示外轮进出艘次、对外贸易总额、港口GDP、集装箱标准箱、港口货物吞吐量为输入变量,运用Libsvm软件构建基于遗传算法参数寻优的支持向量回归模型预测2008年和2009年的交通流。算例结果表明,与BP神经网络和SVM模型相比,组合预测模型是有效和实用的预测工具。     

基于模糊支持向量核回归方法的短期峰值负荷预测

分析了电力系统负荷预测目前采用的方法的不足;在已有研究成果的基础上,根据电网负荷的特点进一步完善了基于模糊支持向量的核回归方法;与目前已有的方法,如神经网络、卡尔曼滤波、最小绝对值参数估计、结合遗传算法的支持向量机、结合模糊小波技术的支持向量机等进行对比实验,实验结果展示了几种方法的性能对比,为该领域的研究提供了参考．     

基于特征选择和支持向量机的异常检测方法

针对异常检测系统虚警率高、检测率低以及冗余特征对检测系统造成负担的问题,提出一种基于特征选择和支持向量机相结合的异常检测方法.该方法通过构造一种基于分类模型分类准确率计算的特征选择算法,筛选出能够获得分类准确率最高的特征组合,并与支持向量机分类算法相结合,实现数据的异常检测.仿真测试结果表明,该方法具有较高的检测准确率和较低的检测时间,并通过去除噪声特征,降低了系统的数据处理难度.     

Support vector regression with chaos-based firefly algorithm for stock market price forecasting

Due to the inherent non-linearity and non-stationary characteristics of financial stock market price time series, conventional modeling techniques such as the Box–Jenkins autoregressive integrated moving average (ARIMA) are not adequate for stock market price forecasting. In this paper, a forecasting model based on chaotic mapping, firefly algorithm, and support vector regression (SVR) is proposed to predict stock market price. The forecasting model has three stages. In the first stage, a delay coordinate embedding method is used to reconstruct unseen phase space dynamics. In the second stage, a chaotic firefly algorithm is employed to optimize SVR hyperparameters. Finally in the third stage, the optimized SVR is used to forecast stock market price. The significance of the proposed algorithm is 3-fold. First, it integrates both chaos theory and the firefly algorithm to optimize SVR hyperparameters, whereas previous studies employ a genetic algorithm (GA) to optimize these parameters. Second, it uses a delay coordinate embedding method to reconstruct phase space dynamics. Third, it has high prediction accuracy due to its implementation of structural risk minimization (SRM). To show the applicability and superiority of the proposed algorithm, we selected the three most challenging stock market time series data from NASDAQ historical quotes, namely Intel, National Bank shares and Microsoft daily closed (last) stock price, and applied the proposed algorithm to these data. Compared with genetic algorithm-based SVR (SVR-GA), chaotic genetic algorithm-based SVR (SVR-CGA), firefly-based SVR (SVR-FA), artificial neural networks (ANNs) and adaptive neuro-fuzzy inference systems (ANFIS), the proposed model performs best based on two error measures, namely mean squared error (MSE) and mean absolute percent error (MAPE).     

Pattern selection for support vector regression based response modeling

Two-stage response modeling, identifying respondents and then ranking them according to their expected profit, was proposed in order to increase the profit of direct marketing. For the second stage of two-stage response modeling, support vector regression (SVR) has been successfully employed due to its great generalization performances. However, the training complexities of SVR have made it difficult to apply to response modeling based on the large amount of data. In this paper, we propose a pattern selection method called Expected Margin based Pattern Selection (EMPS) to reduce the training complexities of SVR for use as a response modeling dataset with high dimensionality and high nonlinearity. EMPS estimates the expected margin for all training patterns and selects patterns which are likely to become support vectors. The experimental results involving 20 benchmark datasets and one real-world marketing dataset showed that EMPS improved SVR efficiency for response modeling.     

快速增量加权支持向量机预测证券指数

传统支持向量机是对小样本提出,对于大样本会出现训练速度慢、内存占用多等问题．并且不具有增量学习性能．而常用的增量学习方法又会出现局部极小等问题．本文阐述了一种改进的支持向量机算法(快速增量加权支持向量机算法)用于证券指数预测．该算法先对指数样本做相空间重构,再分解成若干个工作子集,针对样本重要程度给出不同权重构建预测模型．实验分析表明,在泛化精度保持略好情况下,训练速度明显提高．     

On finite Newton method for support vector regression

In this paper, we propose a Newton iterative method of solution for solving an ε-insensitive support vector regression formulated as an unconstrained optimization problem. The proposed method has the advantage that the solution is obtained by solving a system of linear equations at a finite number of times rather than solving a quadratic optimization problem. For the case of linear or kernel support vector regression, the finite termination of the Newton method has been proved. Experiments were performed on IBM, Google, Citigroup and Sunspot time series. The proposed method converges in at most six iterations. The results are compared with that of the standard, least squares and smooth support vector regression methods and of the exact solutions clearly demonstrate the effectiveness of the proposed method.     

回归型支持向量机的调节熵函数法

基于最优化理论中的ＫＫＴ 互补条件建立支持向量回归机的无约束不可微优化模型,并给出了一种有效的光滑近似解法———调节熵函数方法．该方法不需参数取值很大便可逼近问题的最优解,从而避免了一般熵函数法为了逼近精确解,参数取得过大而导致数值的溢出现象,为求解支持向量回归机提供了一条新途径．数值实验结果表明,回归型支持向量机的调节熵函数法改善了支持向量机的回归性能和效率．