A test of the joint efficiency of macroeconomic forecasts using multivariate random forests

We contribute to recent research on the joint evaluation of the properties of macroeconomic forecasts in a multivariate setting. The specific property of forecasts that we are interested in is their joint efficiency. We study the joint efficiency of forecasts by means of multivariate random forests, which we use to model the links between forecast errors and predictor variables in a forecaster's information set. We then use permutation tests to study whether the Mahalanobis distance between the predicted forecast errors for the growth and inflation forecasts of four leading German economic research institutes and actual forecast errors is significantly smaller than under the null hypothesis of forecast efficiency. We reject joint efficiency in several cases, but also document heterogeneity across research institutes with regard to the joint efficiency of their forecasts.     

Combining expert‐adjusted forecasts

It is well known that a combination of model‐based forecasts can improve upon each of the individual constituent forecasts. Most forecasts available in practice are, however, not purely based on econometric models but entail adjustments, where experts with domain‐specific knowledge modify the original model forecasts. There is much evidence that expert‐adjusted forecasts do not necessarily improve the pure model‐based forecasts. In this paper we show, however, that combined expert‐adjusted model forecasts can improve on combined model forecasts, in the case when the individual expert‐adjusted forecasts are not better than their associated model‐based forecasts. We discuss various implications of this finding.     

Evaluating volatility and interval forecasts

A widely used approach to evaluating volatility forecasts uses a regression framework which measures the bias and variance of the forecast. We show that the associated test for bias is inappropriate before introducing a more suitable procedure which is based on the test for bias in a conditional mean forecast. Although volatility has been the most common measure of the variability in a financial time series, in many situations confidence interval forecasts are required. We consider the evaluation of interval forecasts and present a regression‐based procedure which uses quantile regression to assess quantile estimator bias and variance. We use exchange rate data to illustrate the proposal by evaluating seven quantile estimators, one of which is a new non‐parametric autoregressive conditional heteroscedasticity quantile estimator. The empirical analysis shows that the new evaluation procedure provides useful insight into the quality of quantile estimators. Copyright © 1999 John Wiley & Sons, Ltd.     

Daily volatility forecasts: reassessing the performance of GARCH models

Volatility plays a key role in asset and portfolio management and derivatives pricing. As such, accurate measures and good forecasts of volatility are crucial for the implementation and evaluation of asset and derivative pricing models in addition to trading and hedging strategies. However, whilst GARCH models are able to capture the observed clustering effect in asset price volatility in‐sample, they appear to provide relatively poor out‐of‐sample forecasts. Recent research has suggested that this relative failure of GARCH models arises not from a failure of the model but a failure to specify correctly the ‘true volatility’ measure against which forecasting performance is measured. It is argued that the standard approach of using ex post daily squared returns as the measure of ‘true volatility’ includes a large noisy component. An alternative measure for ‘true volatility’ has therefore been suggested, based upon the cumulative squared returns from intra‐day data. This paper implements that technique and reports that, in a dataset of 17 daily exchange rate series, the GARCH model outperforms smoothing and moving average techniques which have been previously identified as providing superior volatility forecasts. Copyright © 2004 John Wiley & Sons, Ltd.     

Combining forecasts using optimal combination weight and generalized autoregression

In this paper, we consider a combined forecast using an optimal combination weight in a generalized autoregression framework. The generalized autoregression provides not only a combined forecast but also an optimal combination weight for combining forecasts. By simulation, we find that short‐ and medium‐horizon (as well as partly long‐horizon) forecasts from the generalized autoregression using the optimal combination weight are more efficient than those from the usual autoregression in terms of the mean‐squared forecast error. An empirical application with US gross domestic product confirms the simulation result. Copyright © 2008 John Wiley & Sons, Ltd.     

Forecasting volatility

In this paper, we investigate the time series properties of S&P 100 volatility and the forecasting performance of different volatility models. We consider several nonparametric and parametric volatility measures, such as implied, realized and model‐based volatility, and show that these volatility processes exhibit an extremely slow mean‐reverting behavior and possible long memory. For this reason, we explicitly model the near‐unit root behavior of volatility and construct median unbiased forecasts by approximating the finite‐sample forecast distribution using bootstrap methods. Furthermore, we produce prediction intervals for the next‐period implied volatility that provide important information about the uncertainty surrounding the point forecasts. Finally, we apply intercept corrections to forecasts from misspecified models which dramatically improve the accuracy of the volatility forecasts. Copyright © 2006 John Wiley & Sons, Ltd.     

Professional economic forecasts: Are they worth their costs?

Most economic forecast evaluations dating back 20 years show that professional forecasters add little to the forecasts generated by the simplest of models. Using various types of forecast error criteria, these evaluations usually conclude that the professional forecasts are little better than the no-change or ARIM A type forecast. It is our contention that this conclusion is mistaken because the conventional error criteria may not capture why forecasts are ma& or how they are used. Using forecast directional accuracy, the criterion which has been found to be highly correlated with profits in an interest rate setting, we find that professional GNP forecasts dominate the cheaper alternatives. Moreover, there appears to be no systematic relationship between this preferred criterion and the error measures used in previous studies.     

A detailed look at crude oil price volatility prediction using macroeconomic variables

We investigate whether crude oil price volatility is predictable by conditioning on macroeconomic variables. We consider a large number of predictors, take into account the possibility that relative predictive performance varies over the out-of-sample period, and shed light on the economic drivers of crude oil price volatility. Results using monthly data from 1983:M1 to 2018:M12 document that variables related to crude oil production, economic uncertainty and variables that either describe the current stance or provide information about the future state of the economy forecast crude oil price volatility at the population level 1 month ahead. On the other hand, evidence of finite-sample predictability is very weak. A detailed examination of our out-of-sample results using the fluctuation test suggests that this is because relative predictive performance changes drastically over the out-of-sample period. The predictive power associated with the more successful macroeconomic variables concentrates around the Great Recession until 2015. They also generate the strongest signal of a decrease in the price of crude oil towards the end of 2008.     

Volatility forecasts using stochastic volatility models with nonlinear leverage effects

The leverage effect—the correlation between an asset's return and its volatility—has played a key role in forecasting and understanding volatility and risk. While it is a long standing consensus that leverage effects exist and improve forecasts, empirical evidence puzzlingly does not show that this effect exists for many individual stocks, mischaracterizing risk, and therefore leading to poor predictive performance. We examine this puzzle, with the goal to improve density forecasts, by relaxing the assumption of linearity of the leverage effect. Nonlinear generalizations of the leverage effect are proposed within the Bayesian stochastic volatility framework in order to capture flexible leverage structures. Efficient Bayesian sequential computation is developed and implemented to estimate this effect in a practical, on-line manner. Examining 615 stocks that comprise the S&P500 and Nikkei 225, we find that our proposed nonlinear leverage effect model improves predictive performances for 89% of all stocks compared to the conventional stochastic volatility model.     

A threshold factor multivariate stochastic volatility model

A new multivariate stochastic volatility model is developed in this paper. The main feature of this model is to allow threshold asymmetry in a factor covariance structure. The new model provides a parsimonious characterization of volatility and correlation asymmetry in response to market news. Statistical inferences are drawn from Markov chain Monte Carlo methods. We introduce news impact analysis to analyze volatility asymmetry with a factor structure. This analysis helps us to study different responses of volatility to historical market information in a multivariate volatility framework. Our model is successful when applied to an extensive empirical study of twenty stocks. Copyright © 2009 John Wiley & Sons, Ltd.     

Forecasting aggregate market volatility: The role of good and bad uncertainties

We decompose economic uncertainty into "good" and "bad" components according to the sign of innovations. Our results indicate that bad uncertainty provides stronger predictive content regarding future market volatility than good uncertainty. The asymmetric models with good and bad uncertainties forecast market volatility in a better way than the symmetric models with overall uncertainty. The combination for asymmetric uncertainty models significantly outperforms the benchmark of autoregression, as well as the combination for symmetric models. The revealed volatility predictability is further demonstrated to be economically significant in the framework of portfolio allocation.     

Model averaging estimation for conditional volatility models with an application to stock market volatility forecast

This paper is concerned with model averaging estimation for conditional volatility models. Given a set of candidate models with different functional forms, we propose a model averaging estimator and forecast for conditional volatility, and construct the corresponding weight-choosing criterion. Under some regulatory conditions, we show that the weight selected by the criterion asymptotically minimizes the true Kullback–Leibler divergence, which is the distributional approximation error, as well as the Itakura–Saito distance, which is the distance between the true and estimated or forecast conditional volatility. Monte Carlo experiments support our newly proposed method. As for the empirical applications of our method, we investigate a total of nine major stock market indices and make a 1-day-ahead volatility forecast for each data set. Empirical results show that the model averaging forecast achieves the highest accuracy in terms of all types of loss functions in most cases, which captures the movement of the unknown true conditional volatility.     

Combining forecasts based on multiple encompassing tests in a macroeconomic core system

This paper investigates whether and to what extent multiple encompassing tests may help determine weights for forecast averaging in a standard vector autoregressive setting. To this end we consider a new test‐based procedure, which assigns non‐zero weights to candidate models that add information not covered by other models. The potential benefits of this procedure are explored in extensive Monte Carlo simulations using realistic designs that are adapted to UK and to French macroeconomic data, to which trivariate vector autoregressions (VAR) are fitted. Thus simulations rely on potential data‐generating mechanisms for macroeconomic data rather than on simple but artificial designs. We run two types of forecast ‘competitions’. In the first one, one of the model classes is the trivariate VAR, such that it contains the generating mechanism. In the second specification, none of the competing models contains the true structure. The simulation results show that the performance of test‐based averaging is comparable to uniform weighting of individual models. In one of our role model economies, test‐based averaging achieves advantages in small samples. In larger samples, pure prediction models outperform forecast averages. Copyright © 2010 John Wiley & Sons, Ltd.     

Predictive ability and economic gains from volatility forecast combinations

The availability of numerous modeling approaches for volatility forecasting leads to model uncertainty for both researchers and practitioners. A large number of studies provide evidence in favor of combination methods for forecasting a variety of financial variables, but most of them are implemented on returns forecasting and evaluate their performance based solely on statistical evaluation criteria. In this paper, we combine various volatility forecasts based on different combination schemes and evaluate their performance in forecasting the volatility of the S&P 500 index. We use an exhaustive variety of combination methods to forecast volatility, ranging from simple techniques to time-varying techniques based on the past performance of the single models and regression techniques. We then evaluate the forecasting performance of single and combination volatility forecasts based on both statistical and economic loss functions. The empirical analysis in this paper yields an important conclusion. Although combination forecasts based on more complex methods perform better than the simple combinations and single models, there is no dominant combination technique that outperforms the rest in both statistical and economic terms.     

Predicting exchange rates in Asia: New insights on the accuracy of survey forecasts

By linking measures of forecast accuracy as well as testing procedures with regard to forecast rationality this paper investigates aggregated survey forecasts with forecast horizons of 3, 12, and 24 months for the exchange rates of the Chinese yuan, the Hong Kong dollar, the Japanese yen, and the Singapore dollar vis-à-vis the US dollar and, hence, for four different currency regimes. The rationality of the exchange rate predictions is initially assessed utilizing tests for unbiasedness and efficiency which indicate that the investigated forecasts are irrational in the sense that the predictions are biased. As one major contribution of this paper, it is subsequently shown that these results are not consistent with an alternative, less restrictive, measure of rationality. Investigating the order of integration of the time series as well as cointegrating relationships, this empirical evidence supports the conclusion that the majority of forecasts are in fact rational. Regarding forerunning properties of the predictions, the results are rather mediocre, with shorter term forecasts for the tightly managed USD/CNY FX regime being one exception. As one additional important and novel evaluation result, it can be concluded, that the currency regime matters for the quality of exchange rate forecasts.     

Average conditional correlation and tree structures for multivariate GARCH models

We propose a simple class of multivariate GARCH models, allowing for time‐varying conditional correlations. Estimates for time‐varying conditional correlations are constructed by means of a convex combination of averaged correlations (across all series) and dynamic realized (historical) correlations. Our model is very parsimonious. Estimation is computationally feasible in very large dimensions without resorting to any variance reduction technique. We back‐test the models on a six‐dimensional exchange‐rate time series using different goodness‐of‐fit criteria and statistical tests. We collect empirical evidence of their strong predictive power, also in comparison to alternative benchmark procedures. Copyright © 2006 John Wiley & Sons, Ltd.     

Model Uncertainty and Forecast Combination in High‐Dimensional Multivariate Volatility Prediction

In multivariate volatility prediction, identifying the optimal forecasting model is not always a feasible task. This is mainly due to the curse of dimensionality typically affecting multivariate volatility models. In practice only a subset of the potentially available models can be effectively estimated, after imposing severe constraints on the dynamic structure of the volatility process. It follows that in most applications the working forecasting model can be severely misspecified. This situation leaves scope for the application of forecast combination strategies as a tool for improving the predictive accuracy. The aim of the paper is to propose some alternative combination strategies and compare their performances in forecasting high‐dimensional multivariate conditional covariance matrices for a portfolio of US stock returns. In particular, we will consider the combination of volatility predictions generated by multivariate GARCH models, based on daily returns, and dynamic models for realized covariance matrices, built from intra‐daily returns. Copyright © 2015 John Wiley & Sons, Ltd.     

Measuring the market risk of freight rates: A forecast combination approach

This paper addresses the issue of freight rate risk measurement via value at risk (VaR) and forecast combination methodologies while focusing on detailed performance evaluation. We contribute to the literature in three ways: First, we reevaluate the performance of popular VaR estimation methods on freight rates amid the adverse economic consequences of the recent financial and sovereign debt crisis. Second, we provide a detailed and extensive backtesting and evaluation methodology. Last, we propose a forecast combination approach for estimating VaR. Our findings suggest that our combination methods produce more accurate estimates for all the sectors under scrutiny, while in some cases they may be viewed as conservative since they tend to overestimate nominal VaR.     

Forecasting and trading currency volatility: an application of recurrent neural regression and model combination

In this paper, we examine the use of non‐parametric Neural Network Regression (NNR) and Recurrent Neural Network (RNN) regression models for forecasting and trading currency volatility, with an application to the GBP/USD and USD/JPY exchange rates. Both the results of the NNR and RNN models are benchmarked against the simpler GARCH alternative and implied volatility. Two simple model combinations are also analysed. The intuitively appealing idea of developing a nonlinear nonparametric approach to forecast FX volatility, identify mispriced options and subsequently develop a trading strategy based upon this process is implemented for the first time on a comprehensive basis. Using daily data from December 1993 through April 1999, we develop alternative FX volatility forecasting models. These models are then tested out‐of‐sample over the period April 1999–May 2000, not only in terms of forecasting accuracy, but also in terms of trading efficiency: in order to do so, we apply a realistic volatility trading strategy using FX option straddles once mispriced options have been identified. Allowing for transaction costs, most trading strategies retained produce positive returns. RNN models appear as the best single modelling approach yet, somewhat surprisingly, model combination which has the best overall performance in terms of forecasting accuracy, fails to improve the RNN‐based volatility trading results. Another conclusion from our results is that, for the period and currencies considered, the currency option market was inefficient and/or the pricing formulae applied by market participants were inadequate. Copyright © 2002 John Wiley & Sons, Ltd.     

Realized Volatility Forecast of Stock Index Under Structural Breaks

We investigate the realized volatility forecast of stock indices under the structural breaks. We utilize a pure multiple mean break model to identify the possibility of structural breaks in the daily realized volatility series by employing the intraday high‐frequency data of the Shanghai Stock Exchange Composite Index and the five sectoral stock indices in Chinese stock markets for the period 4 January 2000 to 30 December 2011. We then conduct both in‐sample tests and out‐of‐sample forecasts to examine the effects of structural breaks on the performance of ARFIMAX‐FIGARCH models for the realized volatility forecast by utilizing a variety of estimation window sizes designed to accommodate potential structural breaks. The results of the in‐sample tests show that there are multiple breaks in all realized volatility series. The results of the out‐of‐sample point forecasts indicate that the combination forecasts with time‐varying weights across individual forecast models estimated with different estimation windows perform well. In particular, nonlinear combination forecasts with the weights chosen based on a non‐parametric kernel regression and linear combination forecasts with the weights chosen based on the non‐negative restricted least squares and Schwarz information criterion appear to be the most accurate methods in point forecasting for realized volatility under structural breaks. We also conduct an interval forecast of the realized volatility for the combination approaches, and find that the interval forecast for nonlinear combination approaches with the weights chosen according to a non‐parametric kernel regression performs best among the competing models. Copyright © 2014 John Wiley & Sons, Ltd.