N-step combinations of forecasts

While there is general agreement that a linear combination of forecasts can outperform the individual forecasts, there is controversy about the appropriateness of the combination method to be used in a given situation. Hence, in any given application it may be more beneficial to combine different sets of combined forecasts rather than picking one of them. This paper introduces the concept of N-step combinations of forecasts which involves combining the combined forecasts obtained from different combination procedures used at the preceding step. Using quarterly GNP data, evidence supporting the increase in the accuracy of the one-period-ahead ex-ante forecasts as the combination step increases is provided. The MSE, MAE, MAPE and their corresponding standard deviations are used to evaluate the accuracy of the forecasts obtained.     

Choosing among competing econometric forecasts: Regression‐based forecast combination using model selection

Forecast combination based on a model selection approach is discussed and evaluated. In addition, a combination approach based on ex ante predictive ability is outlined. The model selection approach which we examine is based on the use of Schwarz (SIC) or the Akaike (AIC) Information Criteria. Monte Carlo experiments based on combination forecasts constructed using possibly (misspecified) models suggest that the SIC offers a potentially useful combination approach, and that further investigation is warranted. For example, combination forecasts from a simple averaging approach MSE‐dominate SIC combination forecasts less than 25% of the time in most cases, while other ‘standard’ combination approaches fare even worse. Alternative combination approaches are also compared by conducting forecasting experiments using nine US macroeconomic variables. In particular, artificial neural networks (ANN), linear models, and professional forecasts are used to form real‐time forecasts of the variables, and it is shown via a series of experiments that SIC, t‐statistic, and averaging combination approaches dominate various other combination approaches. An additional finding is that while ANN models may not MSE‐dominate simpler linear models, combinations of forecasts from these two models outperform either individual forecast, for a subset of the economic variables examined. Copyright © 2001 John Wiley & Sons, Ltd.     

Combining experts' opinions using a normal-wishart model

This paper examines how a Bayesian decision maker might update her distributions for continuous variables X_i, i=1, 2, …, upon hearing experts' forecasts expressed as quantiles. To utilize the relationship between the decision maker and experts, and to avoid problems associated with different scales and ranges of the variables, we assume that the decision maker transforms the experts' quantiles in terms of her own prior distribution for each X_i. A model using such a transformation is presented and its properties are examined.     

The usefulness of heuristic N(E)RLS algorithms for combining forecasts

There exists theoretical and empirical evidence on the efficiency and robustness of Non-negativity Restricted Least Squares combinations of forecasts. However, the computational complexity of the method hinders its widespread use in practice. We examine various optimizing and heuristic computational algorithms for estimating NRLS combination models and provide certain CPU-time reducing implementations. We empirically compare the combination weights identified by the alternative algorithms and their computational demands based on a total of more than 66,000 models estimated to combine the forecasts of 37 firm-specific accounting earnings series. The ex ante prediction accuracies of combined forecasts from the optimizing versus heuristic algorithms are compared. The effects of fit sample size, model specification, multicollinearity, correlations of forecast errors, and series and forecast variances on the relative accuracy of the optimizing versus heuristic algorithms are analysed. The results reveal that, in general, the computationally simple heuristic algorithms perform as well as the optimizing algorithms. No generalizable conclusions could be reached, however, about which algorithm should be used based on series and forecast characteristics. © 1997 John Wiley & Sons, Ltd.     

Assessing the historical accuracy of regional economic forecasts

This study addresses for the first time systematic evaluation of a widely used class of forecasts, regional economic forecasts. Ex ante regional structural equation model forecasts are analysed for 19 metropolitan areas. One- to ten-quarter-ahead forecasts are considered and the seven-year sample spans a complete business cycle. Counter to previous speculation in the literature, (1) dependency on macroeconomic forecasting model inputs does not substantially erode accuracy relative to univariate extrapolative methodologies and (2) stochastic time series models do not on average, yield more accurate regional economic predictions than structural models. Similar to findings in other studies, clear preferences among extrapolative methodologies do not emerge. Most general conclusions, however, are subject to caveats based on step-length effects and region-specific effects.     

Subjective versus objective combining of forecasts: An experiment

The paper reports results of an experiment conducted to evaluate subjective versus objective combination of forecasts. The subjects were undergraduate students at Texas A&M. The students forecasted two different types of time series. The results found show that the subjective combination of forecasts improves their accuracy as compared with individual efforts. Four ex-ante weighting methods were also used to combine the forecasts. They all improve the accuracy of the forecasts. The best results, though, were from the subjective combination of forecasts.     

A new insight into combining forecasts for elections: The role of social media

This study is devoted to gain insight into a timely, accurate, and relevant combining forecast by considering social media (Facebook), opinion polls, and prediction markets. We transformed each type of raw data into the possibility of victory as a forecasting model. Besides the four single forecasts, namely Facebook fans, Facebook “people talking about this” (PTAT) statistics, opinion polls, and prediction markets, we generated three combined forecasts by associating various combinations of the four components. Then, we examined the predictive performance of each forecast on vote shares and the elected/non‐elected outcome across the election period. Our findings, based on the evidence of Taiwan's 2018 county and city elections, showed that incorporating the Facebook PTAT statistic with polls and prediction markets generates the most powerful forecast. Moreover, we recognized the matter of the time horizons where the best proposed model has better accuracy gains in prediction—in the “late of election,” but not in “approaching election”. The patterns of the trend of accuracy across time for each forecasting model also differ from one another. We also highlighted the complementarity of various types of data in the paper because each forecast makes important contributions to forecasting elections.     

Monetary policy,composite leading economic indicators and predicting the 2001 recession

On 26 November 2001, the National Bureau of Economic Research announced that the US economy had officially entered into a recession in March 2001. This decision was a surprise and did not end all the conflicting opinions expressed by economists. This matter was finally settled in July 2002 after a revision to the 2001 real gross domestic product showed negative growth rates for its first three quarters. A series of political and economic events in the years 2000–01 have increased the amount of uncertainty in the state of the economy, which in turn has resulted in the production of less reliable economic indicators and forecasts. This paper evaluates the performance of two very reliable methodologies for predicting a downturn in the US economy using composite leading economic indicators (CLI) for the years 2000–01. It explores the impact of the monetary policy on CLI and on the overall economy and shows how the gradualness and uncertainty of this impact on the overall economy have affected the forecasts of these methodologies. It suggests that the overexposure of the CLI to the monetary policy tools and a strong, but less effective, expansionary money policy have been the major factors in deteriorating the predictions of these methodologies. To improve these forecasts, it has explored the inclusion of the CLI diffusion index as a prior in the Bayesian methodology. Copyright © 2004 John Wiley & Sons, Ltd.     

An empirical investigation of combinations of economic forecasts

This paper examines the effects of combining three econometric and three times-series forecasts of growth and inflation in the U.K. If forecasts are unbiased then a combination exploiting this fact will be more efficient than an unrestricted combination. Ex post econometric forecasts may be biased but ex ante they are unbiased. The results of the study are that a restricted linear combination of the econometric forecasts is superior to an unrestricted combination and also to the unweighted mean of the forecasts. However, it is not preferred to the best of the individual forecasts.     

Inflation forecasting for aggregates of the EU-7 and EU-14 with Bayesian VAR models

This article applies the Bayesian Vector Auto-Regressive (BVAR) model to key economic aggregates of the EU-7, consisting of the former narrow-band ERM members plus Austria, and the EU-14. This model appears to be useful as an additional forecasting tool besides structural macroeconomic models, as is shown both by absolute forecasting performance and by a comparison of ex-post BVAR forecasts with forecasts by the OECD. A comparison of the aggregate models to single-country models reveals that pooling has a strong impact on forecast errors. If forecast errors are interpreted as shocks, shocks appear to be—at least in part—asymmetric, or countries react differently to shocks. © 1998 John Wiley & Sons, Ltd.     

Sources of error in forecasts and expectations: U.K. economic models, 1984–8

This paper presents a comparative analysis of the sources of error in forecasts for the UK economy published over a recent four-year period by four independent groups. This analysis rests on the archiving at the ESRC Macroeconomic Modelling Bureau of the original forecasts together with all their accompanying assumptions and adjustments. A method of decomposing observed forecast errors so as to distinguish the contributions of forecaster and model is set out; the impact of future expectations treated in a ‘model-consistent’ or ‘rational’ manner is specifically considered. The results show that the forecaster's adjustments make a substantial contribution to forecast performance, a good part of which comes from adjustments that bring the model on track at the start of the forecast period. The published ex-ante forecasts are usually superior to pure model-based ex-post forecasts, whose performance indicates some misspecification of the underlying models.     

Volatility forecasts using stochastic volatility models with nonlinear leverage effects

The leverage effect—the correlation between an asset's return and its volatility—has played a key role in forecasting and understanding volatility and risk. While it is a long standing consensus that leverage effects exist and improve forecasts, empirical evidence puzzlingly does not show that this effect exists for many individual stocks, mischaracterizing risk, and therefore leading to poor predictive performance. We examine this puzzle, with the goal to improve density forecasts, by relaxing the assumption of linearity of the leverage effect. Nonlinear generalizations of the leverage effect are proposed within the Bayesian stochastic volatility framework in order to capture flexible leverage structures. Efficient Bayesian sequential computation is developed and implemented to estimate this effect in a practical, on-line manner. Examining 615 stocks that comprise the S&P500 and Nikkei 225, we find that our proposed nonlinear leverage effect model improves predictive performances for 89% of all stocks compared to the conventional stochastic volatility model.     

The geometric combination of Bayesian forecasting models

A nonlinear geometric combination of statistical models is proposed as an alternative approach to the usual linear combination or mixture. Contrary to the linear, the geometric model is closed under the regular exponential family of distributions, as we show. As a consequence, the distribution which results from the combination is unimodal and a single location parameter can be chosen for decision making. In the case of Student t‐distributions (of particular interest in forecasting) the geometric combination can be unimodal under a sufficient condition we have established. A comparative analysis between the geometric and linear combinations of predictive distributions from three Bayesian regression dynamic linear models, in a case of beer sales forecasting in Zimbabwe, shows the geometric model to consistently outperform its linear counterpart as well as its component models. Copyright © 2008 John Wiley & Sons, Ltd.     

BVAR as a category management tool: An illustration and comparison with alternative techniques

Category management—a relatively new function in marketing—involves large-scale, real-time forecasting of multiple data series in complex environments. In this paper, we illustrate how Bayesian Vector Auto regression (BVAR) fulfils the category manager's decision-support requirements by providing accurate forecasts of a category's state variables (prices, volumes and advertising levels), incorporating management interventions (merchandising events such as end-aisle displays), and revealing competitive dynamics through impulse response analyses. Using 124 weeks of point-of-sale scanner data comprising 31 variables for four brands, we compare the out-of-sample forecasts from BVAR to forecasts from exponential smoothing, univariate and multivariate Box-Jenkins transfer function analyses, and multivariate ARMA models. Theil U's indicate that BVAR forecasts are superior to those from alternate approaches. In large-scale forecasting applications, BVAR's ease of identification and parsimonious use of degrees of freedom are particularly valuable.     

System-based weights versus series-specific weights in the combination of forecasts

System-based combination weights for series r/step-length h incorporate relative accuracy information from other forecast step-lengths for r and from other series for step-length h. Such weights are examined utilizing the West and Fullerton (1996) data set-4275 ex ante employment forecasts from structural simultaneous equation econometric models for 19 metropolitan areas at 10 quarterly step-lengths and a parallel set of 4275 ARIMA forecasts. The system-based weights yielded combined forecasts of higher average accuracy and lower risk of large inaccuracy than seven alternative strategies: (1) averaging; (2) relative MSE weights; (3) outperformance (per cent best) weights; (4) Bates and Granger (1969) optimal weights with a convexity constraint imposed; (5) unconstrained optimal weights; (6) select a ‘best’ method (ex ante) by series and; (7) experiment in the Bischoff (1989) sense and select either method (2) or (6) based on the outcome of e experiment. Accuracy gains of the system-based combination were concentrated at step-lengths two to five. Although alternative (5) was generally outperformed, none of the six other alternatives was systematically most accurate when evaluated relative to each other. This contrasts with Bischoff's (1989) results that held promise for an empirically applicable guideline to determine whether or not to combine.     

Predictive ability and economic gains from volatility forecast combinations

The availability of numerous modeling approaches for volatility forecasting leads to model uncertainty for both researchers and practitioners. A large number of studies provide evidence in favor of combination methods for forecasting a variety of financial variables, but most of them are implemented on returns forecasting and evaluate their performance based solely on statistical evaluation criteria. In this paper, we combine various volatility forecasts based on different combination schemes and evaluate their performance in forecasting the volatility of the S&P 500 index. We use an exhaustive variety of combination methods to forecast volatility, ranging from simple techniques to time-varying techniques based on the past performance of the single models and regression techniques. We then evaluate the forecasting performance of single and combination volatility forecasts based on both statistical and economic loss functions. The empirical analysis in this paper yields an important conclusion. Although combination forecasts based on more complex methods perform better than the simple combinations and single models, there is no dominant combination technique that outperforms the rest in both statistical and economic terms.     

Backtesting Value‐at‐Risk: A Generalized Markov Test

Testing the validity of value‐at‐risk (VaR) forecasts, or backtesting, is an integral part of modern market risk management and regulation. This is often done by applying independence and coverage tests developed by Christoffersen (International Economic Review, 1998; 39(4), 841–862) to so‐called hit‐sequences derived from VaR forecasts and realized losses. However, as pointed out in the literature, these aforementioned tests suffer from low rejection frequencies, or (empirical) power when applied to hit‐sequences derived from simulations matching empirical stylized characteristics of return data. One key observation of the studies is that higher‐order dependence in the hit‐sequences may cause the observed lower power performance. We propose to generalize the backtest framework for VaR forecasts, by extending the original first‐order dependence of Christoffersen to allow for a higher‐ or kth‐order dependence. We provide closed‐form expressions for the tests as well as asymptotic theory. Not only do the generalized tests have power against kth‐order dependence by definition, but also included simulations indicate improved power performance when replicating the aforementioned studies. Further, included simulations show much improved size properties of one of the suggested tests. Copyright © 2017 John Wiley & Sons, Ltd.     

The combination of macroeconomic forecasts

The paper examines combined forecasts based on two components: forecasts produced by Chase Econometrics and those produced using the Box-Jenkins ARIMA technique. Six series of quarterly ex ante and simulated ex ante forecasts are used over 37 time periods and ten horizons. The forecasts are combined using seven different methods. The best combined forecasts, judged by average relative root-mean-square error, are superior to the Chase forecasts for three variables and inferior for two, though averaged over all six variables the Chase forecasts are slightly better. A two-step procedure produces forecasts for the last half of the sample which, on average, are slightly better than the Chase forecasts.     

Combining forecasts using optimal combination weight and generalized autoregression

In this paper, we consider a combined forecast using an optimal combination weight in a generalized autoregression framework. The generalized autoregression provides not only a combined forecast but also an optimal combination weight for combining forecasts. By simulation, we find that short‐ and medium‐horizon (as well as partly long‐horizon) forecasts from the generalized autoregression using the optimal combination weight are more efficient than those from the usual autoregression in terms of the mean‐squared forecast error. An empirical application with US gross domestic product confirms the simulation result. Copyright © 2008 John Wiley & Sons, Ltd.     

Testing for the usefulness of forecasts

Ashley (Journal of Forecasting 1983; 2 (3): 211–223) proposes a criterion (known as Ashley's index) to judge whether the external macroeconomic variables are well forecast to serve as explanatory variables in forecasting models, which is crucial for policy makers. In this article, we try to extend Ashley's work by providing three testing procedures, including a ratio‐based test, a difference‐based test, and the Bayesian approach. The Bayesian approach has the advantage of allowing the flexibility of adapting all possible information content within a decision‐making environment such as the change of variable's definition due to the evolving system of national accounts. We demonstrate the proposed methods by applying six macroeconomic forecasts in the Survey of Professional Forecasters. Researchers or practitioners can thus formally test whether the external information is helpful. Copyright © 2010 John Wiley & Sons, Ltd.