Forecasting the ECB’s main reﬁnancing rate. A ﬁeld experiment. Michael Berlemann ifo Institute for Economic Research, Branch Dresden Dresden University of Technology CESifo March 31, 2006 Abstract. When no appropriate markets to derive expectations on economic variables are available often surveying methods are employed. However, surveys suﬀer from the problem how to aggregate individual expectations in a reasonable manner. We show at the example of a market on the ECB’s main reﬁnancing rate that experimental markets aggregate information quite eﬃciently and thus are a useful tool in this respect. Keywords: expectations, forecasting, ﬁeld experiments, experimental stock markets, main reﬁnancing rate, ECB JEL code: E58 2 Forecasting the ECB’s main reﬁnancing rate 1. Introduction Basically, two basic approaches of macroeconomic forecasting can be distinguished. On the one hand econometric forecasting techniques can be applied to macroeconomic data in order to construct forecasts. On the other hand the expectations approach of fore- casting can be employed by measuring market participants’ expectations on the event or variable to be forecasted. Whenever these expectations are formed rationally they should provide unbiased and eﬃcient forecasts. The basic problem of the expectations approach to forecasting (or simply of measuring expectations) is how to uncover market participants’ expectations. Direct methods of measuring expectations typically rely on some sort of survey in which certain subsamples of the population are asked to reveal their personal expec- tations (see Croushore  or Thomas ). However, then an appropriate way of aggregating the individual responses has to be found. In most cases an arbitrary procedure such as averaging is used for this purpose which is often inappropriate. Besides solving the aggregation problem the surveyor has to overcome the well documented (see e.g. Aaker and Day ) sampling-, non-response- and motivation-problems of respondents. The indirect approach of measuring expectations is to derive expectations from mar- ket participants’ behavior on real world (ﬁnancial) markets. This approach bases on the idea that markets are the most eﬃcient means of aggregating private information (see Lioui and Poncet  and Smith ). Various experimental studies (Smith [1962,1964,1965], Miller, Plott and Smith , Williams [1979,1980], Smith and Williams ) found that initially dispersed information is quite eﬃciently disseminated via prices in market settings. Thus, market prices solve the information aggregation problem quite eﬃciently and can therefore be used to to uncover the underlying expectations of market participants. However, often there are no appropriate markets available which can be used to extract forecasts. And even if these markets exist, the underlying expec- Forecasting the ECB’s main reﬁnancing rate 3 tations can often be extracted from the market data only when using highly simplifying assumptions.1 As an alternative to real-world markets one might use experimental markets to aggre- gate initially dispersed private information on events to be forecasted. The experimental approach to forecasting developed in the late 1980s when political stock markets were employed to generate market data under somewhat controlled circumstances in ﬁeld experiments (see Forsythe et al. ). While these markets were initially intended to allow to study individual market behavior they delivered election-outcome forecasts as by-products. Since these forecasts have proven to be highly accurate the method has been used extensively to forecast election results in the aftermath (see e.g. Berg et al.  or Berlemann and Schmidt ). However, experimental markets can also be employed to generate forecasts of economic variables. As an alternative to surveys, experts could also be asked to take part in appropriately designed experimental markets. These markets aggregate the experts’ expectations in a non-arbitrary way thereby allowing to derive useful forecasts. As an example we report results from an experimental forecasting market on the ECB’s main reﬁnancing rate in this paper. In section 2 we describe the conducted market. Section 3 explains how forecasts can be extracted from the market and presents some empirical results. In section 4 we study in how far the forecasts derived from the market are eﬃcient and thus solve the information aggregation problem in a satisfactory manner. Section 5 concludes. 2. Market description The experimental market we report on was intended to forecast the main reﬁnancing rate of the European Central Bank (ECB) on 15th January 2003. The market opened up on 1 For example, inﬂation expectations can be derived from the prices of CPI futures (see Lovell and Vogel  or Lioui and Poncet ). Since CPI futures markets did not develop in most countries, several authors (see e.g. Fama  or Mishkin ) tried to gauge inﬂation expectations from the term structure of interest rates. While this type of data is typically available, generating inﬂation forecasts from it is far from being easy. Often a number of simplifying assumptions, e.g. on the behavior of the real interest rate over time, have to be made. 4 Forecasting the ECB’s main reﬁnancing rate 13th October 2002 and closed on 14th January 2003. The market was fully computerized. In order to be allowed to take part in the market, participants had to register in a market software via Internet. The market participants were recruited from a university course in macroeconomics. Altogether 51 students took part in the market. After registrating participants got trader-IDs and passwords to login the market. Trader accounts for the participants were created and 100 virtual Euro were transferred to the accounts. These virtual funds could be used by the participants to organize market transactions. The three participants with the highest returns on investment were ﬁnally rewarded by the market organizer with 75, 50 and 25 Euro.2 The market made use of a winner-takes-all design. In the market a set of binary lock-in options were traded. The underlying of these options was the main reﬁnancing rate of the ECB on 15th January 2003. The traded options had a ﬁxed, predetermined payoﬀ of 1 virtual Euro if the underlying was inside the strike range at expiration.3 Table I. Traded contracts in ECB market. Contract number Contract name Interval middle/border Pays oﬀ 1 virtual Euro, if 1 r(2.25−) 2.250 r ≤ 2.25 2 r(2.50) 2.500 r = 2.50 3 r(2.75) 2.750 r = 2.75 4 r(3.00) 3.000 r = 3.00 5 r(3.25) 3.250 r = 3.25 6 r(3.50) 3.500 r = 3.50 7 r(3.75) 3.750 r = 3.75 8 r(4.00) 4.000 r = 4.00 9 r(4.25+) 4.250 r ≥ 4.25 The set of binary lock-in options traded in the market is shown in table I.4 The contracts were designed symmetrically around the main reﬁnancing rate prevailing when 2 Since the market was conducted with students at a university, we decided not to use a real-money setting. As a consequence we can not rule out that at least some players engaged in more risky strategies than he or she would have done under real-money trading. 3 Thus, the type of lock-in options traded in the market were formally identical to what is called pure, Arrow- or Arrow-Debreu securities in ﬁnancial markets literature. See e.g. Copeland and Weston . 4 To understand the design of the contracts it should be noted that the ECB’s main reﬁnancing rate is varied only in multiples of 0.25 percent. During the market period the main reﬁnancing rate was changed once. On 6th December 2002 the rate was lowered by 0.5 percentage points from 3.25 to 2.75 percent. Forecasting the ECB’s main reﬁnancing rate 5 the market opened up (3.25 percent).5 The strike ranges of these options did not overlap and covered the whole range of possible outcomes of the underlying. Since the number of unique linearly independent securities was equal to the total number of alternative states of nature we dealt with a complete market (Copeland and Weston , p. 112). Regardless of the initial distribution of securities it was thus possible to reduce the uncertainty about the value of future wealth to zero by holding unit portfolios consisting of one of each option type. Upon entering the market and any time thereafter participants could buy unit port- folios from the market organizer for the price of 1 virtual Euro until the market closed. Complete unit portfolios could be sold back to the market organizer during the market period for the same price. Market participants could also buy or sell contracts from or to other market participants. This “secondary market” was organized as a double auction market. Market participants could issue oﬀers to buy (bids) or oﬀers to sell (asks) contracts. When using a ﬁrst type of transactions, so-called “limit orders”, traders had to choose the order type (bid or ask), the contract type, the number of contracts he or she wanted to trade, the desired transaction price and ﬁnally the order’s expiration date. Limit orders were maintained in separate bid and ask queues ordered ﬁrst by oﬀer price and then by the time of issuance. Whenever an oﬀer entered one of the queues it remained there until the oﬀer turned out to be unfeasible (e.g. because of a lack of liquidity to realize a buying transaction), was withdrawn by the trader, reached its expiration date or was carried out. Orders were carried out whenever bid- and ask-prices overlapped. The second type of transactions available, the so-called “market orders”, were orders to buy or sell at current market prices which were carried out immediately. Diﬀerent from real world stock markets, short sales and purchases on margin were disallowed. No transaction costs were levied by the market organizer. 5 In order to get a forecast as precise as possible the set of traded options can be increased during the market period. This can easily be done by splitting options into two (or more) contracts with smaller strike ranges (given that the options do not overlap and all possible outcomes of the underlying are still covered). In order not to inﬂuence the values of the participants’ portfolios by contract splits, each holder of a split contract is supplied with one of each newly issued contract in exchange. 6 Forecasting the ECB’s main reﬁnancing rate All primary and secondary market transactions were organized via a market software. Besides serving as a market platform the software provided several facilities for the traders to obtain information on the market. A trader could access personal information on his market account, current portfolio or already submitted orders. The software also delivered information about the highest bids to buy and lowest asks to sell for each contract type or the last prices for which a certain share was traded. The ECB market got liquidated as soon as the realization of the underlying was known. The individual payoﬀ of each participant in virtual Euro consisted of (i) the virtual money the trader held on his market account when the market closed and (ii) the liquidation value of the portfolio of contracts the trader held at the end of the market. 3. Density forecast, mean forecast and forecast uncertainty Using Arbitrage Pricing Theory it can be easily shown that the prices of the pure secu- rities traded in the market are equal to the probabilities, market participants attach to the diﬀerent states of nature.6 Thus, the described experimental stock market directly generates a density forecast for the ECB’s main reﬁnancing rate. However, it does not f directly deliver a mean forecast. The mean market forecast rt at time t can be constructed by multiplying the normalized7 last observed market prices8 with the the middles of the 6 According to Arbitrage Pricing Theory the equilibrium price of a pure security principally depends on three factors: the risk-free rate of return, individuals’ attitudes towards risk and expectations as to the probability that a particular state will occur (Copeland and Weston , p. 116). Since the risk-free rate of return is zero in the described market and there is no non-diversiﬁable risk, the prices of the traded contracts solely depend on market participants’ expectations on the probabilities with which the various states of nature occur. However, this is true only if the market is in equilibrium. It is then when all available information is reﬂected in the current market prices. 7 Experience from previous experimental markets shows that last traded prices do not always add up to one. A possible reason for this observation is that traders in most markets are cash-restrained. Thus, even if some traders would have perfect information on the fair prices of all contracts they will typically not have enough funds to ﬁx the prices to their fair values. While this problem should diminish in markets with a high number of traders it can hardly be ignored in smaller markets. To deal with this problem most market organizers normalize the sum of last traded prices to unity before calculating the mean market forecast. 8 Besides last observed prices (last traded prices (LTP)) market organizers often report daily volume- weighted prices (average traded prices (ATP)). While from a theoretical point of view last observed prices should be superior since they belong to the most actual, marginal transactions, we nevertheless report both sorts of forecasts in the following. Forecasting the ECB’s main reﬁnancing rate 7 strike ranges of the referring options and summing up for all traded contracts,9 i.e. f pr(2.25−),t pr(2.50),t rt = · 2.25 + · 2.50 + . . . Pt Pt pr(4.00),t pr(4.25+),t + · 4.00 + · 4.25. Pt Pt with Pt := pr(2.25−),t + . . . + pr(4.25+),t . Mean forecasts do not provide any information on the uncertainty underlying the fore- cast. Since one and the same mean forecast can principally result from many diﬀerent probability distributions information about the uncertainty surrounding a mean inﬂation forecast is important in addition to the forecast itself. A measure of forecast uncertainty helps to qualify a forecast and gives a richer picture of the expected range of likely outcomes (Ericsson , pp. 88-89). The described experimental market setting allows to assess the mean forecast’s uncertainty directly. Since the normalized market prices can be interpreted as the market’s aggregated evaluation of the probabilities of diﬀerent scenarios, these probabilities can be used to calculate the empirical variance of the daily mean forecast as 2 pr(2.25−),t f pr(2.50),t f 2 σrf ,t = · (2.25 − rt )2 + · 2.50 − rt + . . . Pt Pt pr(4.00),t f 2 pr(4.25+),t f + · 4.00 − rt + · (4.25 − rt )2 . Pt Pt The market allowed to extract forecasts and to assess their empirical variances at any point in time during the market period. The market thus delivered a time-series of ﬁxed- event forecasts. The mean forecasts of the ECB’s main reﬁnancing rate on 15th January 2003 are shown in ﬁgure 1. It is easy to see that the market performed quite well. From the beginning of the market period on the market predicted a decreasing reﬁnancing rate. 9 Most of the strike ranges in the ECB market consisted of only one outcome of the main reﬁnancing rate. However, there are two options in the market which have inﬁnitely large strike ranges (r(2.25−), r(4.25+)). To deal with this problem we use the (lower respectively the upper) bounds of these inﬁnitely large intervals instead of the class middles. Doing so is unproblematic as long as the prices for these contracts are comparatively low. Whenever the market prices for these intervals are high, thus indicating that market participants attach a signiﬁcant probability to the case that the underlying will fall into the contracts’ strike range, the market organizer can easily solve this problem by the earlier described splitting procedure. 8 Forecasting the ECB’s main reﬁnancing rate Initially, the market predicted a decrease by some 0.25 percent. However, well before the reﬁnancing rate was in fact lowered the market forecast further decreased to around 2.90 percent indicating that ECB might take two steps down on the ladder. After the factual change of the reﬁnancing rate on 6th December the market forecast converged quickly to 2.75 percent and remained there until the market closed. 3.300 3.200 3.100 Main refinancing rate in percent 3.000 2.900 2.800 2.700 2.600 2.500 2.400 02 02 02 02 02 02 02 02 02 02 02 02 03 03 20 20 20 20 20 20 20 20 20 20 20 20 20 20 0. 0. 0. 1. 1. 1. 1. 2. 2. 2. 2. 2. 1. 1. .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .0 .0 15 22 29 05 12 19 26 03 10 17 24 31 07 14 Last Traded Prices Normalized Forecast Average Traded Prices Normalized Forecast Actual interest rate Last observed interest rate Figure 1. Mean forecast of ECB’s main reﬁnancing rate on 15th January 2003. Whenever the ECB’s main reﬁnancing rate has some predictable component we should expect empirical variances of the forecasts to decrease in the course of time since the market participants are getting more information over the market period. In ﬁgure 2 we show the variances of the mean forecasts during the market period. In fact, the uncertainty surrounding the forecast showed a decreasing tendency throughout the market period (see ﬁgure 2). 4. Forecast eﬃciency While it is hardly possible to judge the absolute accuracy of the derived forecasts, we can study in how far the used methodology is eﬃcient in aggregating individual informa- tion. As outlined earlier, the aggregation problem of individual expectations, based on Forecasting the ECB’s main reﬁnancing rate 9 0.14 0.12 0.10 Forecast variance 0.08 0.06 0.04 0.02 0.00 02 02 02 02 02 02 02 02 02 02 02 02 03 03 20 20 20 20 20 20 20 20 20 20 20 20 20 20 0. 0. 0. 1. 1. 1. 1. 2. 2. 2. 2. 2. 1. 1. .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .1 .0 .0 15 22 29 05 12 19 26 03 10 17 24 31 07 14 ATP normalized forecast LTP normalized forecast Figure 2. Variance of ECB market forecasts. dispersed information, is one of the major problems within the expectations approach of forecasting. A concept for studying the eﬃciency of ﬁxed-event forecasts has been developed by Nordhaus . Eﬃciency of ﬁxed-event forecasts requires that forecast errors should be uncorrelated with past forecast revisions since such a correlation could be used to decrease forecast errors. Moreover eﬃcient time series of ﬁxed-event forecasts should not show any autocorrelation in forecast revisions. Such an autocorrelation could indicate that newly arriving information is not incorporated into the market prices immediately. However, up to now the concept has rarely been applied since most time series of forecasts are rolling-event forecasts. Nordhaus  himself studied the eﬃciency 4 time series (oil prices, nuclear capacity, energy and real GNP) of ﬁxed-event forecasts. He found signiﬁcant autocorrelation of forecast revisions and thus empirical evidence in favor of the hypothesis of social psychologists that people tend to hold on to prior views too long (see Tversky and Kahneman ). More recently, Bakshi, Kapetanios and Yates  studied the eﬃciency of 7 time series of ﬁxed event inﬂation forecasts of surveys of fund managers conducted by Meryll Lynch. The authors had to reject the hypothesis that the 10 Forecasting the ECB’s main reﬁnancing rate forecast errors are uncorrelated with past revisions for 5 out of 7 time series of ﬁxed event forecasts. Similarly, 2 of the time series exhibited autocorrelation of forecast revisions. In order to study the eﬃciency of the ECB main reﬁnancing rate forecasts derived from the market we ﬁrst analyze in how far the forecast errors are correlated with past forecast revisions. We therefore run the OLS-regression f f f f f rτ − rt = α0 + α1 · (rt−1 − rt−2 ) + α1 · (rt−2 − rt−3 ) + t . (1) The results of regression 1 are shown in table II. We ﬁnd positive constants for both forecasts. However, there is no signiﬁcant correlation between past forecast revisions and forecast errors. Table II. Correlation between forecast errors and past forecast revisions (OLS). f f f f Market Forecast Constant rt−2 − rt−1 rt−3 − rt−2 type (t-value) (t-value) (t-value) ECB LTP 0.19 -1.22 -1.54 (6.83)∗∗∗ (-1.01) (-1.27) ECB ATP 0.17 -1.18 -1.53 (6.11)∗∗∗ (-0.85) (-1.10) Signiﬁcance levels are reported as follows: * for a 90%- signiﬁcance-level, ** for 95% and *** for more than 99%. In a second step we study in how far the forecast revisions are autocorrelated. We therefore run the OLS regression f f f f rt − rt−1 = β · (rt−1 − rt−2 ) + t . (2) The estimation results are shown in table III. While the LTP forecast revisions show no serial correlation the hypothesis of autocorrelated forecast revisions can not be rejected on a 90-percent conﬁdence level for the ATP forecasts. However, past ATP forecast revisions only explain a minimal part of the observed forecast revisions. Nevertheless it seems that forecasts based on last observed prices generate slightly more eﬃcient forecasts than those based on average ﬁgures. Forecasting the ECB’s main reﬁnancing rate 11 Table III. First-order autocorrelation of ECB main reﬁnancing rate forecast revisions. Market Forecast type Coeﬃcient T-statistic p r2 ECB market LTP 0.12 1.23 0.22 0.00 ECB market ATP 0.17 1.67 0.09 0.01 Altogether, we can conclude that the ECB market performed quite well in eﬃciently aggregating individual information. Thus, the experimental markets methodology delivers a highly useful and eﬃcient solution to the information aggregation problem. 5. Conclusions Initially, political stock markets were used to generate market data in ﬁeld experiments in order to study how individual market participants behave in market settings. The high accuracy of the forecasts of election results derived from these markets indicate that the experimental markets methodology also provides a useful tool for forecasting. The results presented in this paper indicate that it might be useful to employ this methodology for example to extract market expectations on the future use of monetary policy instruments by the central bank. References Aaker, D. A. and G. S. Day (1990): Marketing Research; 4th Edition, New York. Bakshi, H., G. Kapetanios and A. Yates (2003): Rational Expectations and Fixed-event Forecasts: An Application to UK Inﬂation; Bank of England Working Paper, No. 176, Bank of England, London. Berlemann, M. and C. Schmidt (2001): Predictive Accuracy of Political Stock Markets - Empirical Evidence from a European Perspective; Dresden Discussion Paper Series in Economics, Dresden 5/01, University of Technology. Berg, J., R. Forsythe, F. Nelson and T. Rietz (2000): Results from a Decade of Election Futures Markets Research; Working Paper, College of Business Administration, University of Iowa. 12 Forecasting the ECB’s main reﬁnancing rate Copeland, T. E. and J. F. Weston (1992): Financial Theory and Corporate Policy; 3rd Edition, Reading/Mass. Croushore, D. (1996): Inﬂation Forecasts: How Good Are They?; Federal Reserve Bank of Philadelphia Business Review, May/June, pp. 1-11. Ericsson, N. R. (2001): Forecast Uncertainty in Economic Modeling; in: D. H. Hendry and N. R. Ericsson (Eds.), Understand- ing Economic Forecasts, Cambridge/Mass, pp. 68-92. Fama, E. F. (1975): Short-term Interest Rates as Predictors of Inﬂation; American Economic Review, Vol. 65 (3), pp. 269-282. Forsythe, R., F. Nelson, G. R. Neumann and J. Wright (1992): Anatomy of an Experimental Political Stock Market; American Economic Review, Vol. 82 (3), pp. 1142-1162. Lioui, A. and P. Poncet (2002): Revealing Inﬂation Expectations : Let the Market Do It; Working Paper, Bar Ilan University, Ramat Gan. Lovell, M. C. and R. C. Vogel (1973): A CPI-Futures Market; Journal of Political Economy, Vol. 81 (2), pp. 1009-1012. Miller, R. M., Ch. R. Plott and V. Smith (1977): Intertemporal Competitive Equilibrium: An Experimental Study of Speculation; Quarterly Journal of Economics, Vol. 91, pp. 599-624. Mishkin, F. S. (1990): What Does the Term Structure Tell us About Future Inﬂation?; Journal of Monetary Economics, Vol. 25, pp. 77-95. Nordhaus, W. D. (1987): Forecasting Eﬃciency: Concepts and Applications; Review of Economics and Statistics, Vol. 69, pp. 667-674. Smith, V. (1962): An Experimental Study of Competitive Market Behavior; Journal of Political Economy, Vol. 70, pp. 111-137. Smith, V. (1964): Eﬀect of Market Organization on Competitive Equilibrium; Quarterly Journal of Economics, Vol. 78, pp. 181-201. Smith, V. (1965): Experimental Auction Markets and the Walrasian Hypothesis; Journal of Political Economy, Vol. 73, pp. 387-393. Smith, V. (1976): Experimental Economics: Induced Value Theory; American Economic Review, Vol. 66, pp. 274-279. Smith, V. (1982): Markets as Economizers of Information: Experimental Evidence on the ”Hayek Hypothesis”; Economic Inquiry, Vol. 20, pp. 165-179. Smith, V. and A. W. Williams (1983): An Experimental Comparison of Alternative Rules for Competitive Market Exchange; reprinted in V. Smith (Ed.), Papers in Experimental Economics, Cambridge (1991). Thomas, L. B. (1999): Survey Measures of Expected U.S. Inﬂation; Journal of Economic Perspectives, Vol. 13 (4), pp. 125- 144. Forecasting the ECB’s main reﬁnancing rate 13 Tversky, A. and D. Kahneman (1981): The framing of decisions and psychology of choice; Science, Vol. 211, pp. 453-458. Williams, A. W. (1979): Intertemporal Competetive Equilibrium: On further Experimental results, Research in Experimental Economics, Vol. 1, pp. 255-278. Williams, A. W. (1980): Computerized Double Auction Markets: Some Inital Experimental Results, Journal of Buisness, Vol. 53, pp. 235-258.