<![if !vml]>
<![endif]>Simple forecasting models
Statisticsreview and the simplest forecasting model: the sample mean (pdf)
Notes on the randomwalk model (pdf)
Mean (constant) model
Linear trend model
Random walk model
Geometric random walk model
Three types of forecasts: estimation, validation, and thefuture
When facedwith a time series that shows irregular growth, such as X2 analyzed earlier, the best strategymay not be to try to directly predict the level of the series at eachperiod (i.e., the quantity Yt). Instead, it maybe better to try to predict the change that occurs from one period tothe next (i.e., the quantity Yt - Yt-1). That is, it may be better to look at thefirst difference of the series, to see if apredictable pattern can be found there. For purposes of one-period-aheadforecasting, it is just as good to predict the next change as to predict thenext level of the series, since the predicted change can be added to thecurrent level to yield a predicted level. The simplest case of such a model isone that always predicts that the next change will be zero, as if the series isequally likely to go up or down in the next period regardless of what it hasdone in the past.
Here's apicture that illustrates a random process for which this model is appropriate:
<![if !vml]>
<![endif]>
In eachtime period, going from left to right, the value of the variable takes anindependent random step up or down, a so-called random walk. If up and down movements are equally likely at eachintersection, then every possible left-to-right path through the grid isequally likely a priori. See thislink for a nice simulation. A commonly-used analogy is that of a drunkardwho staggers randomly to the left or right as he tries to go forward: the pathhe traces will be a random walk.
<![if !vml]>
<![endif]>
Fora real-world example, consider the daily US-dollar-to-Euro exchange rate. Aplot of its entire history from January 1, 1999, to December 5, 2014 (4006observations) looks like this:
<![if !vml]>
<![endif]>
The historicalpattern looks quite interesting, with many peaks and valleys.("Chartists" often try to extrapolate such patterns by fitting localtrend lines or curves, which I do not recommend. On average, 49% of them will correctlyguess the direction in which the market will move between today and some givenfuture date.) Now, here's a plot of the daily changes (first difference):
<![if !vml]>
<![endif]>
Thevolatility (variance) has not been constant over time, but the day-to-daychanges are almost completely random, as shown by a plot of their autocorrelations:
<![if !vml]>
<![endif]>
The autocorrelation at lag k is thecorrelation between the variable and itself lagged by k periods. If the values inthe series are completely random in the sense of being statisticallyindependent, the true values of the autocorrelations are zero, and theestimated values should not be significantly different from zero. The red lineson this plot are significance bands for testing whether the autocorrelations ofthe daily changes are different from zero at the 0.05 level of significance,and overall they are not. In particular, they are completely insignificant atthe first few lags and there is no systematic pattern. (For large samples,autocorrelations are significantly different from zero at the 0.05 level iftheir magnitude exceeds plus-or-minus twodivided by the square root of the sample size. Here the sample size is4006, and 2/SQRT(4006) is approximately 0.03, as seen in the location of thered lines on the plot.)
Theforecasting model suggested by these plots is one that merely predicts no change from the one period to thenext, because past data provides no information about the direction of futuremovements:
Ŷt = Yt-1
This isthe so-called random-walk-without-driftmodel: it assumes that, at each point in time, the series merely takes arandom step away from its last recorded position, with steps whose mean valueis zero. If the mean step size is some nonzero value α, the process issaid to be a random-walk-with-drift, whose predictionequation is Ŷt = Yt-1 + α. The drunkard inthe picture above is missing one shoe, so he was probably drifting.
In generalthe steps could be be discrete or continuous random variables, and the timescale could also be discrete or continuous. Random walk patterns are commonlyseen in price histories of financial assets for which speculative marketsexist, such as stocks and currencies. This does not mean that movements inthose prices are random in the sense of being without purpose. When they go upand down, it is always for a reason! But the direction of the next move cannotbe predicted ex ante: it can only be explained ex post, because if thedirection and magnitude of the next price movement could have been predicted inadvance, then speculators would already have bid it up or down by that amount.Random walk patterns are also widely found elsewhere in nature, for example, inthe phenomenon of Brownianmotion that was first explained by Einstein. (Return to topof page.)
It isdifficult to tell whether the mean step size in a random walk is really zero,let alone estimate its precise value, merely by looking at the historical datasample. If you simulate a random walk process (for example, by building aspreadsheet model that uses the RAND() function in the formula for generatingthe step values), you will typically find that different iterations of the samemodel will yield dramatically different pictures, many of which will havesignificant-looking trends, as shown in the simulationlink mentioned above. In fact, the same model will usually yield bothupward and downward trends in repeated iterations, as well asinteresting-looking curves that seem to demand some sort of complex model. Thisis just a statistical illusion, like the so-called "hot hand inbasketball" and other examples of "streakiness" in sports. Yourbrain tries hard to find patterns, even when they are not there. See the Hot Hand in Sports web site for moreon this.
Inapplications, it is best to draw on other sources of information and ontheoretical considerations in deciding whether to include a drift term in themodel, and if so, how to estimate its value. In the case of exchange rates,there is no reason to assume a long-term trend in one direction or the other,at least, not a trend that would stand out against the noise. The mean dailychange is 0.000012 for this sample of exchange rate data, and the standarderror of the mean is 0.00012, so the sample mean is different from zero by only1/10th of a standard error, which is not significant by any measure.Again, though, the mean value of the steps in a finite sample of random-walkdata generally does not provide a good estimate of the current rate of drift,if any.
Overall,then, it appears that a random-walk-without-drift model is appropriate for thistime series. If the model is fitted to the entire history of the daily data,going back to 1999, the forecasts and 50% confidence limits produced by themodel look like this:
<![if !vml]>
<![endif]>
(Thischart was produced by Statgraphics. 50% rather than 95% limits are shown merelyto make them fit better in the picture. There is nothing special about 95%anyway, apart from convention.) Here is a close-up view of the actual datapoints and forecasts at the very end of the series:
<![if !vml]>
<![endif]>
The keyproperties of the model that are illustrated by this graph are the following:
<![if !supportLists]>a. <![endif]>The one-step-ahead forecasts within the samplefollow exactly the same path as the data, except that they lag behind by one period. (You must lookcarefully to see this: at first glance it may appear that the model fits thedata perfectly, but in fact it is making errors in every period, and thoseerrors are independent random variables.)
<![if !supportLists]>b. <![endif]>The long-term forecasts outside the samplefollow a horizontal straight line anchored on the last observed value,because no upward or downward drift or any other systematic time pattern isassumed. (If non-zero drift was assumed, this line might slope upward or downward.)
<![if !supportLists]>c. <![endif]>The confidence bands for long-term forecasts growwider in a fashion that looks like a sidewaysparabola, for reasons explained below. (Return to top ofpage.)
In therandom-walk-without-drift model, the standard error of the 1-step ahead forecastis the root-mean-squared-value of the period-to-period changes in the datasample, i.e., it is the square root of the average of squared values of thefirst difference of the series. For a random-walk-with-drift, the forecast standard error is the sample standard deviation of the period-to-period changes. (Thedifference between the RMS value and the standard deviation of the changes isusually negligible unless the volatility is very small in comparison to thedrift.)
The errorthat the model makes in a k-step-aheadforecast is the sum of k independently and identically distributed randomvariables, because the model continues to make the same prediction while thevariable takes k random steps. Because the variance of a sum of independentrandom variables is the sum of the variances, it follows that the variance ofthe k-step-ahead forecast error is larger than that of the one-period-aheadforecast by a factor of k. And because the standard deviation of the forecasterror is the square root of its variance, it follows that the standard error of a k-step-ahead forecastis larger than that of the 1-step-ahead forecast by a factor ofsquare-root-of-k. This is the so-called "square root of time"rule for the errors of random walk forecasts, and it explains thesideways-parabola shape of the confidence bands for long-term forecasts: that'sthe shape of the graph of Y=SQRT(X).
For thisvery large data sample, the root-mean-squared value and the sample standarddeviation of the daily changes are both equal to 0.00778 to 3 significantdigits, so the standard error of a k-step ahead forecast error is0.00778*SQRT(k), and confidence limits are calculated from it in the usual way.A 95% interval is (approximately) the point forecast plus-or-minus 2 standarderrors, and a 50% confidence interval is the point forecast plus-or-minustwo-thirds of a standard error.
In thecase of the exchange rate data, it is not really appropriate to use the entiresample to estimate the standard deviation of the daily changes, because itclearly has not been constant over time. A shorter data history could be usedto address this problem, and other kinds of information such as prices offoreign-exchange options could also be considered.
The randomwalk model may look trivial if you have never seen it before: what could bemore simple-minded than always predicting that tomorrow will be the same astoday? This does not even require any knowledge of statistics! For that reasonit is sometimes called the "naive model." It is not at all trivial,however. The square-root-of-time pattern in its confidence bands for long-termforecasts is of profound importance in finance (it is the basis of the theoryof options pricing), and the random walk model often provides a good benchmarkagainst which to judge the performance of more complicated models.
The randomwalk model can also be viewed as an important special case of an ARIMA model ("autoregressiveintegrated moving average"). Specifically, it is an"ARIMA(0,1,0)" model. More general ARIMA models are capable ofdealing with more interesting time patterns that involve correlated steps, suchas mean reversion, oscillation, time-varying means, and seasonality. Thesetopics are discussed in detail in the ARIMA pages of these notes.
For a much morecomplete discussion of the random walk model, illustrated by a shorter sampleof the exchange rate data, see the "Notes on the randomwalk model" handout.
