John Kay on economists' forecasts

policy

I love John Kay's writing (which I accessed direct from his site, http://www.johnkay.com). This one is especially interesting - as somebody working on an econometric tool for 'forecasting', his observation rings so true.

This one is from http://www.johnkay.com/in_action/340

23 June 2004 (Financial Times). If you have one economic forecaster's opinion, you have them all - and most often, it's wide off the actual outcome. As the number of commercial soothsayers grows, it is worth knowing that the only valuable economic forecast comes from those who say "on the one hand ... and on the other."

President Harry Truman reportedly sought a one-armed economist, one who would be unable to say "on the one hand ... and on the other".The joke is that there are more opinions than there are economists. But this is not true of economic forecasters. They all agree with each other. They all make the same mistakes.

In an article almost 10 years ago, I observed that between 1991 and 1994 Britain's growth rate had never once fallen within the range of forecasts made for it - everyone had overestimated growth in 1991 and 1992, and underestimated it in 1993 and 1994. Recently I decided to see whether matters have improved since then. The answers are not encouraging.

The UK Treasury publishes a compendium of economic forecasts. I looked at the 43 forecasts of growth in 1999 that it listed in December 1998. The average forecast was 0.8 per cent: 33 estimates were in the range 0.3 per cent to 1.3 per cent; and the complete range of forecasts extended from minus 0.5 per cent to 2.1 per cent. The actual growth rate in 1999 was 2.8 per cent.

For 2000, every single forecasting group again underestimated the actual growth rate. But the problem is not simply pessimism. Only two estimates for 2001 were too low.

The economic environment since then has been more stable but the degree of clustering has become still more extreme. In December 2003, for example, all but three forecasts of growth for 2004 were between 2 per cent and 3 per cent, and more than half were in the range 2.5 per cent to 2.9 per cent. This nearly universal agreement does not reflect common knowledge of what is actually going to happen. Six months later, there is similar unanimity that growth in 2004 will be above 3 per cent.

So, if you have one economic forecaster's opinion, you have them all. The basis of almost all forecasting is that the economy will revert from where it is now to its trend level. Whether people reach this conclusion by econometric modelling or putting a finger in the air, the answers they reach are similar. Forecasting accuracy has recently improved because the British economy has grown in line with that trend. In fact, if you had simply assumed since 1998 that the British economy would always grow at its long-term average of 2.5 per cent, you would have tracked the outcome more closely than if you had followed the consensus forecast.

But there is always a consensus forecast because the City pundits who form it watch each other and talk to each other. Amid the politics of large financial institutions, it is much safer to be wrong with a crowd than to be isolated and right. The largest bonuses go to the most articulate cheerleaders for the conventional wisdom.

We understand that we can have knowledge of the kind "Manchester United will probably win" but not knowledge of the kind "the score will be 3-1". This is intrinsic to the nature of a complex system whose outcome depends on many individual uncertain events - whether that system is a sporting contest or the progress of thousands of businesses and millions of individuals. The Treasury and the Bank of England now acknowledge this limitation by indicating a range of projections rather than predicting a number. Perhaps they will soon describe that range as a planning assumption, which is its true status, rather than a forecast.

A decade ago, I observed that academic forecasters, less influenced by the herd, were more extreme - though not necessarily more right - than commercial ones. Today, there are almost no academic forecasters left: the activity no longer commands prestige or attracts research funding.

Yet the number of commercial soothsayers continues to grow. There is supply because there is demand: investors and businesses that continue to believe, despite the evidence, that these people really see the future in their crystal balls. Hope continues to triumph over experience, just as it does among the customers of medical quacks and racing tipsters.

The voices of people who admit they do not know the answer are usually drowned by those who claim, however falsely, that they do. But the future is inherently uncertain, and that is why the only economists worth listening to are those who often say "on the one hand ... and on the other"