Expert predictions often off the mark

Only partly, according to a growing pile of research that says economists labor under the same biases and mental shortcuts as the rest of us. For those and other reasons, economists’ forecasts are often off the mark, according to several studies published over the last two decades.

“Both an enormous amount of evidence and anecdotal experience suggests that people are very bad at forecasting. This is often because we all tend to be massively overconfident,” James Montier, with Boston securities firm GMO LLC, said in a 2005 article. One of London’s top analysts, Montier cites findings from behavioral economists to argue that most economists, stock pickers and other people who make forecasts have similar mental blind spots that cause them to make similar mistakes.

For instance, decades of data from the Federal Reserve Bank of Philadelphia show that most economists’ forecasts tend to mimic the recent past.

“Economists are really good at telling you what just happened,” he said. “They constantly seem to lag reality.”

‘Shocks’ shake things up

While Montier is something of a heretic in the financial world, others have raised similar questions. University of Michigan economist Edward Gramlich, who went on to serve on the Federal Reserve’s board of governors, noted in the early 1980s that UM’s national survey of consumers seemed to be a more accurate inflation predictor than the Fed’s long-running survey of economists’ forecasts. Even the Federal Reserve’s own economists have sounded a cautionary note about relying too much on colleagues’ forecasts.

In a 2007 research note, Federal Reserve economists Michael F. Bryan and Linsey Molloy found that forecasters since 1983 typically missed the mark by more than a percentage point when predicting economic growth and the inflation rate for the following year.

That may not sound like much of an error, but it can make many forecasts almost meaningless.

For instance, in its most recent survey of economists, the Philadelphia Fed said economic growth is expected to be about 2.4 percent this year (versus NABE’s 3.2 percent forecast).

But in a related chart, the Philadelphia Fed indicated that if you wanted to be 80 percent sure about the forecast, you would need to be prepared for the economy to grow this year anywhere from a robust 4 percent to a rather anemic 1 percent. “The reason those [uncertainty] intervals are so wide is because the economy is subject to huge shocks” like the financial collapse and housing market crash in 2007, said Tom Stark, assistant director of the Philadelphia Fed’s Real-Time Data Research Center. “Those things are largely unpredictable.”

It also takes time for economists to figure out how such big shocks will play out in the economy, which is continuously evolving, Stark said. In the meantime, they’re also learning how to adjust their forecasts.

“It’s easy to be harsh that people are making persistent errors, but keep in mind that economics is a behavioral science in itself,” Stark said. “It takes time for people to learn what’s going on.”

Not just what but why

He said surveys of dozens of economists usually produce better forecasts than computer models or single economists’ predictions. Surveys also usually do better on forecasts of “choppy” indicators such as changes in economic growth, he said. In the case of slower-moving indicators such as inflation, the previous quarter’s number is usually a better predictor than economists’ forecasts, he said.

Even though they often miss their targets, Stark said it’s evident that economists must provide a valuable service or corporations and universities wouldn’t employ thousands of them.

The NABE, for instance, has about 2,300 members, including top economists at Ford Motor Co., Fidelity Investments, Fannie Mae and other major corporations.

“They don’t just need a raw forecast, but some explanation of why the forecast is what it is,” Stark said. “Nobody gets paid for producing a widget that has little value.”

His advice to people trying to make business decisions this year: Use the forecasts as a reality check on your own guesstimates, but be aware of “huge uncertainty.”

The studies

Decades of academic studies have raised questions about the accuracy of economic forecasts. They also suggest ways to view and use the information in such surveys and forecasts.

Boiled down, the academic studies seem to say:

● The average economist is a more accurate forecaster than the Average Joe.

● Last year’s inflation or GDP numbers are often as good a forecast as any — at least in typical years.

● The median forecast of a group of economists is usually better than one economist’s predictions.

● A survey of a bunch of average people (inflation expectations from a national consumer survey, say) is usually more accurate than surveys of a few dozen economists, mainly because of the difference in sample size.

● Big consumer surveys have problems, too. How questions are worded changes the results.

● No forecasts are accurate when there’s a big shock to the economy, such as the oil embargo in the 1970s or the stock and real estate market crash in 2007.

● Bottom line: Use forecasts with caution, and be ready for surprises.