While short-term forecasts are near impossible, and medium-term is hard and requires a lot of luck, long-term forecasting is easier because market values generally head towards "fair" and mean reversion operates at a lot of levels. The best long-term forecaster that I'm aware of is Jeremy Grantham, partner at GMO. Grantham produces 7 and 10-year forecasts based on fundamental relationships and the assumption that over a long time frame, asset classes will revert to intrinsic value. His current forecast suggests that US equity real returns over the next 7-years will be negative; he thinks US equities are currently 75% overvalued.
He updates the forecasts quarterly to account for changes in things like GDP and investment levels, but most importantly, to account for changes in asset prices. In 1999 he predicted that equity returns over the next 10 years would be zero, a staggering contrarian prediction at the time. After the tech bubble burst, he became moderately bullish in 2002 before again predicting negative long-term equity returns in 2007. He was then bullish again in 2009 before becoming bearish in 2012. He also correctly forecast the outperformance of commodities and emerging market equity in the 2000s. This is not to suggest that his timing is perfect; he became skeptical of equities in 1997, 3 years too early. He seems to have little skill in picking the turning point, but is excellent in identifying when we're far above or below the "trend line" to which we will eventually revert.
The bulk of his work is to simply ignore the narratives of the pundits, and focus on some very basic metrics that ultimately drive returns. For equities, long-term returns come down to revenue and profit margins (which determine net profits), leverage (which converts profits to return to equity), and the earnings multiple (which converts earnings per share to a stock price.) Revenue generally trends upwards with the economy, and the rest is strongly mean reverting. While we can't know what the mean reverting level is exactly, we can be sure that, for example, profit margins won't climb and climb until breaking 100%, and the P/E ratio must ultimately relate to real interest rates.
This simple analysis is just the starting point however, because things do fundamentally change over time. For example, the top quartile of venture capital funds earned annualized returns of over 40% from 1990 to 2000. That's likely a decade that will never be repeated for venture capital, because it was a new, underpopulated asset class investing in a new, underinvested industry. Today venture capital is crowded with ten times the capital chasing a similar number of opportunities.
Long-term secular environments are also critical. We're now 5 years into a period of financial repression with artificially low real interest rates. The further the government pushes real interest rates below equilibrium, the higher asset prices may get above fair value.
Grantham thinks the biggest cause of excess market volatility and mispricing is faulty extrapolation. Investors assume that trends will continue indefinitely, instead of identifying short-term aberrations for what they are. For example, investors consistently overprice "growth" stocks, despite research showing that future earnings of growth and value stocks are actually the same; in other words, the prior revenue growth rate has no predictive value for future revenue growth rates. This is why mindlessly buying stocks with low price to earnings ratios has outperformed most professional investors over the last 90 years. Right now, the biggest false extrapolation is profit margins. It depends on the measure you use, but profit margins are about 35% too high. This means that if absolutely nothing else changes with the economy or corporate fundamentals and valuations, we should expect net profits to fall 35% and stock prices to fall 35%. Of course the adjustment is unlikely to happen instantly, so the market may adjust to lower profit margins over time while other inputs into stock valuation provide support (e.g. stocks may lose 35% from lower margins but gain an offsetting 35% from revenue growth over the next 4 years).
Here is a link to an article about GMO's methodology (which they recently updated), and Grantham's latest quarterly letter: http://www.gmo.com/
websitecontent/GMO_QtlyLetter_ ALL_3Q2013.pdf. You have to register to read it, but it's a quick and painless process and well worth it. I've attached their latest 7-year forecast.
It's worth noting that the forecasts are in real (aka inflation adjusted) terms. So, if we expect inflation to average 3% over the next decade, this forecast then predicts small positive nominal returns for US equities. As you can see, none of the asset classes look attractive. Even top performing timber is projected to return a modest 6%. This is a result of "financial repression" and the major run-up in all asset prices over the last 4 years.
So where should we have our money? Grantham's forecast suggests that neither emerging market equities nor commodities are undervalued, but both areas look far more attractive than US equities. I've been tip-toing in to emerging market equity, particularly China. I've also been continuing to scale into commodity-linked equities. I was a little early getting long natural gas related equities in 2012, but those are now paying off nicely, and I think the sector has a long way to run. I'd also suggest looking at the mining and agriculture sectors. My concern is that the credit situation in China leads to a mini-panic and sell-off in both emerging market equity and commodities across the board. If that happens, I think both asset classes will represent a phenomenal buying opportunity.