Forecasting season is upon us. Anyone who gets to be quoted in print or speak to a reporter is asked an opinion. Expertise not required! It is paradise for pundits.

Many have decried the “folly of forecasting.” People love to laugh at supposed experts, looking back at old forecasts. Since most forecasts are based upon a model, modelers are thrown under the bus as well.

Barry Ritholtz wrote on this topic in his excellent Apprenticed Investor series. There are now over 200,000 blog hits on this phrase.

Background

But please consider this: Most models and forecasts are bad – very bad – but not all. The trick is to figure out which is which. Barry notes the possible exceptions:

There are only two kinds of predictions that have some value to investors: One is probability-based, and the other is risk-based. If you apply the same rules — no one knows the future, they are subject to revision and should not be taken as gospel — then these are sometimes worth considering.

Here are a few examples.

• Millions of people attempt to paint, but only a handful are successful. Could you pick the winners?
• Millions attempt to write, but there are few best-sellers. Could you guess them in advance?
• Worldwide wine consumption is over 30 billion bottles. How many are really good?
• In the U.S., 11 million people are playing baseball in a given year. Fewer than 900 are in the major leagues. There is so little difference that only an expert could identify the best players by watching them bat.

Finding the best in any large field is a real challenge!

The issue is especially important for financial analysis. I have been pondering this question for weeks. How can I best explain an important but unpopular viewpoint? I recently began this theme with by citing a bogus analysis in the New York Times. In simple fashion, I showed that if you only had the long-term average – that the market returned a positive result 2/3 of the time – you would do much better to predict “Up” every year rather than guessing 2/3 up and 1/3 down. This counter-intuitive result should be cause for thought, since it is an expensive and common investor mistake.

Ben Carlson inspired another approach. He publishes consistently strong work at his blog, A Wealth of Common Sense, which I always read and frequently cite. He discusses the difficulty of selecting the best stocks and sectors. This is the updated sector asset quilt he created, followed by his principal conclusion.

Like any asset quilt, there’s no rhyme or reason from one year to the next. I’m sure you could torture the data here using a momentum or value-based strategy to improve upon the results of the S&P 500, but unless you’re using a rules-based approach, you’re really just guessing when attempting to figure out which sectors will perform best over any given time frame.

That conclusion seems persuasive to a very intelligent observer using annual ranking changes. Can a first-rate forecaster add any value?

Finding the Real Experts

Ben is quite correct in noting that most contrived explanations will torture the data. Is this true of every approach? Here are some things I look for in evaluating a model:

• It does not use too many variables compared to the available data
• It has a good record using “out of sample” data as well as in real time
• The underlying method is logical, proceeding from a theory
• The modeler has both experience and expertise

Two sharply contrasting success stories reflect the two most common model types, trend following and mean reversion.

Dr. Robert Shiller is a leading economist at a top university, a Nobel Laureate, author of many papers and books of value to investors, and a popular media guest. Among investors, he is probably best known for his Cyclically Adjusted Price to Earnings ratio (CAPE) method. One of the methods that he endorses is the Barclays ETN, CAPE. Barclays implements CAPE in a mean reversion method. They look at the historical CAPE for each sector choosing the sectors most under-valued by this comparison (throwing out the bottom one). This is a mean-reversion method based upon fundamental data. At the introduction over four years ago, Barclays had promising backtest data. This did not attract many investors, most of whom cite CAPE as a method for timing the overall market – which Dr. Shiller himself does not do. After four years, the fund remains very small (under $34 million).

How has it done in real time?

Probably riskier than buy and hold the market, but much stronger returns. Those choosing to use CAPE as a reason to exit the market (not Dr. Shiller’s recommendation) would have done better to buy the ETN.

Dr. Vincent Castelli is not a professor at a top university, but he could have been. He will not win a Nobel prize, because his best work was top secret. He spent a career making U.S. armed forces safer and more effective, heading a group of other scientists from various disciplines. His modeling is known in quant circles, where he demonstrates, advises, and coaches. He is probably not going to become famous on CNBC.

His approach to sector analysis begins with the time-tested method of trend following. The tricks are in separating signal from noise, recognizing trends in a timely fashion, and exiting while you can protect profits. As an expert in modeling, Vince touches all the bases for sound work — ruthlessly pruning variables, a generous out-of-sample test, and real-time comparisons.

These two brilliant men took quite different approaches to life and later to analyzing stock sectors. Each found a profitable approach where most of us would see nothing.

Conclusion

There are many paths to successful investing. Remain open to profit opportunities by giving open-minded consideration to other approaches. Finding the best experts is just as important as finding the best stocks or sectors.