As investors shift away from actively managed funds towards passive investments, factor investing or smart-beta investing has rapidly gained popularity as passive investors try to replicate the performance of various active strategies at a low cost.

For example, investors seeking to gain exposure to value, which has historically outperformed the market, can buy a value focused smart-beta ETF, rather than a higher cost actively managed value fund, which tend to underperform their benchmarks in general.

However, while there is plenty of evidence that shows some factors do outperform, the explosion in demand for these products has led to rapid growth in the number of factors and smart-beat funds investors can follow. Many of these factors put forward as having generated premiums in the past are merely the result of data mining, and there's no guarantee that these performance figures will ever be realized in the real world.

To investigate this trend, Credit Suisse analysts Elroy Dimson, Paul Marsh and Mike Staunton set out to analyze the performance of various factor and smart-beta strategies over and ultra long holding period (118) across 23 markets. The findings of this study are presented in the bank's Investment Returns Yearbook 2018.

At the beginning of 2018 there were more than 7000 ETFs and ETPs with over 12,000 listings and assets of more than $5 trillion, an enormous market for investors to choose from. According to the report from the Financial Times, assets in products that followed factors surpassed $1 trillion during 2017.

The idea of factor-based investing was really first proposed by Fama and French in 1993. These researchers identified four factors in addition to the market including size, value, profitability, and investment.

Subsequent papers identified other factors including low-risk, momentum, income and low volatility. In total, researchers have identified 316 different factors that can lead to different equity returns.

The problem is that almost all of these factors were discovered in a research environment, and in the real world are unlikely to produce the returns suggested as Dimson and team note: "Harvey, Liu and Zhu (2016) point out that nearly all are unlikely to be robust in independent testing. Novy-Marx and Velikov (2015) and Green, Hand and Zhang (2017) express complementary doubts about the prospective profits from exploiting factors that appear promising on an in-sample basis. The problem of apparently significant in-sample results being non-robust in out-of-sample tests has been discussed for more than a quarter of a century: see, for example, Dimson and Marsh (1990) and Markowitz and Xu (1994). But there is no substitute for genuine out-of-sample testing. Harvey (2017) notes the impracticality of waiting for additional data in order to test a model’s OOS reliability – not to mention the understandable impatience of practitioners."

The Credit Suisse team set out to test the ability of factors to produce outperformance by using out-of-sample data, or in other words, real-world data. Only five of the 316 different factors in existence today have been around for more than three decades, so the researchers focus on these. Size, value, income, momentum, and volatility are among the longest-established and best-documented regularities in the stock market, allowing the team at Credit Suisse to study them on an out of sample basis in in two ways; first, examining recent data, and second, evaluating truly long-term data.

It is this recent data that is perhaps the most interesting. The market's most well-established factors have been around for many decades, but the market environment over the past few years has hardly been what many investors would call normal. What's more, the research underpinning these factors is pre-crisis, with only a limited amount of data available for the post-crisis period. It is interesting then, to see how these factors have performed in the years since the financial crisis and if their outperformance (or underperformance) has continued.

As shown in the chart below it has been a mixed bag. No single factor has generated a predictable or steady return over the ten years since 2008. In the UK market momentum has achieved the best return in four years out of the 10-year sample, size has been the second-best performer. Overall in the UK, momentum produced the best return premium of 13% between 2008 and 2017, followed by size at 6.1%, low volatility at 3.9%, income at 1.8% and value underperformed by 2.8% over the period.

In the US, low volatility was the best performing style between 2008 and 2017. During his period low volatility produced a return premium of 6.2%, closely followed by size at 2.8% and then income at 1.9%, value of -2.7% and momentum at -4.9%. Momentum would have achieved a much stronger performance if it were not for 2009 and 2016 when the strategy significantly underperformed. And even though many decades of research show value is a strong performer over the long-term, it underperformed in most years since the crisis.

Overall, the researchers conclude that while factors will always be important "in investment in terms of helping explain differential performance, there is no guarantee that factor premiums will persist." As a result, smart-beta investors should focus on factors supported by robust evidence and should diversify across multiple factor risk premiums.