A Stock Ranking Backtest

The Mechanical Bull method is a strategy based on theory. The logic is appealing, and it has performed reasonably well over the last 15 months. However, I have had no real way of testing how well it is going to perform over the long term. I would love to be able to backtest the approach. Unfortunately, this is not possible because Stockopedia doesn't provide a backtesting service (at least not yet).

Recently, I made a rather intriguing discovery. I used to be a frequent user of Sharelock Holmes, until Stockopedia launched its subscription service. Over the weekend, I dipped back into the site and stumbled over some new metrics that looked remarkably like Stockopedia's ranking measures. Sharelock now has composite Quality, Momentum and Value scores, as well as the various blended composites, including the "Market Score" which is their version of the StockRank. It uses a similar ranking score from 1 to 100 (although 1 is the best through to 100 for the worst).

Now, I am certainly not advising anyone to cancel their Stockopedia subscriptions and switch to Sherlock Holmes. It is a rather dull and drab website and the screening functionality is fairly limited. More importantly, I suspect the data is not always up to scratch. I've come across errors in the past relation to handling of share splits. However, the one area where it exceeds the Stockpedia offer is in terms of backtesting. So I feel this provides a useful opportunity to see how these composite strategies play out over the longer term.

It is important to note that these Sharelock Holmes rankings do not produce the same set of stocks as the Stockopedia ranking system. Of the current StockRank top decile stocks, about 60 per cent are in the top decile "Market Score" list. Most of the remainder rank are in the second decile although a few got very poor scores.

So these rankings seem to be based on very similar principles, but the implementation differs. There are also clearly differences with some of the data. Just one example I picked out was the Piotroski score for Staffline, which is 8 according to Stockopedia but just 4 according to Sharelock. Not with standing these differences some back testing of these ranking scores seemed too me to be a worthwhile project.

This first chart shows "Market Score" performance by deciles (rebalanced each quarter) going back to Jun 2003:

This gives the familiar and reassuring fanning out pattern seen on the Stockopedia discussion forums, except going all the way back to mid-2003. The top decile stocks blitz the field with an especially impressive performance over the past 5 years. The chart also nicely illustrates the compounding effect over the longer term. This just goes to show how tilting the odds in your favour, even just slightly, can deliver very impressive returns over the longer term.

However, it is also interesting to note that that all deciles fall away during the bear market that started in mid-2007 through to early 2009. I think this raises some interesting questions about market psychology. It seems that the market as a whole acts fairly rationally during bull markets, identifying quality, undervalued and winning stocks and dumping junk, expensive and losing stocks. However, during a bear market, fear takes over and investors seem to be willing to irrationally dump stocks irrespective of fundamentals. So, really, we should be grateful for bear markets since they generate the fertile ground for rational investors to exploit in the next bull run.

My next piece of analysis involved selecting the top decile stocks for Quality, Value and Momentum, along with all the composites (QV, QM, VM, "Market Score"). For each strategy I generated a set of relative performance data for every quarter back to June 2003 for a range of time periods from 3 months to 5 years. For each of these time periods I have then averaged the performance and then annualised the data. For example, for performance data over two years, I halved the average performance to give a figure that could be compared with all the other time periods.

It is only possible to compare the performance across all these various time scales on a fully like for like basis for the first five years (i.e. Jun 2003 to Jun 2009). In other words, to get the 5 year performance results for June 2014, you would have to wait until June 2019 to see how those stocks will eventually perform. So it is important to bear in mind that this data does not include the bull market of the past five years (and also remember this is relative not absolute performance). The results of this comparison is shown below:

This shows that momentum over the short term is is the single most effective strategy. However, performance drops of quickly after 3-6 months so in terms of translating this into an effective strategy you would need to factor in trading costs and spreads etc. Also, momentum tends to do badly in falling markets.

A strategy based on a pure value rank performed worse of all. This result seems pretty surprising considering that value investing is so widely respected. A likely explanation is that a rules based approach to value investing may be susceptible to picking value traps, that is, stocks that are a cheap, but for very good reasons that doesn't appear in the financial data. The only time that a value based rules strategy seems to really pays-off is after the market has completely tanked.

A strategy based purely on quality does consistently well over all time periods. Quality also holds up  best when the market is under pressure. This suggests that quality stocks are not just for buy and hold type investors, but should probably be key consideration for any investment strategy.

The combined QVM (Market) strategy does fairly well but it doesn't particularly stand out as the best approach. I suspect the under performance of value acts as a slight drag its overall performance. On this basis, the greater safety offered by quality would seem to be a price worth paying for only slightly greater short-term returns.

I also produced another version for the full ten years. As explained above, this shows only partial data for the longer time periods, but it is worth showing as it incorporates the bull market since early 2009.

This shows that the returns for both the Market and QM strategies have picked up relative to pure quality. If one considers that the typical holding period for a stock in the MB portfolio is about a year, then either of these strategies would offer at least a 10 percent annual return above the market.

There is a lot more I would like to look at here, but something I am starting to think about is a strategy based on different phases of the stockmarket cycle. This could consist of quality, value and momentum "pots" where the size of the pots would vary according to the state of the market. For example, given where we are at the moment I might have about 70% quality and 30% momentum stocks. If the market drifts down I might go 70% quality 30% value. If another bull market I might go 50% quality and 50% momentum.

Anyway, lots of ideas here and lots more thinking to do.