Clever or Lucky?
As I mentioned in last week's post, I am interested in the question as to whether the impressive performance of the Mechanical Bull (MB) portfolio is the result of a clever investment technique or whether I have simply been lucky. In this post I have done some analysis to try and answer that question using a Monte Carlo simulation.Wikipedia describes Monte Carlo methods as:
"... a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results; typically one runs simulations many times over in order to obtain the distribution of an unknown probabilistic entity"
This technique can be applied to answer my question. The idea is to take the full set of stocks from my investment universe, select a corresponding time frame, create portfolios at random, and then calculate an average return for each portfolio. The full set of returns then provides a distribution against which the performance of the MB portfolio can be measured. This can be used to tell me whether I am clever or just lucky.
Backtesting data
To do this experiment I needed a backtesting database. I used Sharelockholmes, for no other reason then I have used it extensively in the past and the cost of renewing my subscription was affordable (£5.20 a month). There may be better such databases out there, but I did not bother to investigate this further.Sharelockholmes allows you to select a point in time at each quarter and then see how selected UK stocks would have performed over selected future time periods. Since I started the MB portfolio on 24 May 2013, I could not test its performance from the outset. The best I could do was to start from 30 June. Likewise, the way subsequent performance is reported is constrained and so the testing period stops at 31 December. This gave a total test period of six months. The MB return over this period was 31.6 per cent.
Issues to sort out
A common issue with backtesting is company delisting, which for Stockopedia means that the forward performance metrics are missing. There were 43 such stocks in my dataset. I tackled this by searching each of these stocks manually and then identifying the last price at which the stock traded. In some cases this might not have reflected reality, for example, if the stock was suspended and shareholders were later wiped out. However, in one case (RSM Tenon) I remembered what happened so I recorded a nil return.I also noticed some other problems with the performance mertics. For example, Ixico showed a 6 month performance gain of 4240 per cent! Some quick research revealed that the company has a reorganisation of share capital as the result of a reverse takeover, so these figures were not reliable. I readily concede that there may be other errors in the data and as such this experiment may not perfect. However, I simply did not have the time or patience to investigate these further.
Running the simulations
Once I had a full set of performance metrics, I generated portfolios based on a random selection of 15 stocks, which is the same number that I hold in my MB portfolio. I did this in Excel by adding a random number column in the spreadsheet, which I then sorted to generate a new portfolio and an average return metric for each. I repeated this 100 times thus providing 100 hypothetical return values.
The Results
The average return from each of these simulations was 9.4 per cent. This compares with the FTSE Allshare return of 9.7 per cent (sourced from Google Finance) over the same time period, so this seems about right. But the real point of doing all this was to get an estimate of variance, or the degree to which the simulations vary around this average. Now clearly, the MB return of 31.6 per cent is much better than 9.4 per cent, but how can I be sure that this is not just down to luck? The higher the variance, the more difficult it is for me to claim to be clever rather than lucky.The standard deviation is a useful measure of variance as this can be used to construct confidence intervals, which are a commonly used framework for hypothesis testing, For the data I generated the standard deviation was 10.4 per cent. I am not going to do all the maths, but in lay person's terms this all means that we can be 95 per cent confident that the out performance of the MB portfolio over this period was due to something other than luck.
It is important to note that this test does not say anything about the level of out performance. All this is saying is that the MB portfolio is very likely able to deliver out performance. So with a 31.6 per cent return over this period, perhaps I have been both clever and lucky!
