Portfolio Optimisation: Backtesting NSE ETF with mean variance methods

This post explores the performance of select Exchange Traded Funds (ETF) being traded on NSE India using this portfolio optimiser app. For more details on the idea behind the app, see this.

While simple mean variance optimisation might not retain its allure in modern portfolio optimisation methods, given its simplistic assumptions, it is still a great starting point for any quantitative investor. The key ideas of maximising returns while minimising risk are still valid, even though the basic implementations using historical data to calculate key inputs like expected returns and covariances seem a bit naïve. Nevertheless, with real world data, it is worth looking into whether even the simplistic models can somehow be tuned to outperform passive buy and hold strategies.

Basic inputs and formulations of Mean Variance Optimisation

Below workflow chart explains how optimal weights are calculated using returns data for any group of assets.

The above is a very straightforward implementation of the Markovitz Portfolio Theory, using historical returns data to calculate expected returns and covariances.

Our Basket of Assets

For the purposes of this article we will be focusing on a list of 5 ETFs traded on NSE. These assets have been among the best performing ones with CAGR exceeding 12%. So in that sense, this whole exercise seems hacky and reeks of hindsight. But with enough curiosity coupled with shamelessness we can pretend this is meaningful!

ETF	NAME
Motilal Oswal NASDAQ 100 ETF	MON100
Motilal Oswal Nifty Midcap 100 ETF	MOM100
Nippon India ETF Nifty Next 50 Junior BeES	JUNIORBEES
CPSE ETF (PSU investments)	CPSEETF
Nippon India ETF Nifty 50 BeES	NIFTYBEES

Simple Mean Variance Optimisation against Buy and Hold Strategies

First we look at the most basic Mean Var strategy, using the bt Algos WeighMeanVar function and rebalance annually. As explained in our article for recipe implementation, we will contrast this with simple buy and hold for all assets. The backtesting recipe for the weighmeanvar strategy looks like this:

{
    "weighmeanvar": {
        "rebalance_freq": "RunYearly",
        "select": "all",
        "RunAfterDate": "2015-12-24",
        "optimiser": [
            {
                "name": "WeighMeanVar",
                "args": {
                    "covar_method": "standard"
                }
            }
        ],
        "rebalance": true
    }

The buy and hold is implemented using RunOnce, which ensures no rebalancing takes place at any frequency and WeighEqually vacuously assigns the full weight to the only asset chosen using select. Below is an example, with MON100.

    "buy_hold_mon100": {
        "select": "MON100",
        "RunAfterDate": "2015-12-24",
        "optimiser": [
            {
                "name": "RunOnce",
                "args": {
                }
                
            },
            {
                "name": "WeighEqually",
                "args": {
                }
                
            }
        ],
        "rebalance": true
    }

We run the backtest and below is the strategy performance graph, which looks abysmal. MON100 (the NASDAQ ETF) clearly outperforms everyone and out portfolio optimisation is unable to take advantage by allocating more to this.

If the above graph was not a nail in its coffin, the below stats certainly are. It’s kind of funny that our “strategy” manages to be the worst in terms of CAGR! That too, in my opinion is an achievement: to optimise the best performing stocks and come out to be the worst. The only thing going for it is that it’s risk averse: as highlighted by the high calmar ratio (CAGR/max drawdown) which is achieved by an impressively low drawdown.

Strategy	CAGR	Maximum Drawdown	Calmar Ratio	Yearly Sharpe	Yearly Vol
weighmeanvar	11.20%	-36.16%	0.31	0.62	21.40%
buy_hold_mon100	16.85%	-90.27%	0.19	0.74	26.83%
buy_hold_mom100	14.33%	-48.42%	0.30	0.72	22.61%
buy_hold_niftybees	11.93%	-89.92%	0.13	1.20	10.35%
buy_hold_juniorbees	12.89%	-38.68%	0.33	0.85	16.52%
buy_hold_cpsetf	14.85%	-56.45%	0.26	0.63	28.63%

How did the rebalance transactions look for the weighmeanvar strategy? As we can see below, while the strategy did try to allocate more weight to the highest performing assets, the switch in allocations to assets is too abrupt, as we can see that on certain years, all the capital is allocated to a single asset.

This trigger happy approach does not suit any longer term growth strategy with low rebalance frequency. We will try to fix this.

Improvements

The transactions to rebalance a portfolio in the simple strategy above demonstrate what is wrong with the approach. We selected a basket of 5 top ETFs (in a hacky way, no doubt) but with each rebalance we take dramatic measures based on out covariance and expected returns calculations, abandoning any measure of belief in our choices. To reinforce our trust in these assets in a simple way, we impose upper and lower bounds on our allocated weights, thus protecting the portfolio rebalances against extreme allocations. A simple example would be a bound of (0.05, 0.5) which means all assets are at least allocated 5% and none exceeds 50%. Below is the demonstration of modification to the json recipe to achieve this.

"weighmeanvar_bounded": {
        "rebalance_freq": "RunYearly",
        "select": "all",
        "RunAfterDate": "2015-12-24",
        "optimiser": [
            {
                "name": "WeighMeanVar",
                "args": {
                    "covar_method": "standard",
                    "bounds" : [0.05, 0.5]
                }
            }
        ],
        "rebalance": true
    }

Below is what the results look like. While the modified strategy is still not at the top, is an improvement from the previous one which was languishing at the bottom.

The stats confirm that the returns get better with the max drawdown a bit increased but still low enough to be classified as a safer investment strategy.

Strategy	CAGR	Maximum Drawdown	Calmar Ratio	Yearly Sharpe	Yearly Vol
buy_hold_mon100	16.85%	-90.27%	0.19	0.74	26.83%
buy_hold_cpsetf	14.85%	-56.45%	0.26	0.63	28.63%
buy_hold_mom100	14.33%	-48.42%	0.30	0.72	22.61%
weighmeanvar_bounded	14.21%	-41.55%	0.34	0.85	18.56%
buy_hold_juniorbees	12.89%	-38.68%	0.33	0.85	16.52%
buy_hold_niftybees	11.93%	-89.92%	0.13	1.20	10.35%
weighmeanvar	11.20%	-36.16%	0.31	0.62	21.40%

We can play around with other bounds to see how they perform, as below.

Strategy	CAGR	Maximum Drawdown	Calmar Ratio	Yearly Sharpe	Yearly Vol
weighmeanvar_bounded_0_0.5	14.48%	-53.76%	0.27	0.82	19.56%
weighmeanvar_bounded_0.05_1	13.52%	-33.18%	0.41	0.73	21.25%
weighmeanvar_bounded_0_0.3	13.53%	-40.65%	0.33	0.81	18.65%
weighmeanvar_bounded_0.1_0.3	14.43%	-37.34%	0.39	0.88	18.08%

Temporal Sensibilities

If you are an investor, perhaps the most influential parameter in defining your strategy is the time horizon you are looking at. It separates the impulsive/reactive from the patient value driven investor. But in all the above models, we are yet to play with the underlying time periods of key inputs.

1.     Frequency of rebalancing: For all the above cases, we rebalanced the portfolio annually. A different breed of investor (and also for a different mix of assets) might prefer a more proactive approach by adjusting the allocations more frequently like quarterly or even weekly.

2.     Lookback: For variance - covariance calculation using the historical returns it is important to consider what slice of data we need to take into account. This lookback period is set to three months as default in all the previous cases and we might want to look at the impact of other variations.

While the above two are technically independent parameters, for a coherent strategy we might want these to be dependent. In other words, for more frequent rebalancing, we might want to consider shorter lookbacks and longer lookbacks for longer time horizons. For a fair comparison we will use the same bounds of [0, 0.5] for all the variations. Below are the different strategies we will use for our analysis. The naming convention reflects the rebalance period and the corresponding lookback i.e. yearly_2yl means an yearly rebalance with 2 year lookback.

Below are the backtesting results for each of the above strategies.

The statistical summary of the results look better now. The key thing to note is that almost all of these strategies outperform the simple buy and hold for any singular asset (best of which was CAGR of 16.84% by MON100).

Strategy	CAGR	Maximum Drawdown	Calmar Ratio	Yearly Sharpe	Yearly Vol
yearly_3yl	16.85%	-48.41%	0.35	0.99	18.50%
yearly_2yl	17.84%	-49.14%	0.36	0.96	20.43%
yearly_1yl	19.31%	-42.99%	0.45	1.20	17.17%
quarterly_1yl	16.18%	-51.50%	0.31	0.90	19.80%
quarterly_3ml	16.95%	-31.94%	0.53	1.06	17.27%

The best performing strategy, purely in terms of returns in all our experiments turns out to be yearly rebalance with one year lookback for variance covariance calculations. It has the best CAGR of above 19% and coupled with a drawdown that that is not the worst. The quarterly rebalance with a 3 month lookback has a healthier risk adjusted return profile with a respectable calmar ratio above 0.5.

Pitfalls and Biases

When backtesting an optimal portfolio allocation with mean variance optimization, it's essential to be aware of various biases and pitfalls that may be inherent in our backtesting procedure:

1.     Data-Snooping Bias: This occurs when multiple variations of the portfolio allocation are tested against historical data, leading to the selection of a strategy that performs well purely by chance. Experiments with out-of-sample data or cross-validation techniques are needed to rule this out.

2.     Look-Ahead Bias: This was obvious in our asset selection where we chose the best performing assets in our basket. We need a mix of good and bad and see if the best performing strategies can still come up with optimal results.

3.     Transaction Costs: We did not take this very important realistic adjustment in our strategies that will cause overestimation of returns.

4.     Overfitting: Our tuning of parameters like lookbacks, rebalance frequencies and bounds on asset allocations will reek of overfitting unless verified against a host of other asset baskets and time periods.