Beware the Backtest

Backtesting is a popular tool amongst traders and marketers. Traders use it to experiment and uncover new strategies. When one looks promising, the marketers go to work! Creating fancy charts and graphs to sell you on hypothetical performance. But are strong backtests indicative of strong future results?

The world of "Smart-Beta" is filled with promising backtests. Unlike plain-vanilla ETFs that simply track an index, Smart-Beta aims to outperform by adjusting their holdings - sometimes greatly deviating from their index.

A recent study called the Smart Beta Mirage analyzed the performance of these "smarter" funds from 2000 - 2018. Sadly, it found that outperformance only lived in the data.

Smart-beta in backtests showed an above-market returns of 2.77%, but when put into action, they underperformed by 0.44%.

There are several reasons for the disappointing results. The most obvious one, no one can predict the future. The oldest adage in investing is "Past performance is no guarantee of future results". Things change, what worked in the past might not work in the future.

Another reason is backtesting is flawed. It's susceptible to bias and data mining. ETF providers are incentivized to attract as many investors as possible. A great way to do this is to show great performance would've been, even if it sacrifices the integrity of testing.

While backtests are interesting and a great tool if used correctly, like star ratings and analyst targets, they do little in predicting the future. Don't count on them.

Comments

The Art of Giving Feedback

Constructive feedback is an awkward affair. You don't want hurt feelings, but recognize the importance of honesty. You've tried the classic "hoping things will get better on its own" and unfortunately it hasn't played out. When giving feedback, here are a few things that I try to keep it mind. Start with empathy. Step into their shoes and understand their story. If you don't know, ask. Be genuinely curious. Feedback is a dynamic affair. Shared communication with a shared goal towards progress. Take the emotion out of it. Focus on the situation, not the person. Focusing on the person adds unnecessary weight to an already emotionally-bloated event. Be specific. Give clear examples. Vague feedback equals dismissed feedback. Doing above won't de-awkward things fully, but it will dampen it and increase the chance of better outcomes.

Bias For Clarity

Bias for action. Gets things done. Go-getter. Traits companies big and small look for. And for good reason, you're being hired to do things! However, action is a secondary step that often overshadows the primary step, direction. Clear direction is the foundation that enables our actions to takeoff. Without it, we're stuck in the mud. Striving for clarity is an underrated skill. Having the courage to ask ( seemingly ) obvious questions, and to check in, making sure we're all on the same page. "O bvious " questions are a low risk, high reward way to add value. At worst, you'll add confidence to our actions. At best, you discover a misalignment that saves us from a dead-end. The more people, the more clear we need to be. The bigger the initiative, the bigger the risk of reaching the finish line, only to realize expectations were off. Success is always uncertain. But we can be certain about what we want and what everyone's job is. Things that can be clea...

Negative Feedback, Positive Lessons

In the battle against plastic bags, a five-cent tax was shown to be much more successful at deterring usage than a five-cent credit for bringing your own bags. Carrots satisfy but sticks sting, and they sting hard. So we default to the less painful choice of avoiding loss. Loss aversion impacts the way we process information. A 2019 study invited participants to learn through a series of multiple choice questions. Each question only had two options to choose from. Whether guessing correctly or not, they would still learn the right answer. Despite the identical learning opportunity, participants were much more successful at recalling the answers they guessed correctly than those they got wrong. "You're right!" feels good. We savour the moment, analyzing every detail. "You're wrong!" stings. We want to quickly forget, dismiss, and move on. When we succumb to loss aversion, we miss opportunities to learn. Failure is part of the process. We'll experie...

Herman Ye

Search