Backtesting Your First Strategy

Lesson 1 of 5 · 6 min

What Backtesting Actually Tells You

What a backtest can and cannot prove, and why most published results are worthless.

A backtest is a hypothesis test, not a promise

Backtesting runs a set of trading rules against historical data to see how they would have performed. That is genuinely useful — but only for what it actually measures.

A backtest can tell you: whether a strategy had an edge in the past, how volatile that edge was, how deep the losing periods ran, and whether the idea survives different market regimes.

A backtest cannot tell you: that the edge still exists, that you would have held through the drawdown, or that the future will resemble the sample you tested.

Why most backtest results are worthless

• Survivorship bias — testing on today's index members ignores every company that went to zero.
• Look-ahead bias — using data that was not available at decision time (restated earnings, same-bar closes).
• Ignoring costs — commissions, spread, and slippage routinely turn a "profitable" high-frequency strategy negative.
• Too few trades — 20 trades proves nothing. Edge needs a sample large enough to distinguish skill from luck.

The honest standard

Treat a backtest the way a scientist treats one experiment: as weak evidence that justifies further testing, not as a result. The strategies that survive are the ones that keep working on data you did not use to build them.

Lesson 2 of 5 · 6 min

Choosing a Strategy Worth Testing

Start from an economic reason a strategy should work — not from a pattern you found in the data.

Start with a reason, not a pattern

The single biggest determinant of whether your backtest means anything is what you chose to test and why. If you started by mining data for patterns, you will find them — and they will be noise.

Instead, start with an economic explanation for why an edge should exist and persist. Someone must be on the other side of your trade for a reason.

Edges with a real explanation

• Momentum — investors underreact to news, so trends persist longer than they should.
• Mean reversion — forced sellers (margin calls, redemptions) push prices below fair value temporarily.
• Carry — you are compensated for holding an asset others do not want to hold.
• Risk premia — you get paid for accepting a risk (illiquidity, volatility) that others avoid.

Define the rules before you look

Write down the entry, the exit, the position size, and the universe before running anything. If you adjust the rules after seeing results, you are no longer testing a hypothesis — you are fitting one.

Keep it simple. A strategy with two parameters that works is far more trustworthy than one with nine that works better. Every additional parameter is another way to accidentally fit noise.

Lesson 3 of 5 · 7 min

The Metrics That Matter

Why total return is the least useful number, and what to read instead: drawdown, Sharpe, and exposure.

Total return is the least informative number

A strategy that returned 400% tells you almost nothing on its own. Over what period? With how much leverage? Through what drawdown? Concentrated in which few trades?

Read these instead

• CAGR — annualized return, so results across different periods are comparable.
• Maximum drawdown — the worst peak-to-trough loss. This is the number that decides whether you could actually hold the strategy.
• Sharpe ratio — return per unit of volatility. Above 1 is good; be suspicious of anything above 3 in a backtest.
• Sortino ratio — like Sharpe but only penalizes downside volatility, which is what you actually care about.
• Number of trades — your sample size. Fewer than ~100 and your statistics are fragile.
• Exposure — how much of the time you were in the market. A strategy in cash 90% of the time is not comparable to buy-and-hold.

Drawdown is the one that ends careers

A 60% drawdown requires a 150% gain just to break even. More importantly, almost nobody keeps executing a system through a 60% loss — they abandon it at the bottom, converting a paper drawdown into a permanent one.

Before trading anything, ask honestly: if this lost that much, in real money, would I keep going? If not, the strategy is wrong for you regardless of its Sharpe.

Always compare to the boring benchmark

Compare against buy-and-hold on the same universe over the same period. A surprising share of "strategies" underperform simply holding the index once you account for costs and taxes.

Lesson 4 of 5 · 7 min

Overfitting: How Backtests Lie

The failure mode that makes a strategy look brilliant historically and lose money live.

The core problem

Overfitting is building a strategy that describes the noise in your sample rather than the signal. It produces a beautiful equity curve on the data you tested and a flat or negative one on data you did not.

It rarely feels like cheating. It feels like diligent research: trying a 50-day moving average, then 52, then 47, and keeping whichever performed best.

Warning signs you have overfit

• Performance collapses when you change a parameter slightly (50-day works, 45-day does not).
• The strategy has many parameters relative to the number of trades.
• Results depend heavily on one period or a handful of outsized trades.
• You tested dozens of variants and are reporting the best one.
• Sharpe above 3 with no obvious structural explanation.

Defenses that actually work

• Hold out data. Build on one period, test once on another you never looked at.
• Walk-forward testing. Re-fit on a rolling window and test on the period immediately after, repeatedly.
• Prefer parameter plateaus. A strategy that works across 40–60 day lookbacks is real; one that only works at 51 is noise.
• Count your attempts. If you tried 100 variants, the best one looking good is expected by chance.

The uncomfortable rule: every time you look at the test data and adjust, you have spent some of its value. Look as few times as you can.

Lesson 5 of 5 · 6 min

Running Your First Backtest

A practical checklist to take one simple strategy from idea to evaluated result.

Do the whole loop once, simply

The goal of your first backtest is not to find a profitable system. It is to complete the full loop end to end so you learn where the traps are. Pick something almost trivially simple.

A reasonable first test

1. Pick one liquid universe (a broad index's members, or a handful of large-cap names).
2. Pick one simple rule with an economic story — e.g. buy when the 50-day crosses above the 200-day, exit on the reverse cross.
3. Fix your position sizing (equal weight is fine) and write the rules down before running.
4. Include costs — commission plus a realistic spread/slippage assumption.
5. Run it over a period containing at least one bear market so you see the drawdown.

Then interrogate the result

• What was the maximum drawdown, and would you have held through it?
• How many trades? Is the sample big enough to mean anything?
• Does it beat buy-and-hold after costs?
• Does it still work if you shift the parameters ±20%?
• Does it work in a period you did not use while building it?

If it survives all five, you have something worth researching further — not something worth funding. The next step is walk-forward testing and paper trading, in that order.