What our backtest does not tell you
A backtest is a simulation, not a promise. The six limits of ours, written by us, before anyone raises them against us.
We publish the results of a backtest, and we think it is the right thing to do. But a backtest is a simulation of the past, not a promise about the future, and it has precise limits. Here they are, written by us, before anyone raises them against us.
The rule tested, in one sentence
We buy when the score reaches 58. We sell when a trailing stop, set 2.5 times the ATR below the highest price reached, is hit, or after 20 trading days, whichever comes first. Nothing else.
1. An expectation per trade is not a return
The backtest takes every signal, as if capital were unlimited. A figure such as “plus 0.8% per trade” therefore says nothing about what a real portfolio, which cannot buy everything, would have done.
That is why there is a second simulation, with €10,000 and a chronological order: when everything is invested, the next signal is ignored, and counted. The gap between the number of signals and the number of positions taken is information, not a detail. It is in the order of a few percent: the portfolio takes only a small part of what the backtest counts.
2. The portfolio IS the draw
Since only a fraction of the signals is taken, the choice of which ones becomes decisive. Taking them alphabetically has no particular merit. So we replay several random draws and publish the minimum, the median and the maximum: the range is what is honest, not the point.
3. A real bug, and what it cost
The first real calculation produced a positive expectation per trade and a portfolio at minus 96%. An impossible contradiction, so a certain defect.
The cause: when several signals fall on the same day, they have to be ranked. The code used the exit date. With around fifty signals per session, the simulator therefore always took those that exit earliest, in other words the stop-outs, in other words the losers. A selection that reads the future, even in the pessimistic direction, invalidates the whole simulation.
The ranking is now done on the stock's name: arbitrary, known at entry, uncorrelated with the outcome. On the test scenario, the fix moves the result from minus 99.6% to plus 26.4% with exactly the same trades. A test replays the structure of the defect and fails if exit information comes back into the order of selection.
4. The drawdown shown is gentler than the real one
Between entry and exit, positions are valued at cost. The curve therefore shows realised losses, not the dips crossed along the way. The real maximum loss was deeper than the one displayed. We have no good way to measure it without intraday data, so we say so rather than let you believe otherwise.
5. Execution is perfect, and it never is
Every order is filled at the price the backtest uses. In reality there is a gap between the price shown and the price obtained, wider for small stocks and nervous markets. Fees, for their part, are deducted: 0.20% on the purchase and as much on the sale. They penalise first the strategy that takes the most trades, and leaving them out would have flattered the most active variants.
6. The tested past is short, and it has a climate
The history fits in a rolling window of about 200 days per stock. That is enough to calculate every indicator; it is not enough to have gone through several complete market regimes. A method that works during a rising period has not proven it works elsewhere.
And survivorship bias
We score the stocks currently tracked. Companies that left the indices, were delisted or merged are no longer in the universe, and they are rarely the best. A soft deactivation keeps the history of positions, but the bias exists and it pushes results upward.
What remains true despite everything
These limits do not make the backtest useless. They say what it is: an internal consistency test. It answers “would this rule have done better than nothing over this period”, not “how much will I earn”. A method that fails this test has no chance; a method that passes it has no guarantee yet.
The figures live on the Backtest page, where they recalculate. We do not copy them here: a figure in an article ages in silence.
Educational content: these articles explain what the site's measurements calculate, so that you can decide for yourself, knowing what they mean. Investing carries a risk of capital loss. See the legal notice.