Robustness: walk-forward and Monte Carlo

You can explain how walk-forward analysis and a Monte Carlo reshuffle of trades test a rule set's robustness, and use the results to plan for worse drawdowns.

One story, many versions

A backtest tells one story: these rules, on this data, in this order, produced this result. It is easy to forget how much of that story is accident.

Slightly different data might have led to different settings. The same trades in a different order might have produced a much deeper drawdown. Two robustness tests look at exactly these questions, and they often change how much risk a trader is willing to take.

Optimize, test, roll forward

Walk-forward analysis repeats the out-of-sample test many times:

Optimize the rules on a window of history, for example two years.
Test the chosen settings on the following window, for example six months, without changing anything.
Move both windows forward by six months and repeat.
Join all the test windows into one result.

Every test window is unseen data for the optimization before it. The joined result shows how the whole method, including its optimization, performs on new data, period after period. If the joined result is poor while each optimization looked great, the method fits noise. MetaTrader 5's forward option performs a single step of this idea.

Same trades, deeper drawdowns

A Monte Carlo simulation uses random repetition to explore what else could have happened. A simple version reshuffles the order of the backtest trades thousands of times and records the drawdown each time.

Sixty fictional trades, 24 wins of 2R and 36 losses of 1R, at 1% risk per trade:

In their backtest order, the maximum drawdown is 3.0%, with at most three losses in a row.
Across 10,000 random reorderings, the median maximum drawdown is 7.8%, and half the orderings contain six or more losses in a row.
The worst 5% of orderings reach 11.6% or more. The single worst reaches 19.1%.

The trades and the expectancy are identical in every case. Only the order changed. A plan built on the 3.0% would be unprepared for a perfectly ordinary bad run.

Common mistakes

Trusting one backtest path. The historical order of trades is only one of many possible ones.
Re-optimizing until walk-forward looks good. That turns the test itself into a fit.
Sizing for the median drawdown. Plan for the bad tail, not the middle.
Assuming reshuffling covers everything. It cannot simulate a market that changes character.

The takeaway

Walk-forward analysis re-tests rules on data they never saw, again and again. Monte Carlo reshuffles trades to show how bad the drawdown could get.

Check yourself

Quick quiz

4 questions. Instant feedback, nothing is saved.

1Tap the drawdown that a typical reordering of the same trades produces.

Tap the chart.

Half of all reorderings produced a maximum drawdown above 7.8%, more than twice the backtest's 3.0%.
2What does walk-forward analysis add to a single backtest?

Each test window is out of sample for its optimization window. Joined together, they show how the method itself performs.
3Why can the backtest's drawdown be misleadingly small?

Here the backtest order had at most three losses in a row. Half of the reshuffles had six or more.
4How should a Monte Carlo result change your plan?

If a 12% drawdown is plausible, the plan must survive it, financially and emotionally.

of 4 correct.

Sources

Back to Trading Styles & Strategy Development