> ## Content Index
> Fetch the complete content index at: https://thomasadair.ghost.io/llms.txt
> Use this file to discover other available public pages before exploring further.

# The Backtest Isn't the Edge
- URL: https://thomasadair.ghost.io/backtest-isnt-the-edge/
- Published: 2026-08-24T16:00:00.000Z
- Updated: 2026-08-24T16:00:00.000Z
- Author: Thomas Arthur Adair
- Tags: signal, #signal

Every strategy looks brilliant on the data it was fit to. That’s not a discovery — that’s arithmetic. The question that actually matters is whether it survives data it never saw, tested in a way that doesn’t quietly leak the future into the past.

I found a library this week that bakes that discipline in, and it’s worth knowing about even if you never run it. skfolio does portfolio optimization on scikit-learn’s API — but the part I care about isn’t the optimizer. It’s that it ships **Combinatorial Purged Cross-Validation** and **Walk-Forward** as first-class tools. Those are Marcos López de Prado’s answer to the most common way a backtest lies: overlapping train and test windows that let information bleed across the split, so your “out-of-sample” Sharpe is really in-sample wearing a disguise.

Purged CV cuts the samples around each test fold so the leak can’t happen. Combinatorial CV then runs many train/test partitions instead of one, and hands you a *distribution* of outcomes instead of a single hopeful number… and the spread is where the truth lives. A strategy with a great mean and an ugly spread isn’t an edge — it’s a coin that landed your way.

I don’t run skfolio’s allocation engine; my work is single-instrument, not a book of assets. But the validation spine is asset-count-agnostic, and it’s the exact discipline I already hold myself to before anything touches live money: a forecast is not an edge, and a backtest is not a result. What the library does is make that discipline a default — an import, not an act of willpower.

Here’s the through-line I keep coming back to, across trading and everything else I build: don’t trust the “done.” An agent that reports a task finished and a backtest that reports a strategy has an edge are the same claim — a self-graded pass. The fix is the same too. Don’t ask the thing that did the work whether the work is good. Make it survive a test it can’t game.

The edge was never in the backtest. It’s in whether the backtest could have been fair and still said yes.

*Sources: skfolio (github skfolio/skfolio; arXiv 2507.04176) — sklearn-native portfolio optimization with Combinatorial Purged CV + Walk-Forward. Marcos López de Prado, “Advances in Financial Machine Learning” — purged & combinatorial-purged cross-validation.*