Take any strategy, add enough adjustable parameters, and you can make its backtest spectacular. This is not skill; it is memorization. The optimizer stops learning the market's structure and starts learning the historical data's noise — and noise does not repeat. This failure mode, overfitting, is the central occupational disease of quantitative research.
The smell tests
Overfit strategies share a look. Performance that collapses when a parameter moves slightly — 10-period works brilliantly, 9 and 11 lose money — is noise-fitting; real edges live on plateaus, not pins. Too many rules is another tell: each added condition (only on Tuesdays, only above the 63-bar average…) is another opportunity to memorize the past. So is in-sample perfection: an equity curve that never suffers is describing history, not capturing structure. And any edge that exists on exactly one market and one timeframe, but nowhere adjacent, deserves deep suspicion.
Practices that keep you honest
Hold data back. Tune on one segment of history, then verify on a segment the optimization never saw; the out-of-sample result is the only one you are allowed to believe. Prefer parameter plateaus — if performance is decent across a broad range of values, pick the middle of the range, not the historical peak. Count your degrees of freedom and treat every parameter as a cost that must justify itself. Test the same logic on related markets and neighboring timeframes; robustness across contexts is the best cheap evidence of a real effect.
Finally, stress the sequencing: reshuffle the trade list with Monte Carlo resampling and confirm the strategy's viability doesn't depend on the lucky order in which history happened to deal its trades.
Optimize the idea, not the number
The healthiest mindset shift is this: optimization's job is not to maximize the backtest return, it is to discover whether the idea is stable. If honest testing shows a modest but robust edge, you have something worth risk-managing. If it shows a spectacular but fragile one, you have a beautiful description of the past. The market pays for the first and punishes the second — though nothing, including robust testing, guarantees future results. Educational content, not advice.