Quant
Parameter plateaus, not peaks
Every optimizer does the same thing: it hands you the settings that scored highest. That sounds like its job, and it's precisely the problem — the highest score is where the noise happened to be kindest. The most useful habit in strategy research is to stop reading the top row of the results table and start looking at the shape of the surface around it.
Peak or plateau
Why the shape is the signal
Here's the reasoning that makes this more than a rule of thumb. A 13-day lookback and a 14-day lookback are nearly the same strategy. They see almost the same data and take almost the same trades. If a real effect exists, both should capture most of it — the results have to be similar, because the strategies are similar.
So when 14 prints money and 13 and 15 both lose, that's not a parameter working. It's a statement that your result flips sign under a change that barely touches the logic — which means the result was never about the logic. It was about which specific trades happened to land inside the window on this specific slice of history. Change the history and the magic number moves somewhere else.
A plateau makes the opposite statement: a whole neighbourhood of nearby strategies all work. Noise struggles to fake that, because it would have to be kind to all of them in a coordinated way. The smoothness of the surface is evidence; its maximum is not.
A real edge works at 13, 14 and 15 days, because those are basically the same idea. If only 14 works, you didn't find a setting — you found the one spot where luck happened to pile up.
How to look for one
- Plot the surface, always. A heatmap over two parameters tells you more in one glance than any table sorted by return. You are looking for a region, not a cell.
- Pick the centre of the plateau, not the peak. Even if the peak sits at the edge of the good region, take the middle — it has the most room to be wrong in.
- Expect to give up backtest performance. The centre always scores below the peak. That gap is your honesty tax, and paying it is the point.
- Fewer parameters, fewer hiding places. Every added parameter multiplies the search space and gives noise another dimension to produce a spike in — see the multiple-testing trap, since a 20×20 grid is 400 trials.
- Check the plateau moves as little as possible. Re-run on a different period, or a correlated market. A plateau that shifts wholesale between samples is a slower-moving illusion.
The uncomfortable corollary
If the surface is all spikes — no smooth region anywhere — the honest read isn't "pick the best spike." It's that there's no edge here at any setting, and the optimizer is just reporting the shape of this dataset's noise. That's a real result. It saves you the money you'd have spent finding out live.
This is the same instinct as everything else in honest testing: prefer results that are robust over results that are large. It's why walk-forward re-fits on each window and expects consistency, why Monte Carlo shuffles the trades to see if the edge survives a different order, and why the strategy you actually deploy should always be a little less impressive than the one your optimizer fell in love with. Read the backtest for shape, not for maximums.