How to backtest a stock scan, no code needed

Every scan is a claim. "RSI crossing above 30 in an uptrend" claims that such stocks tend to rise afterward. Run the scan without testing the claim and you are trading folklore; test it and you are trading evidence. The difference costs about ten minutes.

This is the method, whether you use the built-in replay here or a spreadsheet and patience.

Step 1: State the claim precisely

"Finds good stocks" is not testable. "Stocks matching this scan outperform over the next 10 sessions" is. Fix three things before looking at any results: the exact conditions, the holding horizon (a swing scan tested on 6-month returns tells you nothing), and what counts as success — beating zero? beating the index?

Step 2: Replay, don't remember

For each trading day in your test window: run the scan as of that day, record the matches, record their forward return over your horizon. Every scan page on this site does this automatically for the past year — the hit-rate panel on, say, the golden cross is exactly this replay. What matters is that the signal list for each historical day was computed from data available on that day. Which brings us to the ways this silently goes wrong.

Step 3: Know the three result-fakers

Survivorship bias. Testing only on stocks that exist today flatters every strategy — the delisted losers are precisely the stocks your scan would have matched on the way down. The dataset here keeps them; a hand-built test in a spreadsheet usually cannot.

Lookahead bias. Any condition computed with information from after the signal date — an adjusted price series that didn't exist yet, a "top 500 by turnover" list drawn from today's rankings. Subtle versions of this inflate results by a few percent, which is more than most real edges.

Overfitting. Tune thresholds until last year looks perfect and you have memorised last year, not learned a pattern. The tell is fragility: if rsi < 32 tests beautifully and rsi < 30 tests poorly, you have curve-fit noise. Real effects are threshold-tolerant.

Step 4: Read hit rate and payoff as a pair

A 45% hit rate with winners twice the size of losers is a good system; a 70% hit rate that gives it all back on the losers is a famous way to bleed. Neither number means anything alone. When you replay a volume breakout scan, the question is not "how often was it green?" but "what did the average win pay, against the average loss?" — and whether enough signals exist per month for the edge to matter in practice.

Step 5: Respect the regime

One year of replay is one market regime. A momentum scan tested across a trending year will flatter itself; the same scan in a sideways year pays whipsaw tax. Read results as "how this signal behaved in this regime," check the failure clusters — do losses bunch in specific months? — and prefer scans whose edge is explainable over ones that merely test well. RSI oversold turning up in an uptrend has a mechanism (one-sidedness resolving in favourable ground); a mechanism is what you fall back on when the recent numbers wobble.

Make it a habit

The workflow that compounds: build the scan in the query language, or start from a preset; replay it before running it live; keep it only if the claim survives; retest when the regime changes. No code at any step — the replay panel does the bookkeeping. The scans that survive this process are few, and they are yours in a way no copied scan ever is: you know why you run them, and you know what their failure looks like before it arrives.