Backtesting Learning Guide

1. Begin with a testable hypothesis

Write the market idea before looking for the most profitable settings. A hypothesis should explain the behaviour you expect, the conditions under which it might occur and what observation would count against it. “Buy when RSI is low” is incomplete. A better research statement identifies the instrument universe, timeframe, entry threshold, confirmation, exit rule and risk boundary.

Predefining the question reduces the temptation to keep changing parameters until the past looks attractive. Record the original rule and every material revision. If a revision is inspired by the test result, it belongs to a new round of research and should be validated on data that did not guide that revision.

2. Use data that matches the question

Intraday and daily strategies need different histories. Check whether the data includes the intended trading session, whether timestamps mark the start or end of a candle, and whether corporate actions are adjusted consistently. Missing bars, symbol changes, illiquid periods and current-only instrument lists can all change results.

Survivorship bias appears when a test uses only securities that exist today and quietly excludes companies that were delisted, merged or failed. Look-ahead bias appears when the calculation uses information that was not actually available at the simulated decision time. Both can make a weak strategy appear stronger than it was.

3. Define entries without ambiguity

Specify whether a condition is checked during a candle or only after it closes. Define “crosses above” differently from “is above”: a cross normally requires the previous observation to be on the other side. State how multiple conditions combine and which timeframe each condition uses. If a daily trend filter and a five-minute entry are combined, confirm that the daily value available at that moment does not include the unfinished day.

A scanner match is not automatically a trade recommendation. It only means the selected data satisfied the selected expression at the evaluation point. Liquidity, spread, news, position sizing and portfolio exposure still require separate consideration.

4. Define exits and risk before measuring profit

An entry rule without an exit model cannot describe a strategy. Document signal exits, stop-losses, profit targets, maximum holding periods and what happens when multiple exit conditions occur in the same candle. With candle data alone, the exact sequence of the high and low is often unknown. Any assumed intrabar path can affect which level is treated as reached first.

Position size determines how a price move becomes a portfolio gain or loss. Compare percentage risk per position, total simultaneous exposure and the effect of correlated holdings. A strategy with a high win rate can still lose money if its occasional losses are much larger than its gains.

5. Model trading frictions

Gross returns are not spendable returns. Real trading may include brokerage, exchange fees, securities transaction tax, stamp duty, bid-ask spread, slippage, market impact and rejected or partial orders. Small theoretical edges are especially vulnerable to these costs. Test more conservative assumptions than the best case and observe whether the conclusion changes.

Liquidity is time-dependent. A quantity that looks small compared with daily volume may still be difficult to execute at a particular moment. Historical candle volume does not guarantee that the desired quantity was available at the modelled price.

6. Read the result as a collection of risks

Net profit alone hides important information. Review trade count, average gain and loss, maximum drawdown, time spent underwater, exposure, streaks and the distribution of results across instruments and periods. A small number of exceptional trades may account for most of the return. Removing one unusually favourable event is a useful sensitivity check.

Drawdown is the decline from a previous equity peak. It is not a fixed prediction of the worst future loss; a live drawdown can exceed the historical one. Profit factor compares gross gains with gross losses, but it becomes unstable with few trades. Win rate says nothing about the size of wins and losses. Each metric needs context.

7. Guard against overfitting

Overfitting occurs when a rule captures historical noise rather than a repeatable effect. Warning signs include many adjustable parameters, performance concentrated around one precise setting, repeated optimization on the same sample and a large gap between training and later results.

Prefer broad, stable regions over a single perfect parameter. Split data into an in-sample period used to develop the idea and an out-of-sample period reserved for evaluation. Walk-forward tests can repeat that separation through time. The reserved sample must remain genuinely unused until the rule is settled, otherwise it gradually becomes part of the training process.

8. Compare market regimes

Markets change between trends, ranges, high volatility, low volatility, liquid periods and stressed periods. Break results into meaningful subperiods rather than relying only on a combined curve. Ask whether the underlying reason for the rule still makes sense and whether the strategy depends on a regime that may not return soon.

Testing more instruments can help, but it does not automatically make observations independent. Stocks in the same sector may respond to the same event. A result repeated across related securities can still represent one concentrated market exposure.

9. Paper trade as an operational check

Paper trading can reveal timing, data and workflow mistakes before capital is involved. It is useful for checking whether signals arrive when expected, whether the rule can be followed consistently and whether the assumed instruments are actually available. It still cannot reproduce the emotional pressure, fills, slippage or liquidity of live trading.

Keep a dated log of the signal, the information available at that time, the intended action and the eventual outcome. Do not rewrite an ambiguous rule after seeing what would have worked; resolve the ambiguity prospectively.

10. A responsible research checklist

• Can another person implement the rule from the written description without guessing?
• Were all indicators calculated only from information available at the decision time?
• Does the data cover weak as well as favourable market conditions?
• Are costs, spread, slippage and position size represented conservatively?
• Was an untouched sample reserved for validation?
• Is performance reasonably stable across nearby parameters and subperiods?
• Would the conclusion survive the removal of the best trade or period?
• Have data gaps, corporate actions and survivorship limitations been reviewed?
• Is the maximum tolerable loss defined independently of historical profit?