Lessons
Our notebook · Lesson 15

How we test honestly

It is easy to make any method look good on past charts. These rules stop us from fooling ourselves.

Two words first

WordSimple meaningDollar example
RThe money risked on one trade, from entry to stop.Our rule bots risk 0.5% of $10,000, so 1R = $50. A trade at +2R made $100. A trade at −1R lost $50.
EdgeThe average result per trade over many trades.+0.1R = +$5 a trade on average. That is our minimum before we pay for a challenge.

A low win rate can still be fine. With a 2R target, winning 1 trade in 3 is break-even.

Risk $50 a trade, so 1R = $502R target, 1 win in 3: break-even$0$0−1R−$50+1R+$50+2R+$100Trade 1−$50Trade 2−$50Trade 3+$100−$50 − $50 + $100 = $0 (before spread)
Results are measured in R. On the right: two losses and one 2R win add up to $0, so a 2R target needs 1 win in 3 to break even, before spread.Teaching diagram, not real market data

Our five rules

  1. Pick on 2025, judge on 2026. We choose the rules on 2025 prices only, then test them on 2026, which they have never "seen". Only the 2026 number counts.
  2. Strict accounting. We pay the spread on every trade. If one candle touches both the stop and the target, we count a loss.
  3. Testing many ideas is a trap. If you try 100 versions, one looks good by luck. So we also look at the average of all versions, not only the best one.
  4. 30+ trades before judging. Under 30 trades, the result is mostly luck. Losing 3 in a row is normal, even for a good method.
  5. Don't change rules after a few bad trades. That is how you fool yourself.
Quick check

Our rule bots risk $50 a trade, so 1R = $50. What does a trade at +2R make?

A method looks great on 2025 prices. Which number counts?