Skip to main content
    All cases
    Own product · Trading / research

    The honest backtester that rejected two strategies, net of cost

    One NautilusTrader engine runs the same strategy code in backtest and live, a judge trusts only out-of-sample data and the public web tool matches the Python result to the digit.

    24Tests

    Covering the OOS judge, evaluate.py

    0Strategies passed OOS
    2Cost levels tested

    0.3% and 0.4% round trip

    1 dayBrief → build
    Honest Backtester: Bitcoin hourly, trend strategy, net-of-cost result
    Bitcoin, hourly, trend (MA-cross): rejected out-of-sample at -34.66% net of cost, Sharpe below the kill threshold.
    1. 1Net result after cost: -34.66%
    2. 2Rejected out-of-sample (OOS)

    What runs automatically

    01 / BACKTEST

    Event-driven backtest

    NautilusTrader runs the identical strategy class in backtest and live, with real event-driven fills and no look-ahead, so there is no separate live version to drift out of sync with what got tested.

    02 / JUDGE

    The OOS judge

    evaluate.py enforces an out-of-sample split, walk-forward validation across six windows, kill criteria and an overfit detector, all covered by 24 passing tests before a strategy gets a verdict.

    03 / WEB

    Honest Backtester (web)

    A self-contained web tool lets anyone pick a coin and a strategy and see the net-of-cost, out-of-sample result; the browser runs its own calculation and matches the Python backtest to the digit.

    Built with Claude Code in 1 day

    Before

    Strategies got judged on the same data they were tuned on, hiding whether an edge was real.

    After

    Every verdict is net of cost and out-of-sample, published even when the answer is no.

    []/More screens
    Honest Backtester: Ethereum daily, mean-reversion strategy, a second net-of-cost result
    Ethereum, daily, mean reversion: also rejected, -43.93% net of cost with too few trades to trust.
    docs/results verdict table: both strategies rejected across all four cost runs
    The published results table: rejected on every row, net of cost.
    Under the hood

    01Context

    A backtest that cheats is worse than no backtest

    Crypto strategy projects usually ship the exciting part first, a strategy that backtests beautifully, then loses money live because the backtest never survived look-ahead, real fees or an out-of-sample test. quant-ring proves the discipline before trusting any strategy: one NautilusTrader engine runs the identical code in backtest and live, so there is no separate live version to drift out of sync with what got tested, and evaluate.py judges only on data a strategy has never seen.

    02How it works

    Walk-forward, net of cost, two fighters judged

    evaluate.py enforces an out-of-sample split, walk-forward validation across six windows, net-of-cost Sharpe, drawdown and an overfit detector, all covered by 24 tests. Two fighters went through it on two years of Binance BTCUSDT hourly data spanning a full cycle, a trend signal on an MA cross and a mean-reversion signal on a z-score, each judged at two cost levels, a 0.3% round trip and a stressed 0.4%. The public face is the Honest Backtester, a web tool where the browser runs its own calculation and matches the Python backtest to the digit.

    03Outcome

    Both fighters rejected, and that counts as a win

    Both fighters failed all four runs: the trend strategy lost 67.8% to 75.5% net of cost, mean reversion lost 52.4% to 66.7% and the published table lists every row rejected. No strategy has passed the out-of-sample bar yet, so nothing here trades real money. A negative result counts as money saved rather than money lost live. This is an internal research project, not investment advice. The next step is one new hypothesis under the same discipline, not a pile of parameter variations on the same data.

    ANM SOLUTIONS / CONTACT US

    Want something like this?

    First 30-minute call is free. We scope it, you decide.

    []/More cases