How we test

Rules before results

A backtest can be made to flatter almost anything, so the rules come first. Each test’s pass rules are written and sealed with a digital fingerprint before a single result is read, and each test is judged only once.

Most ideas stop at judgement. That is the point of judging them.

Process

How a strategy earns its place

Every idea takes the same seven steps, in this order. Most stop long before the end.

  1. 1

    Data

    Prices, corporate actions, filings and quotes, each dated to the moment the market first saw it. A quarterly result counts from the second it was published, not from the quarter it describes.

  2. 2

    Features

    Signals built only from what was known at the time, each set named, versioned and frozen before it is tested.

  3. 3

    Models

    Gradient-boosted trees, neural networks and ensembles, trained only on history that had finished, and trained again on a fixed schedule.

  4. 4

    Rulebook

    Pass rules written and sealed before the first result exists. A rule changed afterwards makes a new attempt, never a better result.

  5. 5

    Judgement

    One look at the result, against a bar that rises with every attempt. The verdict goes in the run ledger, passed or failed.

  6. 6

    PaperThe A6 family is here

    A paper account at live prices, to watch the strategy behave in the real market.

  7. 7

    Own capital

    Real orders, sent with the owner’s capital, and only by the owner.

Rulebook

Sealed before the first result

The pass rules, the code that judges and the data it reads are each fingerprinted before a test runs. Change one byte and the fingerprint no longer matches.

Rulebook

SHA-256

Pass rules
Version 3
Sealed
1 Nov 2026, 01:16 IST
Registered first
A test without a sealed registration is screening, and screening can never pass.
One look
Each test is judged once. Its verdict stands, whatever it says.
New rules, new version
Rules change only forward, as a new version with its own seal. The old ones stay as they were written.

Passing

What a pass needs

A strategy passes only when every gate in its sealed rules holds at once.

Beat its own list

After every cost, the book must beat the average stock of the very list it chose from, by a margin no lucky attempt could reach at the current count.

Big enough to matter

A minimum size sits beside the statistics, because a tiny edge can look significant over enough years.

Add something

A blend of models must add something its simplest member does not, or the simpler one is kept.

Hold up lately

The edge must hold in recent years too, not only on average across the decades.

Standards

Nine standards, every test

These hold for every test, in every segment, whatever it is testing.

Every attempt counted

The pass bar rises with every attempt the research makes, so a lucky result cannot pass as skill. Failures count too, and closely related tries count by how alike they are.

No hindsight

Each decision uses only what was known at that moment, and the models learn only from history that had already finished. A month’s stock list is fixed before the month begins.

Failures kept in

Delisted and failed companies stay in every test until their last trade, exactly as they traded.

Every cost paid

Brokerage, taxes, exchange fees, stamp duty and slippage are charged on every simulated order, at the account’s real size. Old years pay old tax rates, old ticks and wider spreads.

The market’s rise removed

Results are measured against the very stocks a strategy could have chosen, month by month, so a rising market is never mistaken for skill.

Luck tested from every side

Random rankings run through the same engine as a placebo, shuffled data and random portfolios check that a result is not a fluke, and a book started on another day shows how much the start date alone moves it.

Audits judged again

When an audit finds a fault in the data or the code, we fix it, retrain every model from scratch and judge again under the same rules. The new verdict goes in the ledger beside the old one.

Untouched years

A strategy is judged again on years it was never judged on, with models trained only on the years before, back to NSE’s first files of 1994.

Sealed forward tests

Some tests are sealed with their rules and code frozen, then judged only on months that had not happened yet.