Docs
Run the gate yourself
Install #
git clone --depth 1 https://github.com/ben06kn-droid/observable-garden pip install -e . # Python 3.10 or later
One minute #
quixote example
This audits a bundled example, a scripted search on a small synthetic panel, and prints the verdict. Every verdict on this site comes with the command that reproduces it. For example:
quixote audit examples/logs/scripted_session.json --panel examples/data/tiny_panel.npz --class examples/class.json --B 2000 --seed 0 --json
The panel format #
A panel is a single .npz file (see quixote/panel.py). quixote run, quixote audit and
quixote sancho all read it.
| array | shape | meaning |
|---|---|---|
returns | (T, M) | simple returns; NaN = not tradable in that row |
features | (T, M, K) | what a rule may use in row t (data up to row t only) |
feature_names | (K,) | names of the K features |
periods_per_year | scalar | 252 for daily rows |
delay | scalar, default 1 | features at row t earn returns[t + 1 + delay] |
cost_bps | scalar, default 5 | one-way cost per unit of turnover |
borrow_bps_yr | scalar, default 50 | yearly borrow cost on short weight |
row_labels, asset_names | (T,), (M,) | optional; split accepts a row label |
A rule's book is dollar-neutral with gross exposure 1 over the tradable assets. quixote.panel.save(...) writes
the file. Sancho reads the same file. It builds its return-structure inputs from returns,
treats features as its 40 registered features only when K is 40, and needs a few years of rows: with the default
levers, its first scored row is 763.
What a verdict means #
| PASS | p is below α: the submitted rule beats what the search would find on noise. |
|---|---|
| FAIL | p is at or above α, and the test had the power to tell: the result is what this search finds by luck. |
| INADMISSIBLE | p is at or above α, and the search was too wide to certify anything. |
| UNDECIDABLE | the record alone cannot license a correction. |
| DEGENERATE | the test statistic broke on this menu. Do not read its p. |
The code reports a pass as CERTIFIED; this site shows it as PASS.
How this site runs it #
Nothing is computed on a server. Your browser downloads Python (Pyodide 314.0.7), numpy, pandas and the
observable-garden package, built from the repository's own source. All are served from this site, with their hashes
recorded. A Web Worker then runs quixote.audit.audit_log, the function behind quixote audit.
Two functions that the package normally takes from compiled libraries are replaced in the browser by ports, which tests
hold equal to the originals: the Politis–White block length (from arch) and the normal distribution
functions (from scipy). An equivalence test runs the browser build against the command line on every
example.
The record on a result #
The audit id is the SHA-256 of a JSON object with the keys B, alpha,
class_sha256, log_sha256, panel_sha256, seed and wheel,
written in code-point order with no spaces. It covers the hashes of the inputs, the replicate count, the seed, α and the
exact package build. Two runs with the same audit id test the same search on the same data with the same code.
Sancho's config hash is different: it is the hash of its lever file, shown in the
Sancho section and in the viewer.
The paper: The Observable Garden: fixed and adaptive candidate sets under the DSR #
Knaster (2026). On SSRN: papers.ssrn.com/sol3/papers.cfm?abstract_id=7616559.
A backtest is usually the best of many attempts, so its Sharpe ratio has to be judged against what the best of many attempts would find on pure noise. When the search is adaptive, testing against the list of strategies it happened to try gives the wrong error rate. A class of strategies declared before the search, or a log of every decision that can be re-run on noise, restores a valid test. Quixote is the gate built on that answer.
Where it stands #
The gate kept its error rate in every registered test. Its limit is power. On real data it refused nearly everything, including 240 LLM agent sessions on liquid ETFs. It issued one certificate, on a long history of industry portfolios, and that certificate's lower bound held over the six years that followed. This is evidence that the gate works, not a source of trading signals.
Tracing a claim #
Every claim in the paper maps to its registration, result and read in research/NOTE_MAP.md. The archived release: doi:10.5281/zenodo.23332377.
Licence #
The paper is licensed CC BY; the code is MIT.