JW Lab

If it can be verified, it can be looped

Any research process whose output can be checked can be turned into a loop that improves the process by which it improves — recursive self-improvement. I think most hard problems are better attacked that way than head-on, because a loop compounds and a direct attack does not. So these are notes on building one: what to automate, what to leave to judgment, and how to keep its conclusions checkable.

The logistic map, fed its own output. Left to right: how hard each step pushes. Reload — the seed changes, the picture does not.

Loop for Quant Research

Failure modes we encountered while building RSI for quantitative research — and what each one taught us.

  1. 2026-08-10 · Loop for Quant Research Deterministic and Probabilistic

    Which of a system's decisions are allowed to be a sample from a distribution? Here, zero of the fifty-eight files that run the pipeline call a model. The interesting part is the failure that leaves: a gate documented as doing something it does not do.

    agent design · gates · the honest failure

  2. 2026-08-10 · Loop for Quant Research Fat Skill, Thin Multi-Agents

    Two thousand lines of resident procedure against eighty-five lines of orchestration, and an ablation showing the procedure is worth +2.4 on a ten-point judgment task. Meanwhile the fan-out's diversity guarantee is a syntax check that a competent fan-out satisfies while producing copies.

    skills · multi-agent · information hiding

  3. 2026-08-10 · Loop for Quant Research Overfit and Dynamic

    You can only build a verifiable target out of the past, and the thing the past describes keeps moving. Following that mismatch to its conclusion arrives at a slime mould — and then at the exact point where a borrowed mechanism stops working, because a maze never becomes obsolete.

    search · stigmergy · nonstationarity

  4. 2026-08-10 · Loop for Quant Research Abstraction Layer, but Think First

    A search that records what failed learns nothing that transfers; it has to record why a kind of thing fails. But the component that generalises will assert a shared structure between any two findings, fluently. Four tests that break a bad one — and nothing mechanical that runs them.

    ontology · falsifiers · what is not enforced

  5. 2026-08-10 · Loop for Quant Research Look-Ahead Bias and Path Dependency

    How do you test a program when you cannot state what its output should be? You assert a relation between two runs instead of a value from one. That is metamorphic testing, and the first version of it here passed a program written to cheat.

    backtesting · integrity · blind spots