2026-08-10 · Loop for Quant Research · 2

Fat Skill, Thin Multi-Agents

The easy claim is that one thick skill beats a swarm. The evidence in this system says something narrower and more useful: fan out for information hiding, never for throughput. Everywhere a subagent genuinely earned its keep here, the reason was that a fresh context cannot see something the main context cannot un-see. Everywhere I fanned out for volume, I got correlated work and paid for it.

The loop, and the question about it

Some background, then the question. The loop behind this note hunts for stock-market trading strategies with nobody supervising it. Each pass — a trial — writes an economic hypothesis, writes the code for it, replays sixteen years of market history against that code (a backtest), and records what it learned. A script decides pass or fail, and the model never sees the score it is being graded on. Roughly a thousand trials to a campaign, about four per cent kept.

This note is about one design question inside that: the model's side of the loop has to hold a great deal of accumulated method — what to read first, what counts as evidence, which mistakes to avoid — and there are two places to put it. You can write it down as resident instruction that one agent reads every iteration. Or you can distribute it across subagents, each with its own brief and its own context.

The two are not interchangeable, and the usual argument for the second one is the weakest available. Parallelism is sold on throughput: ten agents, ten times the work. But throughput is rarely the binding constraint in a research loop — the loop is bounded by how much it learns per attempt, not by how many attempts it can issue per hour. Optimising the wrong term buys you failures faster.

So the interesting question is what else a subagent could be for, and whether that thing is worth its cost. This note is what I found by building it the expensive way round first.

The proportions

Two skills carry the research procedure. The one that generates the next hypothesis and writes the strategy file is 532 lines. The one that interprets the backtest and writes the reflection is 353. Around them sit the loop's step-by-step procedure file and the domain document describing the backtester's API and its correctness invariants.

That ratio was not planned. It is what a year of trimming the fan-out and growing the skills converged to.

What "fat" means here

It is not length. A 532-line skill spent describing the API would be a manual. What makes the proposal skill fat is that judgment is pre-loaded — decisions a competent agent would otherwise re-derive from scratch, badly and differently, every iteration.

It fixes the read-set as literal shell commands. The agent does not decide what context it needs; it runs the commands and reads what comes back. Improvised orientation is the largest source of run-to-run variance I have seen in an agent loop, because the agent's sense of "what matters here" is itself sampled.

It defines a closed verdict vocabulary and makes some verdicts binding. Each hypothesis in the running catalogue carries a status. A thesis marked dead — thesis is ceiling or dead — refuted is a hard self-block: the next proposal may not continue that thesis. live — expression leaky keeps it open, because the idea might be right and the code wrong. suspect — no story means do not refine it — refining something you cannot explain is score-chasing with extra steps. Four words, and they constrain the next move more tightly than any amount of encouragement to "be rigorous".

It names a specific failure mode and pre-declares the defence. The skill tells the model, in the second person, that its own spontaneous ideas come from its training prior, that the prior over-represents famous published anomalies, and that an anomaly published fifteen years ago has been arbitraged for fifteen years. The gate kills those correctly; each one still costs a full batch. Two counter-rules follow, both defined as file reads rather than things to remember — "not a counter to track in your head — read it off the file." The same boundary as the previous note, applied inside the prompt.

It bakes a worked failure incident into a procedure step. One step exists because a specific past trial went wrong in a specific way, and the skill says so. It is justified rather than asserted, which matters because an agent that understands why a step exists will apply it to the case its literal wording does not cover.

It partitions the code surface explicitly. The 73-line template the agent fills in carries a comment marking the surrounding skeleton as correctness invariants ONLY — the book above is yours; this plumbing is not. Freedom is scoped, in-line, at the point of temptation.

It repeats one clause verbatim in three places, because examples degrade into menus. The clause is: the parenthetical is an example, not a menu — "because it is simple", "because it is common", "because it was in the example" are not reasons. Written once, it is decoration. Written at each of the three points where the agent is about to pick off a list, it changes behaviour.

And the skill states what it deliberately does not mechanise. On when to stop refining an idea, it says plainly that this gate is judgment, not a counter: no refinement-depth ceiling and no "did the score improve enough?" threshold, because either would collapse the loop into variant-climbing. A fat skill is not one that automates everything. It is one that has decided, per decision, which side of the line that decision sits on.

The analysis skill, and the moment it protects

The interpretation skill is shorter and, I think, the better piece of writing, because almost all of it is about one moment. It forces a read order first: re-read your own code before looking at any metric, so the interpretation is grounded in what the code does rather than reverse-engineered from the Sharpe ratio. Reading the score first contaminates everything after it; the agent will find a story that fits the number, and it will be a good story.

Then it names the highest-value moment in the whole loop — the moment when what you expected and what you observed disagree — and pre-labels two of the three available responses as errors. Rejecting the observation ("noise") loses the discovery. Rejecting the prior ("the literature is wrong") throws away accumulated knowledge. The remaining move, the one it marks correct, is to assume both are true and go looking for the distinction you did not know existed. Almost every genuinely new thing this system has learned came out of that third branch, and it is not the branch a model takes by default, because the first two are cheaper and both feel like resolution.

Against self-deception it gives a concrete test rather than an exhortation: write the falsifiable sentence. If you cannot produce a sentence of the form "if this were merely repackaged momentum, this specific number would look different, and it does not" — then the story is not pinned. Mark it overfit-suspect and do not refine it. Worked examples of good and bad output follow, the bad ones labelled "black box" and "fabricated" — more instructive than any rubric.

It also argues against its own extension. Any internal confidence score, it points out, even one you later collapse to a boolean, becomes an optimisation target — so there is no scale. A story is pinned or it is not.

Free prose, five hard fields

None of this would survive contact with a loop if the output were prose all the way down. The reflection is free text, but five fields in it are validated by a script at commit time.

Validation failure refuses the commit outright. And exactly one structured bit — that boolean — feeds a deterministic downstream policy that forces the following iteration onto a different data family. One bit of judgment, escaping into machinery that acts on it without further interpretation, is the whole interface between the thinking third of the system and the counting two-thirds. Its narrowness is deliberate.

Does the resident procedure actually do anything?

Everything above is a description of a design. It is not evidence that the design works, and a 24:1 ratio is only impressive if the 2,009 lines are carrying something. So the documents were ablated.

The test approximates one iteration's judgment as a selection problem. Rewind a real run to a point in its history, hand over only the records that existed at that moment, and show forty candidate strategies that were in fact tried next — as code only, comments stripped. Then ask where to spend ten backtest slots. The answer key is what those forty candidates actually scored when they were run, so the metric is: how many of your ten picks cleared the gate. Same model in every arm. The arms differ only in which documents the model was given.

The two n's are different things and it is worth saying which is which. The 55 and 44 in the figure are repetitions: five rewind points in one terrain and four in the other, eleven runs at each. That is nine points in total, and the test is over those nine — not over the 297.

Read the p-values against what such a test can produce. Nine paired differences give 2^9 = 512 sign assignments, so the smallest attainable p is 1/512 ≈ 0.002 and 0.004 is two of them. That is the floor, not a coincidence, and it is the honest ceiling on how strong this evidence can be: nine timepoints is a small paired design, and three p-values are reported here without any correction for reporting three.

It was also expensive. Three conditions over the two terrains at 55 and 44 repetitions each is 297 graded runs, every one of them a full judgment over forty candidates on a frontier model. That is why this was run once as a set-piece rather than standing as a regression check — and it is most of the reason local models are now being evaluated for the grading arm. A check you cannot afford to repeat is a measurement, not a guardrail.

Two things in the result are worth separating. The plan is the larger single step, and the plan is mostly definitions — what counts as a pass, what the bar is, what the loop is for. The rejection ledger is a record of what has already failed, and it adds a further step on top of a model that already knows the objective. Knowing the target helps more than knowing the history; knowing the history still helps, on top.

The gap between the panels is the more interesting half. Where the code alone carries signal, the bare model already starts well above chance and the documents add less. Where it does not, the bare model sits at chance and the documents carry nearly all of the lift. Resident procedure is not a uniform multiplier on capability. It is worth the most exactly where the problem is least legible from the artefact in front of you — which is also where an unassisted agent's confidence is least warranted.

The one place it fans out

There is a workflow for taking a single hypothesis and covering it broadly. One agent decomposes the thesis into N orthogonal expression angles, ten by default. Then N drafters are spawned, each shown only its own angle and not the others — blind to each other by construction. Two JSON schemas constrain both tiers, and the drafters are told plainly that a draft violating the code contract is discarded.

The workflow's own description forbids the obvious use of it: this is coverage, not a tournament — never pick a winner to chase. The procedure file repeats it from the other side: write one synthesis reflection covering all the drafts, never crown a single winner. I wrote that twice because the pull towards treating ten parallel results as a leaderboard is strong, and a leaderboard over ten samples of one idea is a machine for manufacturing overfit.

Note how little of the pipeline is actually parallel. Decomposition is one agent. Only drafting fans out. Execution is strictly sequential — the runner's comment says sequential execution is the concurrency barrier and is therefore CPU-safe, and that drafting was the only thing parallelised. Synthesis is one reflection; the thesis is registered exactly once. The swarm is a thin band in an otherwise serial process.

And its central guarantee does not hold

The fan-out's value rests entirely on the drafts differing from each other, and the system checks that with the duplicate gate. The duplicate gate parses only the scoring function. It never looks at the portfolio-construction function.

So drafts that differ only in how the book is built collapse as clones before execution. They consume the batch's slots and produce nothing. The methodology log records this five separate times across roughly 1,180 trials: first as an observation, then twice more, then promoted to a validated lesson after three independent recurrences — and then it claimed another victim on a case that had been explicitly declared in advance. Writing the lesson down did not stop the lesson recurring.

That is a fixable bug. The next fact is not.

One batch produced four structurally different syntax trees — a median threshold, a descending rank count, a log-z cutoff, a rank fraction — that were bet-identical. The log's summary is the sharpest sentence in the whole methodology record: syntax-tree variation defeats the duplicate gate but not the bet. Elsewhere, five drafts from one batch all landed at residual correlation between 0.986 and 0.997 against already-banked strategies.

So the fan-out's diversity is enforced by a syntactic check, and a competent fan-out satisfies that check while producing ten expressions of one idea. Diversity of expression is not diversity of bet, and only one of the two is measurable before you have spent the compute. You can improve the check; you cannot make it into the thing you wanted, because the thing you wanted — are these ten bets economically distinct? — is the research question itself.

Where subagents earn their keep

Here is the turn. The system asks for subagents in three other places, all outside the fan-out, and every one gives the same justification — which has nothing to do with work per hour.

Verifying your own draft. The skill's reasoning: you are anchored on your own draft, and a verifier that has not seen your reasoning catches what you cannot. It then restricts what the verifier gets — the code file, the plan, the one-sentence claim, and explicitly not the proposer's reasoning. The restriction is the mechanism. A verifier handed the justification will verify the justification.

Refuting your own story. After the causal explanation is written, a refutation agent is spawned with a fresh context, on the stated grounds that a same-context re-read shares its blind spots. Its brief is adversarial by design: find the strongest fact-based refutation, attack rather than assess plausibility. "Does this seem reasonable?" returns yes.

Escaping the shared prior. When the search must go outside what the model already knows, several agents are dispatched at different angles — because, in the skill's words, the web is the only participant that does not share your weights.

None of those three is about throughput. All three construct a reader who cannot see what you saw — something a single agent cannot do for itself at any context size, and the real, narrow case for multi-agent design. The fan-out is the one place this system uses parallelism for volume, and also the one place the parallelism does not reliably deliver what it promises.

Where the fat skill is straining

The post would be dishonest if it stopped there. The fat skill is visibly failing at four things a swarm would handle.

532 lines cannot stop the model re-treading ground. Avoidance of explored territory was deliberately left to judgment rather than mechanised. The log records the model licensing an experiment on the grounds that the question had not yet been measured, when it had been measured about a hundred trials earlier — and the new results landed at residual correlation 1.0 against the old ones. Judgment over a long history degrades in a way a lookup does not. The build-order lesson above recurred five times for the same reason.

A cheap critic would have paid for itself many times over. A dedicated pre-flight critic whose only job was to ask "does any draft in this batch differ solely in portfolio construction?" would have cost about one cheap call per batch and saved at least fifteen named drafts. That is a subagent used for neither throughput nor information hiding, but for attention — a check nobody in the main context is going to run, because everyone in the main context is busy.

The swarm created integration debt in single-agent machinery. A component that maintained the search coverage map processed only the last committed result, since it was written when there was only ever one. Batch drafts bypassed it entirely. Adding parallelism to a serial system is never only an addition.

And the sharpest one, which cuts both ways. The 85-line thin workflow carried a stale description of the backtester's API for months, instructing drafters to write code shapes the linter rejects — a mistake the 532-line fat skill had already corrected. Thin components drift precisely because nobody reads them; there is no daily pressure keeping them true. That argues for consolidation. It also shows what consolidation costs: the fat skill is now a single point of truth with no redundancy, and nothing would catch it being wrong either.

The rule, restated

The denominator all of this runs against: one campaign ran 1,203 trials and banked 48 strategies. Four per cent. Most of what such a search does is fail informatively, and the design question is only ever how much you learn per failure.

So: put procedure where it is read every time, and put agents where a second pair of eyes is otherwise impossible to obtain. The ratio here ended up at 24:1 not because thick prompts beat orchestration, but because most of what I wanted from orchestration — consistency, accumulated judgment, not repeating yesterday's mistake — turned out to be procedure, and procedure is cheaper resident than distributed. What was left was small, and all of it had one shape: a reader who has not seen what I have seen.

If you are about to spawn ten agents, the question worth asking first is not whether they will be faster. It is what each of them is prevented from knowing. If the answer is nothing, you have bought ten copies of one opinion and a bill.