Stop paying for the same reasoning twice

The MacroKit Pattern in one sitting: why your agent re-derives an identical plan on every request, what it costs you, and the compiler-shaped fix that let a 7B local model hold 94.5% routing accuracy at roughly 1/5000 of the price.

The bill you are actually paying

Most production agents work like this: a request arrives, a frontier model reads the prompt stack, reasons its way to a plan, executes, and forgets everything. The next, nearly identical request pays for all of that reasoning again. If a workflow runs a thousand times a day, you are buying the same chain of thought a thousand times — at frontier prices, roughly $0.50 a run for the workloads we measured.

An analogy you already trust: the compiler

Nobody re-compiles a program every time they run it. You compile once — slow, expensive, done by heavy machinery — and then the binary runs cheaply, forever, on modest hardware. MacroKit applies exactly that split to agents:

• Design time = compile time. A frontier model works through the task once and its reasoning is captured as a macro — an explicit, replayable encoding of the workflow;
• Runtime = execution. A small local model (7B-class) does two cheap jobs: route the incoming request to the right macro, and replay it — filling in the request-specific details.

What the numbers said

On a 100-task intent-routing benchmark, a 7B local model replaying distilled macros reached 94.5% routing accuracy with zero structural failures — at about $0.0001 per run versus ~$0.50 with a frontier model in the loop: a ~5000× cost gap on the workloads measured.

Where it works — and where it honestly doesn't

The pattern earns its keep on narrow, recurring workflows: the same shape of task arriving over and over, where a frontier model's reasoning can be captured once and replayed with different parameters. That is precisely why the SDK ships a distillation gate — a static analyzer that flags which workflows are worth encoding as macros and, just as importantly, which are not.

Try it / verify it

Everything above is checkable — the benchmark, the SDK, and the macro format are public.