ruvnet/meta-harness-architecture.md

## meta-harness-architecture.md

      
    Raw
  

              meta-harness-architecture.md
            
          
    Freeze the model, evolve the harness — Agentic-Flow's move to a meta-harness architecture

A plain-language explainer of where Agentic-Flow is headed and why.
The one-sentence version

Most AI tooling gets better by reaching for a bigger model. Agentic-Flow is betting on a different lever: keep the model the same and make everything around it smarter. That "everything around it" is called the harness, and a system whose job is to build and improve harnesses is a meta-harness.
First, what's a "harness"?

When people say "an AI agent," they usually picture the model (Claude, GPT, a local model). But the model is only the engine. The harness is the car built around it:

Planner — how a task gets broken into steps.
Context builder — which files/snippets the model actually sees.
Reviewer — how the model's output gets checked before it's trusted.
Retry policy — what happens when something fails.
Tool policy — which tools/commands are allowed, and in what order.
Memory — what's worth remembering for next time.
Score policy — how "did this go well?" is measured.

Two systems can use the exact same model and get wildly different results, purely because one has a better harness.
So what's a "meta-harness"?

A meta-harness is a runtime whose product is the harness itself. It does four things:

Routes each request to the right model.
Evolves the harness (and repairs code) so results improve over time.
Orchestrates agents, tools, memory, and swarms on top.
Verifies every change with a safety gate and signed provenance.

Why this is the right bet

It's measured, not marketing. Across recent work (the DRACO/Darwin findings behind the @metaharness/* packages):

A cheap model in a good, self-improving harness can match a frontier model — at a fraction of the cost.
The biggest, most reliable gains come from changing the harness, not swapping the model.

If the harness is the lever, then the most valuable thing a runtime can do is operate and improve that harness well. That's the meta-harness.
The four pillars, with concrete examples

🧭 Route — pick the cheapest model that's good enough

Instead of sending everything to your most expensive model, learn from your own evaluation logs which model is "good enough" for each kind of request, and route accordingly.
import { CostOptimalRouter } from 'agentic-flow/router/cost-optimal';

const router = CostOptimalRouter.fromDataset(myEvalLogs, {
  'anthropic/claude-haiku-4.5': 1,    // $/Mtok
  'anthropic/claude-sonnet-4.5': 3,
  'anthropic/claude-opus-4': 15,
}, { qualityBar: 0.8 });
// → routes each query to the cheapest model predicted to clear 0.8
Measured: ~28.5% cheaper than always using the top model, while ~98% of answers still clear the quality bar. Routing decision takes microseconds.
🧬 Evolve — improve the harness and fix code automatically

"Freeze the model, evolve the harness": the system mutates one part of the harness at a time, tests it in a safe sandbox, and keeps only changes that measurably help.
# Hand it a repo; it repairs it, gated by the repo's own tests
npx agentic-flow-repair ./your-repo --generations 3
# Deterministic dry-run, no Docker:
npx agentic-flow-repair ./your-repo --mock
The sandbox is shell-free and runs with a scrubbed environment (no secrets/tokens leak to a variant) — safety is part of the design, not an afterthought.
🤝 Orchestrate — run real work on top

The familiar Agentic-Flow capabilities — 66 specialized agents, 200+ MCP tools, persistent memory, and multi-agent swarms — are "what the harness runs."
🔏 Verify — trust what shipped

Every harness change passes a frozen scorer + safety gate. You can also sign your agent/harness config and later verify it hasn't been tampered with:
import { generateKeyPairPem, signFiles, verifySignedManifest } from 'agentic-flow/harness/provenance';

const { publicKey, privateKey } = generateKeyPairPem();
const signed = signFiles(['.claude/agents/coder.md', 'harness/policy.json'], privateKey, publicKey);
// later, on a consumer machine:
const report = verifySignedManifest(signed);
// → { signatureValid, filesIntact, drift: [...] }  (Ed25519, built on Node crypto)
What's real today (agentic-flow@2.1.0)


✅ Cost-optimal routing (agentic-flow/router/cost-optimal)
✅ Autonomous repair engine + CLI (agentic-flow-repair, agentic-flow/repair)
✅ Harness MCP tools (harness_repair / harness_manifest / harness_verify)
✅ Ed25519 provenance (agentic-flow/harness/provenance)
✅ Repositioned around the four pillars (recorded in ADR-073/074/075/076)

Honest caveat: the in-package repair engine is fully working and tested. The headline SWE-bench-Lite "Test-Driven Repair" product numbers (~58–68% of real issues fixed for pennies) come from the upstream @metaharness/darwin Docker harness — that's the documented deployment path, not bundled into agentic-flow.
Where it's heading


Route: turn live usage into routing training data automatically; native FastGRNN routing by default.
Evolve: evolve agentic-flow's own agent policies against its benchmark suite; wire the full SWE-bench TDR path.
Verify: harness verify as a CI/pre-publish gate, with key-management guidance.
Docs: an end-to-end "build your own meta-harness" guide.

Links


npm: agentic-flow · @metaharness/router · @metaharness/darwin
Decision records: docs/adr/ADR-073…076 in ruvnet/agentic-flow


The short version, one more time: don't just buy a bigger engine — build a better car, and let it keep improving itself.
No results found