Home / Work / harness-go

harness-go — an autonomous multi-agent coding pipeline that actually ships.

Proctor plans. Coders implement in isolated git-worktree generations. Reviewers score and reject. The pipeline maintains opinionated memory across sessions and hot-reloads policy without dropping in-flight work.

Role
Architect · author
Status
Active · Nerds Run maintains
Stack
Go · NATS · BadgerDB · Bleve
Evaluated on
128GB Strix Halo · Vulkan

01 · Thesis

A model given one prompt and told to just produce code has no separate step that catches its own mistakes — it plans, writes, and ships in a single uninterruptible pass, and whatever comes out is what you get. harness-go takes a different bet: a long-running pipeline where planning, implementation, and review are separate roles with separate memories and separate failure modes. The pipeline can reject its own output, regenerate, and learn across sessions. That's the part that lets it run on a real codebase instead of a demo.

The claimAn agent pipeline that can reject its own output, keep policy hot, and maintain cross-session memory is categorically different from one that can't. The difference shows up in the eval, not the demo.

02 · Architecture

Three agent roles — Proctor, Coder, Reviewer — communicate over NATS JetStream. Each generation N produces 3–5 candidate implementations in isolated git worktrees, scored and either merged or rejected. Rejection feeds back into the Proctor with structured rationale, not an unguided retry.

harness-go multi-agent pipelineThe harness-go autonomous coding pipeline. Work enters from Linear issues, Slack commands and GitHub webhooks. A Proctor plans, decomposes and assigns each task. Successive generations of coder agents run in a pool of parallel git worktrees producing candidate changes. A Reviewer scores each candidate and either merges it or rejects it, and a rejection returns to the Proctor with structured rationale rather than an unguided retry. Underneath sits a durable substrate: NATS JetStream for at-least-once work queues and acknowledgements, BadgerDB as a key-value LSM store, Bleve for full-text recall, and OpenTelemetry for traces, metrics and logs. Local models run on 128GB hardware; the memory and index layer is embedded with no network dependency of its own, though the pipeline as a whole still reaches Linear, Slack, GitHub and, when a task calls for it, frontier-model APIs. // HARNESS-GO · AUTONOMOUS MULTI-AGENT CODING PIPELINE LINEAR issue ingest SLACK ops · commands GITHUB webhooks PROCTOR plans · decomposes · assigns GENERATION N worktree pool · parallel candidates CODER A git-wt-0 CODER B git-wt-1 CODER C git-wt-2 CODER D git-wt-3 CODER E git-wt-4 CODER F git-wt-5 spec · ctx REVIEWER scores · rejects · merges ↺ reject · regenerate ORCHESTRATION + MEMORY durable substrate NATS JetStream work · events · ack BadgerDB kv · lsm store Bleve full-text recall OpenTelemetry traces · metrics · logs Hot-reload policy · atomic pointer swap Compaction window · safe reconfiguration GITHUB PR merged · signed LINEAR closed · notified CI / DEPLOY gated · observed * every signal enters the same durable queueLOCAL + FRONTIER MODELS · 128GB STRIX HALO · EVAL HARNESS · EMBEDDED MEMORY, NO NETWORK DEP · AUDITABLE

Why NATS JetStream

We need durable, ordered, replayable work queues with consumer-group semantics, and JetStream gives us that without the operational weight of Kafka. Delivery is at-least-once: a NAK or a missed ack redelivers a message up to a configured MaxDeliver. JetStream's own role there stops at the count — it tracks delivery attempts and can emit a MAX_DELIVERIES advisory event once a message exhausts them, but it does not forward anything anywhere on its own. The move to a dead-letter subject is our consumer's own handler: on the final delivery attempt it republishes the work item to a dead-letter subject and terminates the original, and a separate DLQ consumer picks it up from there. An explicit ack reports successful handling back to the server, not merely receipt — and even that alone doesn't buy exactly-once execution, any more than looking a work item up does: a lookup just confirms the item still exists, and neither rules out a duplicate external effect. Idempotent handling of a redelivered “generation produced valid candidate / needs regeneration” event is a separate, explicit check keyed off the work item, applied at the application layer before a Coder or Reviewer step runs.

03 · Memory model

Two stores, one purpose:

  • BadgerDB — key-value, LSM-backed, for structured episodic memory: run artifacts, decision trees, per-agent scratchpads.
  • Bleve — full-text, for lexical recall across sessions: keyword and phrase search that answers “have we tried this refactor before? what happened?” against prior runs that used similar words. It's standard-analyzer full-text search, not embedding-based semantic similarity.

Both stores are embedded in the same Go binary, with no network dependency of their own — that matters for local-model evaluation where we're running on one box. The pipeline around them still reaches out over the network, for Linear, Slack, GitHub, and any frontier-model calls a task needs.

04 · Hot-reload policy via atomic pointer swap

Policy (scoring weights, reviewer thresholds, tool allow-lists) is a struct behind an atomic.Pointer. Updates build a new policy graph, validate it, then swap the pointer during a safe compaction window. In-flight work holds references to the old policy until it completes; new work reads the new policy. No drain, no stop-the-world.

Why this mattersLive-tuning a running agent pipeline is the difference between “useful R&D toy” and “thing we iterate on during a client engagement.” Restarting to change a threshold is not a serious operating model.

05 · Evaluation harness

The Reviewer scores each generation's candidates against a fixed internal task suite — bug fixes, small features, refactors, test authoring — and we don't have a public, reproducible version of that scoring to publish numbers from yet, so we're not asserting a task count or a frontier-vs-local comparison here. What we can point to is the hardware and the throughput we've actually measured on it: local weights run on our 128GB Strix Halo box, mostly via llama.cpp/Vulkan, with several profiles running on ROCm/HIP as well. A gfx1151 ROCm hang specific to Qwen 3.5 models pushed that one model family to Vulkan (ROCm/ROCm#6027, filed 10 March 2026, closed 1 April 2026) — see the measured throughput write-up for the numbers and the backends they ran on.

06 · Takeaway

Three things came out of building harness-go that are shaping how we scope AI-systems engagements: (1) agent pipelines need a way to reject their own output or they're just slower autocompletes; (2) cross-session memory is underrated and underbuilt; (3) local models on serious hardware are increasingly viable for real coding work, which matters for organizations that can't send prompts to a third party.

Let's talk

Tell us what you're trying to ship —
we'll tell you what it looks like to get there.

We take one or two new engagements per quarter. No published pricing — every engagement is scope- and duration-dependent, so pricing is a conversation. Bring the problem. We'll bring the questions.

Start a conversation abanna@nerds.run