hiveloom

Apache-2.0 · Python 3.11+ · v0.5.0

Build durable agent harnesses so smaller models can perform repeatable, verifiable tasks.

A model is only one part of an agent. Tools, context, loop policy, guardrails, and verification often decide whether the same model succeeds or fails. hiveloom makes that surrounding system a self-contained folder that can be validated, versioned, run anywhere, measured, and deliberately improved.

uv add hiveloom

or pip install hiveloom

Five-minute quickstart View on GitHub

Your harness is the product

The hive is the collective memory of runs. The loom turns task intent and that evidence into an improvable harness.

A durable artifact

harness.yaml and its code hooks replace fragile, conversation-only setup.

One validated construction path

CLI edits and model-generated plans use the same transactional API; invalid changes roll back.

Closed-loop evidence

Every run produces a version-hashed trace, so stats can show whether a change improved success, cost, or turns.

Safety outside the model

Cost limits, tool allowlists, redaction, verification, and frozen evolution fields are enforced in code.

Open and portable

Builtins, extension packs, custom providers, and MCP tools share one runtime contract. A harness stays a normal folder.

Agent-facing by default

mcp serve exposes each harness as an MCP tool, carrying its measured success rate so an agent can delegate on evidence.

Measured, not assumed

Three checked-in evaluations ship with their harnesses, scoring code, and committed results. They answer different questions — and the answers are not the same.

Task success by model, raw versus the same model inside a hiveloom harness
Same model, same prompt, same tool — the only difference is the harness. Claude Haiku 4.5 goes from 3% to 65%.
article-extractor · 32 live URLs × 3 epochs (96 runs per arm)
Model / arm Task success Hallucination Cost per success
Claude Haiku 4.5, raw3%96%$0.2212
Claude Haiku 4.5 + hiveloom65%11%$0.0186
Claude Sonnet 5, raw baseline100%0%$0.0073
Qwen 3 4B, raw / harness58% / 69%19% / 16%local
Qwen 3.6 35B, raw / harness75% / 84%16% / 0%local
Gemma 4 12B, raw / harness92% / 90%1% / 0%local

Read the whole table before generalising. Haiku's is the only delta that survives a paired test over the 32 URLs (p < 0.0001). The three local models move by 10 points or less — noise at this sample size — and Gemma is slightly worse harnessed. Scaffolding rescues a model that cannot hold the output contract. It does not improve one that already can, and it did not beat raw Sonnet.

On frontier models, it buys the tail

In article-digest, both arms run through hiveloom with the same prompt, tool, and loop policy — the raw arms only drop the validators. Opus 5 and Sonnet 5 each go from 80% to 100% success at roughly unchanged cost. Nothing was fabricated either way; what the raw arms lost was the contract.

And a wrong answer arrives labelled

page-audit makes a complete answer unreachable from the tool alone. Raw Opus 5 was silently wrong 5 of 6 times. Harnessed, it was silently wrong 0 of 6 — it either recovered on retry or exited verify_failed. That is the property a downstream automation can build on.

Full results, sample sizes, and caveats →

Five minutes, start to evidence

Every mutation is validated and rolls back on error. The first two commands need no API key at all. 

# Explore the contract without an API call
hiveloom schema --annotated
hiveloom catalog tools

# Construct a harness; every mutation validates and rolls back on error
hiveloom init ./summarizer --name summarizer \
  --task "Summarize a text file into JSON."
hiveloom add tool --builtin file_read --dir ./summarizer
hiveloom add validator --builtin regex_match --pattern '"summary"' \
  --dir ./summarizer
hiveloom validate ./summarizer --json

# Assemble the first call without contacting the model
hiveloom run ./summarizer --input notes.txt --dry-run --json

# Run for real — or point it at any other lab
export ANTHROPIC_API_KEY=sk-...
hiveloom run ./summarizer --input notes.txt --json
hiveloom set model openai/gpt-4.1-mini --dir ./summarizer

# Inspect evidence and propose a gated improvement after failures
hiveloom stats ./summarizer --json
hiveloom evolve ./summarizer --propose --json

A harness is a folder

my-harness/
├── harness.yaml          # declarative runtime contract
├── tools/                # optional code tools
├── validators/           # task-specific verification
├── schemas/
├── skills/
├── .hiveloom/traces/     # append-only run memory
├── .env.example
└── requirements.txt

Safety invariants

  • Evolution cannot change guardrails, model, logging.redact, extensions, hooks, mcp_servers, or evolution.auto_propose.
  • The cost guardrail defaults on at $1.00.
  • The shell tool is disabled unless explicitly configured, and stays allowlist-only.
  • Redaction runs before trace persistence.
  • Foreign harness code is trust-gated before loading.

Prefer model-driven construction? hiveloom generate "Summarize a text file into JSON." -o ./summarizer

Read the documentation

docs.hiveloom.cloud github.com/FrancescoMrn/hiveloom

Harness spec Architecture Models and providers Extending Deploying and evolving Control plane