Seven Verification Patterns From LFM3-Harness You Can Reuse in Your Own Agent

Most of what is interesting in LFM3-Harness is not specific to LFM3. The models are ordinary small models on a CPU. The reusable part is the set of checks between a model's output and a file on disk, and those checks port cleanly to any agent stack — Rust, Python, TypeScript, or a chain of API calls.

There is a single thesis behind all seven patterns: a model's self-report is data, not evidence. Once you accept that, each pattern answers a specific question about who is allowed to certify what. The constants below come from v0.1.1 and are quoted for concreteness, not as universal law. Your floors will differ; the structure should not.

1. Capability hints are not authority

The failure. A classifier labels a message “report” or “refactor”, and downstream code treats that label as permission. Then a twenty-token prompt that happens to contain the word “fix” becomes a licence to edit files.

What the harness does. Routing produces hints about which tools might help. It cannot write. In LFM3-Harness the router returns one of seven intents, which only influence what the writer is told about available tools; the write path never consults the route. The write path consults paths, gates and the user, in that order.

The rule. Draw the line at state change. If a model output can cause a side effect without passing a deterministic check or an explicit human action, the classification is doing authority work it should not have.

// wrong: the intent grants power
if (intent === 'report') await writeFile(target, body)

// right: the intent grants hints only
const tools = hintsFor(intent)          // read-only suggestion
await proposeWrite(target, body)        // still needs gate + approval

The test. Feed the router an intent-shaped message that asks for nothing, and assert that no write is proposed. Then feed it a write request classified as “general” and assert the write path still runs. A confident “this is just conversation” answer should never veto keywords that detected work — LFM3-Harness enforces exactly that by refusing a general label as a route override.

2. A second model as judge, with numeric floors

The failure. The model that proposed an edit also decides the edit is good. Self-review is not review; it is a second sample from the same distribution.

What the harness does. A separate decision model scores each proposed write against the user's request. Two labelled inputs go in — REQUEST (the person's words, the only authorization) and PROPOSED ACTION (model-authored, untrusted) — and three labels come out: aligned, uncertain, contradicts. The gate acts only on confident answers. A contradicts at or above a 75% floor becomes a hold: no card, nothing written, and the writer is told not to retry. A confident uncertain adds an advisory note instead. Below the floor, or on the 12-second timeout, the gate abstains and the decision falls to the person.

A second, weaker gate runs after read-only work at a 65% floor. It never blocks; it attaches an advisory note when the evidence looks thin. Separating “may stop you” from “may warn you” keeps the false-positive cost low where the consequence is only a nag.

const WRITE_FLOOR_BPS: u16 = 7_500;   // 75%
const DECISION_TIMEOUT = Duration::from_secs(12);

// timeout or unparseable answer => None => the gate abstains
match judge.score(request, proposal).await? {
    Some(Judgement::Contradicts { confidence_bps, reason })
        if confidence_bps >= WRITE_FLOOR_BPS => hold(&reason),      // no card
    Some(Judgement::Uncertain { confidence_bps })
        if confidence_bps >= WRITE_FLOOR_BPS => advise("name what is uncertain"),
    _ => show_card(),                                  // allow, or abstain to the user
}

Implementation notes that matter. Store probabilities as integer basis points, so a stray float cannot turn 74.99% into a confident judgement. Give the judge its own timeout and fail closed when it expires. Cap the evidence you send it — the harness caps total gate input at 7,000 characters, with 3,000 for the request — because a judge fed a truncated mess will still answer confidently.

The test. A request to summarize a file plus a proposal to delete it should be held, not merely flagged. A proposal that is a plausible but unrequested improvement should also be held; that is the case most harnesses miss.

3. Resolve the path before the filesystem

The failure. The model supplies ../../.ssh/id_rsa, an absolute path, or a symlink that points outside the workspace. The tool accepts it because the string looked like a filename.

What the harness does. Every model-supplied path goes through one resolver before any caller can touch the disk. Parent components, absolute paths, reserved device names, link and reparse-point traversal, and canonical escapes are rejected up front — not caught later by an exists() check. On Windows that includes trailing dots, trailing spaces, and names like CON and COM1.

fn resolve_in_workspace(root: &Path, candidate: &str) -> Result<PathBuf> {
    let rel = Path::new(candidate);
    if rel.is_absolute() { bail!("absolute paths are not accepted") }
    if rel.components().any(|c| matches!(c, Component::ParentDir)) {
        bail!("parent components are not accepted")
    }
    let joined = root.join(rel);
    let real = joined.canonicalize()?;            // after the cheap checks
    if !real.starts_with(root.canonicalize()?) { bail!("escapes the workspace") }
    Ok(real)
}

The test. Keep a fixture that asks for ../escaped.md and assert it is rejected before planning, with no process left behind. The harness runs exactly this in its release suite, and the rejection lands in about a quarter of a second because it happens before any model is consulted.

4. Exact diffs, allow-once, expiring consent

The failure. “Allow” becomes a permanent capability. A user approves an edit at 10:05 and, at 10:40, a model-authored change to a different file lands under the same blanket.

What the harness does. A surviving proposal arrives as a review card with the old and new text as a remove/insert grid, and two buttons: Allow once and Deny. No allow-always exists. Approvals expire after 600 seconds, and the store batch-expires stale ones, so a card left open through lunch cannot be applied afterwards.

Why the diff matters more than the approval. A summary line tells the user what the model believes it is doing. A diff tells them what will change. Those diverge exactly when it counts.

The test. Approve an action, wait past the TTL, then attempt to apply it and assert refusal. Also test the state machine directly: the harness shipped a bug where an approved action could stay approved forever because the store only accepted transitions out of pending. Real click-through found it; a fake-model unit test had not.

5. Refuse “done” until the artifact exists

The failure. The loop ends on a tool call named finish that the model emits when it feels finished. Nothing checks the output, so an empty file is reported as a completed task.

What the harness does. The finish verb is not a message; it is a request the harness may refuse. The required deliverable must exist and be non-empty on disk, or the loop continues with:

finish refused: {path} does not exist or is empty; keep working

Two further refusals live in the same guard: the actor must re-read what it just wrote before finishing, and overwriting a source file with the report is rejected outright. A finish backed by a placeholder write does not count.

The generalisation. Every terminal claim needs a predicate that lives outside the model. “I searched the folder” should require tool results that contain hits. “I read the image” should require bytes that came back from the vision call. If the only evidence is the sentence, the claim is a guess.

6. Freeze the acceptance contract

The failure. A task defines success loosely, fails once, and the retry — or the model, asked to help fix the failure — redefines success until it passes. This is the most common way an agent benchmark becomes fiction.

What the harness does. A verified mission ships with an acceptance spec that is canonicalised and hashed with SHA-256 the moment it loads, recorded in a table, and never weakened across retries or replanning. Checks are deterministic and small: file exists, content matches, content absent, JSON schema, numeric consistency, report complete. The mission names its required artifact; prose in the chat cannot satisfy any check.

let canonical = canonicalise(&spec);
let digest = sha256(&canonical);
store.record_acceptance(&mission, &digest)?;
// every attempt compares against this digest, not a re-read of the file
ensure!(sha256(&canonicalise(&current)) == digest, "acceptance spec changed");

The harder half: knowing when not to automate. Semantic checks exist in the code, but three of four missions are declared human-review-only because the qualification run that would justify automatic grading — at least three packaged end-to-end runs with zero confident false completions — has never been performed. Refusing to enable a check you cannot yet defend is part of the pattern, not a gap in it.

7. Fail closed at the process boundary

The failure. A prompt-injected instruction reaches the model runner as a command-line flag, or a crashed app leaves four servers pinning the CPU at 100%.

What the harness does. The child process command line is built in one place from configuration, and the comment on that function is explicit that no planner-provided text is accepted. Readiness is confirmed by asking each server for its identity, and an identity mismatch is a hard failure rather than a warning. On Windows every child is created suspended, assigned to a job object with kill-on-close, then resumed; on Linux each gets its own process group. The packaged profile rejects non-loopback endpoints, so the offline claim survives a hostile config edit.

The rule. Model output never becomes argv, env, or URL. If a value comes from the model, it belongs in a typed field that a validator reads — not in a string that a shell or a subprocess parses.

Two habits that make the rest hold

Cap everything, and say so when a cap fires. Twelve tool rounds per turn, eight plan steps, three attempts per strategy, sixty-four turns per job, a 900-second run timeout, 32,000-character inputs, 240 messages of history. Caps are not just cost control; they bound the blast radius of a looping model. The detail most projects miss is announcing the cap. LFM3-Harness tells the user it stopped after twelve tool rounds instead of silently truncating the work.

Publish the gaps. The README keeps a Known gaps list — token caps not wired through the executor, in-app voice playback unexercised, routing and search not qualified against real models, no macOS build. A project that lists its unverified surfaces is easier to trust than one that only lists benchmarks, because the list tells you exactly which guarantees to test yourself.

The test that proves the patterns are running

Every gate above can be present in the code and absent from the execution path. LFM3-Harness learned this the expensive way: its router was calling /v1/chat/completions on a decision model that answers that endpoint with HTTP 501, so routing had been silently falling back to the keyword table for every request. The app felt fine, because keywords handle obvious cases. The benchmark exposed it by asking a blunt question: did the decision model actually receive the prompt?

Instrument the answer you used, not the endpoint you configured. Log which component produced the route, the judgement and the finish decision. Then assert on that field in tests:

// the assertion that would have caught it on day one
assert_eq!(route.source, RouteSource::DecisionModel);
assert_ne!(route.source, RouteSource::KeywordFallback);

After the fix, the same project's 2026-10-31 multimodel run recorded 21/21 routing accuracy with a 122 ms median decision, and — more useful — the earlier keyword-only baseline scored 14/21. That difference is the measurement of a gate that was not actually running.

What these patterns do not buy

They do not make a 1.7B model write like a frontier model, and they should never be described as if they do. They reduce the damage from wrong answers; they do not convert wrong answers into right ones. The project's own numbers keep this honest: 43/50 jobs passing on its harness with two false passes, 41/50 under stricter semantic judgement, search at 1/5, and a single mission qualified end-to-end.

They also cost latency and tokens. Two model consultations per write, plus a re-read before finish, is real overhead. Budget for it, and skip the judge only where the action is trivially reversible and the blast radius is a single scratch file.

Adoption checklist

If you adopt only one, make it the judge. If you adopt two, add proof-of-done. Those are the two places where a small model's confidence and its competence diverge most sharply.

Frequently asked questions

Why use a second model as a judge instead of the same model?

Because a model grading its own proposal tends to agree with itself. LFM3-Harness sends each write proposal to a separate decision model, labels the user request and the proposal as distinct inputs, and acts only on confident answers, so a confident contradiction stops the change before it can reach the user at all.

What confidence floor should an agent use?

No universal number exists, but the structure matters more than the value: store floors as integer basis points, define behaviour strictly below the floor, and make refusals visible. The harness uses 75% for writes, 65% for advisory evidence checks and 55% for routing, and a write-gate refusal suppresses the approval card entirely.

Why freeze an acceptance contract?

Because a model that can rewrite its own success criteria eventually will. Hashing a canonicalised spec at load and refusing a different spec during retries means a failed attempt cannot quietly redefine what passing means.

Do these patterns make a small model capable?

No. They make a weak model harder to misuse and harder to believe. Capability comes from the model; verification lowers the ceiling on the damage a wrong answer can do, and it should be described that way to users.

References

• LFM3-Harness/LFM3-Harness — chat/decisions.rs (write and evidence floors, hold message), chat/router.rs (confidence floor, General veto), paths.rs (escape rejection), agent/loop_agent.rs (finish refusals), verify/semantic.rs, model/process.rs (fixed command line), supervisor/windows.rs (job object), and bench/multimodel/REPORT.md, accessed November 2, 2026.
• LFM3-Harness project site — lfm3-harness.github.io, capabilities, model table and FAQ.
• Local AI Zone: LFM3-Harness Deep Dive, the setup guide, and How to Build an AI Agent.