Validate LLM output before persisting

Repair generated Markdown shape with autofix, then require human-owned state before saving or rendering.

TARS: "I can reformat the checklist. I cannot truthfully check the box."

Generated Markdown is safest when it has an airlock: fix shape automatically, then refuse to persist anything whose content, attestations, or external facts still fail. This recipe compiles one contract, runs validateSyntax before autofix, then runs validateState before storage.

The contract

contracts/incident-summary.mdv.md

<!-- mdv: document noFenceWrapper noLLMResidue noTruncation -->
<!-- mdv: block required minWords=25 noPlaceholder -->
## Summary

Explain what happened, who is affected, and the current status.
<!-- mdv: endblock -->

<!-- mdv: evidenceList required minItems=2 each.falsifiable error=each.falsifiable -->
## Evidence

- Include a concrete observation that someone can verify.
<!-- mdv: endevidenceList -->

<!-- mdv: taskList required allChecked exactLabels -->
## Human Review

- [ ] Source data checked.
- [ ] Reviewer approved persistence.
<!-- mdv: endtaskList -->

noFenceWrapper, exact labels, and missing headings are shape problems. minWords, noPlaceholder, falsifiable evidence, and checked review boxes are state problems: they describe truth, not formatting. (See Evidence lists for how each.falsifiable decides whether a bullet is concrete enough to verify.)

The TypeScript workflow

src/validate-generated-markdown.ts

import { readFile } from "node:fs/promises";
import { autofix, compile } from "mediva";

type Generate = (prompt: string) => Promise<string>;

type Route =
  | { ok: true; markdown: string }
  | { ok: false; reason: "shape"; errors: unknown }
  | { ok: false; reason: "state"; errors: unknown };

export async function validateGeneratedMarkdown(input: {
  markdown: string;
  contractPath: string;
  generate: Generate;
  persist: (markdown: string) => Promise<void>;
}): Promise<Route> {
  const contract = await readFile(input.contractPath, "utf8");
  const schema = compile(contract);

  let markdown = input.markdown;
  let syntax = schema.validateSyntax(markdown);

  if (!syntax.success) {
    const repaired = await autofix({
      document: markdown,
      schema: contract,
      generate: input.generate,
      maxAttempts: 2,
    });

    if (!repaired.ok) {
      return { ok: false, reason: "shape", errors: repaired.error };
    }

    markdown = repaired.document;
    syntax = schema.validateSyntax(markdown);
  }

  if (!syntax.success) {
    return { ok: false, reason: "shape", errors: syntax.error.flatten() };
  }

  const state = schema.validateState(markdown);
  if (!state.success) {
    return { ok: false, reason: "state", errors: state.error.flatten() };
  }

  await input.persist(markdown);
  return { ok: true, markdown };
}

autofix is only called after validateSyntax fails, and its validator-in-the-loop repair uses syntax diagnostics. State failures go to a human review queue, a rejection path, or a regenerate-with-better-context path. Persist or render only after validateState passes.

Variations

Require a linked tracker item with a block issueKeyword=Fixes issueState=open — issueKeyword checks the Fixes #N reference is present, and issueState=open (the rule that consumes --context / context.issues) checks the issue is actually open.
Add  when the generated document uses reference links or footnotes that must resolve.
When the model is asked for a code answer, gate the fence itself. A code section can require a real language and a properly closed fence — models love to emit a bare ``` or forget the closing fence:
```

## Patch

```ts
export const fixed = true;
```

```
lang=required rejects a fence with no language (use lang=ts to pin one exactly, or lang.oneof=ts,js,py for a set); codeFenceClosed catches the truncated block where the model dropped the closing ```.

Validate LLM output before persisting

The contract

The TypeScript workflow

Variations

On this page