Conformance scenarios
Language-neutral cases any mediva implementation can run: a contract, the documents checked against it, and the verdict each must get.
A scenario pins what mediva says about a document: a contract, the documents checked against it, and
the verdict each document gets, written as Markdown data. Any implementation can run them. One
runner, scenarios., turns every case into a test, so pinning a behaviour needs no
TypeScript.
A scenario file
A file is one behaviour (checklists/). Its # heading names the behaviour. Each case
is a heading written as the input, never "should":
# Bare ticks and evidence
## A ticked item with nothing to show for it is a bare tick
```mdv.md
<!-- mdv: section required -->
## Verification
<!-- mdv: list items.none=bare-tick -->
- [ ] Ran the suite
<!-- mdv: endlist -->
```
```md
## Verification
- [x] Ran the suite
```
```expect
exit 1 at state
error state tick-without-evidence@3 in `Verification > list[1]` by `items.none=bare-tick`
```
```json readout
{
"fields": [
{
"block": "Verification > list[1]",
"kind": "list",
"present": true,
"empty": false,
"items": [{"label": "Ran the suite", "line": 3, "ticked": true, "evidence": null, "disposition": "claimed", "reason": null}]
},
{"block": "Verification", "kind": "section", "present": true, "empty": false}
]
}
```
### A note nested under the tick is its evidence
```md
## Verification
- [x] Ran the suite
- 412 passed, 0 failed on CI run 8814
```
```expect
exit 0
```
```json readout
{
"fields": [
{
"block": "Verification > list[1]",
"kind": "list",
"present": true,
"empty": false,
"items": [{"label": "Ran the suite", "line": 3, "ticked": true, "evidence": "412 passed, 0 failed on CI run 8814", "disposition": "done", "reason": null}]
},
{"block": "Verification", "kind": "section", "present": true, "empty": false}
]
}
```The second case is a ###: it reuses the contract above it.
The blocks
| Block (info string) | What it is |
|---|---|
mdv. | The contract, for its own case and every case after it, until the next one. Contract: [name](path/ on a line of its own reads the contract from a file instead. |
md | The document. A heading whose section holds one is a case. |
expect | The verdict. The first line is exit <n>, plus at <gate> when a gate failed (lint, shape, state, could-not-run). Then one line per diagnostic, in the order mediva reports them: <severity> <gate> <code>@<line>, then in and the block, and by and the constraint, each as a code span, when the diagnostic names them. The message is not pinned. |
json readout | The readout: what the document claims. It is there whenever shape passed, and is then pinned. |
<lang> file=<path> | A file beside the contract: a source (team.), a parent contract (base.), or a change set (changes., passed as --change-set-file). In the section that sets the contract, it goes with the contract. In a case's own section, it belongs to that case. |
context | What the host supplies: now: <ISO date> (the clock; default 2026-10-06T00:00:00.) and title: <text> (a PR or issue title). |
md fixed | mdv fix's exact output for the document. Fixing that output again must change nothing. |
Any other text is commentary. A block is fenced with more backticks than any fence inside it. A document always ends with exactly one newline.
Running
bun test tests/scenarios # check every case
SCENARIOS=update bun test tests/scenarios # write every expect, readout and fixed block from a real runTo pin a new case, write the heading, the contract and the document, then run update mode and read
the diff before you commit it. A failure names the file, the line and the case, and prints expected
beside actual (~ a changed line, - expected only, + actual only).
A case runs through the public API (compile(contract).). A case that needs the
host runs through mdv check -f json instead, in a temp directory: one with files, or one whose
contract composes others (extends=, include, override). The CLI is what reads files. That
the CLI and the API agree in general is not this suite's job.
Coverage
Every live diagnostic code needs at least one scenario that expects it. bun run lint:testing
enforces this (scripts/). Its baseline lists the codes that have
no scenario yet. The baseline can only shrink.
Core specification
The normative mediva core: data model, directives, targets, props, sources, and diagnostics.
Markdown Crimes
How often generated Markdown goes wrong, and why a static check misses it. A variance-aware benchmark of structured-Markdown failure modes in cheap, fast language models: method, results and how they were produced.