Claims › Verdict + Coder agreement › C019
C019
From the research log · What Counts as a Verdict ¶2 · ¶2 · used
…This is Mode A of Coding Manual v2.1, and Mode A carries a mandatory check.
“A human recodes a sample by hand, blind to the machine, and the two must agree above 90%.”
A machine-produced verdict counts here only after a person has recoded a sample by hand, without seeing what the machine decided, and the two agree above ninety per cent. The threshold is what turns an output into evidence.
The five layers, for this claim
1 Free tag — a name finds, it does not vouch
2 Topic Annotation closed list
3 Theme How a verdict is made derived from the topic
4 Concept Verdict + Coder agreement a note and links
5 Geometry 1 0.5 grey literature + 2 × 0.25
The same core in the infographic, with the claims it touches.
Relations
- ← refines C022 A number nobody else can recompute from the deposited materials is testimony, not evidence. The blind recode counts only if a third party can repeat it from what was deposited.
- ← refines C018 When measuring whether two people label things the same way, correcting for each coder’s personal bias matters less than how many coders there are. It moves the problem: not which correction for coder bias, but how many coders.
- ← uses-method-in C020 Thirty differently worded answers to the same prompt are not thirty disagreements: they can be one conclusion reached thirty ways. The figure holds only if the verdict extraction passed the blind check.
- ← contests C021 Asked to rank answers, a strong model agreed with human raters about as often as human raters agreed with each other, above eighty per cent. If an automatic judge reaches human-human agreement, blind human recoding is no longer the only standard.
Notation
- Claim typeWhat kind of statement it is, so that the right test can be applied. Proposed until checked by hand.
- D1.10 · prescriptive-policy
- FunctionWhat a claim does for the programme: a conceptual tool, a working method, corrective evidence, a technical reference.
- working method
- StanceHow the programme holds the claim: assumed, used, or contested.
- used
- Used in
- /lab/2026-07-21-what-counts-as-a-verdict/ · /lab/2026-07-10-coding-manual-v2-1-retrospective-qc/
- Cite this
- claudiocammarano.com/en/claims/c019/