Claims › Verdict + Machine judge › C021

C021

In their own words · Zheng et al. 2023 · §1 Introduction · contested

…We show that some of the biases are minor or can be mitigated.

“Once addressed, our results from 3K controlled expert votes and 3K crowdsourced human votes in the wild verify that GPT-4 judge match human evaluations at an agreement rate exceeding 80%, achieving the same level of human-human agreement (4.2, Table 4).”

Consequently, this suggests LLM-as-a-judge is a scalable method to swiftly evaluate human preference, serving as a promising alternative to traditional human evaluations.…

Asked to rank answers, a strong model agreed with human raters about as often as human raters agreed with each other, above eighty per cent. What was measured is a rate of agreement, which is not the same thing as competence to judge.

The five layers, for this claim

1 Free tag — a name finds, it does not vouch
2 Topic Evaluation closed list
3 Theme How a verdict is made derived from the topic
4 Concept Verdict + Machine judge a note and links
5 Geometry 1.5 0.75 proceedings or preprint + 1 × 0.25 × 1.5 contested

Relations

Notation

Claim typeWhat kind of statement it is, so that the right test can be applied. Proposed until checked by hand.
D1.2 · empirical-quantitative
FunctionWhat a claim does for the programme: a conceptual tool, a working method, corrective evidence, a technical reference.
corrective datum
StanceHow the programme holds the claim: assumed, used, or contested.
contested
Used in
/lab/2026-07-21-what-counts-as-a-verdict/
Cite this
claudiocammarano.com/en/claims/c021/