Claims › Measurement artefact + Inverse scaling › C024

C024

In their own words · Srivastava et al. 2023 · §3.4.2 · assumed

“Breakthrough behavior is consistent with the model suddenly gaining new skills in an abrupt way. Careful analysis of task behavior, however, suggests that the underlying change in model capabilities is generally more smooth.”

These smooth improvements can be exposed by decomposing tasks in ways that better capture partial progress. We describe some of those ways in the remainder of this section.…

Abilities that look as though they switch on all at once with scale often improve smoothly underneath: the jump belongs to an all-or-nothing metric. Change the metric and the cliff turns back into a slope.

The five layers, for this claim

1 Free tag Srivastava a name finds, it does not vouch
2 Topic Evaluation closed list
3 Theme How a verdict is made derived from the topic
4 Concept Measurement artefact + Inverse scaling a note and links
5 Geometry 1.25 1 peer-reviewed + 1 × 0.25
SrivastavaSrivastavaEvaluationEvaluationHow a verdict is madeHow a verdict is madeMeasurement artefactMeasurement artefactAlso: Inverse scalingC024C024 · Abilities that look as though they switch on all at once with scale often improve smoothly underneath: the jump belongs to an all-or-nothing metric.

The same core in the infographic, with the claims it touches.

Relations

No relation to another claim has been declared yet.

Notation

Claim typeWhat kind of statement it is, so that the right test can be applied. Proposed until checked by hand.
D1.2 · empirical-quantitative
FunctionWhat a claim does for the programme: a conceptual tool, a working method, corrective evidence, a technical reference.
corrective datum
StanceHow the programme holds the claim: assumed, used, or contested.
assumed
Used in
/lab/2026-02-25-horizontal-validation-31-claims/
Cite this
claudiocammarano.com/en/claims/c024/