Claims
A scientific claim is not a headline: it withholds itself until you have the context, and no two readers arrive with the same context. So this is not a list to scroll — it is 15 questions, and the claims come through them.
These are not claims in general. Every one of them is on file because this programme stands on it, borrows a method from it, or has to answer to it. Nothing is here for completeness, and the archive is not trying to cover a field: it is showing what its own work rests on, including the parts that push back.
The claims, cored
Every claim is a core drilled through the five layers of ENTIA, from the least to the most exposed to error. Read down a column for one claim, across a band for one layer; the arcs underneath are the relations between claims.
Read it
Point at a column, or tap it, to read one claim through its five layers. Each mark says what kind of warrant the claim lives on: • observed, ∴ derived, ∃ what there is, ! what should be done. Green comes from the literature, bordeaux from the research log; a ring means the programme contests it.
What a machine cannot be said to do
Meaning
- C007Form–meaning gapAgent modelThe dispute over machine understanding is not one question but two pictures of what understanding is: tracking statistical regularities, or holding a causal model of how things work.Mitchell & Krakauer 2023 · Mitchell · Krakauer
- C008Form–meaning gapA system trained only on the shape of language has nothing in its input that could tell it what the language is about.Bender & Koller 2020 · Bender · Koller
- C006Agent modelPredicting the next word can require the model to build a partial picture of what the writer believed and wanted.Andreas 2022 · Andreas
Claim types
- C001Claim typeBefore asking whether a statement is true, you have to settle what kind of statement it is: the test that fits a measurement does not fit a value judgement.Building the Taxonomy ¶4 · Cammarano
- C046Claim typeConceptual frameworks are made, not given, and can be rebuilt critically.Lakatos 1978
- C002Claim typeTwo claims of different kinds share no standard of truth, so a procedure that works on one returns a meaningless verdict on the other.Building the Taxonomy ¶4
- C003Claim typeOne dimension of the taxonomy decides what a claim is; the other six describe how it behaves once that is settled.Building the Taxonomy ¶3 · Cammarano
- C004Claim typeA measurement, an experiment and an argument are accountable to different things, so they cannot be asked to share one document and one standard of proof.What Counts as One Paper ¶10
- C005Claim typeMeasurement artefactTwo independent findings pointing the same way is not evidence that they are connected.What Counts as Introspection ¶11
Introspection
- C009Epistemic self-classificationThe programme used to assert flatly that the producing system performs no metacognition over its own outputs.What Counts as Introspection ¶6
- C010Epistemic self-classificationThe premise the whole framework rests on, in its narrow form: a model may represent that a passage is in Spanish, or that it is being evaluated, without representing that the assertion it is about to make is causal rather than correlational.What Counts as Introspection ¶8 · Cammarano
- C011Epistemic self-classificationWhat a model reports about its own workings is not a log of them.What Counts as Introspection ¶10
How a verdict is made
Annotation
- C019VerdictCoder agreementA machine-produced verdict counts here only after a person has recoded a sample by hand, without seeing what the machine decided, and the two agree above ninety per cent.What Counts as a Verdict ¶2
- C022VerdictDeclared incompletenessA number nobody else can recompute from the deposited materials is testimony, not evidence.Coding Manual v2.1 ¶2
- C018Coder agreementWhen measuring whether two people label things the same way, correcting for each coder’s personal bias matters less than how many coders there are.Artstein & Poesio 2008 · Artstein · Poesio
- C020Coder agreementMeasurement artefactConvergent pathsThirty differently worded answers to the same prompt are not thirty disagreements: they can be one conclusion reached thirty ways.Vertical Validation ¶11
Evaluation
- C021VerdictMachine judgeAsked to rank answers, a strong model agreed with human raters about as often as human raters agreed with each other, above eighty per cent.Zheng et al. 2023
- C024Measurement artefactInverse scalingAbilities that look as though they switch on all at once with scale often improve smoothly underneath: the jump belongs to an all-or-nothing metric.Srivastava et al. 2023 · Srivastava
- C027Declared incompletenessAn evaluation should publish the distance between what it set out to measure and what it actually measured.Liang et al. 2022 · Liang
Severity
- C012SeverityAgreement between data and a claim is not evidence when the method could not have found the claim false.Mayo 2018 · Mayo
- C013SeverityThe positive counterpart of the weak requirement: evidence is graded by how capable the test was of finding flaws.Mayo 2018 · Mayo
- C014SeverityConvergent pathsConvergent, independent checks can warrant a conclusion more strongly than any single one of them.Mayo 2018 · Mayo
- C015SeverityAbsence of a detected error counts only if the procedure would have signalled that error had it been there.Mayo 2018 · Mayo
- C016SeverityMayo treats inferential methods as instruments for finding things out and for grading others' work, not as after-the-fact justifications.Mayo 2018 · Mayo
- C017SeveritySeverity is applied one level up, to methods rather than to claims: a method is judged by its capacity to catch erroneous interpretations.Mayo 2018 · Mayo
- C030SeverityNeither of the two dominant philosophies of statistics, assigning degrees of belief or securing long-run error rates, asks whether a test could have caught the error.Mayo 2018
- C031SeveritySeverity comes from Popper but arrived without a working definition and without a place in any statistical method.Mayo 2018
- C032SeverityA significance level is valid only relative to how data and hypotheses were selected; selective reporting makes a hard result look easy to explain away as chance.Mayo 2018
- C033SeverityA severe test is built from prior knowledge of specific mistakes: a probe that would reveal a given error if present, and stay silent otherwise.Mayo 1996
- C036SeveritySeverity is not tied to one statistical school: any inference can be judged by whether it withstood severe testing.Mayo 2018
- C037SeverityPeirce justified induction not by the truth of its conclusions but by the method’s capacity to correct its own errors over time.Mayo 2005
- C038SeverityInduction is justified by the method, not by any single conclusion: persisting in it corrects whatever error a stage contains.Peirce 1931–1958
- C039SeverityProbability attaches to the method’s track record, not to the hypothesis.Peirce 1931–1958
- C040SeverityA test does not collect agreement; it looks for consequences that would be surprising if the hypothesis were false.Peirce 1931–1958
- C041SeverityA lucky agreement is exposed by persisting with the same method: replication is the correction.Peirce 1931–1958
- C042SeverityConvergent pathsTwo selections made by different methods err in different ways, so their agreement is evidence about the truth.Peirce 1931–1958
- C043SeverityInduction needs no uniformity of nature, only the absence of an overwhelming hidden bias, and lesser biases are themselves detectable by the method.Peirce 1931–1958
Falsification
- C045FalsificationDogmatic falsificationism treats the experimenter as Nature’s neutral voice, delivering verdicts the theory cannot touch.Lakatos 1978
- C051FalsificationOnce ceteris paribus clauses are allowed, a theory that survives every challenge looks more scientific, not less: the reductio of dogmatic falsificationism.Lakatos 1978
- C034FalsificationFalsificationism explains why failing a test counts against a hypothesis, but not why passing one counts for it.Mayo 1996
- C035FalsificationBeating rivals is not passing a severe test: the test has to probe the ways this particular hypothesis could be wrong.Mayo 1996
- C044FalsificationIf every universal theory has zero probability on any evidence, probability cannot be what separates good theories from bad ones.Lakatos 1978
- C047FalsificationEvery test bottoms out in basic statements that someone decides to accept; logic alone never forces the stop.Popper 1959
- C048FalsificationWhat is decided by agreement is the singular observation, not the theory, and no experience justifies it: it is accepted.Popper 1959
- C049FalsificationA theory forbids an event only under a ceteris paribus clause, so a failed prediction never refutes it alone.Lakatos 1978
- C050FalsificationNeither proof, nor probability, nor refutation can be read off the facts: all three fail together.Lakatos 1978
- C052FalsificationAn observation counts as basic because a trained technique makes it decidable, not because it is given.Lakatos 1978
- C053FalsificationEvery test decides what is under test and what is taken for granted, and that line is a decision, not a fact.Lakatos 1978
- C054FalsificationMethodological falsificationism keeps hard criticism while admitting every statement is fallible, by making the decisions explicit.Lakatos 1978
What scale does not buy
Truthfulness
- C026Inverse scalingImitative falsehoodOn questions written to echo common misconceptions, the larger models answered falsely more often than the smaller ones, reversing the usual pattern.Lin et al. 2022
- C028Imitative falsehoodAcross the field, hallucination means generated content that is nonsensical or unfaithful to the source it was given: a definition about fidelity to the input, not about truth in the world.Ji et al. 2022
Architecture
- C023Machine judgeHarmlessness was trained by having the model criticise and revise its own answers against a written list of principles, instead of having people label every harmful output.Bai et al. 2022 · Bai
- C025Convergent pathsWhen several different chains of reasoning arrive at the same answer, that agreement is itself evidence for the answer.Wang et al. 2022
- C029Sequence without recurrenceThe architecture behind current language models dropped step-by-step processing of a sequence in favour of computing every relation between positions at once.Vaswani et al. 2017 · Vaswani
The 15 concepts
Layer 4 is where ideas are asserted and linked. Each concept has a page of its own: the question it answers, where it sits in the other layers, and its claims.
What a machine cannot be said to do
The limit is argued from how these systems are trained, not from what they fail at today.
- Form–meaning gapCan a system trained only on form learn what the language is about?2 claims
- Agent modelDoes predicting text require representing the mind that wrote it?2 claims
- Epistemic self-classificationCan a model report what kind of claim it is about to make?3 claims
- Claim typeDoes the kind of a claim decide how it can be checked?6 claims
How a verdict is made
A judgement counts only with the procedure that produced it attached.
- VerdictWhat has to hold before a judgement on an output counts?3 claims
- Coder agreementWhat does agreement between coders actually measure?3 claims
- Machine judgeCan a model stand in for a human rater?2 claims
- Measurement artefactWhen does a jump in a curve belong to the metric rather than to the thing measured?3 claims
- Declared incompletenessDoes an evaluation have to publish what it did not measure?2 claims
- Inverse scalingDoes a larger model get better at everything?2 claims
- SeverityWhen does passing a test count as evidence?18 claims
- FalsificationWhat does it take for a test to count against a theory?12 claims
- Convergent pathsIs agreement between different derivations evidence about the answer?4 claims
What scale does not buy
Size improves many things monotonically and some things not at all.