Readings › Staley 2004 › Extract

Extract · a self-contained text inside a work

Strength of evidence and security of evidence claims

Staley, Robust Evidence and Secure Evidence Claims (2004), §4, pp. 470–471.

Extract published · 0 claims drawn

What it says, in one sentence

How strongly a result indicates a hypothesis is one question; how exposed the claim “this result is evidence for H” is to the failure of an auxiliary assumption is another. Robustness may not answer the first, but it can answer the second.

The two dimensions, line by line

Each line restated in my notation. e ▷ H reads “e is evidence for H”; A is any auxiliary assumption the claim rests on (theory of the experiment, apparatus, recording, interpretation of the data).

Two dimensions of an evidence claim

SStrengthstr(e ▷ H)

How strongly e indicates H. Usually said in words (suggestive, compelling, conclusive); significance levels, chi-squared fits and likelihood ratios give partial numbers, not the confirmation measure philosophers looked for.

SecSecuritysec(e ▷ H) = resistance of e ▷ H to the failure of some A

Not how strong the evidence is, but how likely the claim that it is evidence is to be defeated by an assumption that turns out false.

Why security is a separate question

UUsed, not testedstr(e ▷ H) is computed given A

The assumptions serve in the evaluation of strength and are not evaluated in the same step.

FFallibleconfidence in A ≠ A true

Even absolute confidence in an assumption could be misplaced. That is the only reason security matters, and it is reason enough.

Two questions to ask of each assumption

Q1Supporthow strong is the evidence for A?

Security grows if the claim rests on assumptions that are themselves very well supported.

Q2Sensitivityhow much does e ▷ H depend on A?

Security also grows if the claim rests on A lightly, so that its failure would weaken the claim rather than destroy it.

QTwo routessec ↑ ⇐ evidence for A ↑ ∨ dependence on A ↓

The second route counts even when the first looks settled, because the judgement that A is well supported can itself be wrong.

Where robustness comes in

R1Stronger?robust ⇒ stronger : open

Two statistically equivalent results, one from several experiments resting on different assumptions, one from a single experiment: whether the first is stronger evidence, Staley leaves unresolved.

R2More securerobust ⇒ more secure, under conditions

The thesis of the paper: robustness can answer either question, and guard against at least two kinds of error.

Where it sits in the work

469470471472

pp. 469–472 · 8 marks

Claims drawn from this extract

No claims drawn yet.

Paraphrase and notation CC BY-NC 4.0, Claudio Cammarano. The formulations restate the author’s; they are not quotations. Back to the card of the work.