Verbatim. The claim, and the passage around it, are the source’s own words. Where the quotation skips something, the gap is marked and the omitted words can be put back: a cut you cannot inspect is an argument you have to take on trust.
Running text only. Never the abstract, never the keywords, never a blurb. An abstract is the author’s own restatement of a conclusion with the argument removed; quoting it means quoting the summary instead of the thing.
One paper, one claim. A book can hold several, one per chapter actually used. A claim exists here because something on this site uses it.
Twelve concepts, 14 links between them. The circle is the concept, the bar is the weight of the claims under it, the arc is a link it declares. Pick one to read its proposition and to narrow the wall below.
Andreas 2022§1 Introduction, (C1)
In particular, I claim:
In the course of performing next-word prediction in context, current LMs sometimes infer approximate, partial representations of the beliefs, desires and intentions possessed by the agent that produced the context, and other agents mentioned within it.
(C2) Once these representations are inferred, they are causally linked to LM prediction, and thus bear the same relation to generated text that an intentional agent’s state bears to its communciative actions.
topic Meaning
concept Agent model
- contests form-alone-cannot-yield-meaning — If the model infers representations of intentions, form alone does not stay mute, as Bender and Koller claim it must.
Zheng et al. 2023§1 Introduction
We show that some of the biases are minor or can be mitigated.
Once addressed, our results from 3K controlled expert votes and 3K crowdsourced human votes in the wild verify that GPT-4 judge match human evaluations at an agreement rate exceeding 80%, achieving the same level of human-human agreement (4.2, Table 4).
Consequently, this suggests LLM-as-a-judge is a scalable method to swiftly evaluate human preference, serving as a promising alternative to traditional human evaluations.
topic Evaluation
concept Verdict, Machine judge
- contests mode-a-needs-a-blind-recode — If an automatic judge reaches human-human agreement, blind human recoding is no longer the only standard.
Mitchell & Krakauer 2023p. 1
This debate is not just academic; the extent and manner in which machines understand our world has real stakes for how much we can trust them to drive cars, diagnose diseases, care for the elderly, educate children, and more generally act robustly and transparently in tasks that impact humans.
Moreover, the current debate suggests a fascinating divergence in how to think about understanding in intelligent systems, in particular the contrast between mental models that rely on statistical correlations and those that rely on causal mechanisms.
Until quite recently there was general agreement in the AI research community about machine understanding: while AI systems exhibit seemingly intelligent behavior in many specific tasks, they do not understand the data they process in the way humans do.
topic Meaning
concept Form–meaning gap, Agent model
Artstein & Poesio 2008§5.2 Choosing a Coefficient
Regarding the question of annotator bias, there is an overwhelming consensus in CL practice: K and α are used in the vast majority of the studies we reported.
We agree with the view that K and α are more appropriate, as they abstract away from the bias of specific coders. But we also believe that ultimately this issue of annotator bias is of little consequence because the differences get smaller and smaller as the number of annotators grows […](Artstein and Poesio 2005).
We believe that increasing the number of annotators is the best strategy, because it reduces the chances of accidental personal biases.
topic Annotation
concept Coder agreement
Bai et al. 2022§1 Introduction
We also aim to develop methods that encode desirable AI behavior in a simple and transparent form, and that make it easier to understand and evaluate AI decision making.
In this paper we develop a method we refer to as Constitutional AI (CAI), depicted in Figure 1, and use it to train a non-evasive and relatively harmless AI assistant, without any human feedback labels for harms. The method therefore improves upon, and partially replaces reinforcement learning from human feedback […][Christiano et al., 2017].
The new assistant ‘RL-CAI’ is preferred by crowdworkers over those trained with previously collected [Bai et al., 2022, Ganguli et al., 2022] human feedback labels for harmfulness.
topic Architecture
concept Machine judge
Ji et al. 2022§2
Similar to psychological hallucination, which is hard to tell apart from other “real” perceptions, hallucinated text is also hard to capture at first glance.
Within the context of NLP, the above definition of hallucination, the generated content that is nonsensical or unfaithful to the provided source content […][72, 177, 198, 325], is the most inclusive and standard.
However, there do exist variations in definition across NLG tasks, which will be further described in the later task-specific sections.
topic Truthfulness
concept Imitative falsehood
Liang et al. 2022§2, Recognition of incompleteness
Different benchmarks make different decisions on what to prioritize, how to make these decisions, and to what extent these processes are made clear in presenting the benchmark.
Since our aim is holistic evaluation, we believe it is necessary to be explicit on the relationship between what we aspire to evaluate and what we actually evaluate.
The construction of HELM starts top-down with a taxonomy over scenarios and metrics (see Figure 2). The taxonomy not only facilitates the systematic selection of scenarios and metrics, but it also make explicit what is missing.
topic Evaluation
concept Declared incompleteness
Srivastava et al. 2023§3.4.2
Breakthrough behavior is consistent with the model suddenly gaining new skills in an abrupt way. Careful analysis of task behavior, however, suggests that the underlying change in model capabilities is generally more smooth.
These smooth improvements can be exposed by decomposing tasks in ways that better capture partial progress. We describe some of those ways in the remainder of this section.
topic Evaluation
concept Measurement artefact, Inverse scaling
Programme note§ The governing principle
The governing principle, which became the paper’s title, is this: ontology precedes epistemology.
You cannot determine whether a claim is true without first establishing what type of claim you are dealing with.
A statistical claim and a normative claim share nothing in their truth conditions — treating them as interchangeable is the error the framework is designed to prevent.
topic Claim types
concept Claim type
Programme note§ The gap
The gap is the finding.
Surface-level variance […]— which looks like inconsistency — masks high semantic convergence.
An LLM that produces thirty different textual responses to the same prompt is not thirty times unreliable; it may be converging on the same epistemic conclusion through thirty different linguistic paths. Distinguishing these two phenomena requires a framework. That is what this project provides.
topic Annotation
concept Coder agreement, Measurement artefact, Convergent paths
Programme note§ Start with the trigger
This is Mode A of Coding Manual v2.1, and Mode A carries a mandatory check.
A human recodes a sample by hand, blind to the machine, and the two must agree above 90%.
topic Annotation
concept Verdict, Coder agreement
Bender & Koller 2020§1 Introduction
From our perspective, these are overclaims caused by a misunderstanding of the relationship between linguistic form and meaning.
We argue that the language modeling task, because it only uses form as training data, cannot in principle lead to learning of meaning.
We take the term language model to refer to any system trained only on the task of string prediction, whether it operates over characters, words or sentences, and sequentially or not.
topic Meaning
concept Form–meaning gap
Lin et al. 2022§1 Introduction
Across different model families, the largest models were generally less truthful (Fig. 2). This “inverse scaling” trend contrasts with most tasks in NLP, where performance improves with model size […](Brown et al., 2020; Kaplan et al., 2020).
One explanation of this result is that larger models produce more imitative falsehoods because they are better at learning the training distribution. Another explanation is that our questions adversarially exploit weaknesses in larger models not arising from imitation of the training distribution.
topic Truthfulness
concept Inverse scaling, Imitative falsehood
Vaswani et al. 2017§2 Background
End-to-end memory networks are based on a recurrent attention mechanism instead of sequence-aligned recurrence and have been shown to perform well on simple-language question answering and language modeling tasks [34].
To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.
In the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as [17, 18] and [9].
topic Architecture
concept Sequence without recurrence
Programme note§ The governing principle
You cannot determine whether a claim is true without first establishing what type of claim you are dealing with.
A statistical claim and a normative claim share nothing in their truth conditions — treating them as interchangeable is the error the framework is designed to prevent.
topic Claim types
concept Claim type
Programme note§ Dimension D1
Dimension D1 — Content Type is the load-bearing axis. It currently includes eleven active categories (D1.1 through D1.10, plus D1.14), covering the full range from historical-factual and empirical-quantitative claims to ethical-normative and metaphysical-ontological ones.
The remaining dimensions […]— D2 (temporality), D3 (epistemic certainty), D4 (causal structure), D5 (framework-dependency), D6 (verifiability mode), D7 (falsifiability) — operate as modifiers that describe how a given claim type behaves epistemically.
topic Claim types
concept Claim type
Wang et al. 2022§1 Introduction
It has been observed that chain-of-thought prompting significantly improves model performance across a variety of multi-step reasoning tasks (Wei et al., 2022).
Self-consistency leverages the intuition that complex reasoning tasks typically admit multiple reasoning paths that reach a correct answer […](Stanovich & West, 2000).
The more that deliberate thinking and analysis is required for a problem (Evans, 2010), the greater the diversity of reasoning paths that can recover the answer.
topic Architecture
concept Convergent paths
Nothing under this filter yet.