Claims › Sequence without recurrence › C029
C029
In their own words · Vaswani et al. 2017 · §2 Background · assumed
…End-to-end memory networks are based on a recurrent attention mechanism instead of sequence-aligned recurrence and have been shown to perform well on simple-language question answering and language modeling tasks [34].
“To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.”
In the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as [17, 18] and [9].…
The architecture behind current language models dropped step-by-step processing of a sequence in favour of computing every relation between positions at once. It is the engineering fact the rest of the debate stands on.
The five layers, for this claim
The same core in the infographic, with the claims it touches.
Relations
No relation to another claim has been declared yet.
Notation
- Claim typeWhat kind of statement it is, so that the right test can be applied. Proposed until checked by hand.
- D1.1 · historical-factual
- FunctionWhat a claim does for the programme: a conceptual tool, a working method, corrective evidence, a technical reference.
- technical reference
- StanceHow the programme holds the claim: assumed, used, or contested.
- assumed
- Used in
- /lab/2026-02-03-the-seven-dimensional-taxonomy-v2/
- Cite this
- claudiocammarano.com/en/claims/c029/