Emotion-word rating norms
The psycholinguist’s workhorse, and new to this wiki with Ferré et al. (2024). Take a large word set, split it across questionnaires, have ~20 people rate every word on one dimension, average, publish. The resulting norms — valence, arousal, concreteness, age of acquisition, and now interoception — become shared infrastructure that other labs regress against for decades (ANEW; Warriner et al. 2013; Stadthagen-Gonzalez et al. 2017; Emopro).
What kind of measure this is
The placement problem is the same one embody has, one level further removed, and it is worth being blunt about.
| method | asks | ground truth |
|---|---|---|
| heartbeat-detection-task | can you count your heartbeats? | physiology |
| maia | how do you relate to your body? | none (sensibility) |
| embody | where do you feel this emotion? | none (felt topography) |
| rating norms | is this word about internal body sensations? | none (semantics) |
emBODY at least asks about the rater’s own felt body. Rating norms ask about a word — a fact about the lexicon, averaged over people. On the interoceptive-taxonomy the method does not fit anywhere, because it is not measuring the rater’s interoception at all. It is measuring what a linguistic community takes interoception to be relevant to.
That sounds like a fatal disqualification for an interoception wiki and is not, for one reason: the semantic layer is where the word-cue objection lives. If you want to know whether word-cued bodily maps could be driven by conceptual knowledge, you need to know what the concepts contain — and that is a question about words, answerable only by a method like this one. See emotion-prototypicality.
The design details that carry weight
Three features distinguish a serious norming study from a survey, all present in Ferré et al.:
- Calibrators. Seven words spanning the full scale range are shown first, so raters do not discover the extremes halfway through and silently rescale. Chosen per variable (for interoception: hambriento/hungry at the high end, imaginación/imagination at the low).
- Repeated control words. Twenty-two words appear in every version of a variable, so the versions can be correlated against each other. This is the study’s real validity check and the reason its interoception weakness is visible at all — without control words, an ICC of .80 would have looked fine.
- Correlation-based trimming. Raters correlating < .10 with their version’s mean are dropped and replaced (1% of data here); near-zero correlations indicate idiosyncratic responding, negative ones indicate a reversed scale.
The reliability lesson
Worth generalising beyond this paper. Ferré et al. report two reliability metrics that disagree: interoception’s ICC (.80) is unremarkable and its cross-version control-word correlation (.654) is the worst in the study by a distance. ICC and cross-version agreement are answering different questions — the ICC asks whether raters within a version rank items consistently, the control-word correlation asks whether different groups of raters converge on the same values for the same items.
For a norming study, whose entire product is per-item means intended for reuse by other people, the second is the one that matters, and it is the one that is reported in prose rather than in the reliability table. When a method’s headline variable is its least reproducible one, that belongs in the limitations section.