Why picture naming?

• Testing theories about how people plan and output language; i.e. grammatical encoding (V. S. Ferreira and Slevc 2007; Wheeldon and Konopka 2023; Roeser, Torrance, and Baguley 2019).
• Control / manipulating the utterance that is going to be produced.
• Clinical assessment of linguistic impairments to do with lexical retrieval of actions and object naming (Caramazza and Hillis 1991) / forming of syntactically correct sentences (Yan, Martin, and Slevc 2018).
• For example, syntactic priming (Bock 1986; Pickering and Ferreira 2008; Garcia, Roeser, and Kidd 2023) …

• The chef is chasing the ballerina.

The ballerina is being chased by the chef.

To describe an image we need to…

• Visually encode image
• Build conceptual representation
• Decide for a starting point (based on contextual factors [image, intention] and grammatical constraints of language).
• Encode enough lexical information to start sentence and maintain fluency
• Encode phonological / orthographic form when needed

Where do I start? – starting point and syntactic choice

• Start with the doer (agent)? – doer depends on verb, hence action, hence perception: The ballerina is running away from the chef
• Reversibility in transitive actions: is chasing vs is kissing
• Animacy: humans > animals > plants > objects (e.g. Branigan, Pickering, and Tanaka 2008); visual saliency (e.g. F. Ferreira and Rehrig 2019; but see Bock, Irwin, and Davidson 2004); vividness (Bock, Irwin, and Davidson 2004).
• Using an active (The chef is chasing the ballerina) or passive (The ballerina is being chased by the chef) sentence.
• Name agreement of either or both of the two actors (dancer vs ballerina) and the action (chasing, running away from, harassing), ease of encoding (girl vs ballerina) (Gleitman et al. 2007; Kuchinsky, Bock, and Irwin 2011).
• Direction of action because of scan path preferences (Buswell 1935): tendency to describing images from left to right (reference?)

Direction of action

Left to right

Direction of action

Right to left

Existing naming norms

• Primarily single object naming (see e.g. Torrance et al. 2018).
• Actions / verbs are interesting for many reasons (see Bonin et al. 2004).
• Norms exist for various languages:
  • French: Bonin et al. (2004); English: Szekely et al. (2004); Fiez and Tranel (1997); Spanish: Cuetos and Alija (2003);
  • More recently Dutch: Shao, Roelofs, and Meyer (2014); Polish: Wolna et al. (2022); Russian: Akinina et al. (2015); Moroccan Arabic: Rami et al. (2022); German: Busch et al. (2021); Turkish: Bayram et al. (2017); Gulf Arabic: Khwaileh et al. (2018); Kannada: Ahmed, Giridhar, and Krishnan (2022)
• Typically recorded:
  • Most commonly used verb for intransitive / transitive / distransitive actions but not complete sentences (I think).
  • Response latency
  • Frequency, age of acquisition of most common response (verb) [also useful number of syllables / characters / sounds / words]
  • Familiarity, name agreement, imageability, image agreement
  • Visual complexity

Naming disfluency

• Response onset: _^The chef is chasing the ballerina.
  • sentence-level planning: scene encoding, syntax, thematic dependencies, content of message
• Before-word keystrokes: The _^chef _^is _^chasing _^the _^ballerina.
  • word-level encoding: lexical retrieval, phrase dependencies, spelling, pronunciation
• Within-word keystrokes: T^h^e  c^h^e^f  i^s  c^h^a^s^i^n^g  t^h^e  b^a^l^l^e^r^i^n^a.
  • parallel planning of upcoming lexical material; incomplete orthographic / phonological encoding

Naming disfluency

Hypothetical keystroke-interval data

Novelty

There are no naming norms that including information about the production timecourse of full sentences.

• We will collect a large data set of written image descriptions in which we manipulate the direction of irreversible transitive actions (to chase, to push, to bite).
• The elicited utterance will inform us about various psycholinguistic aspects (sentence structure, word order, name agreement, and words used) as well as the time course of encoding image properties into utterances.

To-do

• Decide for images; check existing image sets; maybe Druks and Masterson (2000) which is here. I’ve got action images (Holly Branigan, Robert Slevc, Mikhail Pokhoday, Rowena Garcia); ideally a large variety of actions. I’m waiting to get access to the CLT (Haman et al. 2017; Haman, Łuniewska, and Pomiechowska 2015).
• What other image / sentence properties do we need (check literature)?
• Some images need to be resized and renamed following action_agent_patient_direction.jpg, for example: chasing_chef_ballerina_LR.jpg; direction labels are missing.
• Need to flop pictures (LR > RL, RL > LR).
• Add ditransitive and intransitive actions as fillers to avoid people committing to only actives / passives.
• Gorilla: presenting images and capturing keystrokes.
• Counterbalance so that participants never see both the LR and RL version of an image.
• Markup collected data as needed (annotating sentences).