Mind-Brain Lecture: Pamela Perniss (Nijmegen)
Communicative expression in the visual-spatial modality, as in sign language and co-speech gesture, has been claimed to be similar in the domains of action and motion representation due to shared access to the iconic affordances of the modality. This claim is consistent with theories of embodied language and cognition, which emphasize correspondences between language and action through mental simulations of motor and visual imagery. Within a simulation-based framework, the use of the body and space in the visual-modality allows direct expression of these simulations, such that expression of action and motion should resemble the actual events [cf. the Gesture as Simulated Action (GSA) framework (Hostetter & Alibali 2008)].
In this talk, I present a study which investigates these claims by comparing narrative encoding of events containing both action and motion components (e.g. carrying a suitcase down stairs) in German Sign Language (Deutsche Gebärdensprache, DGS) and German co-speech gesture. Our findings show that the use of the visual-spatial modality does not in itself lead to similarity of expression in these domains. Rather, sign and gesture expressions differ significantly in both amount and type of representation of action and motion. Moreover, we find that sign, but not co-speech gesture, representations reflect patterns predicted by simulation-based theories of action and motion representation. We argue that the difference between sign and gesture encoding of action and motion lies in the difference between a two-channel vs. one-channel system, whereby linguistic and imagistic elements are encoded in distinct channels (vocal and visual) in the case of co-speech gesture but in the same channel (visual) in the case of sign language. As a result, we argue that speech encoding of action and motion drives gestural representations away from what would be expected from a simulated-based theory of representation, whereas sign representations allow the conventionalization of linguistic expressions that are directly motivated by embodied representations.
I will conclude the talk with a discussion of the relationship between iconicity and embodiment, specifically arguing for the role of iconicity in how language comes to engage our perceptual and motor systems, thus giving rise to embodied language (i.e. language grounded in our sensori-motor experience) (Perniss, Thompson, & Vigliocco 2010).
Dr Pamela Perniss, Radboud University & MPI for Psycholinguistics, Nijmegen
http://www.dcal.ucl.ac.uk/team/pamela_perniss.html
Zertifiziert durch Ärztekammer Berlin: VNR 2761102010091980004, 2 Punkte
All are welcome!
