Poster presentation at X-PPL by Margot Berthelin on the adult interpretation of infant pointing and its multimodality
Infant pointing sits at the intersection of two modalities: gestural and vocal. We disentangle the contributions of the two channels in intentionality expression by conducting two experiments.
We extracted points from Shipibo-Conibo 10-to-12-month olds and asked Swiss German adults to interpret each point as declarative or imperative.
The first experiment looks at the participant-level variance in interpretations across conditions. Results show that variance was comparable in the audiovisual and audio-only conditions, but substantially higher in the video-only condition, suggesting that vocalisations played a key role in the adult's identification of the infants' communicative intent.
Our second study looks at the disagreement of participants with a ground truth, when the pointing act is manipulated. An incongruent manipulation results in disagreement (versus no effect in congruent manipulation case).
Our results suggest that vocalisations play a decisive role in the adult interpretation of intentionality. We highlight the efficiency of multimodal integration.