The Neuroscience of Social Interaction: Decoding, imitating, and influencing the actions of others CHRISTOPHER D. FRITH DANIEL M.WOLPERT Editors OXFORD UNIVERSITY PRESS The Neuroscience of Social Interaction Wolpert-FM.qxd 11/18/03 7:52 PM Page i Wolpert-FM.qxd 11/18/03 7:52 PM Page ii This page intentionally left blank The Neuroscience of Social Interaction Decoding, imitating, and influencing the actions of others Edited by CHRISTOPHER D. FRITH Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, London and DANIEL M. WOLPERT Sobell Department of Motor Neuroscience and Movement Disorders, Institute of Neurology, University College London, London Originating from a Theme Issue first published by Philosophical Transactions of the Royal Society, Series B. 1 Wolpert-FM.qxd 11/18/03 7:52 PM Page iii 3 Great Clarendon Street, Oxford OX2 6DP Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Bangkok Buenos Aires Cape Town Chennai Dar es Salaam Delhi Hong Kong Istanbul Karachi Kolkata Kuala Lumpur Madrid Melbourne Mexico City Mumbai Nairobi São Paulo Shanghai Taipei Tokyo Toronto Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York © The Royal Society, 2003 The moral rights of the author have been asserted Database right Oxford University Press (maker) First published by the Royal Society 2003 First published by Oxford University Press 2004 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose this same condition on any acquirer A Catalogue record for this title is available from the British Library ISBN 0 19 852925 2 (Hbk) 0 19 852926 0 (Pbk) 10987654321 Typeset by Newgen Imaging Systems (P) Ltd., Chennai, India Printed in Great Britain on acid-free paper by Biddles Ltd, Guildford and King’s Lynn Wolpert-FM.qxd 11/18/03 7:52 PM Page iv Preface A key question for science to explore in the twenty-first century concerns the mechanism that allows skilful social interaction. Although enormous advances in our understanding of the links between the mind, the brain, and behaviour have been made in the last few decades, these are based on studies in which people are considered as strictly isolated units. For example, studies might typically examine the brain activity when volunteers press a button when they are aware of seeing a visual stimulus. Outside the laboratory, in contrast, we spend most of our time thinking about and interacting with other people rather than looking at abstract shapes and pushing buttons. One of the major functions of our brains must be to facilitate such social interactions. It is the mental and neural mechanisms that underlie this social interaction which forms the main theme of this book. We have concentrated on two-person social interactions in which one person, either implicitly or explicitly, tries to ‘read’ the hidden mental states of the other; their goals, beliefs or feelings. In this book we have brought together scientists from many different disciplines, but all concerned with the same problems. These problems include how goals and intentions can be read from watching another person’s movements, how movements that we see can be converted into movements that we make, and how our own behaviour can be used to influence the behaviour of others. The book reviews the general principles concerning the cognitive and neural bases of social interactions that have emerged. Within this framework the authors discuss many different aspects of social interaction, demonstrating the excitement and vigour of this emerging discipline. This book was originally published as an issue of the Philosophical Transactions of the Royal Society, Series B, Phil. Trans. R. Soc. Lond. B (2003) 358, 429–602. Christopher D. Frith London Daniel M. Wolpert August 2003 Wolpert-FM.qxd 11/18/03 7:52 PM Page v Wolpert-FM.qxd 11/18/03 7:52 PM Page vi This page intentionally left blank Contents List of Contributors ix Introduction: the study of social interactions xiii T. Singer, D. M. Wolpert, and C. D. Frith Biological motion: decoding social signals 1. Electrophysiology and brain imaging of biological motion 1 A. Puce and D. Perrett 2. Teleological and referential understanding of action in infancy 23 G. Csibra 3. Development and neurophysiology of mentalizing 45 U. Frith and C. D. Frith 4. Mathematical modelling of animate and intentional motion 77 J. Rittscher, A. Blake, A. Hoogs, and G. Stein Mirror neurons: imitating the behaviour of others 5. What imitation tells us about social cognition: a rapprochement between developmental psychology and cognitive neuroscience 109 A. N. Meltzoff and J. Decety 6. Action generation and action perception in imitation: an instance of the ideomotor principle 131 A. Wohlschläger, M. Gattis, and H. Bekkering 7. The manifold nature of interpersonal relations: the quest for a common mechanism 159 V. Gallese 8. Imitation as behaviour parsing 183 R. W. Byrne 9. Computational approaches to motor learning by imitation 199 S. Schaal, A. Ijspeert, and A. Billard Mentalizing: closing the communication loop 10. Detecting agents 219 S. C. Johnson Wolpert-FM.qxd 11/18/03 7:52 PM Page vii 11. Facial expressions, their communicatory functions and neuro-cognitive substrates 241 R. J. R. Blair 12. Models of dyadic social interaction 265 D. Griffin and R. Gonzalez 13. Dressing the mind properly for the game 283 D. Sally 14. A unifying computational framework for motor control and social interaction 305 D. M. Wolpert, K. Doya, and M. Kawato Index 323 viii Contents Wolpert-FM.qxd 11/18/03 7:52 PM Page viii List of Contributors Harold Bekkering, Nijmegen Institute for Cognition and Information, University of Nijmegen, Montessorilaan 3, NL-6525 HR, Nijmegen, The Netherlands Aude Billard, Computer Science and Neuroscience, University of Southern California, 3641 Watt Way, Los Angeles, CA 90089-2520, USA and School of Engineering, Swiss Federal Institute of Technology, Lausanne, CH 1015 Lausanne, Switzerland R. J. R. Blair, Unit on Affective Cognitive Neuroscience, Mood and Anxiety Disorders Program, National Institute of Mental Health, National Institute of Health, Department of Health and Human Services, 15K North Drive, Bethesda, MD 20892-2670, USA Andrew Blake, Microsoft Research, 7 JJ Thomson Avenue, Cambridge CB3 0FB, UK R. W. Byrne, School of Psychology, University of St Andrews, St Andrews, Fife KY16 9JU, UK Gergely Csibra, Centre for Brain and Cognitive Development, School of Psychology, Birkbeck College, Malet Street, London WC1E 7HX, UK Jean Decety, Center for Mind, Brain and Learning, University of Washington, Seattle, WA 98195, USA Kenji Doya, ATR Human Information Science Laboratories and CREST, Japan Science and Technology Corporation, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan. Christopher D. Frith, Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, Queen Square, London WC1N 3AR, UK Uta Frith, Institute of Cognitive Neuroscience, University College London, Queen Square, London WC1N 3AR, UK Vittorio Gallese, Istituto di Fisiologia Umana, Università di Parma, Via Volturno, 39, 43100 Parma, Italy Wolpert-FM.qxd 11/18/03 7:52 PM Page ix [...]... al 2000) These STS cells are sensitive to the form of the hand performing the action, and are unresponsive to the sight of tools manipulating objects in the same manner as hands Furthermore, the cells code the spatio-temporal interaction between the agent performing the action and the object of the action For example, cells tuned to hands manipulating an object cease to respond if the hands and object... representation of actions and their consequences in the world Crucially the properties of the frontal mirror system indicate that we may understand actions performed by others because we can match the actions we sense through vision (and audition) to our ability to produce the same actions ourselves The actions of others are not always fully visible, for example someone may become hidden from our sight as they... Pickering and Garrod (2003) The book is organized in terms of three stages in the interaction between an ‘observer’ and an ‘actor’ First, the observer watches the movements of the actor and infers goals, beliefs, and feelings Second, the observer generates behaviour in response to that of the actor In the simplest case the observer imitates the actor Successful imitation often indicates some understanding of. .. understanding that others are ‘like me’ They hypothesize that the primitive ‘like me’ understanding of infants is a vital building block for the later ability to adopt the perspective of others a fundamental mechanism for empathy The authors emphasize not only on the similarity between self and other (the focus of debates about ‘mirror neurons’) but also on how humans differentiate their own acts and. .. some understanding of the goals of the actor Third, the communicative loop is closed so that the actor, in turn, interprets and responds to the behaviour of the observer Within this framework the authors discuss many different aspects of social interactions, demonstrating the excitement and vigour of this emerging discipline Here we will highlight some of the key ideas that emerge in the chapters that... of focused conferences, special issues of journals, and books (e.g., Adolphs 2003; Allison, Puce, and McCarthy 2000; Cacioppo et al 2001; Harmon-Jones and Devine, in press; Heatherton and Macrae 2003; Ochsner and Lieberman 2001) The agenda of social cognitive neuroscience has been described in terms of seeking ‘to understand phenomena in terms of interactions between three levels of analysis: the social. .. place in the neighbouring discipline of cognitive psychology during the 1960s and 1970s (the first issue of the journal Social Cognition’ appeared in 1982, the first edition of the ‘Handbook for social cognition’ in 1984) Theoretically, and methodologically, the intellectual movement of social cognition strongly relied on the information-processing approach and the new experimental paradigms developed... shapes and pushing buttons It is this social interaction which forms the main theme of this volume Humans, like other primates, are intensely social creatures One of the major functions of our brains must be to enable us to be as skilful in social interactions as we are in recognizing objects and grasping them Furthermore, any differences between human brains and those of our nearest relatives, the great... or the simultaneous recording of dyadic brain interactions using techniques such as EEG or fMRI (Montague et al 2002) Mechanisms of social interaction It is not our aim in this book to represent the whole field of social cognitive neuroscience We have concentrated on two-person social interactions in which one person, either implicitly or explicitly, tries to ‘read’ the hidden mental states of the other;... trial and error learning Even better imitation can be achieved if the movement primitives are used to make predictions about the behaviour of the robot teacher (see also Chapter 14) C) Closing the communication loop The most remarkable feature of social interactions is how skilled we are in correctly inferring the goals, beliefs, and feelings of others How is it possible to read these mental states? They . The Neuroscience of Social Interaction: Decoding, imitating, and influencing the actions of others CHRISTOPHER D. FRITH DANIEL M.WOLPERT Editors OXFORD UNIVERSITY PRESS The Neuroscience of Social. to influence the behaviour of others. The book reviews the general principles concerning the cognitive and neural bases of social interactions that have emerged. Within this framework the authors. Introduction: the study of social interactions Tania Singer, Daniel Wolpert, and Chris Frith In the last few decades there have been enormous advances in our under- standing of the links between the mind,