O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

This is Your Brain on Virtual Reality

300 visualizações

Publicada em

Presentation from June 21, 2015 to researchers and PhD students at the Center for Biological Cybernetics at the Max Planck Institute in Tübingen (Germany).

Publicada em: Ciências
  • Seja o primeiro a comentar

This is Your Brain on Virtual Reality

  1. 1. This is Your Brain on Virtual Reality Lynda Joy Gerry 21st of June 2016
  2. 2. Agenda:  1. Spatial Processing and the Brain  2. Agency and the Brain  3. Avatar Effects  4. First-Person POV “Embodied Experiences”  5. Ex Nihilo: Stimulating Creativity and Empathy  6. Future Directions: Social Interactions in VR 2
  3. 3. Spatial Processing and Wayfinding in Virtual Environments 3
  4. 4. Boldly Going Where No Rat Has Gone Before… (Mayank Mehta, 2008, UCLA) 4
  5. 5. VR For Rats 5
  6. 6. Virtual Reality Affects Firing Patterns of Brain’s GPS Cells  Background: Selective spatial firing of neurons in the hippocampus  Research Question: Do virtual environments activate a mental map in the same way as the real world?  Findings: VR world stimulates response more like a pedometer – the rat’s brains were tracking how many steps they were taking, rather than incorporating multisensory cues to integrate into a mental map 6
  7. 7. Measures of Presence and Neuronal Activation 7
  8. 8. Presence: Definition  Presence is the “…subjective experience of being transiently unaware of actions and cognitions linked to the real environment while concurrently perceiving oneself as situated in and perceiving action possibilities associated with the virtual environment (VE)” (Wirth et al., 2007) 8
  9. 9. High Vs. Low Presence Ratings 9 Baumgartner et al., 2008
  10. 10. Presence and The Brain  There are individual differences in how easily drawn into the virtual environment a person feels (Ijsselsteijn and Riva, 2003; Wirth et al., 2007)  Children and adults differ in the quality and speed with which presence is evoked by VEs (Schaik et al., 2004)  Individual differences in susceptibility towards presence might be modulated by the Prefrontal cortex, which is linked to executive functions and inhibitory control over emotions and behavior (Aron et al., 2004; Garavan et al., 1999; Koechlin et al., 2003; Ridderinkhof et al., 2004) 10
  11. 11. The Amphibian SCUBA Diving Simulator (Dhruv Jain, MIT Media Lab) 11
  12. 12. Body Ownership, Agency, and Avatar Effects 12
  13. 13. Is that hand my hand? 13
  14. 14. Body Ownership Illusions 14
  15. 15. Body Ownership Illusions and Agency  Body Ownership Illusions: Paired visual-tactile stimulation (Rubber Hand Illusion) or paired visual- motor stimulation  Tracked and rendered full-body motion in fully- immersive virtual environments creates sense of Agency  Bottom Line: Under specific multisensory conditions, we can experience artificial body parts, fake bodies, or virtual bodies as our own body parts or whole body. 15
  16. 16. 16
  17. 17. Counseling Oneself: Embodying Sigmund Freud Avatar 17 Osimo, Pizarro, Spanlang, & Slater (2015)
  18. 18. Avatar Effects Examples  Embodying a body of a different race reduces implicit racial biases (Groom, Bailenson, and Nass, 2009)  Embodying a tall avatar increases self-confidence in negotiation tasks (Yee and Bailenson, 2007)  Embodying an attractive avatar increases self-disclosure (Yee and Bailenson, 2007)  Embodying a stressful posture in a virtual body can increase stress even though physical body is not in that posture/position (Bergström, Kilteni, and Slater, 2013)  Embodying a child avatar body causes subjects to overestimate object sizes (Banakou et al., 2013) 18
  19. 19. Ahn, Bailenson, & Park (2014) 19
  20. 20. Ahn, Bailenson, & Park (2014) – Pro- Environmental Self-Efficacy  Environmental locus of control is the perception that one’s behaviors directly impacts the wellbeing of the environment.  This is one of the most powerful drivers for environmental behaviors. 20
  21. 21. Core Problem: Knowledge-to-Action Gap “the time-delayed, abstract, and often statistical nature of the risks of global warming does not evoke strong visceral reactions.” – Weber (2006) 21
  22. 22. Promoting Helping Behavior in VR 22 Ahn et al., 2015
  23. 23. Benefits of Avatars  Controlled environment + Stimuli  Ability to have live interaction with your own and another person’s avatar  Why use avatars when you can use real people? 23
  24. 24. Agency and the Brain 24
  25. 25. Medial Prefrontal Cortex and Agency (David et al., 2006)  Distinct neural substrates underlying representations of the self and others  The medial prefrontal cortex is crucial for a neural bases of the self and agency  Perspective taking and agency represent independent constituents of self- consciousness 25
  26. 26. FPP Hand Movement in VEs (Adamovich et al., 2009) 26
  27. 27.  Time-variant increase in activation in the left insular cortex during observation of avatar hand  Imitation of avatar showed activation in angular gyrus and extrastriate body area  Virtual hand avatar serves as disembodied training tools in observation condition and as embodied “extensions” of subject’s own body in the imitation condition 27FPP Hand Movement in VEs (Adamovich et al., 2009)
  28. 28. Background: First-Person Point-of-View “Embodied Experiences” in VR 28
  29. 29. “Think about it: there is no experience you have had that you are not the absolute center of. The world as you experience it is there in front of YOU or behind YOU, to the left or right of YOU, on YOUR TV or YOUR monitor. And so on. Other people’s thoughts and feelings have to be communicated to you somehow, but your own are so immediate, urgent, real.” - David Foster Wallace, “This is Water” (2005 Kenyon Commencement Speech) 29
  30. 30. Machine to Be Another: Live Stream Perspective Swapping 30
  31. 31. The Machine to Be Another (MIT Media Lab) 31
  32. 32. THE MACHINE TO BE ANOTHER: An Empathy Machine? 32 “Empathy is created the moment we try to put ourselves in another person’s shoes.” – Aspen Baker, TED Radio Hour “The machine to be another is designed to stimulate empathy through embodied interaction between individuals.” (Bertrand et al., 2014) To what extent can we experience and understand what it is like to be someone else?
  33. 33. Simo Ekholm: Meeting Myself from Another Person’s Perspective 33
  34. 34. Makropol: “Art House” VR using POV storytelling 34
  35. 35. Skammakrogen: How it Was Made 35
  36. 36. How to Combine These Experiences?  Be Another Labs use live stream into the DK2 Benefit: Real-Time Interaction Con: Low visual resolution + not stereo  Original plan: Live Stream in Stereo – Low visual resolution + Delay  Used Stereo and Pre-Recorded Footage with Binaural audio 36
  37. 37. Painter Project: Stereo Footage with Binaural Audio 37
  38. 38. Ex Nihilo: Exploring the Stimulation of Creativity and Empathy through Embodied Experiences in Virtual Reality 38
  39. 39. Gallagher et al. (2015) Neurophenomenology of Awe and Wonder 39
  40. 40. Video of Experimental Task 40
  41. 41. Video of Experimental Task 41
  42. 42. 42
  43. 43. 43
  44. 44. 44
  45. 45. 45
  46. 46. Introducing Ex Nihilo In The Republic, Plato asks, "Will we say, of a painter, that he makes something?" and answers, "Certainly not, he merely imitates." 46
  47. 47. “Imitated” Paintings by Subjects 47
  48. 48. Subject Quote  “It almost feels like collectively discovering something that was in her head all along. The experience allowed this completely different way of connecting to someone that doesn’t rely on facial expressions or anything.” 48
  49. 49. Measuring Social Interaction in VEs 49
  50. 50. Measuring Social Interaction in VR  How do you measure two people’s ability to communicate effectively?  What does communicating “effectively” mean?  Suzanne Dikker – “shared semantic understanding” and neural synchrony 50
  51. 51. Perceptual Crossing (Auvary et al., 2009) 51
  52. 52. Perceptual Crossing (Auvary et al.,2009)  Task: Participants recognize one another while avoiding shadow and static objects  Mutual tactile interaction (trackball mouse and haptic feedback)  Static object = too stable and predictable to be human  Shadow = too unstable since it moves but does not respond  Other subject’s avatar can react with communicative intent 52
  53. 53. Froese and Di Paolo (2011)  Hypothesized that the lack of personal recognition of the other in the perceptual crossing paradigm was due to lack of a genuinely social experimental task  The mark of the social is the co-regulation of mutual interaction 53
  54. 54. Primary Intersubjectivity in Infants  2-month old infants get distressed when shown a pre-recorded video of an interaction with their mother, as opposed to a live streamed video.  Simple recognition of a pattern of movements is not sufficient, and instead social action perception requires an understanding of how other's movements are related to our own.  Infants display movement synchrony with adult speech (Condon & Sander, 1974)  Still face studies (Toda & Fogel. 1993; Tronick, Als, Adamson., Wise. & Brazelton, 1979) 54
  55. 55. Thank You! www.khora-vr.com lynda@khoravr.com