Exploring the Brain Computer Interface
•
11 gostaram•5,084 visualizações
This document provides an overview of brain-computer interfaces and their applications. It discusses the science of reading brain activity through various technologies like EEG, MRI, and ultrasound. It also covers direct brain input methods such as tDCS and TMS. The document outlines several consumer brain-computer interfaces currently available and demonstrates using a brain interface to control a quadcopter. It concludes by discussing future applications of brain interfaces such as enhanced reality, thought identification, and uploading consciousness.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (20)
Semelhante a Exploring the Brain Computer Interface
Semelhante a Exploring the Brain Computer Interface (20)
Mais de Jim McKeeth
Mais de Jim McKeeth (14)
Último
Último (20)
Exploring the Brain Computer Interface
- 1. Android and the Brain-Computer Interface JIM MCKEETH EMBARCADERO TECHNOLOGIES JIM.MCKEETH@EMBARCADERO.COM @JIMMCKEETH
- 2. ‹#› About Jim McKeeth • Lead World Wide Developer Evangelist • Host of Podcast at Delphi.org • Longtime developer ◦ Object Pascal, Java, C#, JavaScript, Objective-C, etc. • Invented and patented swipe to unlock in 2000 ◦ US Patent # 8352745 & 6766456, etc. • Improvisational performer • I am not a neuroscientist, but I find this all fascinating! 2
- 3. ‹#3› Embarcadero Technologies • Founded 1993 • Provides Architect, Database and Developer Tools • 3.2 Million Customers including 97% of Fortune 2000 • 35+ Product Awards – Consistent Innovation • 500+ Employees in 29 Countries •www.embarcadero.com
- 4. ‹#› Agenda • The Science of Reading the Brain • The Science of Direct Brain Input • Consumer Brain-Computer Interfaces • Demonstration - Volunteer flying the Quadricopter • Summary & MindTunes Images from Wikimedia Commons or Product websites. 4
- 5. -Claus Torp Jensen (@ClausTorpJensen) Senior Technical Staff Member and Chief Architect API Economy at IBM (@ibmapimgt) ‹#› “In five years, science fiction will be real!” 5 What will the world be like in five years?
- 6. ‹#› Brain-Computer Interface ST:TNG Episode 93 (4x19): The Nth Degree (1991) 6
- 7. ‹#› Brain Reading and Writing SQUIDS from Strange Days (1995) X
- 8. The Science of Reading the Brain
- 9. ‹#› Vocabulary and Roots • Encephalon = Latin. The brain. • -graphy (sufix) = Greek. One who draws or writes. • Encephalography = Drawing (or picture) of the brain • Poly- (prefix) = Greek. Many • Tomos = Greek. Cut into parts or sections. • Neuro- (prefix) = Greek. Relating to nerves or nervous system • Hemo- (prefix) = Greek. Pertaining to blood. • Dynamics = Greek. Active; in motion X
- 10. ‹#› Understanding the Brain X Delta 4 Hz Hemodynamics (Blood flow) Theta 4-‐7 Hz Alpha 7-‐14 Hz Mu 8-‐13 Hz Beta 15-‐30 Hz Gamma 30-‐100 Hz Neural Oscillation
- 11. ‹#› Polygraph (aka Lie Detector) • Measures physical manifestations of stress that come with lying • Blood pressure, pulse, respiration, skin conductivity, etc. • Looks for changes. • Requires specialized training. • Invented in 1921 8
- 12. https://developer.bluetooth.org/TechnologyOverview/Pages/HRP.aspx ‹#› Heart Rate Monitors on Android • Bluetooth GATT Profiles supports all Bluetooth Smart Heart Rate Monitors. • Your app doesn’t need to support individual heart rate monitors. • Assigned Number: 0x180D 9 https://developer.bluetooth.org/gatt/profiles/Pages/ProfilesHome.aspx
- 13. ‹#› Magnetic Resonance Imaging (MRI) • Uses oscillating magnetic fields at different resonant frequencies. • Functional MRI (fMRI) uses software to analyze images. • Shows activity (blood flow) in different parts of the brain. • Combined with brain mapping can reveal thought information. 10
- 14. CT / CAT Scan (Computed Tomography X-ray) • Faster & cheaper than an MRI • Uses ionizing X-ray radiation • Lower detail images • Sometimes combined with Positron ‹#› Emission Tomography (PET) scans which use gamma radiation. X
- 15. ‹#› Near-Infrared Spectroscopy (NIRS) • Spectroscopic use of near-infrared (700-900nm) light through the skull and detecting degree remerging light is attenuated. • Detects blood oxygenation levels in different areas of the brain. • More portable than fMRI but only scans cortical tissue. X
- 16. ‹#› Electroencephalography (EEG) •Measures electrical activity along the scalp. • Detects voltage fluctuations resulting from ionic current flows within neurons of brain. •Millisecond-range resolution (faster than CT or MRI) • Typical clinical usage has 19+ input electrodes as well as ground and reference. • Invented between 1875 and 1924 11
- 17. ‹#› Electroencephalography (continued) • Series of electrodes placed next to scalp. • Difference in voltage is measured between each electrode and the reference. X
- 18. ‹#› Magnetoencephalography (MEG) 12 Uses Arrays of SQUIDs (Superconducting QUantum Interference Devices)
- 19. The Science of Direct Brain Input
- 20. ‹#› Change Perception of Reality The “headjack” from The Matrix movies -‐ 1999 14
- 21. Transcranial Direct-Current Stimulation (tDCS) • Constant, low current delivered via electrodes. • Results: ◦ Increase cognitive performance ◦ Enhance language & math ability ◦ Improve attention span ◦ Aid problem solving, memory & coordination • Recent DIY “Brain Hacking” Craze ◦ http://www.reddit.com/r/tDCS 15 ‹#› Accelerated Learning tdcsplacements.com http://thebrainstimulator.net/
- 22. ‹#› tDCS with Mobile • foc.us just launched new tDCS headsets with Bluetooth API • Developer devices are available for preorder • Targeting athletes and gamers • http://www.foc.us/index.php/v2-tdcs-developer- edition • http://www.foc.us/bluetooth-api 16
- 23. ‹#› Transcranial Magnetic Stimulation (TMS) • Similar strength to MRI •Magnetic induction of electrical currents • Used to treat neurological conditions • Produces: ◦ Flashes of lights in vision (phosphene) ◦Muscular activity 17
- 24. Ultrasonic Transcranial Stimulation Device • 2000 patent from Sony (US6584357 B1) ◦ Finer targeting than TMS ◦ Theoretically invoking sensory experiences ◦ Continued research and patents (US7350522 B2) 18 ‹#› http://www.newscientist.com/article/mg18624944.600 Speculative work, “There were not any experiments done.”
- 25. ‹#› Ultrasonic Neuromodulation for Spatial Resolution • Virginia Tech Carilion Research Institute study published in Jan 2014 Nature Neuroscience • Used “targeted ultrasound” to boost “sensory perception” • Targeted an area of the brain as small as the size of an “M&M” http://www.eurekalert.org/pub_releases/2014-‐01/vt-‐udt011014.php 19
- 26. ‹#X› "In neuroscience, it's easy to disrupt things. We can distract you, make you feel numb, trick you with optical illusions. It's easy to make things worse, but it's hard to make them better. These findings make us believe we're on the right path." -- William "Jamie " Tyler, Assistant Professor Virginia Tech Carilion Research Institute
- 27. 2‹#0› Using More than 10% of Our Brain? We all use 100% of our Brain Capacity LUCY 2014
- 28. Science Fiction Becoming Reality Interesting Milestones
- 29. ‹#› Brain-to-Brain Input The “Hat” from Brainstorm -‐ 1983 22
- 30. ‹#› Brain-to-Brain Input at University of Washington in 2013 • Input from Computer • EEG from brain • Transmitted over campus internet • Replayed through TMS to another brain • Output to computer More information http://bit.ly/EEG2TMS & http://bit.ly/bci2cbi 23
- 31. ‹#› Cybernetic Limbs Captain America: The Winter Soldier -‐ 2014 X
- 32. ‹#› DARPA’s Brain Controlled Prosthetic Arm • Part of DARPA’s Innovation Pathway. • Uses microchip implanted in brain. • 27 different fine grained controls to arm, hand and fingers. • Expected on market by 2016 X http://www.fastcompany.com/1725799/darpas-‐mind-‐controlled-‐ prosthetic-‐arm-‐could-‐be-‐market-‐four-‐years
- 33. ‹#› See What You See - Hear What You Hear The “Monitor” from Ender’s Game – 2013 24
- 34. ‹#› See What You See • In 2010 another team lead by Shinji Nishimoto partially reconstructed YouTube videos based on brain recordings. • Used fMRI • Overlaid similar videos to create composite video. • http://youtu.be/nsjDnYxJ0bo & http://youtu.be/EdGjB6XCXOA http://bit.ly/ReconstructingVisual 25
- 35. ‹#› See What You See X • In 2008 IBM used a feedback loop based on the fusiform face area of the observer’s brain for facial recognition. • Patent # US20100049076 A1
- 36. ‹#› Hear What You Hear • 2012 - UC Berkeley & Brian Pasley decoded internal neural processing of auditory information. • Reconstructed sound on computer from electrical signals in observer’s brains. • Resulting sound was recognizable when compared to original. http://bit.ly/ReconstructingSpeech 26
- 37. ‹#› Identifying Thoughts • 2008 & 2009 Using fMRI recognized thoughts within the human brain. • Discovered similar thoughts appear similar is different subject’s brains. • Was able to discern which of 10 objects subject was thinking about. • Also predicted subject intentions up to 10 seconds before subject expressed them. http://bit.ly/IdentifyThoughts X
- 38. ‹#› Data Entry via BCI ST:TNG Episode 93 (4x19): The Nth Degree (1991) 27
- 39. ‹#› Typing via BCI • Brain researcher Jonathan Wolpaw at Wadsworth Center in Albany, NY •Wear EEG while looking at a flashing matrix of characters • Each character flashes with a different pattern • ~15 seconds to type a character • The Verge http://embt.co/typebci 28 . . . . . . . . . A B C D E F . . G H I J K L . . M N O P Q R . . S T U V W X . . Y Z 1 2 3 4 . . 5 6 7 8 9 0 . . . . . . . . .
- 40. ‹#› Uploading Your Consciousness Transcendence -‐ 2014 29
- 41. ‹#› Human Connectome Project •Map of neural connection the brain • The neurological version of the genome • Unique to each person, changes over time • Representation of memories, thoughts, etc. •www.ted.com/talks/sebastian_seung • humanconnectomeproject.org • humanconnectome.org Image by Xavier Gigandet et. al. 30
- 42. ‹#› OpenWorm Project •Mapped the 300+ neurons and 7000+ connections of a worm (C. elegans) • Created a software model of connectome • Connected to LEGO robotic representation • Behaved like worm (without training or programming) • http://bit.ly/openwormlego & http://www.openworm.org/ 31
- 43. As of July 2014 ‹#› Wearable User Interfaces expectations Speech-to-Speech Translation Hype Cycle Brain-Computer Interface Internet of Things Autonomous Vehicles Disillusionment time Plateau of Productivity Slope of Enlightenment Trough of Peak of Inflated Expectations Innovation Trigger Plateau will be reached in: less than 2 years 2 to 5 years 5 to 10 years more than 10 years obsolete before plateau http://www.gartner.com/newsroom/id/2819918 32 Gartner's 2014 Hype Cycle for Emerging Technologies August 11, 2014 BCI
- 44. Read-Only Consumer Brain-Computer Interfaces
- 45. ‹#› Consumer BCI Devices • Emotiv EPOC, Insight & EPOC+ www.emotiv.com • NeuroSky’s MindWave neurosky.com • OpenBCI www.openbci.com • Neuroelectrics Enobio www.neuroelectrics.com • InteraXon’s Muse choosemuse.com X
- 46. NeuroSky’s MindWave (also in XWave headsets) 3‹#4› • EEG with 1 dry electrode • Measures “attention,” “meditation” and eye-blinks • Bluetooth communication • SDK for iOS, Android, PC & Mac • $129.99 to $79.95 retail neurosky.com • Hacking: ◦ http://makezine.com/2011/05/10/behind-the-brain-blinker/ ◦ http://hackaday.com/2012/12/20/modifying-an-eeg-headset-for-lucid- dreaming/
- 47. ‹#› Emotiv EPOC • 16 wet electrodes ◦ 14 EEG electrodes – read brain waves + 2 reference electrodes • Gyroscope to read head movements • 4 mental states, 13 conscious thoughts & facial expressions – 4 Processing suites • Wireless (encrypted) connection to Windows, Linux or Mac • $399 desktop or $499 for Bluetooth Smart (coming soon) • www.emotiv.com 35
- 48. ‹#› InteraXon’s Muse • 7 dry EEG sensors via Bluetooth • Focuses on “brain fitness” • 4 hour battery • Basic Mac & Windows SDK now, full SDK coming soon (incl. iOS & Android) • Works with Windows, OS X, iOS, Android, etc. • $299 available now • choosemuse.com 36
- 49. ‹#› OpenBCI • Open source hardware & software • Bluetooth, Arduino, EEG • 8 wet Electrodes per board (can daisy chain boards) • Full open EEG & BCI platform • $449 - $799 (now shipping) • www.openbci.com • No iOS or Android yet 37
- 50. ‹#› Emotiv Insight • Expected 2015 • Starting at $299 • 5+2 dry sensors • Bluetooth 4.0 LE (Smart) • Battery Life: 4+ hours • SDK: Android, iOS, Mac, Linux and Windows Platforms www.emotiv.com -‐ Comparison http://bit.ly/1cfliTn 38
- 51. ‹#› No More Woof (“work in progress”) • EEG for Dogs • Recognize brain patterns and match with mental states: • Hungry, Playful, Curious, etc. • Translates to human • By Nordic Society for Invention and Discovery (NSID) • Raised $22K on IndiGoGo http://www.nordicinvention.com/nmw.html 39
- 52. More Details on the Emotiv EPOC
- 53. ‹#› Further Specs • Channels: ◦ AF3, AF4, F3, F4, F7, F8, FC5, FC6, P3 (CMS), P4 (DRL), P7, P8, T7, T8, O1, O2 • Battery: ◦ 12 hours operation • Sampling rate: ◦ ~128Hz (2048Hz internal) • Resolution: ◦ 16 bits (14 bits effective) 1 LSB = 0.51μV • Bandwidth: ◦ 0.2 - 45Hz, digital notch filters at 50Hz and 60Hz • More information: ◦ http://emotiv.wikia.com/wiki/Emotiv_EPOC X Int. 10-‐20 locations
- 54. ‹#› Head Rotation Suite • Built in solid state gyroscope • Tracks angular velocity of head movement ◦ Yaw & Pitch ◦ No roll detection • Separate from EEG X
- 55. ‹#› Expressiv Suite • Detect facial expressions ◦ Eyelid & eyebrow positions – Wink, blink, furrow ◦ Horizontal eye movement ◦ Smile, laugh, clenching & smirking • EEG sensors picking up signals to muscles (not brain waves) • Very fast (10ms) 41
- 56. ‹#› Uses for Facial Detection • Realistic avatar expressions • Anything requiring fast input • Facial gesture detection • Basic eye tracking • Fast input & responses 42
- 57. ‹#› Affectiv Suite • Detect emotions / mental states ◦ Excitement ◦ Engagement or Boredom ◦ Meditation ◦ Frustration • Passive detection • Variable levels of each • Short & long term tracking 43
- 58. ‹#› Uses for Emotion Tracking • Realtime media evaluation • Stress analysis and management •Mood monitoring • For more accuracy combine with other data sources • Eye tracking, heart rate, etc. 44
- 59. ‹#› Cognitiv Suite • Detect conscious thoughts • Requires training • Detect 13 thought patterns with intensity level • Left, right, up, down, forward, pull • CW, CCW, left, right, sway-back, sway-forward, Disappear • Not as fast as Expresiv 45
- 60. ‹#› Uses for Cognitive Thought Identification •Whole new input method • Variable intensity for fine control • Observe subconscious responses •More training improves accuracy • Improves the model • Trains your brain to focus 46
- 61. ‹#› The Pieces 47 Emotiv EPOC Brain-Computer Interface Google Glass Wearable Computer Parrot.AR Drone 2 Simi-Autonomous
- 62. ‹#› About Appmethod • Integrated Development Environment • Multi-platform visual designer • Native compiler for Win32, Win64, OS X, iOS & Android • Framework for rapid prototyping and accelerated development with C++ or Object Pascal • Architecture allowing shared code & UI across platforms • Advanced language features like ARC, Generics, etc. 48 www.appmethod.com Starts at $25/mo per platform or C++ Android Phone for Free
- 63. ‹#› Laptop EDK My App How it Works 49 Brain Electrical Signals Wireless Signal AT*REF=1,290718208 Feedback Provides WiFi Hotspot
- 64. Demonstration Emotiv EPOC SDK + Parrot AR.Drone + Google Glass Tw eet @ A ppm e t h o d and #IoT or G+, LinkedIn, Facebook, etc.
- 65. ‹#› Emotiv EPOC SDK • Platforms ◦ Windows ◦Mac OS X ◦ Ubuntu ◦ Fedora ◦ iOS & Android coming soon 51 • Languages ◦ C / C++ / Objective-C ◦ C# (.NET) ◦ Python ◦ Object Pascal / Delphi ◦ Java coming soon
- 66. ‹#› MindTunes • Three disabled musicians • Collaborated with DJ Fresh and Julien Castet (tech) • Created dance track • http://youtu.be/HyF4ZxGhPHw • Full documentary • http://youtu.be/PgfxKZiSCDQ 52
- 67. More Information • Email: jim.mckeeth@embarcadero.com • Emotiv EPOC: www.emotiv.com • Parrot AR.Drone: projects.ardrone.org • Appmethod: www.appmethod.com • My blog: delphi.org • Full Slide Deck: http://www.slideshare.net/jimmckeeth/jim-mc-keeth- ‹#› wearable-thought-input • Code: https://github.com/jimmckeeth/Delphi-Emotiv-EPOC 53