Review of recent findings in cognitive neuroscience and

1. A COGNITIVE NEUROSCIENCE PERSPECTIVE ON
MEMORY FOR PROGRAMMING TASKS
Chris Parnin, Georgia Institute of Technology

2. In the year 2042, super
programmers strapped in SQUIDs
(superconducting quantum
interference devices) build
programs to save the earth from
certain doom.

3. SQUID SOUND LIKE
SCIENCE FICTION?
NO!!
These devices exist right
now -- just waiting to be
used!

4. COGNITIVE NEUROSCIENCE
Understanding how the functions
of the physical brain can yield
the thoughts and ideas of an
intangible mind.
George Miller and Michael Gazzaniga

5. Benefits?
Image no more keyboards.
Direct measures of function
and understanding.
Better predictions and
constraints from models.
COGNITIVE NEUROSCIENCE

6. Background
Anatomy, techniques,
findings with focus on
memory
Insights
Insights from models
old and new
Impact
Tools, theories,
experiments
OVERVIEW

7. LOBES
Frontal Parietal
OccipitalMedial
Regions divided
by deep fissures

8. LIMBIC SYSTEM
Deep structures
within brain.

9. FUNCTION LOCALIZATION

10. HUMAN TECHNIQUES
Electro/magno-based
Measures change in neuron activity
Affects visible within milliseconds.
Location within centimeters.
Examples
• EEG
• MEG
Blood oxygen-based
Measures change in blood activity.
Affects visible within seconds.
Location within millimeters.
Examples
• fMRI
• Optical imaging

11. ANIMAL TECHNIQUES
Gene Therapy
Bred mutant rats
Dye
Discover neural pathways
Lesions
Damage neural pathways
Habitation
The kitten who only saw vertical
lines
Cats, Monkeys,
Mice Oh My!

12. Stimulate neurons
with electricity
LONG-TERM POTENTIATION (LTP)
Effect found even after one year!

13. PLACE CELLS

14. VIRTUAL NAVIGATION
Participants navigate a
virtual town.
Recall paths.
Place cells fire.

15. NEUROGENESIS
Larger volume by
years of experience!
Size of taxi driver
hippocampus?

16. HIPPOCAMPAL ACTIVITY
Forgotten words Remembered words
Forgotten words Familiar words
(Entorhinal cortex)

17. AUTOMATIC MEMORY
We cannot anticipate important events.
We cannot always devote top-down resources to encode and select memory.
Initial memory is largely formed without conscious effort.

18. HIPPOCAMPUS ANATOMY
Grand Central of the senses
Connections to sensory
processing and semantic
recognition. Connections to
frontal regions.
Brain Hashtable
Hippocampal neurons cannot
store directly, index to original
source.
Recording factors
Novel, engaging,
high-frequency…

19. SNOWBALL TO BOULDER
Hippocampal LTP
Formed and tagged
for storage.
Consolidated memory
Replayed and stored at
original regions.
Top-down associations
Properties our bound
to memory. “Rehearsal” is
activation of speech motor.

20. INSIGHT
So what?
A review, and new perspectives.

21. Sensory Short-term Long-term
Attention Encoding
Rehearsal
MODAL MODEL OF MEMORY
(1968)
Patients with no short-term memory still have long-term memory…

22. Sensory Short-term Long-term
Episodic
Buffer (2000)
Lifetime of seconds!
“Temporarily infinite”
WORKING MEMORY (1974)
Phonological
loop
Visio-Spatial
Sketch pad Central
Executive

23. LONG-TERM WORKING MEMORY
(1995)
Good insight, theory formed without any neuroscience basis.

24. MEMORY 2.0
• Distributed
• Specialized
• Retained and Replayed
• Automatic without Rehearsal
• Immediate and Long-Lasting

25. MEMORY TYPES
Spatial
Prospective
Familiarity
Recency
Source
Semantic
…
See paper for more details!

26. Attention
Maintenance
Manipulation
Dynamic filtering
Auto
associative
retrieval
Auto
associative
encoding
Retrieval Encoding
TASK MEMORY MODEL (beta)
Indices
Spatial
Goals
Attention
Bindings
Visual
Frontal
Associative
Perceptual
ProcessRegions
Note, top-down iteration is a
“brain design-pattern” found in
many neural circuits.

27. PROGRAMMING ENVIRONMENTS

28. Often do not associate name with memories of programming.
NAME IS NOT ENOUGH

29. TOOL DESIGN: MEMORY
• Auto-association
– Better distinct cues for learning?
• Spatial memory?
– Spatial stability, isolation
• Prospective memory?
– Better reminders than a compile error or TODO?
• Forgetting?
– Code history and provenance

30. THEORIES (BETA)
Visual Chunks
Initial formation of memory.
“Below the method with
lots of strings”
Iterative Comprehension
Top-down experiences
will bind and update partial
representations.
Concepts form.
Navigational Jitter
Tabs, scrolling
huge files,
little understanding
-- need memory aids.

31. EXPERIMENTS
Gender differences in brain activity during programming?
Neurogenesis in programmer’s brain?
Place cells (spatial) during programming navigation?
Your idea?

32. MUSCLES OF THE MIND
Subvocal
speech
EMG
(electromyography)
Cognition

33. 0 100 200 300 400 500 600 700 800 900
Subvocal
Debug
Select
Click
Search
Edit
Formulating
Problem
Found
useful code
Formulating
solution
EditingFound
useful code
Evaluating
Moments of interest and difficulty

34. QUESTIONS?
Indices
Spatial
Goals
Attention
Bindings
Visual

35. Goal Memory
space-goal binding.
Lesioned monkeys
Forget food location.
Variation
Change location,
keep consistent cue