Presented by: Trish Ververs, Engineering Fellow

1. Brain-based Technologies for Design
and Evaluation of Complex Systems
Trish Ververs, Engineering Fellow
October 2013

2. Honeywell.com
2
Overview
• Neurotechnologies
• Why brain-based systems
• Closed loop cognitive systems
• An example of use in design and evaluation

3. Honeywell.com
3
What is Neurotechnology
• Neurotechnology is a field of science that studies the body and
mind through the nervous system by electronics and mechanisms
• Neuroergonomics is the application of neuroscience to ergonomics
through the understanding biological explanations

4. Honeywell.com
4
Chaotic Situations
Air France
A330 Flight 447
Accident,
June, 2009
US Airways
A320 Flight 1549
Accident,
January, 2009

5. Honeywell.com
5
Human Computer Interaction
PROBLEM
• Increased system complexity
• Lack of transparency
• System provides visual, aural, tactile information
via displays, alerts, graphics on system state
SOLUTION
• Leverage human capabilities
• Make a system easy to learn and easy to use
• Provide a real-time assessment of user’s cognitive effort using the
system
• Provide a direct and natural means of interaction with the system;
not limited to mouse clicks and buttons presses
Note: Large pipeline from system (A),
small pipeline to system (C)

6. Honeywell.com
6
Real Time Cognitive Assessment Technologies
Eyetracking/PupillometryNeuro-imaging (fMRI)
EEG assessment
Physiological measures
Speech

7. Honeywell.com
7
Examples of Measures
Transcranial Doppler
temporal resolution
spatial resolution
direct measure of brain activity
transferable to real world applications
fMRIEEG Spectral Power

8. Honeywell.com
8
Generic Neuro-based System Overview
Behavioral Protocol
System State
Video/Audio
System Inputs
…
Sensing
EEG
ECG
fNIR
Signal Processing
Artifact Detection
Artifact Correct.
Spectral Transf.
Estimation
Working Memory
Drowsiness
Attention
Info Proc Speed
Cognitive State Assessment
Performance
Task ExecutiveCognitive
State
during
Work
Intelligent
Assistance
Context
User Models
Situation Models

9. Honeywell.com
9
Augmented Cognition Test Event
• Aberdeen Test Center, Mulberry Point
• Two-week, full platoon MOUT training
– North Carolina National Guard 252nd Combined Arms
Battalion
– Force on force exercises with simunitions
– Multiple task performance under time pressure and
fatigue
• 24-hour exercise (7 hr blocks)
– Exercises: Route Reconnaissance, Cordon and Search,
Long Hasty Defense
– Mission: Low & high workload events
• Purpose:
– Assess cognitive workload in real-time of team leaders
based on task engagement
– Test ruggedness of sensors
• Mission Goals: Improve survivability and
increase mission effectiveness
Aberdeen Test
Center
USARIEM
Human
Research
Engineering
Directorate
Development Test
Command
Battle Lab
Integration Team,
Natick Soldier
Systems
NC Army National
Guard
Real-time Cognitive State Assessment of the Dismounted Warfighter
A
T1
B
C
D E
F
G
H
I
J

10. Honeywell.com
10
Characterization of Workload
• Tested classifiers on other high and low periods to evaluate our ability
to classify mental workload during operational exercises
• Focus: Platoon Leader (PL) and Platoon Sergeant (PSG)
– Results fused information from EEG spectral power, ECG spectral power,
and ECG inter-beat interval (IBI)
– Presented for 2 classification sampling techniques (red= 2 fold, green= 10
fold) across different temporal smoothing windows (seconds on X axis)
base 2.5 5 10 20 40 60
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
classification as a function of temporal smoothing
smoothing window (seconds)
Az
2-fold x-val
10-fold x-val
Platoon Leader Platoon Sergeant
Classification performance on Y axis (note: Az = .50 chance)

11. Honeywell.com
11
The Future of Neurotechnologies for Aviation
Where do we go from here?
• Electronic pilot assistant
• Single pilot operations – Reduce General Aviation accident rate
• Transoceanic flights – Reduce crew requirements, monitor
fatigue
• Unmanned aerial vehicles – remotely piloted vehicles

12. Honeywell.com
12
Where Will We See Neurotechnology…
• Specialized Training
• Military and Defense Systems
– Intelligence Analysts
– Unmanned Aerial/Ground Systems
– Cognitive Rehabilitation
– Readiness for Duty
• Automotive Systems
• Gaming
• TSA Monitoring
• Medical or Behavioral Treatments
– Alzheimer’s
– Spinal cord injury
– Traumatic brain injury
– Addiction
– ADHD