Knowledge workers increasingly use multiple devices such as desktop computers, laptops, cell phones, and PDAs for personal information management (PIM) tasks. This paper presents the results of a study that examined users' mental workload in this context, especially when transitioning tasks from one device to another. In a preliminary survey of 220 knowledge workers, users reported high frustration with current devices' support for task migration, e.g. accessing files from multiple machines. To investigate further, we conducted a controlled experiment with 18 participants. While they performed PIM tasks, we measured their mental workload using subjective measures and physiological measures. Some systems provide support for transitioning users' work between devices, or for using multiple devices together; we explored the impact of such support on mental workload and task performance. Participants performed three tasks (Files, Calendar, Contacts) with two treatment conditions each (lower and higher support for migrating tasks between devices.) Workload measures obtained using the subjective NASA TLX scale were able to discriminate between tasks, but not between the two conditions in each task. Task-Evoked Pupillary Response, a continuous measure, was sensitive to changes within each task. For the Files task, a significant increase in workload was noted in the steps before and after task migration. Participants entered events faster into paper calendars than into an electronic calendar, though there was no observable difference in workload. For the Contacts task, time-on-task was equal, but mental workload was higher when no synchronization support was available between their cell phone and their laptop. Little to no correlation was observed between task performance and both workload measures, except in isolated instances. This suggests that neither task performance metrics nor workload assessments alone offer a complete picture of device usability in multi-device personal information ecosystems. Traditional usability metrics that focus on efficiency and effectiveness are necessary, but not sufficient, to evaluate such designs. Given participants' varying subjective perceptions of these systems and differences in task-evoked pupillary response, aspects of hot cognition such as emotion, pleasure, and likability show promise as important parameters in the evaluation of PIM systems.

1. Mental Workload at
Transitions between Multiple
Devices
in Personal Information
Management
Manas Tungare
& Manuel Pérez-Quiñones

2. Highlights from
Mental Workload at
Transitions between Multiple
Devices
in Personal Information
Management
Manas Tungare
& Manuel Pérez-Quiñones

3. Personal information,
Multiple devices

4. Problem Domain
• Is there a difference in mental workload
when users perform tasks using multiple
devices?
• Increased perception of task difficulty?
• Does switching to alternate strategies
result in lower workload?

5. Mental workload
• [...] “that portion of operator information
processing capacity or resources that is
actually required to meet system
demands.” [O’Donnell and Eggemeier, 1986]
• Subjective measure: NASA Task Load Index
(TLX)
• Physiological measure: Task-Evoked
Pupillary Response (TEPR)

6. Experiment design
Files Calendar Contacts
Participant Code: Date: Treatment: Session: W T 2009S
January 5 to January 11, 2009
January 2009 February
M T W T F S S M T F S
Home Calendar
1 2 3 4 1
Week 1 5 6 7 8 9 10 11 2 3 4 5 6 7 8
12 13 14 15 16 17 18 9 10 11 12 13 14 15
January 2009
19 20 21 22 23 24 25 16 17 18 19 20 21 22
26 27 28 29 30 31 23 24 25 26 27 28
PIM Study - Home
Monday 5 Tuesday 6 Wednesday 7 Thursday 8 Friday 9 Saturday 10 Sunday 11
8 AM
9 AM
10 AM
Participant Code: Date: Treatment: Session: W T 2009S
January 5 to January 11, 2009
January 2009 February
11 AM
M T W T F S S M T F S
Home Calendar
1 2 3 4 1
Week 1 5 6 7 8 9 10 11 2 3 4 5 6 7 8
NOON
12 13 14 15 16 17 18 9 10 11 12 13 14 15
Level 0
January 2009
19 20 21 22 23 24 25 16 17 18 19 20 21 22
1 PM 26 27 28 29 30 31 23 24 25 26 27 28
PIM Study - Home
2 PM
Team Outing Monday 5 Tuesday 6 Wednesday 7 Thursday 8 Friday 9 Saturday 10 Sunday 11
3 PM
8 AM
4 PM
Dentist's appoint!
ment 9 AM
5 PM
10 AM
6 PM
Michael's Little
League game (tenta! 11 AM
7 PM tive; confirm with
Alex) NOON
8 PM
1 PM
9 PM
2 PM
Team Outing
Page 1/1
3 PM
4 PM
Dentist's appoint!
ment
5 PM
6 PM
Michael's Little
League game (tenta!
7 PM tive; confirm with
Alex)
8 PM
9 PM
Page 1/1
No support for Multiple paper No support for
le migration calendars synchronization
Level 1
System supports Devices support
Online calendars
le migration synchronization

7. Files task, L0 — TEPR
B
B B
20
L0
L1
AB
AB
AB
!
!
10
!
A
Adjusted Pupil Radius
A
! A !
0
!
!
!
! Transition Step
−0
1
1 2 3 4 5 6 7 8 9

8. Calendar task — TEPR
Not
signi cant
L0
L1
20
Adjusted Pupil Radius
10
! !
! ! !
! ! ! !
!
! !
! !
!
!
!
! !
0
! ! !
! ! ! !
!
!
!
!
−0
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Step within Task

9. Contacts task — TEPR
0.00623 0.01158 0.0224
0.03476
0.007127 L0
L1
0.04138
25
20
Adjusted Pupil Radius
15
! !
!
! !
10
!
5
!
! !
!
!
!
0
−
5
1 2 3 4 5 6
Step within Task

10. Files L0: Migration Step
Files Task, Participant P8, Level L1
30
Migration Step
20
Migration Step
Pupil Radius (eye image pixels)
10
0
S1
S2 S3 S4 S5 S6 S7 S8 S9 S10
−0
1
−0
2
−0
3
200 400 600 800

11. Files L0: Migration Step
Files Task, Participant P18, Level L0
30
20
Pupil Radius (eye image pixels)
10
0
S0 S1
S2 S3 S4 S5 S6 S7 S8 S9S10
−0
1
Migration Step
−0
2
−0
3
200 400 600 800 1000

12. Files — Time
600
Without Sync Support
With Sync Support
500
F(1,34)=8.8318
p=0.005404
400
Time Taken (s)
312 (276)
300
200
109 (83)
!
100
!
!
! !
! ! !
!
!
0
1 2 3 4 5 6 7 8 9 10

13. Calendar task — Time
F(1,34)=4.1842
p=0.0486 Both 2 & 6 required
Paper Calendar
Online Calendar
scheduling an event
80
!
F(1,34)=4.3345
p=0.04495
!
60
!
!
Time Taken (s)
!
40
!
!
!
20
! !
!
!
! ! !
!
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

14. Contacts task — Time
No Sync Support
Sync Support
150
!
Time Taken (s)
!
100
!
50
!
!
! !
1 2 3 4 5 6 7

15. Calendar task — TEPR, Time
No Sync Support
Sync Support
150
!
L0
L1
25
20
Time Taken (s)
!
100
Adjusted Pupil Radius
15
! !
!
! !
10
!
5
!
50
!
! !
!
!
!
!
! ! !
0
−
5
1 2 3 4 5 6 1 2 3 4 5 6 7
Step within Task
Step #
Same time, but higher workload in L0:
More mental workload for same performance

16. Workload — Time
• TLX correlation signi cant only for Files.
• No signi cant correlation between TEPR
and time.
• Need to evaluate both separately.

17. TEPR: Files
Files Level L0 Files Level L1
!
! !
!!
!
!
!
!!
20
! !
!!
!
! ! !
! !!
60
!! !
!! ! !
!! !
! !
!
!! ! !
!!
!! ! !
! !!
10
!! !! ! !
!
!
!! ! ! !
! !
!! ! !
!
40
!! ! ! ! !
! ! ! !
Adjusted Pupil Radius
Adjusted Pupil Radius
!! ! ! !! !
!
! !
! ! ! ! ! !
!! !
0
! ! !! !
!
! !! !
! ! !!
! ! !
! ! !
! ! !!
! ! ! !
!
! ! ! !! !
! !
!
! !
! ! !
20
!! ! !
! ! !! ! ! !
!! !
−0
! ! ! !!
!
! !
1
! !
! ! ! !!
! !!
! ! ! ! ! ! ! !
! !!
! !
! !! ! !!
!
!
! !
! ! !
! ! ! !
!! ! ! !
! !
! !! ! ! !!
!
!! !
! ! ! !
0
! ! !
! !
−0
! ! !!
! ! ! !
2
! ! ! !
! ! !!! ! !
!
! !
!
! ! ! ! !
! ! ! ! !
!! ! !! !!
!
!
! ! ! ! ! !! !
! !
!! !! ! !
!
!! !
−0
! !
−0
3
! !
2
!
! !
0 200 400 600 800 1000 1200 0 50 100 150 200 250 300
Time on Task (seconds) Time on Task (seconds)
r=-0.062, p=0.46 r=0.15, p=0.06

18. TEPR: Calendar
Calendar Level L0 Calendar Level L1
60
! !
! !
! ! !
!
40
! ! !
!
!
!! !
! ! ! !
!
40
!
! ! !
!! !! !
!
!
! ! !
! ! ! !
Adjusted Pupil Radius
Adjusted Pupil Radius
!
! !
!! ! ! !
!
!
20
! ! ! ! ! !
! !
! ! !
! ! !
! ! !
! ! ! ! !
! !
20
! !
!! ! ! ! ! !
! ! ! !
!! ! ! ! ! ! ! !
! ! ! ! ! !! !
! ! ! ! !! ! !
! ! !!! ! !
! ! !
!
!! ! !! ! ! ! ! ! ! ! ! !
! ! !
! !! !!!
! ! ! !! !!! !!! ! !
!
! ! !
! ! ! ! !! !
!! ! ! !!! ! ! ! ! !
0
! ! ! ! ! !! ! ! ! !
!
!! !! !! ! !
! ! !! !! !
! ! ! !
! ! ! !
! ! !! ! ! ! ! !! ! ! !
! !
!!!!!
! !! ! ! ! ! !
! ! ! !
! ! ! !!!! ! ! !!! !
!!! !
!
!
!! ! ! !
! ! ! ! ! !
! ! ! ! ! ! !
! ! ! ! ! ! ! ! !
! ! ! !! ! !! ! !! !! !! ! !!!! !!
!! ! ! ! !! ! !! ! !
! ! !! ! ! ! !
!!!!! ! ! !!!!!
! ! ! !
0
! ! !
!! !! !
!! ! !
! ! !
! !!
! ! ! ! ! ! ! !
!!!
!!
!
!
! ! ! !! ! ! !
!
!
!
! ! ! ! !
! ! !! !! !
! ! ! ! ! ! ! ! !
!! ! ! !!! ! ! !
!!!
!! ! ! ! !!
! !
! !! !
! ! ! !
! ! ! !
!
! !
! !
! !
−0
! ! ! ! ! !
2
! !! ! ! !!
! ! !
! ! !! !
! !
! ! ! ! ! !
! ! ! !
!!
! !
−0
! !
2
0 20 40 60 80 100 120 0 20 40 60 80 100 120
Time on Task (seconds) Time on Task (seconds)
r=–-0.11, p=0.08 r=–0.067, p=0.28

19. TEPR: Contacts
Contacts Level L0 Contacts Level L1
! !
! !
! ! !
!
60
! ! !
! ! !
!
! !
! !
!
10
! !
! ! !
!! !
!
!
!
!
!
!!
! !
!! ! ! ! !!
!
40
! ! ! !
Adjusted Pupil Radius
Adjusted Pupil Radius
! ! !
!
! !
! !!
! ! !
! ! !
!
! !! !
! ! ! !
0
!! !
!! ! !
!
!
! !
!
! ! !
!
! ! ! ! !
!!
20
! ! !! ! ! !
! ! !! !
! !! ! ! !
! !
!
! ! !
! ! !!
! ! ! !
!
! ! !! ! ! !
! ! ! ! ! ! ! !
!
−0
! ! ! !
! ! !! !
1
!
! ! ! ! !
! ! !
! !! ! !!
! !
!! ! !!
! ! ! ! !
! ! !
! ! !
! !
0
!! !
! !
! ! !
! !
! ! !!
! ! !
−0
2
50 100 150 200 50 100 150 200 250 300 350
Time on Task (seconds) Time on Task (seconds)
r=–0.013, p=0.18 r=0.415, p=0.68

20. Mental Workload in PIM
• Workload is a useful metric to understand
PIM at a sub-task level.
• Provides alternative to performance
measurement (e.g. same performance but
high workload.)
• Continuous measure of workload provides
important information about sub-tasks that
subjective measures could not capture

21. Holistic Usability
• Computing is situated, cognition is
distributed
• ∴ Must evaluate tools & tasks in situ.
• Device transitions are not designed well.
• Traditional usability: effectiveness,
efficiency and satisfaction (ISO de nition)

22. Future work: Hot Cognition
• Maslow’s hierarchy applied to interaction:
• 1: Function, 2: Usability, 3: Pleasure.
• Usability is necessary, but not sufficient.
• Emotion, pleasure, frustration, likability
• Quality of life, Perceived ease of use/
usability metrics, happiness.

23. Questions & comments
?
!

24. Questions & comments
?
!
Thank you!