1. Pervasive behavior
tracking for cognitive
assistance

2. Reference
 Sylvain Giroux , Jérémy Bauchet, Hélène Pigot, Dany
Lussier-Desrochers, and Yves Lachappelle, “Pervasive
behavior tracking for cognitive assistance”
PETRA'08(PErvasive Technologies Related to Assistive
Environments), July 15-19, 2008, Athens, Greece

3. Outline
 Introduction
 Autonomy and Cognitive Deficits
 Smart Homes
 Sensors network
 Middleware
 Applications
 Behavior Tracking
 Cognitive Assistance to ADLs
 Prototypes
 Experimentation
 Future Works
 Conclusion

4. Introduction
 In many cases, people with cognitive impairments
would can stay at home if a light assistance was
provided, but resources are scarce
 The current demographic trends bring to forecast a
dramatic increase of demand for care resources
from people with cognitive impairments
 Smart homes are environments augmented with
networked sensors, communicating objects,
embedded computers, and information appliances

5. Autonomy and Cognitive Deficits
 In a study made both in Quebec and France [10], four
categories of cognitive deficits [11] were identified as
primary responsible of autonomy loss in the daily life of
cognitively impaired people:
Initiation deficits
Planning deficits
Attention deficits
Memory deficits

6. Smart Homes

7. Smart Homes

8. Sensors network
 electromagnetic contacts
 electronic tags by the mean of Ultra Wide Band
(UWB) enabling their localization in 3D.
 RFID tags
 Flow meters
 Powerline communication devices
 infrared sensors movement detectors(big) or
sensitive rugs(small)

9. Middleware

10. Applications
 Medical assessment
 Cognitive assistance
 Tele-monitoring
 This paper focus on services dedicated to cognitive
assistance for activities of daily living (ADL)

11. Behavior Tracking
 Carberry identifies three main issues:
system robustness in the face of noise in the input
effective discrimination among competing hypothesis
recognition algorithms that scale up to large domains
 More theoretical and long term approach are based on
lattice-based models enhanced with probabilities to recognize ADLs
and to anticipate erroneous plans classified according to cognitive
errors
Bayesian networks
Petri nets combined to a tangible user interfaces approach based on
the “Token and constraints” model
 rule-based approaches and ad hoc modeling

12. Cognitive Assistance to ADLs

13. Cognitive Assistance to ADLs
 The interaction modality (visual, vocal, video, for instance
highlighting an object with a LED) also have to be chosen,
according to the user profile and the assistance strategy
 need to set some principles to guides the development of
cognitive assistants for cognitively impaired people
The assistant has to foster the autonomy of the person
The system is not intended to replace caregivers so there is always a
person at the end of the system
The user should have control, so it is a mixed control of the
interaction
The hardware and software setting should be as unobtrusive; so as
least sensors as possible will be involved.

14. Prototypes
 Two prototypes were implemented for assisting
cognitively impaired people at home.
 The first one is monitoring ADLs related to the
morning routine
 The second one is focusing a specific ADL, namely
meal preparation

15. A general pervasive cognitive
assistant

16. A general pervasive cognitive
assistant

17. Archipel
 simplified illustrated recipes books; the Visual Assistant
provided by AbleLink
Contextual information is neither used nor available
 Archipel is a cognitive assistant relying on the
hierarchical structure described previously and
interacting with the environment in a manner
 through the IO Event server → advance automatically to
the next step

18. Archipel

19. Archipel

20. Experimentation
 12 people with mild intellectual disability has been
performed in the smart apartment of DOMUS
 Some of them were not able to read
 For each participant, the experimentation was performed
over a three-day period
 Help needed has been compared for both experimental
conditions.
 Preliminary data analysis suggests that Archipel reduced
human assistance by half

21. Future Works
 Developing deep cognitive modeling to better anticipate
errors and analyze their causes to provide for more
subtle assistance.
 Developing multi person localization services because
currently our localization systems works best when
there is just one person in the apartment which is not a
realistic setting.
 Extending Archipel towards a Virtual Community
Kitchen

22. Conclusion
 Smart textiles, sensor networks, ubiquitous input and
output devices could combine to provide what can be
considered a computer-based cognitive prosthetics
 proposed a cognitive prosthetic which interacts with the
person and assist him through the environment
 The cognitive assistant supports the person during the
completion of ADLs in a non-intrusive way