This work presents DemaWare, an Ambient Intelligence platform that targets Ambient Assisted Living for people with Dementia. DemaWare seamlessly integrates diverse hardware (wearable and ambient sensors), as well as soft- ware components (semantic interpretation, reasoning), involved in such context. It also enables both online and offline processes, including sensor analysis and storage of context semantics in a Knowledge Base. Consequently, it orchestrates semantic interpretation which incorporated defeasible logics for uncertainty handling. Overall, the underlying functionality aids clinicians and carers to timely assess and diagnose patients in the context of lab trials, homes or nursing homes.

1. The DemaWare Service-Oriented
AAL Platform for People with
Dementia
Thanos G. Stavropoulos*
Georgios Meditskos
Efstratios Kontopoulos
Ioannis Kompatsiaris Centre for Research and
Information Technologies
Institute
Technology Hellas
This work has been supported by the FP7 project Dem@Care: Dementia Ambient Care - Multi-Sensing Monitoring for Intelligent
Remote Management and Decision Support (No. 288199)

2. Outline
1. Introduction
2. State of the art
3. DemaWare Components
4. DemaWare Architecture
5. Semantic Interpretation

3. Introduction
• Goals
• Timely diagnosis and assessment of People with
Dementia
• Support various target pilot scenarios
• Labs, Homes, Nursing Homes
• DemaWare is an AAL platform to unify
• Data retrieval from heterogeneous sensors
• Multi-modal analysis algorithms
• Semantic knowledge storage and interpretation

4. Related Work
• OpenAAL [8], FamiWare [9]
▫ Support knowledge management, Service
composition, fusion etc.
▫ Yet lacking diverse hardware support
• Various existing middleware e.g. for smart homes
▫ AIM [12], Hydra [13], aWESoME-S [11], [14]
▫ Do not provide higher-level functions

5. DemaWare Components
• SleepClock (Gear4) logs sleep states, time, duration
and interruptions
• Wristwatch (Philips DTI2) logs moving intensity, skin
conductance and temperature
• An ambient depth camera (Asus or Kinect) is used for
detecting the user’s location (within zones) and
performed activity (Complex Activity Recognition)
• A camera closer to the user is used for activity
recognition using different models (Human Activity
Recognition)

6. DemaWare Components (2)
• A wearable camera (GoPro) is used for object, room and
activity detection
• Wireless microphones are used for Offline Speech
Analysis, which returns various dementia indicators
• The Knowledge Base (KB) Manager stores all detected
events, measurements and activities in a semantically
enhanced format
• The Semantic Interpretation (SI) module performs
analysis on collected data to infer higher-level
information, sensor fusion and complex event detection

7. DemaWare
• Some components work on various, remote platforms
(OS), some online and some offline
• Need for
▫ complex data transfer under common schema
▫ Uniform, platform-independent API
• DemaWare unifies all components under WSDL/SOAP,
using an XML/XSD Exchange Model
• Meanwhile real-time events are streamed to the KB-service
• Various GUIs visualize data according to scenarios
• GUI backend modules coordinate data collection and
processing

9. Semantic Interpretation
• Need to integrate and process data of different
sensors and modalities
▫ e.g. contact sensors, cameras, microphones
• By combining different modalities we can infer more
about the context
▫ any information that can be used to characterise the situation of
an entity (e.g. the condition of the patient)
• Use of OWL ontologies and rules
▫ Ontologies provide the domain vocabulary for representing
activity-related contextual information (representation layer)
▫ Rules define the structure and semantics of the complex
activities (interpretation layer)

10. Event Ontology

11. Problem Ontology

12. Interpretation Layer
• Hybrid of OWL reasoning and SPARQL rule execution
for inference of complex activities
• Key inferencing tasks:
▫ Temporal reasoning: SPARQL rules to identify
temporal dependencies
▫ Complex correlations: SPARQL rules to overcome
OWL’s tree model property (composite activities)
▫ Assertion of new individuals: SPARQL rules to
generate composite activity individuals

13. Example
• Night sleep monitoring scenario in an ambient assisted
living environment
▫ Atomic activities: {NightSleep, OutOfBed,
InBathroom}
▫ Complex Activities: {BedExit, Nocturia}
x BedExit: An OutOfBed incident during a NightSleep
(classification of OutOfBed in the BedExit class)
x Nocturia: An incident that involves a BedExit incident and an
InBathroom incident (composition of a Nocturia incident out of a
BedExit incident and an InBathroom incident)

14. Rule for classifying an OutOfBed into the BedExit class
CONSTRUCT {
?y a BedExit;
hasClassifier ?x.
}
WHERE {
?x a NightSleep;
hasStartTime ?st1;
hasEndTime ?et1;
hasActor ?p.
?y a OutOfBed;
hasStartTime ?st2;
hasEndTime ?et2;
hasActor ?p.
FILTER(
:contains(?st1, ?et1, ?st2, ?et2)
)
}

15. Rule for composing a Nocturia instance
CONSTRUCT {
?new a Nocturia;
hasStartTime ?st1;
hasEndTime ?et1;
hasActor ?p;
hasSubActivities ?x;
hasSubActivities ?y.
}
WHERE {
?x a BedExit;
hasStartTime ?st1;
hasEndTime ?et1;
hasActor ?p.
?y a InBathroom;
hasStartTime ?st2;
hasEndTime ?et2;
hasActor ?p.
FILTER(
:contains(?st1, ?et1, ?st2, ?et2)
)
BIND(:newURI(?x, ?y) as ?new)
FILTER NOT EXISTS {?new a [] .}
}

16. Conclusion
• The system so far enables:
▫ Both real-time and offline data collection and
processing
▫ Collection and processing of multi-modal data
▫ Fusion and Semantic Interpreation of data
▫ Various assessment scenarios
▫ Data visualization

17. Future Work
• Enrich real-time data collection
▫ Energy data for powered appliances
x Detect cooking, lighting, watching tv etc.
▫ Motion from objects
x Detect book reading, watering plants, taking pills etc.
▫ Wearable wristband
x More acceptable 24/7
x Detect daily physical activity patterns
• Extensive pilots to infer patterns based on data
• Extended AAL @Homes

18. Thank you

19. References
1. Anna Fensel, Slobodanka Tomic, Vikash Kumar, Milan Stefanovic, Sergey V Aleshin, and Dmitry O
Novikov, ‘Sesame-s: Semantic smart home system for energy efficiency’, Informatik-Spektrum, 36(1),
46–57, (2013).
2. Richard Etter, Patricia Dockhorn Costa, and Tom Broens, ‘A rule-based approach towards context-aware
user notification services’, in Pervasive Services, 2006 ACS/IEEE International Conference on, pp. 281–
284. IEEE, (2006).
3. Markus Eisenhauer, Peter Rosengren, and Pablo Antolin, ‘A development platform for integrating
wireless devices and sensors into ambient intelligence systems’, in Sensor, Mesh and Ad Hoc
Communications and Networks Workshops, 2009. SECON Workshops’ 09. 6th Annual IEEE
Communications Society Conference on, pp. 1–3. IEEE, (2009).
4. Nikolaos Georgantas, Valerie Issarny, Sonia Ben Mokhtar, Yerom-David Bromberg, Sebastien Bianco,
Graham Thomson, Pierre-Guillaume Raverdy, Aitor Urbieta, and Roberto Speicys Cardoso, ‘Middleware
architecture for ambient intelligence in the networked home’, in Handbook of Ambient Intelligence and
Smart Environments, 1139–1169, Springer, (2010).
5. Antonis Bikakis, Grigoris Antoniou, and Panayiotis Hasapis, ‘Strategies for contextual reasoning with
conflicts in ambient intelligence’, Knowledge and Information Systems, 27(1), 45–84, (2011).
6. L. Klein, J. Kwak, G. Kavulya, F. Jazizadeh, B. Becerik-Gerber, P. Varakantham, and M. Tambe,
“Coordinating occupant behavior for building energy and comfort management using multi-agent
systems,” Autom. Constr., vol. 22, pp. 525–536, 2012.
7. Z. Wang, R. Yang, and L. Wang, “Multi-agent control system with intelligent optimization for
smart and energy-efficient buildings,” in IECON 2010-36th Annual Conference on IEEE Industrial
Electronics Society, 2010, pp. 1144–1149.
8. T. G. Stavropoulos, D. Vrakas, D. Vlachava, and N. Bassiliades, “BOnSAI: a smart building
ontology for ambient intelligence,” in Proceedings of the 2nd International Conference on Web
Intelligence, Mining and Semantics, 2012, p. 30.