Supporting Article: Fahim Kawsar, Gerd Kortuem and Bashar Altakrouri "Supporting Interaction with the Internet of Things across Objects, Time and Space "; Internet of Things 2010 Conference (IoT-2010), Nov 29 - Dec 1, Tokyo, Japan.

1. Supporting Interaction with the Internet of Things
across Objects, Time and Space
Fahim Kawsar Gerd Kortuem and Bashar Altakrouri
Bell Labs and Lancaster University Lancaster University

2. “A
computa.onally
instrumented
tangible
object
with
an
established
purpose
that
augments
human
percep.on
and
is
aware
of
its
opera.onal
situa.ons
and
capable
of
providing
supplementary
services
without
compromising
its
original
appearance
and
interac.on
metaphor
significantly.”
-­‐
(Kawsar,
2007)
Supplementary
Services
Smart Device
Centric
Objects Situa.onal
Awareness
[Beigl 2001] [Ishii, 1997] [Ambient Device]
Connec.vity
Perceptual
Augmenta.on
[Kawsar, 2005] [Tokuda, 2004] [Intelligent Spoon, MIT]
Smart Objects

3. Implications

4. Internet of Things

5. Things + Web
Internet of Things

6. Things + Web + People
Internet of Things

7. Interactions

8. Object Centric
Spatially Fixed
Temporally Constrained
Current IoT Interactions

9. Activity Centric
Spatially Distributed
Temporally Dispersed
People’s Interactions

10. Supporting Activity Driven IoT Interactions distributed
across Objects, Time and Space.
Research Goal

11. - Supporting Spatially Distributed Tasks - Supporting Dynamic Work Environment
- Interface Consistency - Situated Task Guidance
- State Persistency - Better Support for Exception Management
- Seamless Interaction - Seamless Interaction
- Better User Experience - Better User Experience
Implications

12. - Declarative Modelling Technique to model Activity.
- Software Infrastructure to Support Task Distribution and Intra-
Object Communication.
- User Interface to enable Seamless Interaction.
Requirements

13. Situated Flow
“A situated flow is a sequential model that consists of a set of actions, stitched
together by a plan that specifies how the actions should be performed to achieve
a goal under certain constrains. In other words, a flow formalizes and maps our
activities to certain tasks to achieve a goal. It is situated and context-aware.”
Activity Model

14. o Micro Activity: This type of activity is not
decomposable, so a flow cannot be refined on
this
activity.
o Macro Activity: This type of activity is
decomposable and contains a link to another
flow. During flow association (static
refinement) or execution (dynamic
refinement), this activity is replaced with the
linked flow’s activity or sequence of activities
Flow Representation and Distribution

15. Blood
Pressure
Monitor
Flow Glucose
Meter
Flow
(Opera.ng
Procedures) (Opera.ng
Procedures)
Situated
Discovery
Step
1 Flow
Discovery
Flow
Associa.on Situated
Adapta.on
Step
2
Step
3 Flow
Refinement
Step
4 Flow
Execu.on Situated
Interac.on
Flow Driven Interaction

16. Pro]oy:
Kawsar
et
al.,
Mobiquitous
’05,
EUC
’05 Persona:
Kawsar
et
al.,
MUM
’07,
ACM
MM
’09
Requirement Specific, Simple and
Robust to address a single objective.
FedNet:
Kawsar
et
al.,
Mobiquitous
’08,
UbiComp
’08
Why Another Architecture

17. RESTful Software Architecture

18. Flow Adaptation

19. Flow Adaptation

20. Bob Bob
Record
Record
Result Result
Alice
Alice
Check
Move
to
next
ac.vity Check
Blood
Blood
Pressure Receive
New
UI Sugar
Intra-Object Communication

21. User Interface

22. Mobile Graphical UI

23. Mobile Augmented Reality UI

24. 90
Spatial UI with Mobile Projector

25. 90
Spatial UI with Wearable Projector

26. ➡
Interac.on
Framework
for
IoT
➡
Situated
Flow
for
Dynamic
Ac.vity
Modelling.
➡
Flow
Driven
RESTful
Architecture.
Situated
Flow
for
Ac.vity
Modelling
and
a
corresponding
RESTful
Soaware
Architecture
enabling
Dynamic
Discovery,
Adapta.on
and
Push
Interface
can
support
Distributed
Interac.ons
with
physical
objects
dispersed
across
Time
and
Space.
Summary

27. Fahim
Kawsar
eMail:
fahim.kawsar@gmail.com
web:
h]p://www.fahim-­‐kawsar.net
Gerd
Kortuem
eMail:
gerd@comp.lancs.ac.uk
web:
h]p://www.kortuem.com
Thank
You