Lecture at ICTAC School 2021: 18th International Colloquium on Theoretical Aspects of Computing, Nazarbayev University, Nur-Sultan, Kazakhstan, 1st September 2021.
https://www.alandix.com/academic/talks/ICTCS-School-2021/
The second part focuses on a specific area, the formal specification and analysis of systems that have both physical and digital aspects. This will incude using physigrams, a extension of finite state networks for describing interactions with physical devices such hand-held controllers. We will also look at the ways formal analuysis contributed to the design of a internet-enabled ‘cafe open’ sign – IoT in action!
2.
Part II – getting physical
physigrams
sensor-rich interactions and IoT
ICTAC 2021
3. modelling physical things
have a go …
– what was hard or easy?
– did you need to include the environment?
physigrams
– modelling the device ‘unplugged’
– connecting digital and physical
IoT shop open sign
– modelling human processes
… including uncertainty
4.
5. multiple feedback loops physical–logical
mappings
physical aspects
knobs, dials,
buttons, location,
orientation
virtual aspects
screens,
lights,
buzzers, speakers
(ii) physical effects
(iii) virtual effects
show message,
turn light on
motors, effectors
(a) physical manipulation (i) sensed inputs
logical
system
A B
C
(c) felt feedback
(d)‘electronic’feedback
(b) perceived state
see message on screen
resistance,
? physical sounds ?
turn knob, press button
effects on
logical objects
device
physical
environment
D
(iv) physical effects
controlling
external things
(e) physical feedback
notice light is on,
kettle boils
semantic
feedback
7. multiple feedback loops
physical aspects
knobs, dials,
buttons, location,
orientation
virtual aspects
screens,
lights,
buzzers, speakers
logical
system
A
(ii) physical effects
motors, effectors
(i) sensed inputs
B
(c) felt feedback
physical–logical
mappings
(b) perceived state
(a) physical manipulation
turn knob, press button
device
8. multiple feedback loops physical–logical
mappings
physical aspects
knobs, dials,
buttons, location,
orientation
virtual aspects
screens,
lights,
buzzers, speakers
(ii) physical effects
motors, effectors
(a) physical manipulation (i) sensed inputs
logical
system
A B
(iii) virtual effects
show message,
turn light on
C
(c) felt feedback
(b) perceived state
(d)‘electronic’feedback
see message on screen
resistance,
? physical sounds ?
turn knob, press button
device
9. multiple feedback loops physical–logical
mappings
physical aspects
knobs, dials,
buttons, location,
orientation
virtual aspects
screens,
lights,
buzzers, speakers
(ii) physical effects
(iii) virtual effects
show message,
turn light on
motors, effectors
(a) physical manipulation (i) sensed inputs
logical
system
A B
C
(c) felt feedback
(d)‘electronic’feedback
(b) perceived state
see message on screen
resistance,
? physical sounds ?
turn knob, press button
effects on
logical objects
device
physical
environment
D
(iv) physical effects
controlling
external things
(e) physical feedback
notice light is on,
kettle boils
10. multiple feedback loops
the GUI fallacy … semantic feedback is NOT enough
physical–logical
mappings
physical aspects
virtual aspects
screens,
lights,
buzzers, speakers
(ii) physical effects
(iii) virtual effects
show message,
turn light on
motors, effectors
(a) physical manipulation (i) sensed inputs
logical
system
A B
C
(c) felt feedback
(d)‘electronic’feedback
(b) perceived state
see message on screen
resistance,
? physical sounds ?
turn knob, press button
effects on
logical objects
device
physical
environment
D
(iv) physical effects
controlling
external things
(e) physical feedback
notice light is on,
kettle boils
semantic
feedback
knobs, dials,
buttons, location,
orientation
11. multiple feedback loops physical–logical
mappings
physical aspects
knobs, dials,
buttons, location,
orientation
virtual aspects
screens,
lights,
buzzers, speakers
(ii) physical effects
(iii) virtual effects
show message,
turn light on
motors, effectors
(i) sensed inputs
logical
system
B
C
(d)‘electronic’feedback
see message on screen
resistance,
? physical sounds ?
(a) physical manipulation
A
(c) felt feedback
(b) perceived state
turn knob, press button
effects on
logical objects
device
physical
environment
D
(iv) physical effects
controlling
external things
(e) physical feedback
notice light is on,
kettle boils
semantic
feedback
the GUI fallacy … semantic feedback is NOT enough
12.
13. model physical device states
the device ‘unplugged’
the device ‘unplugged’
switch
UP
DOWN
user pushes
switch up
and down
two visible …
and feelable …
states
physigram
18. compliant interaction
(1) system state visible through control
(2) system and user have similar effects
press
down
UP
DOWN
press
up
kettle switch
system
down
system
down
BOILING
Temp
< 100
POWER
OFF
POWER
ON
system state
20. 20
initial pressure on exposed state switch
UP
DOWN
PART
DOWN
PART
UP
press
down
press
down
press
up
press
up
switch
‘gives’
switch
‘gives’
press
down
UP
DOWN
press
up
shorthand
21. time-dependent devices (continued pressure)
21
CENTRE
IN
LEFT
IN
RIGHT
IN
twist
left
twist
right
CENTRE
OUT
LEFT
OUT
RIGHT
OUT
twist
left
twist
right
pull out
pull out
pull out
minidisk
27. island projects – slow research
Frasan - mobile heritage app
OnSupply – renewable energy awareness
Projected touch-table
TireeConnect – island communication
gossip is not enough!
TireeDashboard
General pattern
understand – act – reflect
… takes time
28. the chip van that tweets …
… and the internet connected Open sign
31. tidying
up
prepare
to open
prepare
to open
open sign off
open sign on
café
open
café
open
tidying
up
café
empty
(i) arrive
at café
café
empty
(a) forget
sign
(iii) open
café doors
(ii) switch
on sign
(b) remember
sign
(iv) switch
off sign
(v) close
café doors
(d) remember
sign
(c) forget
sign
(vi) go home
33. tidying
up
prepare
to open
prepare
to open
open sign off
open sign on
café
open
café
open
tidying
up
café
empty
(i) arrive
at café
café
empty
(a) forget
sign
(iii) open
café doors
(ii) switch
on sign
(b) remember
sign
(iv) switch
off sign
(v) close
café doors
(d) remember
sign
(c) forget
sign
(vi) go home
34.
35. takeaways – physical–digital systems
you can model them
models need to encompass
– physical interactions
– digital interactions
– aspects of external context (physical and social)
embrace and understand uncertainty