1. The Artificiality of Natural User Interfaces
Alessio Malizia, PhD
Dept. of Information Systems and Computing, Brunel University

2. Synopsis
Research areas
Background
Problem(s)
Question(s)
Hypothesis
Contribution
Expected result

3. Research areas
Human Computer Interaction
Natural User Interfaces
Interaction/UX Design

4. The Artificiality of Natural User
Interfaces
1960s –CLI 1980s –GUI 2000s –NUI
Command Line Interface Graphic User Interface Natural User Interface
Memorization/Recall Recognition (recall = shortcuts) Recall? Gestures are like shortcuts

5. Exploration
Don’t touch me!
Demo @ACM Int. Conference AVI 2010
TESIS: Turn Every Surface into an
Interactive Surface
Demo @ACM Int. Conference ITS 2011

6. Problem(s)

7. Question(s)
• How can I create an interface in which users
may quickly gain expertise using actions that
feel natural to them?
(research question 1)
• How can I be sure that I succeeded in creating
an interface that feels natural to the user?
(research question 2)

8. Hypothesis
• Natural properties are related to universals
• The concept of affordance is crucial for
defining a widget-level architecture for NUI
– Interface as a Service
– Widgets expose their affordances
• Personalization and adaptation/evolution as a
step toward natural interaction

9. I’d like to convince you
(thesis)
• Gestures (as signs) are something alive, which
change depending on cultural and human
aspects, time and context. Therefore users’
personalization can be a step to address
natural interaction.
• Refined research question 1: to what extend
personalization of multi-touch gestures can be
valuable for enabling natural interaction?

10. Natural properties and Universals
• Concept of perfect natural properties from metaphysics
– Perfect natural properties are shared among all the elements of
a set
• For we humans, perfect natural properties are our
perception faculties (hearing, feeling by touch, seeing,
tasting, smelling)
– There are also acquired skills...
• Could there exist such a thing as a natural gesture?
– Take into account users innate and learned abilities

11. Which one is natural?
iPhone:
pinch for zooming
Samsung Galaxy s2:
holding down on
the screen with
Zooming in real life: two fingers and
bring the newspaper closer moving your phone
back and forth

12. Affordance
• Why affordance is important for natural
interaction?
– Gibson (1977) defined affordances as all action
possibilities latent in the environment
– Affordances as perceived action possibilities
(Norman, 1988)
– Affordance is the visual clue to the function of the
object (an opportunity for action)

13. Design without affordance
You can move an object in single-pixel
increments by nudging it.
Touch and hold the object with one finger,
and then use another finger to swipe across
the page in the direction you want the object
to move.
From iPad’s Pages gestures list

14. A good affordance

15. What happens if you attempt to make
users interact in a way which is not
natural to them?

16. Beyond affordance languages
Form factor
Adaptation/
Medium/High
Evolution
Personalization
Small
Display
Personal/ Public/
Private Shared

17. Expected result

18. References
• Alessio Malizia and Andrea Bellucci. 2012. The artificiality
of natural user interfaces. Commun. ACM 55, 3 (March
2012), 36-38.
• Andrea Bellucci, Alessio Malizia, and Ignacio Aedo. 2011.
TESIS: turn every surface into an interactive surface. In
Proceedings of the ACM International Conference on
Interactive Tabletops and Surfaces (ITS '11). ACM, New
York, NY, USA.
• Andrea Bellucci, Alessio Malizia, Paloma Diaz, Ignacio Aedo,
"Human-Display Interaction Technology: Emerging Remote
Interfaces for Pervasive Display Environments," IEEE
Pervasive Computing, pp. 72-76, April-June, 2010.