1.
Cyborg Design:
IA Summit
Bennett King
@skunkwUrX
Multimodal Interactions, Information, and
Environments for Wearable Computing
Opinions expressed in this presentation are my own and do not
necessarily reflect those of my company

2.
source: wearable technology database | Vandrico Inc
2
There is a lot of hype about wearable devices with even predictions of a $30B market over the
next 2 years. The reality…most wearable devices will fail
2

3.
Wearable Computing Spaces:
3

4.
The focus of this presentation is Information wearables: connected devices that are worn for
extended periods of time to enhance the user’s capability. Specifically, smartglasses &
smartwatches. These have the best chance of sustained use.
4

5.
5
fitness/health
smartwatches
smart others
smartglasses
source: Strategy Analytics
20132013 2017
Wearable Device Shipment
by Category
But why should we care?

6.
Information wearables represent the first major change in computing in the last 30 years,
breaking away from devices that require you to conform to them. Information wearables
conform to our needs at any given time and enhance our capabilities in the real world.
6

7.
photo: Steve Mann | eyetap.blogspot.com
”
“
Steve Mann
Chief Scientist, Meta
Information wearables become an “extension of ourselves”. We interact with them in more
natural ways and the content provided is relative to our given tasks.
7

8.
8

9.
Current forces behind wearable devices
9

10.
photos: samsung.com; sony.com; getpebble.com; moto360.com. apple logo TM of Apple Inc.
Qualcomm Toq
Galaxy Gear
Sony Smartwatches
Pebble
Moto 360
(Mythical) iWatch
10

11.
photo courtesy: google.com/glass; laforgeoptical.com; buhel.comphoto courtesy: spaceglasses.com; lumus-optical.com
Meta
Lumus
ODG
Google Glass
LaForge
Buhel
11

12.
photo: Laforge Optical photo: Google
Even with more palatable designs, acceptance and sustained use are formidable issues.
Less than half of people under 30 wear watches. If you don’t already wear glasses, what would
make you put them on day after day? How do we affect changes in behavior?
12

13.
photo: tomsguide.com
Another major problem facing wearable devices is that for every well-designed device, there
are 10 that are…let’s say not as well designed.
13

14.
Blogs, market reports, tech reviews, editorials claim success lies with “additional use cases”,
“meeting user needs”, “creating meaningful & engaging experiences”. These terms should not
be unfamiliar to our industry and are key to modifying existing habits and creating the desire
for continued use.
14

15.
What’s missing from the picture is you. If we cannot create meaningful experiences to go along
with form and function then wearables will join the long list of failed tech.
15

16.
changes...
16

17.
17
+

18.
If mobile device
content is about
truncation, wearable
device content is
about encapsulation.
• Smartwatches are limiting – space, colors, content, readability.
• Smartwatches should allow consumption and interaction at a glance.
• You’ll need to use every trick in the book to enhance the experience – visual affordances,
LEDs, haptics.
18

19.
• Monocular devices limit you to roughly a 15º FOV.
• Usually fixed location.
• Varying levels of opacity and background noise.
19

20.
• Binocular devices are currently maxed at about 40º FOV (but are improving).
• Stereoscopic display allows for 3D design – the x y and z can be used in your information,
interaction, and interface design.
• Varying levels of opacity and background noise.
20

21.
The differentiator for binocular glasses is the possibility of 3D augmented reality interfaces –
augmenting content out of cyberspace and into the physical space. Standing alone, AR and
smartglasses are underwhelming but together they can enhance adoption and retention.
21

22.
touch voice gesture
22

23.
micro
gestures
eye gaze
& dwell
enhanced
gestures
body
movement
23

24.
natural
language
brain-computer
interfaces
emotion
24

25.
”
“
Bill Buxton
Principal Researcher, Microsoft
Flikr: Hamed Mousami
25

26.
Photo: wired.com
26
When designing in a new area, research and prototyping are more valuable than ever. But
research requires building new platforms. Luckily our work corresponds with the makers
movement: “If you can think it, you can build it”.
• ID: FabLabs, makers clubs, mfg.com
• SW: Google wear
• HW: Arduino kits, Dragon boards, sensor
platforms
26

27.
Build (whatever you can) and learn. This is our first AR research rig made out of reflective
plexiglass, metal rods, a Pico projector and a used marching band drum harness. You can
quickly move to 3D printed prototypes and iterate further.
27

28.
28

29.
29

30.
30

31.
31
* I cannot take credit for AEIOU but it’s been around so long that I am not sure who to credit. Feel free to say it was you.

32.
photos: moto360.motorola.com
Wearable interfaces should work with your activity. The interfaces and apps we are used to on
our phones and tablets do not apply while riding a bike or running. Subtle, contextual,
glanceable…create designs that understand activity.
32

33.
We have spent a considerable time understanding how user’s interact with their wearable
device in different environments…and it’s rarely constant. You devices and designs should
support multiple modes of interaction and understand their context of use.
33

34.
Understand the object and its intended use. While talking to your watch is cool in the movies, it
is less so in the real world. Does this seem right to you?
34

35.
35
> Display
> Creating media
> Communication
> Consumption
> Text Entry
> Content creation
> Communication
> Display
> Consumption
> Biometrics
> Natural Input
> Display
> Consumption
While some wearables are stand-alone, there is greater benefit if they work together. Each
device has its strengths for interaction, information, and input. Multiple sensors on multiple
devices can work together for better accuracy and experience (not to mention benefits in
processing and battery life).

36.
36

37.
37
One of the keys to designing for immersion is understanding instance. Fitness wearables have
had some success by understanding a specific instance. If we want to create devices that
people will adopt an continue to use we need them to juggle multiple instances. What is the
experience while shopping with family, while out with friends, while at home alone relaxing…
37

38.
38
Many of the wearable demos you’ll see are overkill. While you are walking down the street you
probably don’t need a constant overlay for navigation with live updates.
38

39.
Instead, augmented reality allows us to overlay objects fixed to specific locations. So navigation
now appears where and when you need it, attached to the world around you.
39

40.
40

41.
The Glance mode represents the always-on interface that is the backbone of engagement
(think notifications). It should be discreet and not detract from the user’s real world activity. It
should inform without monopolizing attention.
41

42.
photo: howto.com
The Assist mode (best with but not limited to smartglasses) is the home of apps. It can
partially occupy the screen, follow and update based on tasks, and autohide or turn off based
on need. The input should also be multimodal to allow voice and hands-free interaction.
42

43.
The Augment mode is the home of AR. Markers, information and interfaces augmented over
the real world. AR will be one of the greatest challenges for Information Architects going
forward – the vast amount of information available online presented exactly when and where it
is needed without distraction or annoyance to the user.
43

44.
The final mode, Immerse, is used for interfaces that are intended to monopolize our attention –
reading a book, gaming, looking at photo galleries…
44

45.
45

46.
Always on and always listening; this is the minimum standard for wearable devices. If the user is
interacting with the world around them, they expect the wearable device to react in real time.
While there are technical challenges to this, the multiple sensors allow for us to provide the
illusion of always on. Our first goal should be to remove the smartphone glance.
46

47.
photo: flikr: Shawn Willis
The camera is one of the best examples. During our research this feature consistently stood
out for expected immediacy – look and shoot.
47

48.
48

49.
Our devices should not create peculiar social interactions like the Glass Gloss. Remember that
we will be interacting with others with these things on. We must find ways to use the
technology without straying from social norms.
49

50.
Example: We’ve all forgotten someone’s name. We can replace the awkward interaction caused
by us wandering off into the ether to find them online with an AR overlay that appears in our
line of sight. In a wearable system, you have cameras, connectivity, CV and AR running in
conjunction to enhance our forgetful self.
50

51.
photo credit: www.nbcbayarea.com
Perceived personal privacy has also become one of the biggest issues we need to design for.
The lifelogging capabilities of wearable devices have caused a backlash based on worries of
surveillance. How can we make these actions noticeable and acceptable to those around us?
51

52.
52
Wearables are just the tip of the iceberg. Today’s
technology is advancing exponentially faster since the
birth of the smartphone.

53.
BCI units are already on the market and should be commercialized in devices within the next
five years. There are already several projects looking to add the input to smartglasses. With the
availability of Open BCI (EEG capture platform) the technology is now available to makers
everywhere. Your next interaction – Think and make it happen.
535353535353

54.
54
Why wear technology when you can just make it part
of you?

55.
55
photo: brown.eduphoto: givenimaging.comphoto: illinois.edu
Wireless Optogenetics:
using injectable lights to
stimulate targeted neural
pathways in the brain
SmartPill: an ingestible
capsule that wirelessly
measures pressure, pH
and temperature
Implantable BCI: a fully
implantable and
rechargeable wireless
brain sensor

56.
sight
At the beginning of the talk I said “If it can be done it will be done”. While current wearables
and implantables are being designed as assistive devices, they could easily start to enhance us
well beyond current human capabilities (my coworkers already clamor for a brain chip). This
moves us to a gray ethical area. With projects advancing so rapidly, we are often the only voice
for humans in tech discussions. I wonder if ethics, philosophy and even neurosciences will
become part of our future skillset.
Actual meme
at my office
56565656565656

57.
bennettmking@gmail.com
@skunkwUrX
*