The fourth lecture from the Augmented Reality Summer School talk by Mark Billinghurst at the University of South Australia, February 15th - 19th, 2016. This provides an overview of prototyping techniques for AR interfaces.

1. LECTURE 4: PROTOYPING
FOR AUGMENTED REALITY
Mark Billinghurst
AR Summer School
February 15th – 19th 2016
University of South Australia

2. Interaction Design Process

3. Three Phase Model

4. Elaboration and Reduction
• Elaborate - generate solutions.These are the opportunities
• Reduce - decide on the ones worth pursuing
• Repeat - elaborate and reduce again on those solutions
Source: Laseau,P. (1980) Graphic Thinking for Architects & Designers. John Wiley and Sons

5. How can we quickly
prototype Augmented
Reality experiences
with little or no coding?

6. Why Prototype?
▪ Quick visual design
▪ Capture key interactions
▪ Focus on user experience
▪ Communicate design ideas
▪ “Learn by doing/experiencing”

7. Prototyping Tools
▪ Static/Low fidelity
▪ Sketching
▪ User interface templates
▪ Storyboards/Application flows
▪ Interactive/High fidelity
▪ Wireframing tools
▪ Mobile prototyping
▪ Native Coding

8. Typical Development Steps
▪ Sketching
▪ Storyboards
▪ UI Mockups
▪ Interaction Flows
▪ Video Prototypes
▪ Interactive Prototypes
▪ Final Native Application
Increased
Fidelity &
Interactivity

9. Sketching
Sketching is not about drawing
It is about design.
Sketching is a tool to help you:
• express
• develop, and
• communicate design ideas
Sketching is part of a process:
• idea generation,
• design elaboration
• design choices,
• engineering

10. Figure 41: A Sketch of a Dialogue with a Sketch
The “conversation” between the sketch (right bubble) and the mind (left
bubble). A sketch is created from current knowledge (top arrow). Reading,
or interpreting the resulting representation (bottom arrow), creates new
knowledge. The creation results from what Goldschmidt calls “seeing
that” reasoning, and the extraction of new knowledge results from what
she calls “seeing as.”
sketch
representation
mind
(new) knowledge
Create
(seeing that)
Read
(seeing as)
Sketching is about the
activity not the result
– Bill Buxton
10

11. From Sketches to Prototypes
• Sketches: early ideation stages of design
• Prototypes: capturing /detailing the actual design

12. Sketch vs.Prototype
Sketch
Prototype
Invite
A)end
Suggest
Describe
Explore
Refine
Ques;on
Answer
Propose
Test
Provoke
Resolve
Tenta;ve,
non
commi)al
Specific
Depic;on
The primary differences are in the intent

13. Sketched Interfaces
▪ Sketch + Powerpoint/Photoshop/Illustrator

14. GlassSim – http://glasssim.com/
▪ Simulate the view through Google Glass
▪ Multiple card templates

15. GlassSim Card Builder
▪ Use HTML for card details
▪ Multiple templates
▪ Change background
▪ Own image
▪ Camera view

16. GlassSim Samples

17. Application Storyboard
▪ http://dsky9.com/glassfaq/google-glass-
storyboard-template-download/

18. Application Flow

19. Limitations
▪ Positives
▪ Good for documenting screens
▪ Can show application flow
▪ Negatives
▪ No interactivity/transitions
▪ Can’t be used for testing
▪ Can’t deploy on wearable
▪ Can be time consuming to create

20. Transitions

22. ▪ Series of still photos in a movie format.
▪ Demonstrates the experience of the product
▪ Discover where concept needs fleshing out.
▪ Communicate experience and interface
▪ You can use whatever tools, from Flash to iMovie.
Video Sketching

23. UI Concept Movies

24. Pop - https://popapp.in/
• Combining sketching and interactivity on
mobiles
• Take pictures of sketches, link pictures together

25. Using Pop

26. Interactive Wireframing
▪ Developing interactive interfaces/wireframes
▪ Transitions, user feedback, interface design
▪ Web based tools
▪ UXpin - http://www.uxpin.com/
▪ proto.io - http://www.proto.io/
▪ Native tools
▪ Justinmind - http://www.justinmind.com/
▪ Axure - http://www.axure.com/

27. Proto.io - http://www.proto.io/
▪ Web based mobile prototyping tool
▪ Features
▪ Prototype for multiple devices
▪ Gesture input, touch events, animations
▪ Share with collaborators
▪ Test on device

28. Proto.io - Interface

29. Wireframe Limitations
▪ No access to sensor data
▪ Camera, orientation sensor
▪ No multimedia playback
▪ Audio, video
▪ Simple transitions
▪ No conditional logic
▪ No networking

30. Hardware Prototyping

31. Fake Display
3D print Thingiverse model
see http://www.thingiverse.com/thing:65706
Have the social impact of Google Glass without the cost

32. Build Your Own Wearable
▪ MyVu display + phone + sensors

33. Beady-i
▪ http://www.instructables.com/id/DIY-
Google-Glasses-AKA-the-Beady-i/

34. Rasberry Pi Glasses
▪ Modify video glasses, connect to Rasberry Pi
▪ $200 - $300 in parts, simple assembly
▪ https://learn.adafruit.com/diy-wearable-pi-near-eye-kopin-video-
glasses

35. Physical Input Devices
▪ Can we develop unobtrusive input
devices ?
▪ Reduce need for speech, touch pad input
▪ Socially more acceptable
▪ Examples
▪ Ring,
▪ pendant,
▪ bracelet,
▪ gloves, etc

36. Prototyping Platform
Arduino Kit Bluetooth Shield Google Glass

37. Example: Glove Input
▪ Buttons on fingertips
▪ Map touches to commands

38. Example: Ring Input
▪ Touch strip, button, accelerometer
▪ Tap, swipe, flick actions

39. How it works
Bracelet
Armband
Gloves
1,2,
3,4
Values/
output

40. Summary
▪ Prototyping for wearables is similar to mobiles
▪ Tools for UI design, storyboarding, wireframing
▪ Android tools to create interactive prototypes
▪ App Inventor, Processing, etc
▪ Arduino can be used for hardware prototypes
▪ Once prototyped Native Apps can be built
▪ Android + SDK for each platform

41. Other Tools
▪ Wireframing
▪ pidoco
▪ FluidUI
▪ Rapid Development
▪ Phone Gap
▪ AppMachine
▪ Interactive
▪ App Inventor
▪ WearScript

42. App Inventor - http://appinventor.mit.edu/
▪ Visual Programming for Android Apps
▪ Features
▪ Access to Android Sensors
▪ Multimedia output
▪ Drag and drop web based interface
▪ Designer view – app layout
▪ Blocks view – program logic/control

43. App Inventor Designer View

44. App Inventor Blocks View

45. Orientation Demo
▪ Use wearable orientation sensor

46. Prototyping Case Study

47. Google Glass

48. Google Glass Prototyping

50. The Road to Glass

51. AR Design Case Study

52. ITERATIVE DESIGN PROCESS
Design
(redesign)
Prototype
Test
Design
with
Users
(Note
problems)
Evaluate
(Fix
Issues)

53. MOBILE AUGMENTED REALITY FOR
SPATIAL NAVIGATION
Sharon Brosnan
0651869
Bachelor of Science in Digital Media Design

54. BUNRATTY FOLK PARK
Sharon Brosnan
0651869
Bachelor of Science in Digital Media Design

55. BUNRATTY FOLK PARK
• Irish visitor attraction run by Shannon Heritage
• 19th century life is recreated
• Buildings from the mid-west have been relocated to the
26-land surrounding Bunratty Castle
• 30 buildings are set in a rural or village setting there.

56. AUGMENTED REALITY
In Bunratty Folk Park:
• Allows the visitor to point a camera at an exhibit, the
device recognises it by it’s location and layers digital
information on to the display
• 3-dimensional virtual objects can be positioned with
real ones on display
• Leads to dynamic combination of a live camera view
and information

57. NAVIGATIONALAID
Smartphone Platform
Most people carry mobile phones
Ideal Augmented Reality Technology
• Global tracking tools
• Wireless communication capabilities
• Location based computing
• Large display for interaction

58. DESIGNING
FOR THEVISITOR

59. HUMAN CENTRED DESIGN
Goal of the Navigational Aid
• Easy to use, clear and understandable
• Useful to visitors
• Creating interaction between the visitor and the aid
through the user interface
Engage the visitor
• It is necessary to understand the visitor to Bunratty
• Identify visitor motives and goals while going through
the Folk Park.

60. HUMAN CENTRED DESIGN
Understanding the technology and Related Works
! Literature
! Similar Technologies
! Electronic Tours in Museum Settings
! Interactions design in Outdoor Museums
Understanding the User over time
! Observations
! Interviews

61. HUMAN CENTRED DESIGN
Findings
• Most visitors do not use the map
• Most visitors have mobile phones
• Visitors want more information
• View the Folk Park at their own pace
• Information should be straight to the point
• Large social interaction within groups

62. NEXT STEPS FROM RESEARCH
• DefineVisitors Goals
• Define Functionalities of the Aid
• Develop Personas – visitors who use the Aid
• Develop Scenarios – how the persona uses the navigational aid
in the Folk Park
• Draw up Storyboards on scenarios

63. FUNCTIONALITY
• View Options
• CameraView
• MapView
• ListView
• Sub-Options
• Places
• Events
• Restaurants
! AR Features
! Text Information
! 3D Objects
! 3D Tour Guide
! 3D Placement of Buildings

64. STORYBOARD

67. ITERATIVE DESIGN PROCESS

68. ITERATIVE DESIGN PROCESS
Prototyping and User Testing
• Low Fidelity Prototyping
• Sketches
• Paper Prototyping
• Post-It Prototyping
• PowerPoint Prototyping
• High Fidelity Prototyping
• Wikitude

69. INITIAL SKETCHES - Photoshop
Pros:
• Good for idea generation
• Cheap
• Concepts seem feasible
Cons:
• Not great feedback gained
• Photoshop not fast enough
for making changes

70. POST IT PROTOTYPING
First
DraJ
Camera
View
with
3D
Second
DraJ
Third
DraJ
•
Selec;on
highlighted
in
blue
•
Home
bu)on
added
for
easy
naviga;on
to
main
menu

71. POWERPOINT PROTOTYPING
Benefits
• Used for User Testing
• Interactive
• Functionalities work
• Quick
• Easy arrangement of slides
User Testing
• Participants found
• 15 minute sessions screen captured
• ‘Talk Allowed’ technique used
• Notes taken
• Post-Interview

72. WIKITUDE
• Popular AR browser
• Mapping
• Point of Interest abilities
• Multiplatform
• Shows points of interest
• Markers can be selected in and
information pop-up appears

73. FINAL CONCEPT
DESIGN

74. FINAL DESIGN CONCEPT
Key Issues
• Fix issues found in previous sessions
• Design with guidelines in mind
• Appealing to the Mental Model
• Icon Design
• Aesthetic Design
• Colour/Font
• Buttons
• Look

75. VIDEO PROTOTYPE
" Flexible
tool
for
capturing
the
use
of
an
interface
" Elaborate
simula;on
of
how
the
naviga;onal
aid
will
work
" Does
not
need
to
be
realis;c
in
every
detail
" Gives
a
good
idea
of
how
the
finished
system
will
work

76. www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au