Msc. Interaction Design and Technologies Master Thesis Presentation
A design solution is proposed in this thesis, which aims to establish a convenient way for controlling autonomous cars and at the same time enables the user to control tertiary features unrelated with the driving task.
Touch Drive - A touch-based multi-function controller for autonomous driving
1. A T O U C H - B A S E D M U LT I - F U N C T I O N
C O N T R O L L E R F O R A U T O N O M O U S D R I V I N G
J U N T I M A N A W I L A I J A R O E N & VA S I L E I O S G O L E M A T I S
I X D C T H C H A L M E R S U N I V E R S I T Y O F T E C H N O L O G Y - S E M C O N - A I M M I T
TOUCH DRIVE
2. O U T L I N E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
We planned. We worked. Tirelessly.
3. R E S E A R C H Q U E S T I O N S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Problems were found. Hypothesis was then formed.
Will a tablet-based tactical controller, augmented with
an overlay cover increase the efficiency of the interface,
providing a clear distinction between the input
and output functions?
How can a tablet interface be designed to keep the driver in
the loop of the autonomous driving task by functioning as a
tactical and tertiary controller, providing an easy switch
between these modes?
4. S C O P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Great work came with great responsibilities.
Semi-autonomous
Tactical features & Tertiary menu
Focus on visual modality
Driver as supervisory operator
Given
design problem
5. D I V E R G E N C E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Literature Review - Requirement Elicitation - User Studies
6. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
What are things we are going to deal with?
+ +
Automation IVIS & ADAS Touch - Based Interface
7. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in automation
Levels of Automation
8. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in automation
Interaction between human & automation
Levels of Automation
9. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in automation
Interaction between human & automation
Levels of Automation
Trust and Controllability
10. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in automation
Interaction between human & automation
Levels of Automation
Trust and Controllability
Driver in the loop
11. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
In-Vehicle Information Systems (IVIS) and Advanced Driver Assistance Systems(ADAS)
Interference & Distraction
Complexity & Over-Reliance
Loss of situation awareness
12. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Why touch - based interface?
13. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Why touch - based interface?
why tablet?
14. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Why not?
15. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Rapidly increasing uses of tablet
in everyday contexts
16. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Rapidly increasing uses of tablet
in everyday contexts
then,
What is the problem of using tablet in car?
17. L I T E R AT U R E R E V I E W
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
design challenges and limitations in touch - based interfaces
Utilization of tablet affordance
Intuitiveness & efficiency
Overlay cover for better affordance
18. R E Q U I R E M E N T E L I C I TAT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
listed out system requirements and use cases
19. R E Q U I R E M E N T E L I C I TAT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
listed out system requirements and use cases
Highway Driving
City Driving
Change Lane
Accelerate / Decelerate Stop
Take next exit
Turn left / right
Roundabout Stop Park
U - Turn
20. R E Q U I R E M E N T E L I C I TAT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
listed out system requirements and use cases
Input
Turn left / right
Change lane
Action related information :
available actions, confirmation
of actions, executing action
Output
Speed
(accerelate / decerelate / stop)
Switching to infotainment
Speedometer
Alert Notifications
(fuel and oil level)
21. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
22. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
Build Try Express
23. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
24. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
What did we get from this workshop?
25. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
Patterns were found
Blank space to draw a gesture
Needs of additional info.
- ETA
- Environmental Hazards
26. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
There are also interesting findings
27. U S E R S T U D I E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
participatory design - 'Build - Try - Express'
Highway Driving
Participants say:
There's nothing much to do here.
I will trust the automation.
Facing obstacle
Participants say:
I concern about my safety. I need
do something. I want more control.
I want to know everything!
Then, participants also say
28. R E Q U I R E M E N T E L I C I TAT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
we then realized that...
Feedback is vital.
Design solution therefore should provide a
sufficient amount of feedback.
29. T R A N S F O R M AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Design Iterations - Expert Evaluation
30. A F F I N I T Y D I A G R A M
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Beginning of 1st design iteration. An emergence of dominant ideas.
31. 1 S T P R O T O T Y P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
first batch of prototypes were crafted.
32. 1 S T P R O T O T Y P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
first batch of prototypes were crafted.
33. E X P E RT E VA L U AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Explore the possibilities with the help from experts.
ฉ พ ร ะCriteria
ฉ พ ร ะLayout ฉ พ ร ะContent ฉ พ ร ะAccessbility
ฉ พ ร ะNavigation ฉ พ ร ะSatisfaction ฉ พ ร ะLearnability
ฉ พ ร ะProgress and Interaction
34. E X P E RT E VA L U AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Explore the possibilities with the help from experts.
ฉ พ ร ะAlignment is very subjective
5IxD
experts
ฉ พ ร ะ
Combination of both menus
can be a good thing to consider
ฉ พ ร ะMore visual affordances!
ฉ พ ร ะRevisit drag & drop gestures
35. 2 N D P R O T O T Y P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
time passed. things changed.
36. 2 N D P R O T O T Y P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
time passed. things changed.
37. C O N V E R G E N C E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Implementation - Validation - Result Analysis
38. I M P L E M E N TAT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
We designed. And we crafted thoroughly.
+ +
Android Studio Sketch 3 Git & Bitbucket
39. F I N A L P R O T O T Y P E
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
We designed. And we crafted thoroughly.
40. TA C T I C A L C O N T R O L
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
41. S P E E D S L I D E R
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
The evolution in designing for speed slider
Operational Tactical Tactical & Strategic
42. T E RT I A RY M E N U
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
We designed. And we crafted thoroughly.
43. O V E R L AY C O V E R
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Tacticle feedback and better affordances
44. ฆ
VA L I D AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Wizard and his magic
Speed
Disabled Commands
Processing Commands
Environmental Alerts
Issued Commands
45. M O R E
VA L I D AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
What to validate?
U SUsability
Efficiency Effectiveness Satisfaction
Moreover,
46. VA L I D AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
What to validate?
U SUsability
Efficiency Effectiveness Satisfaction
U SSemi-Autonomous Driving Experience
Trust Controllability
47. VA L I D AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Simulate the semi-autonomous driving experience
48. VA L I D AT I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Simulate the semi-autonomous driving experience
Questionnaire
Manual Driving
Semi-Autonomous Driving
Free drive
Drive with given tasks
System Usability Scales
Interview
49. R E S U LT S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Rumors had it all...
Both system were efficient.
There is no overall significant difference between two system.
50. R E S U LT S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
Rumors had it all.
Pros
Easy to use
Functions are useful
Some functions are hard to access
(Eco-slider, 2nd turn)
Cons
Easy and fast switching
between modes
Misinterpretation of alerts
Unintuitive disabled commands
Divided attention
Fast to learn
51. D I S C U S S I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
There's always a room for improvement.
Driver in the loop
Driver as a supervisory operator
Gradual transition to autonomous cars
Tertiary control
Multimodalities
52. C O N C L U S I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
conclusion - ethical issues - future work
53. C O N C L U S I O N
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
To sum it up...
Feedback is vital.
Mutimodalities is a must.
(1) Dedicated tactical touch-based interface
with overlay cover.
(2) Multi-function touch-based interface
54. E T H I C A L I S S U E S
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
There always two sided of a coin
Deskilling
Misplacement of trust
Replacement of human
55. F U T U R E W O R K
I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
One another iteration
Tablet placement
Audio & Haptic Feedback
More strategic functions
Touch drive, a touch based multi-function controller for autonomous driving.
Right now, we can see that there’s a competition over autonomous car technology. Car companies like Volvo, BMW and Audi, even Google are competing over fin`ding best solution for the car that drive on its own.
In this thesis, with the help from Semcon and Volvo Car Company, we were able to design multi-function controller using tablet. This tablet device is expected to used instead of a steering wheel.
In our thesis work, we followed traditional Jone’s model closely. We divided our work into three phases namely: Divergence, transformation and convergence.
Hence, in this presentation, in the next 25 mins, we going to talk briefly about our work scope, the three phases as showed here and ending with conclusion, future works and some ethical issues that may arise.
Before we begin, we want to show you our formal research questions. You will find an answer to these question through our presentation.
We first started off with some literature reviews combined with design problems given by Semcon and Volvo, we were able to form these research questions.
Our problem is that how can a tablet interface be design to keep the driver in the loop and also function as both tactical and tertiary controller?
And also, will tablet that augmented with overlay cover increase the efficiency of the tablet interface?
Since comparing a tablet with steering wheel, it obvious that tablet has less affordances. So we wish to investigate that tablet would keep the driver in the loop of driving or not? and also if we put some highlighter like overlay cover, would it help in increasing affordance?
From the research question and given design problems, we set up our scope of work as listed here.
First, we considered our work to be used in ‘semi-autonomous’ situation.
Second, the tablet should be driver as a supervisory operator which means the car does most of the task like adjusting the wheel. Role of the driver is that giving more advance command like changing lanes, turn left or right.
Third, we wish to focus on tactical features. As I said earlier, like change to left lane. Not like setting up destination and go. We wish also to investigate the possibility of including tertiary feature like playing music and movies.
Lastly, we will focus only in visual modality as it is the strength of tablet device modality.
After you know a little bit of our project. I will begin with our first phase, Divergence.
To answer the research question we’ve set. We realized that we need to deal with these things. Automation, IVIS & ADAS and design principle for touch-based interface.
Here, we want to give you a very short description on what exactly we are dealing with. Firstly, an automation part.
Before we begin the project. We need to know on what level we are going to design for.
You can see here in this figure that there are several levels of automation. Starting from manual to fully automated. On manual side, human driver would be a driver as we’ve seen today. On the other side, fully automated, human would be just a passenger, just like riding a taxi. You set the destination and go.
Our work will lie on these two levels, operator and supervisor. Human driver still in charge of controlling the car by giving a set of commands. But the car will judge if the command is valid or not in terms of safety.
Next, we need to concern also an interaction between human and automation. Normally, we will give out command to machine like do this and that. And the machine will give us back an information if command is successfully performed or not. These interaction are essential as it lead to next issue. Trust and controllability.
As machine speak different language. We can not talk to machine like we talk to human. So we had an issue of should we trust a machine or not. And how much should we control it. Will it go mad if we sent out a command this way. This issue need to be solve in a good way.
Moving to the next topic, IVIS and ADAS.
IVIS stands for in vehicle information system which is the screen as you can see here. GPS, music, secondary feature like temperature control and stuff.
ADAS is advanced driver assistance system. I don’t know if you guys have tried this system before or not. But car nowadays had some sort of auto pilot mode where it could park on its own.
But why these are problems? Because we had these issues. Distraction when interact with the screen. over-reliance when on auto-pilot mode. and as a consequence loss of situation awareness.
Now, we are going to deal with touch based interface. You may ask why touch based interface over steering wheel?
even more, why tablet?
Then we asked, why not?
Since people nowadays get so used to with mobile technology. The uses of tablet is rapidly increasing.
So, what’s the problem of using tablet.
We learned that tablet, comparing to steering wheel, has less affordance.
Here’s we need to utilize the tablet affordance,
As well as increase its intuitiveness and efficiency.
Lastly, we wish to test if tablet affordance could be increase by using overlay cover.
Then we run a first round of system requirement elicitation. We listed interesting findings from literature.
Combined with our own experience in driving. We then got…
We set a participatory design workshop where participant immerse themselves into semi-autonomous driving situation and help us design the tablet interface.
We divided the workshop into three phases.
This is a sample of the interface design from participant.
And what did we ger from this workshop.
what we’ve got from ‘try’ section:
Observe that, in highway
in facing obstacle
Then we realized that feedback is vital.
Participant want to know everything. car status. speed. environmental information.
Design solution…
After that, we get into transformation. We tranform data into ideas.
ideate through affinity diagram.
tablet orientation
main difference is icon representing available commands.
Putting more strategic functions such as
Android Programming
Graphical Design
Update repository (keep track of progress/revert)
Tactical Map
Strategic Functions (Involve route planning)
Automation status
current task performed/Car Mode (By default)
Car Status
Speed/Fuel
Environmental alerts
signs/ obstacles
Route info
Destination / ETA (PD workshop)
Confirm/Cancel command
Set a mode, let the automation handle the speed according to environment/laws
Less control, more convenience
switch to tertiary (full screen considered - receive output through audio/haptic cues)
wizard app - disabled commands/current performed tasks by automation , alerts
Usability criteria
Trust-Controllability
Safer Simulator - Science Park
1 pilot (10 subjects) 1hour sessions
Questionnaire
Manual driving - Get the feeling of the steering wheel
Freeway driving, Think Aloud - Get familiar with interface
Instructed driving, SUS x2 (Usability)
Closing interview (Experience)
No significant difference, but tablet with overlay performed better overall with less error rate
and TCT. Felt more dedicated to tactical control
Driver in the loop - driver controls and knows
Tactical level of control - automation handles tedious tasks
Not Replacement of steering wheel! But complement for
convenience
Separation of devices or multimodalities
(for tactical/tertiary)
2 paths
Not just sufficient feedback, but also proper feedback!
Multimodality (relieve visual load, reduce divided attention)
Placement (in the front, share between passengers)
Audio & Haptic (tailor to specific functions)
More strategic (navigation map)
Deskilling (reduce human skill in driving, inability to be efficient in manual driving)
Misplacement of trust (blame automation/car industry/manufacturer)
Replacement of human (professional drivers/taxi/chauffer etc.)
this list of requirement.
We separated driving experience into two main phase. Highway driving. Nothing much to do here except for changing lanes, take next exit, speed up and down and stop the car.
City driving is more complex, you need to make a turn, driving along roundabout, stop and parking lot.
So, from here, we see that driver need to provide input to the car.
As you can see here, these are input commands that was expected from driver. And the car will start an interaction with human driver by providing these output.
We felt this list of system requirements is not enough. But we can not observe and find more requirements from real situation as the technology doesn’t exit yet. So, we run a user studies.
Build try and express.
We started off by describing our work scope and the idea of semi-autonomous car. We then asked participant to filled out questionnaire as we wish to collect the ethnography data as well.
After that, we begin build. Participant were asked to used design tools given. as you can see here. to build up interface on a tablet.
Next, participant were asked to try out their design and see if anything need to be fix or not. In here, we set up the situation of combination between highway driving and city driving. Participant were asked what if there is traffic light, what they are going to do.
Lastly, express, in here we capture the thought behind their interface design. we asked the reason why they design like this. We gathered a very valuable data from here.