The document provides a summary of lessons learned for improving user experience in open source software. It lists the top 10 lessons as: 1) think about the user's entire experience, 2) evangelize user experience, 3) work within community processes, 4) conduct user experience research with limited budgets, 5) recruit users from within the community, 6) use appropriate research methods for each project, 7) measure effectiveness, 8) make results actionable, 9) share results with the community, and 10) consider free and open source tools to conduct research. The document describes each lesson in more detail and provides recommendations for applying the lessons to improve user experience in open source projects.

1. Ten Lessons Learnt to Drive and
Transform Open Source Software User
Experience, and How to Get There
Piet Kruithof, IBM Hybrid Cloud
Ju Lim, Red Hat
Melissa Meingast, HPE
14 Oct 2019

2. Top 10
1. Transform UX with End-to-end Thinking
2. Evangelize User Experience
3. Work within community processes
4. UX on a shoestring
5. Recruiting within the community
6. Know which methods are appropriate and when
7. Measure effectiveness
8. Make results actionable
9. Share results with the community
10. Tools to start with

3. Transform UX with End-to-end Thinking
Siloed thinking misses the big picture
● User’s touchpoint with the project (“moment of truth”)
● User’s Journey
● Environment in which the first 2 take place

4. Evangelize User Experience
Evangelizing UX is a never-ending process and requires
perseverance
● Share ownership of UX
● Influencers and decision makers (e.g. Foundation, Leads) buy-in is
important
● Communicate and sell the UX message across the
project/community at all levels
The best way to get people
invested in UX is to get them
involved in it” — John Ferrara

5. Work Within Community Processes
Be aware of and attentive to the unique culture of your
community
● Embrace and participate in community/project processes, tools, and governance
● Take the time to establish trust and cultural understanding
● Be welcoming and inclusive
● Embrace diverse opinions, ideas, differences, dissent, and curiosity
● Propose and drive change in a collaborative manner
● Thank members for their contribution!

6. UX on a Shoestring
Large enterprises prefer to use their budget for internal
projects rather than communities, so consider alternatives
● Look for partnerships and sponsorship opportunities
● Work with local academic programs, UX societies, etc.
● Leverage open source research tools or use free accounts
Some research is better than no
research but understand the
compromise you’re making!

7. Recruiting within the Community
Clearly identify your users and ensure sample reflects the
range of end user technical sophistication
● Work with a panel of users with deep experience
● Leverage community venues, conferences, interest groups, mailing lists,
user groups, etc. to recruit for a given project
● Set expectations upfront on who you are representing and what your
goals are

8. Know Which Methods are Appropriate
● There’s not 1 right method
● Consider research questions,
goals, & resources
● Be creative & open to hybrid
approaches
● Ideally research should begin
before coding
There are methods for every point in the design process

9. Measure Effectiveness
Without clear results and figures, it can be more difficult to
promote UX and get support and budget
● Use data to drive decisions
● Measure KPIs when they are relevant & valid
● Do not over extrapolate
Behavioural UX KPIs Attitudinal UX KPIs
Task success rate System Usability Scale (SUS)
Time-on-task Net Promoter Score (NPS)
Search vs navigation Customer satisfaction (CSAT)
User error rate Single Ease Question (SEQ)
“What gets measured, gets
managed” — Peter Drucker

10. Make Results Actionable
Help identify solutions, not just issues
● Be specific
● Don’t blame the user
● Don’t blame the designers
● Look at the bigger picture
● Prioritize
● Stress the need for iterative testing and multiple approaches

11. Share Results with Community
Be transparent
● If you are from a corporation, use the community’s preferred
communication channels, not your own
● Use communication techniques that help your audience process findings
and focus on next steps
● Follow-through by proactively working with community leaders and
stakeholders to build ownership for addressing issues.
● Build UX brand and trust
“Unseen research is wasted
research” — Gregg Bernstein

12. Tools to Start With
There are a number of design and research tools that offer free accounts. You
should try to use open source tools although you may not have a choice. In
those cases, anticipate some pushback from the community.
● Balsamiq - wireframing/design, free license for open source projects
● Concept.ly - web based collaboration and prototyping tool
● Evernote - keeps track of notes, information, photos, artefacts, etc.
● FreeMind - open source mind mapping tool
● Gimp - open source graphics tool
● Google Docs - study planning, surveys (GoogleForms)
● Inkscape - open source drawing tool
● Invision - design collaboration tool
● LucidChart - diagramming tool
● Marvel - interactive web, tablet and mobile prototypes from imported screen designs
● OptimalSort - Card Sorting
● Origami - free prototyping tool
● Peek - user testing, 3 free tests a month
● SurveyMonkey - online surveys
● yWorks - open source diagramming tool

13. Questions?
Thanks for your time!

14. Abstract
Attendees should bring a deep curiosity about their users, and how they can make their lives just a little easier through
research.
The workshop will provide participants with an overview of some of the key considerations and steps for designing effective
user research. This includes a comparison of key research methodologies, and the factors to consider in order to determine
which research method to employ, how to identify and recruit study participants from open source communities, best practices
when implementing primary user research methods, data analysis tools and practices, and recommendations as to how to
effectively share the study results with the larger open source community. Along the way, we will share some of the tips and
tricks we have learned from our years of conducting user research both within and outside of open source communities. More
importantly, we’ll also suggest ways to collaborate with the overall community to ensure that the results are actionable.
The specific research methodologies covered in the workshop will include:
● Remote usability studies (moderated & unmoderated)
● 1:1 Interviews
● Card sorts
● Focus groups
● Surveys
Attendees will have the opportunity to practice designing and executing a hypothetical study with other workshop attendees,
and share findings and feedback with the group.