Fast, Cheap, and Actionable: Creating an Affordable User Research Program (The Director's Cut)
â˘Transferir como KEY, PDFâ˘
5 gostaramâ˘1,468 visualizaçþes
Done a usability study? Ready for the next step? Today we have an abundance of fast, affordable user research methods, many of which can be done remotely with real users. Learn about available user research options and how one university runs successful research projects that lead to actionable insights.
Recomendados
Recomendados
Mais conteĂşdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Fast, Cheap, and Actionable: Creating an Affordable User Research Program (The Director's Cut)
Semelhante a Fast, Cheap, and Actionable: Creating an Affordable User Research Program (The Director's Cut) (20)
Ăltimo
Ăltimo (20)
Fast, Cheap, and Actionable: Creating an Affordable User Research Program (The Director's Cut)
- 1. Fast, Cheap, and Actionable: Creating an Affordable User Research Program Michael Powers Director of Web Services Indiana University of Pennsylvania eduWeb 2010 â July 27, 2010, Chicago, IL
- 3. In Answer to Your First Question... Buffalo Erie Chicago Cleveland Philadelphia Pittsburgh Morgantown Indianapolis
- 4. In Answer to Your First Question... Buffalo Erie Indiana, Pa. Chicago Cleveland Philadelphia Pittsburgh Morgantown Indianapolis
- 5. In Answer to Your First Question... Buffalo Erie Indiana, Pa. Chicago Cleveland Philadelphia Pittsburgh Morgantown Indianapolis
- 7. What kinds of user research have you done?
- 8. It all started with a usability test
- 14. Today â˘Getting Permission for Research â˘A Process for User Research â˘Inventory of Actionable User Research You Can Do Fast and Cheap
- 15. User Research Study â User Research Program
- 16. 0
- 17. 0 Creating Permission
- 18. Donât Ask
- 20. âA research study? âLook, just make a red box on the home page, and put this text in there. âI just need to make sure students donât miss it.â
- 21. Nobody wants a research study.
- 22. Nobody wants a website.
- 23. What We Really â˘Applicants â˘Successful Students â˘Academic Research â˘Donations
- 24. Create a Research â˘Do a usability test and invite stakeholders to watch â˘Make research part of every project â˘Apply previous research to new projects â˘Show results early and often
- 25. Research Process 1. Frame a Good 4. Find Question Participants 2. Know What You 5. Run Study and Already Know Analyze Results 3. Choose Study 6. Take Action Type 7. Measure and Repeat
- 26. 1 2 3 4 5 6 7 Frame a Good Question
- 27. Good questions are related to your goals
- 28. âStudents can't ďŹnd stuff on our website.â
- 29. Focusing a How can we 1. Can students find information about decrease support services? attrition? 2. Can students find out whatâs happening on campus?
- 31. Focusing a How can we get 1. Can prospectives find our list of majors? more/more Our tuition qualified information? applicants? 2. Can prospectives register for a visit? 3. Can prospectives successfully apply on line?
- 32. Goal-related questions lead to faster, cheaper, more actionable user research
- 33. 1 2 3 4 5 6 7 Know What You Already Know
- 34. What Do You â˘Best Practices â˘Server Logs â˘Published â˘Analytics Studies â˘Past Studies â˘Data about Your Institution and Students
- 35. Get Organized Quantitative Data Qualitative KPI Data Spreadsheet User Personas
- 36. Key KPI Performance Indicator
- 37. KPIs measure progress toward your goals.
- 40. KPI Spreadsheet â˘Focus on the KPIs â˘Track over time you've chosen without running multiple tools â˘Give specific reports to â˘Include anything stakeholders who numeric: survey request them results, 4Q survey, analytics, server logs, etc.
- 42. User Personas
- 45. Include Scenarios Site Motivation Scenario Features Behaviors Used Search Looks for Find the Checks to â˘Site ⢠school with see if her â˘Undergrad âPre-Lawâ the right major is Admissions major, pre-Law available Website does not program for â˘Academics ďŹnd it. me Link
- 46. User Personas â˘Fictionâbut a â˘If it is wrong, team useful fiction or clients will let you know â˘Puts what you've learned into a â˘Helps bring new format you can team members up share with your to speed team and clients
- 47. Use these two tools to track your progress over time.
- 48. 1 2 3 4 5 6 7 Choose Study Type
- 49. If youâve never done one, do a usability test first.
- 50. Focus Groups
- 51. Focus Groups
- 52. Some Types of User â˘Usability Test â˘Card Sort â˘Surveys â˘Navigation Test â˘Pop-up Survey â˘Layout Test â˘KJ Session â˘A/B Test
- 54. Some Types of User â˘Usability Test â˘Card Sort â˘Surveys â˘Navigation Test â˘Pop-up Survey â˘Layout Test â˘KJ Session â˘A/B Test
- 55. Usability Test Photo source: http://www.ďŹickr.com/photos/l-i-n-k/ 3654390818/
- 56. Usability Test What You Do: Ask people to do things on your website while they talk about what they do and why they do it Good For: Finding things to ďŹx When To Use: As early as possible and regularly thereafter Participants As few as 3 (3 groups of 5-7 is Needed: better)
- 57. Usability Test Tools â˘No tools are needed Available: â˘Silverback (Live) â˘Morae (Live (Live) â˘Any screen-sharing tool (Skype, GoTo Meeting, WebEx) (Remote) Cost: â˘UserVue (Remote, no longer $90â$1,000+ available) Record In: â˘User ProďŹles
- 58. Usability Test Positive Negative â˘Can be cheap â˘Behavior of â˘Holistic view of participants will website issues not be entirely natural â˘Takes practice to run good tests
- 59. Analytics
- 60. Analytics What You Do: Analyze the behavior of users on your website using your analytics tool Good For: Seeing real behavior on your website Not So Good Understanding user intent For: When To Use: Collect 24/7, analyze monthly or as needed Participants n/a Needed:
- 61. Analytics Tools â˘Google Analytics Available: â˘Mint â˘ClickTracks â˘Omniture â˘Many others Cost: â˘$0â$10,000+
- 62. Analytics Positive Negative â˘Can be free â˘Canât understand â˘You see actual intent use of your site â˘Can lead to just â˘Can track looking at changes over time pageviews
- 63. Survey
- 64. Survey What You Do: Ask people questions about their browsing habits or your website. Good For: Background information about your usershave a question to When To Use: When you answer, possibly yearly Participants Calculate your sample size: Needed: http://www.surveysystem.com/ sscalc.htm
- 65. Survey Tools â˘Survey Monkey Available: â˘Google Docs â˘Existing campus survey product â˘Many others Cost: â˘$40â$10,000+
- 66. Survey Positive Negative â˘Tools are cheap â˘Shows you or free attitudes, not â˘Offer a prize and behavior, so not you can get a great for respectable understanding number of how users use summissions your website â˘Lots of numbers
- 67. Pop-Up Survey
- 68. Pop-up Survey What You Do: Ask site visitors questions about their experience on a particular page or the whole Good For: website. user intent; settling Capturing arguments about what belongs When To Use: whereyou have a question to When answer, or all the time Participants Calculate your sample size: Needed: http://www.surveysystem.com/ sscalc.htm
- 71. Pop-up Survey Tools â˘4Q Survey Available: â˘iPerceptions â˘CrowdScience â˘Roll your own Cost: â˘$0â$10,000+
- 73. 4Q Survey 1. Based on today's visit, how would you rate your site experience overall? 2. Which of the following best describes the primary purpose of your visit? 3. Were you able to complete the purpose of your visit today? 4. Comment ⢠If yes: What do you value most about the website? ⢠If no: Please tell us why you were not able to fully complete the purpose of your visit today
- 74. âI am âI can never always able easily find to find what what I'm I am looking looking for, for. The if I can find website is it at all.â
- 75. Coding Comments ⢠Visual Design ⢠Navigation Positive Positive ⢠Navigation Negative ⢠Visual Design Negative ⢠Content Quality Positive ⢠Usability Positive ⢠Content Quality ⢠Usability Negative Negative
- 76. 637,'809':-%&-'*(##%+,';+<' NF,)/+O)-./" (:L8")/2"M*:/58" J+8I)4"K:8+,/"" @8)34:" 1+/2"A:.=4:" 2(405-' 9./5:/5"HI)4+56" (:,)-*:" <+5:"<:)F>G" A.8+-*:" ;@A"B:38+5:C"/./D9E<" <=:>+?>"=),:08+5:" 7)8"9./5:/5";"(::2" ()*+,)-./01+/2)3+4+56" !" #!" $!" %!" &!" '!!" '#!" '$!" '%!" '&!" !"#$%&'()'*(##%+,-'.%+/(+0+1'230-'2(405'
- 77. 67'57"3#4'5&"8'9":&*;&"/#00*12" <#&2"%#&'()*=>+,*-.()*=>"?1@" SK1.40T.234# -?Q=#.47#R/?4:=# O0=N.9#P?=014## E=.89?# 6047#F?3B9?# 3#4'5&" >34:?4:#MN.90:;# A0:?#A?.KCL# -?1.2/?# @EF#G?8=0:?H#434I>JA# F3=02/?# AB?C0DC#B.1?5=0:?# <.=#>34:?4:#@#-??7# -./01.23456047.8090:;# !"# $!"# %!"# &!"# '!"# (!"# )!"# *!"# +!"# ,!"# $!!"# !"#$"%#&'()*+,*-.()*"/#00*12&"
- 78. Action â˘Changed our yearly goals to spend more time on these issues
- 79. Pop-up Survey Positive Negative â˘You get info from â˘Some users real users respond â˘Easy to set up negatively to being interrupted â˘Gives you a window into user â˘Coding can take intent time
- 80. Card Sort Photo Source: http://www.ďŹickr.com/photos/ rosenfeldmedia/3343501403/
- 81. Card Sort What You Do: Give participants cards with website topics on them, then ask them to put them into piles Good For: that make sense to mental Understanding the them models of your users When To Use: When creating or improving your information architecture/ Participants navigation face 15 face to Needed: 50 online
- 82. Card Sort Tools â˘Index cards Available: â˘OptimalSort (remote) â˘WebSort (remote) â˘xSort Cost: $40-$600
- 83. Card Sort Positive Negative â˘Get a picture of usersâ â˘Hard for participants to mental model of your understand activity content remotely â˘Understand what labels â˘Some participants give your users use for your up because the sort can content take a while â˘Settle arguments about â˘Data analysis can be what belongs where complex
- 85. Open Closed â˘Users stack â˘Give users cards then header cards â˘Give each pile a â˘They assign name cards to the header cards
- 88. Where do we put counseling?
- 91. Action â˘Complete restructuring of the Student Life section of our website â˘Move counseling websites out of âHealthâ and into âSupportâ
- 92. One Subsite, Before and After Change between Compared to Comparison Whole Site Periods Pageviews 76% 140% Unique 81% 142% Pageviews Bounce Rate -21% 27%
- 93. KJ Session Photo Source: http://www.ďŹickr.com/photos/drewm/ 370231088/
- 94. KJ Session What You Do: Gather 4-6 participants Ask: what do people come to our website to ďŹnd or do? Put answers on sticky notes on the wall Organize and name the groups Good For: Getting the opinion of site owners; learning about users from site owners When To Use: When creating a new information architecture Participants 4â6, or more if you divide into Needed: smaller groups
- 95. KJ Session Tools â˘Walls and sticky notes Available: Cost: $0â$40 Learn More: http://insidenewcity.com/blog/ view/unlock-the-wisdom-of- your-group-host-a-kj-session- for-site-architecture/
- 96. KJ Session Positive Negative â˘Get feedback â˘Not real users from people who know your users â˘Get buy-in from stakeholders
- 97. Navigation Test
- 98. Navigation Test What You Do: Give participants a proposed site structure/navigation and let users drill down to ďŹnd Good For: different things. a more Verifying an IA in realistic way (not everything is When To Use: visible); ďŹnding problem areas To verify a new information architecture Participants 50 Needed:
- 99. Navigation Test Tools â˘Treejack Available: Cost: $150â$700
- 103. Navigation Test Positive Negative â˘Tests your â˘Doesnât look like sitemap/ your site, so on- navigation site results might without having to be different build it â˘Can locate problems with your IA
- 104. Action â˘Rearranged Admissions Site navigation to â˘Combine repeated material â˘Put âCosts and Tuitionâ higher in the site tree
- 105. Layout Test
- 106. Layout Test What You Do: Participants respond to a static image of your site Good For: Getting reactions to visual designs and wireframes without building the whole thing. When To Use: Before building out a new design Participants About 50 per test Needed:
- 107. Layout Test Tools â˘Paper Available: â˘PowerPoint â˘Five-Second Usability Test â˘Chalkmark â˘Usabila Cost: $40â$600
- 108. Layout Test Positive Negative â˘Get reactions to â˘No interactivity real designs â˘Users can be without building confused when out a whole site doing this remotely
- 112. A/B Test
- 113. A/B Test What You Do: Test two (or more) versions of your live website Good For: Testing page variations When To Use: Testing calls to action or other pages that have a clear destination you Participants Tool will tell Needed:
- 114. A/B Test Tools â˘Google Website Optimizer Available: â˘Reedge.com Cost: $0â???
- 115. A/B Test Positive Negative â˘Test real â˘Harder to set up variations on your in a CMS live site with real â˘Harder to use users when a page has many destinations
- 116. Easy to do with Google Website Optimizer A Go al B
- 117. Harder to do with Google Website Go Optimizer A Go Go B Go Go
- 118. 1 2 3 4 5 6 7 Find Participants
- 119. The Right Number â˘Different methods have different requirements â˘Plan to recruit more than you need â˘But donât test more than you need
- 120. The Right Kind of â˘Steve Krug: âRecruit loosely and grade on a curveâ â˘Good advice, but when possible try to recruit participants that are more like your users â˘For instance: faculty and students see your website very differently
- 121. Finding â˘People you â˘Local high know schools (talk to a guidance â˘Students, counselor) faculty, and staff at your school â˘On your website (link on homepage or pop-up)
- 122. Photo source: http://en.wikipedia.org/wiki/ File:Ethnio_recruit_ad.png
- 123. Offer a prize
- 124. 1 2 3 4 5 6 7 Run Study and Analyze Results
- 125. Keep track of results in KPI spreadsheet and personas
- 126. 1 2 3 4 5 6 7 Take Action
- 127. Share results and proposed solutions with stakeholders
- 128. 1 2 3 4 5 6 7 Measure and Repeat
- 129. Share results
- 130. Challenge: Do some sort of user research each month
- 131. Research Process 1. Frame a Good 4. Find Question Participants 2. Know What You 5. Run Study and Already Know Analyze Results 3. Choose Study 6. Take Action Type 7. Measure and Repeat
Notas do Editor
- Hi, my name&#x2019;s Mike Powers.
- You&#x2019;ll find us in the town of Indiana, in the state of Pennsylvania. We are a 14,000-student doctoral university in Western Pennsylvania. One hour from Pittsburgh. Because of the confusion with our name, we like to be known as IUP.
- You&#x2019;ll find us in the town of Indiana, in the state of Pennsylvania. We are a 14,000-student doctoral university in Western Pennsylvania. One hour from Pittsburgh. Because of the confusion with our name, we like to be known as IUP.
- You&#x2019;ll find us in the town of Indiana, in the state of Pennsylvania. We are a 14,000-student doctoral university in Western Pennsylvania. One hour from Pittsburgh. Because of the confusion with our name, we like to be known as IUP.
- You&#x2019;ll find us in the town of Indiana, in the state of Pennsylvania. We are a 14,000-student doctoral university in Western Pennsylvania. One hour from Pittsburgh. Because of the confusion with our name, we like to be known as IUP.
- Usability Test Surveys 4q Survey KJ Session Card Sort Navigation Test Layout Test A/B Test
- I got started with this user testing stuff back in 2002 with usability testing--straight out of Steve Krug&#x2019;s Don&#x2019;t Make Me Think. It was a revelation. Everyone who runs a website should do it.
- But the more user-centered I got, the more I wanted to know about about our users. That led to surveys, analytics, card sorts. And then this happened...
- I found myself in charge of a newly centralized university website. 40,000+ pages. It was hard to find the time to do the research we needed. Then this happened...
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- An explosion of online tools for learning more about your users. They promise fast, cheap, actionable results. But if you&#x2019;re like me, you still don&#x2019;t have a lot of time or money. How do you decide what to do? This is the story of how we&#x2019;ve using these tools at IUP.
- Here&#x2019;s what&#x2019;s on the agenda today.
- The idea is to move on from the isolated user research study to an ongoing user research program. Let&#x2019;s get started!
- The research process starts with step 1, but before we get there, let&#x2019;s start with step 0: creating permission to do user research.
- Of course the first rule is: Don&#x2019;t ask for permission. You don&#x2019;t need permission. User research is what web professionals need to be doing. If you&#x2019;re a web designer or web developer or content strategist, this is your job. But others might not see it that way. You&#x2019;re here because you are interested in user research. Our clients and bosses aren&#x2019;t always as excited about it. So at some point you&#x2019;re going to hear this...
- This might come from your boss or you&#x2019;re boss&#x2019;s boss&#x2019;s boss. Or it might come from an internal client if you work that way. And if you ask them want they want, they might say...
- This is kind of typical of initial reactions. But&#x2014;we know this is wrong, don&#x2019;t we? By this theory, the perfect website would be one big red box.
- It can help to reframe this this way: &#x201C;You&#x2019;re right. Nobody wants a research study.&#x201D; Furthermore...
- None of your (internal) clients really want a website. They always have other goals that they have restated as &#x201C;we need this on the website.&#x201D;
- So back them up to what they really want. These are institutional goals. They are more than the website can provide. But we need to ask these if we are to keep our web tactics in line with institutional goals. If you can back them up to what they really want, you&#x2019;ve got a better shot at creating permission to do research.
- Second tactic: You need to get your clients/bosses/whoever they are used to the idea that research is a necessary part of every project. Luckily, since we tend to work on the same website(s) over and over, you can often apply previous research to new websites. So you don&#x2019;t have to do EVERYTHING for every website. To get there, you need to show results early and often.
- 1. Frame a good question: you won&#x2019;t end up with anything actionable unless you&#x2019;re asking the right questions 2. Know What You Already Know: You already know a lot, you just need to keep track of it 3. Will include the inventory of research techniques as well as some advice about the design of each type. 4. How to find the people you need to participate And by then we&#x2019;ll be set up to blast through the last three steps
- This is where fast, cheap, and actionable starts
- These are your institutional goals. And&#x2014;if you want to be fast, cheap, and actionable&#x2014;you need to spend your limited research time investigating issues that relate to institutional goals.
- Here&#x2019;s what happened when we took this statement and got it focused.
- When we looked into this, it turned out to be a question about student success and support. Esp. finding info about support services. Question 1 gives us a good place from which to start our research. It also suggests (correctly) that there are non-web aspects to this problem.
- A second example.
- The purpose of the admissions site is much more clear. And we already had a pretty good understanding of what prospectives are looking for (thanks in part to eduWeb!).
- These are your institutional goals. And&#x2014;if you want to be fast, cheap, and actionable&#x2014;you need to spend your limited research time investigating issues that relate to institutional goals.
- Once you got your questions, you&#x2019;ll need to pose some preliminary answers.
- You probably know a lot already. The trick is getting it all organized so you&#x2014;and others&#x2014;can make use of it. Noel-Levitz study Usability studies (like Jakob Nielsen&#x2019;s) Existing data about your students Once you start looking, you&#x2019;ll find a lot. So it&#x2019;s time to...
- Get organized. You need to start synthesizing what you know into a format you can use and share: with your team, with stakeholders, with colleagues. I&#x2019;ll demonstrate an approach for capturing a lot (not of all) of what you already know.
- A conversion rate is a classic KPI, but you can use anything so long as it shows your progress toward your larger goals.
- See Brian Clifton&#x2019;s Advanced Web Metrics with Google Analytics for a good discussion of setting your KPIs and the germ of the spreadsheet I&#x2019;m about to show you.
- Allows you to collect raw data and format it easily into reports.
- Check my blog site for the whole spreadsheet to download.
- Qualitative data is harder to collect and synthesize. User personas can be really helpful. Does anyone use personas?
- Some of the things you&#x2019;ll put in here come from basic demographic data, etc., but the point is to make them personal. This gives you the chance to embody a &#x201C;real&#x201D; user--and you can talk about what &#x201C;Harold&#x201D; might do on this page. Really useful is this section at the bottom with tasks.
- The scenarios section is really important. If you&#x2019;re a developer, and you&#x2019;ve done test-driven development, it works a bit like that. You record scenarios--successful or not--and then develop the site to make them happen. As a check, go through and try these scenarios yourself. It&#x2019;s like having a series of unit tests. You can make sure that when you fix one thing, you don&#x2019;t break another
- So: what are you going to do?
- First thing to keep in mind: if you haven&#x2019;t done a usability test, start there. It&#x2019;s the foundation for everything else.
- If you&#x2019;ve read Steve Krug, you know focus groups are a bad idea. People will ask if you&#x2019;re going to do a focus group. They think you need to. You don&#x2019;t. You&#x2019;ll only get some ideas about what looks nice, or what people think they should do. But your boss might think that focus groups or surveys are the only options. I&#x2019;ll admit: I have let my boss call some of this stuff focus groups. It was just easier. He told his boss and didn&#x2019;t have to explain anything. But once people arrived, we didn&#x2019;t have a focus group: we did user research.
- These are (mostly) the kinds of research we&#x2019;ve been doing. There are many other kinds, such as...
- eyetracking. But the equipment for that is more expensive (and, you know, painful) so I haven&#x2019;t tried it yet.
- Given the time we have, I&#x2019;ll be focusing on these three, showing you the tool, how we used it, and what action we took. I&#x2019;ll skim over the others. If you go my website or slideshare to get the presentation, you&#x2019;ll find more info about the ones we skipped, or feel free to ask questions later.
- Steve Krug is the place to start. Also, Chas Grundy did a great presentation on this at last year&#x2019;s eduWeb, and he&#x2019;s done some online seminars for Higher Ed Experts.
- A fantastic source of live info about your users. Place, browser, and connection speed. 20% on ie6. Progressive enhancement. What you don&#x2019;t get is intent. That is, if your time on page goes up, is it because they were engaged or frustrated? For intent, you might try...
- A survey. Regular surveys have their place, and can give you good background information on your users. For instance, we were able to see in 2007 that 80% of our students were logging into Facebook at least once a week&#x2014;and that faculty weren&#x2019;t. Doesn&#x2019;t work out so well for questions about your website. You&#x2019;re asking about what they remember about your website, not how it is. For that, try a...
- Surveys have their place, esp. in finding out background info. For instance, we were able to see in 2007 that 80% of our students were logging into Facebook at least once a week&#x2014;and that faculty weren&#x2019;t. For specific questions about your website, it doesn&#x2019;t work as well.
- Work out your sample rate from here. I&#x2019;ve found that 1% to 2% is sufficient.
- Work out your sample rate from here. I&#x2019;ve found that 1% to 2% is sufficient.
- Avinash Kaushik is one of the people behind 4Q, and one of the great things about it, besides being free, is that it is short.
- A lot of the value lies in the comments. But, just reading them isn&#x2019;t enough. These are two real comments.
- You need to read them twice. Once to come up with categories and once to code.
- This is a really powerful technique for developing information architecture. But, it&#x2019;s pretty time consuming unless you do it online.
- Why so many more online? You get less data per participant online. Also: lots of abandonments online. People have a little trouble understanding the activity when you aren&#x2019;t there to explain it for them.
- Why $40? you&#x2019;ll need to offer a prize, because this can take a while.
- Donna Spencer&#x2019;s book on card sorting is the best reference for how to do this well.
- I&#x2019;ve been using Treejack for the closed sorts.
- How have we used it? Ran an online card sort to reorganize student life section of our website. 50 cards 384 participants. This was one of the questions. Counseling center is part of our health center, so that&#x2019;s where is was on the website. B/c mental health is part of health. But did students see it that way?
- Results of the reorganization for one subsite.
- A great technique for getting at a lot of the same things you&#x2019;d use a card sort for, except with a group. Learned this from David Poteet of New City Media last year at eduWeb.
- Tasks come from your user personas.
- Not good task wording here: the answer (&#x201C;visit campus&#x201D;) is in the question.
- You will get people saying &#x201C;Why didn&#x2019;t you get smarter participants?&#x201D; I&#x2019;ve done tests with all Ph.D.s, and they didn&#x2019;t really do better in usability tests. So it helps to be able to say that the users were at least sort of representative
- Ethnio provides an on-site recruiting service. You set up screener questions and then you can either call or e-mail your recruits. You get people who are just like your users because they are your users. You could just use participants who have never been to the site before. Cost was too much for my budget this year. And speaking of cost...
- You can&#x2019;t forget the prize. We find iPod shuffles to work well, but gift cards are an option. Having an incentive, even if it is just a drawing for a prize, not a guarantee, makes a huge difference for recruiting.
- Not everything you try will work. But do try to fail faster.