18. Creating your Test:
Questions
• figure out what to test
• create a question that tests that
• write specific questions
19. Test Description
• “The goal of this study is to evaluate how
people find information using our website. I will
ask 15 questions and would like you to think out
loud while looking for the answer. I will time
you, and will stop you after 3 minutes have
passed. Don’t worry if you can’t find the answer
every time–we are testing our Web site,
not you! Each answer can be found using the
library’s Web site. The test should take no more
than 20 minutes. Do you have any questions?”
21. Administer
the test
• Think of this step as the consumer
“taste test”
• Administering the test is the easiest
part of a usability study.
• read the explanation, whip out your
stopwatch, and say “go.”
A “website” especially a “library website” tends to be an older, almost more traditional model. Here’s what this older model focuses on: [say the stuff on the slide, dude] What’s going on here?
A traditional library website is about the library - it’s not THE library. let’s look at a few websites - is it a website or a digital branch? Is it THE Library, or About the library?
I’m picking on some library websites here... about their services, policies, about their programs, a list of staff, links... announcements ABOUT the library - not THE library
How about this one? Nice clean design... Personal touch, making a connection, easy to read & look at. Helpful suggestions – get started, ask a librarian, search the catalog. Still an about the library section too – you need a mix of both. So - what’s a digital branch? It’s... [click]
digital branches are just like a physical library building. They have each of these things covered... they can be done digitally
ok - we don’t have a building... or do we? website IS the building - and it has upkeep like a real building does: - there is coding upkeep, data and content upkeep - there are “paint jobs” and “touch up” work to be done clearly laid out places to go to do things commenting and interactive pieces are clear 2.0 stuff is in place - rss, commenting, multimedia, etc...
how about staff? touches the public - librarians can answer questions touches the code - there is clean up work to be done, is there not? someone still HAS to be in charge, make decisions
real staff = staff interaction right on the site IM, email someone responding to comments someone overseeing a community forum...
needs to be something for customers to do - read, watch, listen to someone needs to organize the “digital” collection so it’s easy to discover and interact with our ILS’s - wonderful example. you can check stuff out - our bread and butter
real collection = reading/viewing right there catalog, ebooks, audio books, magazines librarian created content - videos, podcasts patron created content...
OK David, can we really do THIS ONE online? Well... actual people go there you can meet people (forums, discussion lists, IM...) staff interact with customers you can attend programs (screencasts, online book clubs) ok - the coffee bit doesn’t work too well yet... :-)
and, real community: real place to hang out, connect (via comments, social networks, book clubs, forums, etc) real place to attend meetings (livemeeting, tutorials, live feeds, video) ways to facilitate two-way interaction
next section - usability
What is usability & usability testing? Website usability is a measure of ease of use. Usability is measured by asking customers to complete a task on a website. This week’s lesson will focus on website usability and on creating a simple usability test for your website. How does this affect sites? There are really two types of usability: bad and good. Bad usability means that your patrons won’t be able to find information using your library website. For example, if your library databases are hidden under a “library resources and services” link, patrons might not be able to find the databases page. If this is the case, library customers will probably complain about your website. Good usability, on the other hand, means that library customers will be able to find everything your website offers. The first time visitor can easily navigate and find information. Good usability also means little complaining! People might complain that they don’t like a certain feature, but they will still be able to navigate within that feature and find the information. A usability test is a way to find out if your website has good or bad usability. The test consists of a series of well-thought-out questions designed to test parts of your website for usability.
The first thing to do when creating a usability test is to figure out what parts of the website to test. This can usually be a simple process of writing down major features and services currently presented on your site, because the goal of the questions is to test the most important features on your website for usability. Once you have decided what parts of the website to test, you need to create a question from each of those parts. When I create usability questions, I usually try to test the usability of information presented on the main page, and the usability of information found off the main page. For example, one of the questions asked the testee to find information on joining our Friends of the Library program. This question not only tested if testees could find the appropriate information, but it also tested all our links under the main About the Library link. When writing usability questions, remember to write specific questions rather than open-ended questions. For example, on a previous usability test, I created a question that tested our Children’s website. The question was: “Your daughter’s grades are slipping, and she needs help with her homework. Does our library have a Web site that will help her?” This question tested two things: It tested main page navigation, link wording, and link placement. It tested sub-level navigation, link wording, and general design on the KidsLinks page (since the answer to the question could be found by clicking the link to “Homework Help” on the main KidsLinks page).
A description of the test and test-taking instructions also needs to be created. This text will actually be read to your test-takers. Proper instructions will help set them at ease. Make sure the test description and instructions place emphasis on Web site faults, rather than on the volunteer getting correct/incorrect answers on the test. Here’s the paragraph we read to our volunteers: “ The goal of this study is to evaluate how people find information using the Library’s Web site. I will ask 15 questions and would like you to think out loud while looking for the answer. I will time you, and will stop you after 3 minutes have passed. Don’t worry if you can’t find the answer every time–we are testing our Web site, not you! Each answer can be found using the library’s Web site. The test should take no more than 20 minutes. Do you have any questions?”
Have you ever been stopped by clipboard-carrying market researchers at the mall? Here’s your chance to be one! Now that the test is ready to go, you just need to find volunteers willing to participate. I’ll get to that word “volunteer” in a minute. First, let’s decide how many participants are needed for a successful test. Jakob Neilsen, Steve Krug, and other usability experts all agree: Testing sets of five people three times (different people) will provide more and better information than testing 15 people once. Here’s why: With a one-time 15 user test, you’ll find 100 percent of your website usability problems. But when three five-user tests are given, you get these results: Test 1: 85% of usability problems are found. This allows you to redesign to fix those problems, and then test again. Test 2: Redesign problems are found, as well as the first 15% of the usability problems missed during the first test. Plus, this second test “will be able to probe deeper into the usability of the fundamental structure of the site, assessing issues like information architecture, and task flow.”4 Then redesign again, and test one more time. Test 3: This test catches anything missed during the first two tests. So, three tests with a smaller number of volunteers allows for faster turnaround, since you’re redesigning during the testing period. It also allows deeper probing into the usability of your site, thus creating a better site than if you tested 15 users once. Now, let’s look more closely at the term “volunteer.” Do you need to pay volunteers? I think not. A university library can easily find volunteers (students) willing to take a quick test. Pass out candy bars or something similar as an added incentive to volunteer. At a corporate library, ask people you work with to take the test. Most likely, you can find five people willing to do something other than their normal routine. At my library, I called the manager of the branch or department where I’d be doing testing, to find out their busiest times. Then I went to that department/branch, “hung out” by the computers, and asked people to volunteer for the test. We gave volunteers a library mouse pad for taking the test. Most people I asked agreed to take the test. You don’t really need to worry about finding a cross-representative sample of users when testing only five customers. But, in some cases, it is preferable to find representative users for your test. For example, are you testing a corporate intranet? Then, by all means, use employees that use the intranet. Are you a university? Then make sure to test students, and maybe a professor or two.
Think of this step as the consumer “taste test” where you measure consumer preference. Your questions are ready, and your volunteers are lined up to go. Sit back and relax! Administering the test is the easiest part of a usability study. At this stage, sit down with the volunteer, read the explanation and directions for taking the test, whip out your stopwatch, and say “go.” Take notes! Write down the path the volunteer takes when he or she is trying to answer the question. Note more than just the URL of the final answer. Also note where the person actually clicks – if the volunteer goes back to the main page to choose another link, etc. Record whether or not an answer is found for each question. Record the amount of time each question takes to answer. There’s a huge difference between answering a question in two seconds versus three minutes. Record the page where the answer, if any, is found. Also record anything said during the search process, if the volunteer visually scans a page, or if the person almost clicks on something and then doesn’t. Record anything you notice that may prove significant. Some people actually capture usability tests on video, so they can review later for the more subtle nuances of usability (visual cues might be missed while you are writing down a web path).
Once all the usability tests have been taken, it’s time to figure out what to do with the information that was gathered. For starters, I usually provide a summary of what was found for each question. I note how many people correctly answered the question, write down the paths each person took to find the answer, and any other appropriate comments that they or the proctor noted during the test. Next, examine that summary for good nuggets of usability information. Start with the volunteer’s final answers. How many found correct or acceptable answers to the questions? If all five volunteers found an answer relatively quickly, then the item being tested probably functions correctly. If, however, one volunteer answered correctly and the other four didn’t find an answer, something needs to be fixed. Compare notes on that question for each volunteer. An easy way to compare is to group paths and important notes for each question. Then, scan the paths and other notes together. This is where subjectivity comes into play. You need to find out if your volunteers had similar hang-ups. Did some of them go down the same wrong path? Did they make it to the right page, but then couldn’t find the link that answered the question?
Wording Problems: All of us, no matter what type of job we’re in, use work-related lingo. We assume everyone inherently understands that lingo. But that’s not true! You need to watch for these jargon problems. For example, in the library world, do your customers really understand terminology like “Reference,” “Interlibrary Loan,” “OPAC,” “audio/visual material,” or “monograph?” Probably not. Our users got stuck on words like “Reference,” “Online Catalog,” and “Policy.” Design Problems: You’ll also find a myriad of design flaws that you didn’t know existed. You might be used to your Web site, but testing users quickly provides a fresh perspective on visual design and information flow. Here are some things that might occur during the test: Volunteer missed the link because it was in the middle, right-hand side of the page, surrounded by other links. Since many people scan from left to right, upper to lower, make sure to group the most important navigational aids to your Web site on the upper left-hand side of the page. Volunteer missed the link because it’s “under the fold.” You have to scroll down to see it. Don’t hide important links under the screen. Many of your users will miss the links, since they quickly scan pages and move on. Volunteer missed the answer because it was hiding under a link that provided no explanatory text. To fix this, include short explanations with important navigational links. For example, under our new Resources link, we provide a summary that says “find magazine and newspaper articles, genealogy, local history, unique collections, Web links and more.” This explanation gives the user more information, and therefore helps him quickly decide on his destination. Now, there’s one more step to take. Complete your usability test, redesign, and then … do it all again. Remember the three five-person tests? Go that route, and you’ll save yourself many usability headaches later on.
Card sorting is handy - put web pages on index cards, then ask people to arrange them. You quickly get a feel for how your customers would arrange your stuff But you can also do a similar thing live. Have someone stand at the circ desk, and ask them stuff - like what do you call library databases? What would you callthe form where you can make purchase suggestions for us? That type of thing... Great for getting beyond jargon!
talk about it!!!!!! – Usability test
next section - how do I gather info about making a website? Business decision stuff - what to focus on is the question. Customer and library priorities. I get a lot of this by meeting with staff and with patrons. Here’s what I do
talk about it - my process, what I did The questions, who I met with How I did it Summarize staff findings summarize patron findings
How I summarize findings mindmaps! put all 5 or so questions down, then typed each response - for each meeting then I combined like stuff From that, I summarized the major things.
next section - how do I gather info about making a website? Business decision stuff - what to focus on is the question. Customer and library priorities. I get a lot of this by meeting with staff and with patrons. Here’s what I do
explain sitemap first mindmapped it out then went through each item and consolidated/removed/renamed Still need to work on labeling!
This is different from the experience brief. Instead of explaining the experience you want someone to have, this answers more of the why people would come here, what should be here, and what should it look like, how should it function stuff. For each section of you website, answer these questions: Purpose: why does this section exist? Goals: What’s it doing? How do we know when we reach the goal? Primary Audience: who? adults? Kids? 35-year old moms? - can’t be everyone... why viewers use section: why will patrons come here? Tasks section accomplishes: self explanatory Content Requirements: what types of content are required? Functionality Requirements: what functionality is needed? Read lilly’s stuff for a section!!!
an experience brief - 1 page story about experience, then build what you wrote about as the bible for your website - thinking about experience is better than not!
wireframing - here’s what it is, how it works visual outline of the website not focused on graphics, etc Focused on where stuff goes, what space it should fill, etc. This can then go to the graphic designer who can fill in each part and make it look great... then you have to actually start building the thing.
speed, not focused on graphics, etc Focused on where stuff goes, what space it should fill, etc. This can then go to the graphic designer who can fill in each part and make it look great... then you have to actually start building the thing.
hugely important. Here’s our process - our creative group works through stuff, gets it how we want it, then we start sharing I share with director and deputy director, we change big things then I share with management council, and change big things, then share again, etc until it’s good then I share with the staff ... change big things, and explain lots of stuff then we share with patrons, and change big things, repeat...
you should set one as a goal you should also release it unofficially 2-3 weeks prior, tell patrons about it, and let them play/comment sort of a beta test remember - it’s a branch library what do you need on opening day? content, staff, a building lots of coffee and ibuprofen :-)
Better than a real building - because you can and should continuously build! Fix stuff that doesn’t work well, etc. - continuous writing/creating. - schedule writing time - training. posting. writing. making a video. - schedule regular reading times, make suggestions/critiques - periodically touch base with content developers. documents: - style guide - discussion policy redesign, usability, etc!
check out top 10 pages, and time on site see if anything odd sticks out we use google analytics - it’s free & easy to use. And pretty detailed, too.
most popular content shows what people are doing #1 is deceptive - all our PCs are set to that #2 is funny - people are looking for jobs ... not library shelver jobs. We need to reword something #3 is cool - summerfest! #4 is weird - do that many people really want to know how many items to check out??? #11 - yay. databases made the list. Can we figure out how to get those moved up int he rankings? I’ll bet we can... etc - that’s how we go through these...
landing pages this can be interesting. what’s the “door” are people using to enter your digital branch? This is the page that tells you... Jobs? weird summerfest - flyers and newspaper articles etc bounce rate, landing pages where are they finding these pages?
what’s next? Do all of the stuff I’ve been talking about over again! big changes - maybe every year or so. most websites change something big every 18 months, because the web is still changing that fast. Gotta keep up! small changes - continually. This is the cool thing about a digital branch - it’s DIGITAL. You can make quick changes and tweeks whenever you need to!
next - online community facebook, twitter, etc - think of your main website as your home base. Think of facebook, twitter, etc as outposts you want to staff the outpost and fill it with stuff, so there’s something to do there... you also want to be where your patrons gather. Here’s what you can do in those spaces:
#1 - STOP. Before you do ANYTHING ELSE. It’s great to jump in as an individual. But don’t brand it as your library until you ... ... strategize a bit. For example, why do you want to have a presence in Facebook? you need some reasons, a bit of strategy, and you need to set some goals So - figure out a plan for starters. what content, who does the work, what next?
While you’re stopped, use your personal account and start listening. See what people say, and figure out how the tool or service works. Then...
are they talking about you topics close to your heart what they say in general about where you live, issues of the day how they say it - lingo, slang, etc this helps you figure out the lay of the land in that social network - how you can fit in, how to add your own stuff
This part’s easy. set up google alerts, youtube searches set up twitter searches Watch the activity feed in whatever social network you’re interested in (twitter, facebook, blip.fm, etc all have this)
Twitter search for my library. You can also do a location-based search using a zipcode. And a word, like library, reading, watching, etc.
ViralHeat - viralheat.com – is a cool tool that just created a free version. So stuff a couple of your SM accounts in it, and see what happens. Twitter – measures growth, total tweets, retweets, links shared, who mentions you, etc.
Socialmention.com is a cool tool - it searches twitter, facebook, friendfeed, youtube, digg, google, etc for mentions of you, aggregates that, and makes a feed you can subscribe to (or turns it into an email alert with links, too). It’s a cool way to see what people are saying about your library without having to visit lots of different places.
you want friends. numbers are good, patrons can talk privately with you - think confidentiality, privacy friend them back! focus on customers/patrons, focus on people living in your service area, whatever that means. Use tools like wefollow or the Find People searches in tools like Facebook or Twitter and friend people living your area. Pay for Facebook Ads and ask for friends that way. A “David likes the library” ad. $10 per day. Be active.
so ... think of status updates as short conversation starters. Make sure to invite responses by asking for them - ie., what do you think? what’s yours? etc. share your stuff - new materials, events, thoughts - interesting local news, add a library twist if you can answer questions - real questions, questions about you that they didn’t ask you about, real questions that you know the answer to and aren’t about you (remember - you’re their friend).
also conversation - with ways to comment share what’s happening - events, your collection, new tech or services - be a reporter. share your staff! who are those friendly faces? teach things - screencast, short and fun
be slightly informal - think business casual rather than formality - it helps you seem more real and human improvise - don’t memorize lines - are you really a good actor? - instead, outline, storyboard, then wing it Get used to it so you can be yourself. mics and lighting - read up on those a bit so you can still sound and look good. Michael Porter and I will be teaching this in a preconference at Internet Librarian ...
be active - add links, add tips, add info about you claim your space if you can - foursquare has yet to email me back Facebook has a check-in function too. Use this stuff if your community uses it!
free coffee on Fridays gets people in that space to be a bit more active, provides incentive
reply in twitter, comment and like in facebook leave comments on your patron’s blogs don’t be a corporate schnozz. Instead, be a real person goal here? - answer questions, correct info, generally be helpful - make the library seem real, friendly ... - and a service I can’t live without!
So - let’s say that next week, there’s a new service that’s making news, & lots of people are joining. What do you do? 1. don’t write it off - how many wrote off twitter or facebook ... and are now addicted? 2. immerse yourself - friend 50-100 people, interact a couple times a day for a month or so - this is the best way to get a feel for a service 3. see if there’s stuff you can do to connect those users to your library - tips, replies, broadcasting out info, etc. Most likely, there is
Then improve the service by looking at trends! More friends/followers. More views. Use stats - lots of these services have stats. Some examples: This is from flickr - shows number of views, number of comments.
Facebook - how many interactions happened - likes, comments, wall posts Male/female breakdown, etc. So - take a peek at your stats. Then figure out what to do with them: - attracting a certain type of person? IE., 35 year old women on Facebook? Can you focus interactions on that group? - getting unexpected results (weird age ranges maybe)? Look over your last month’s tweets/videos/updates and adjust accordingly. Experiment!
just stuff to think about for the next version of your site, or for additions: CMS - use one! it will be easier for staff to publish. and get you more staff content creators, too ... Drupal, Wordpress, Expression Engine, Joomla Thin about Facebook as expansion - the like button, driving traffic to your website using status updates, etc. patron content as expansion!