edUi 2013 presentation by Jackie Zajac in Richmond, VA on Nov. 4, 2013.

1. @jackiesauter
Usability Within Limits
Jackie Zajac
Director, Strategic Communications

2. Ground Rules
•
•
•
•
This is not uber technical
$500 over 18 months
Colleagues not familiar with UX
Significant improvements AND mixed results

3. Inspiration
•
•
•
•
PSU Web Conference
User Interactions with Information Systems
Declining graduate enrollment
Bickering over web strategy

4. AU’s Kogod School of Business
• 1,200 undergraduate,
600 graduate students
• 120 Faculty & Staff
• 8 Graduate Programs
• 40,000+ visits/month

5. “1,000 More Grad Students in 10 Years”

6. Chronology
First
study
done
Presented at
PSUWeb 13
06/12
09/12
Changes
implemented
Changes
implemented
06/13
07/13
Follow-up
study
done
10/13

7. The Players
8 Faculty Program
Directors
6-person Admissions
Team
Dean, Provost,
University Marketing Advisory Council,
Web Steering Committee...

8. Study Parameters
• Time (< 3 weeks)
• $250 budget
• Silverback ($69.95)
• Gift cards
• Technical limitations due to CMS
• Only tested American.Edu/Kogod

9. Magic Numbers
6 Tasks, 5 Users*
*Jakob Nielsen said it was OK!

10. Magic Numbers
Source: NIELSEN NORMAN GROUP

11. Test Design
Homepage Impressions
Tasks (6)
• Task Completion Survey
• Final Thoughts?
• Less than 30 Minutes
•
•

12. Tasks
1. How many credits are in the MS in Accounting program?
2. Find the average GMAT score for a Full-time MBA
student
3. What is the cost of the Part-time MBA program?
4. When is the next Information Session being held?
5. Find the graduate admissions application
6. Find the graduate admissions’ office phone number

13. Completion Survey
After each task, the user was asked:
1. Did you complete the task? (Yes/No)
2. How satisfied are you, on a scale of 1 to 5*?
3. How easy was it, on a scale of 1 to 5*?
*5 being very satisfied

14. Task completion rate
Task completed from testers' perspective
Task completed from users' perspective
100%
80%
60%
80%
60%
80%
60%
80%
80%
80%
60%
40%
Find the number Find the average Find the cost of the Find when the Find the graduate Find the graduate
of credits in the GMAT score for a Professional (Part- next information
admissions
admissions’ office
MS of Accounting. Full-Time MBA
time) MBA
session is being
application.
phone number.
student.
program.
held.

15. Users’ Final Impressions
My mistakes were easy to correct.
I always felt I knew what was possible
to do next.
Maybe
No
Yes
Information is easy to read.
Site was easy to navigate.
Site is attractive and has a good
balance of graphics/text.
0
1
2
3
4
Number of user responses

16. Key Finding:
Too Many Navigational Options

17. What’d We Do
With All This?

18. Recommendations
1. Added an Admissions tab to the top nav (Duh.)
2. Linked directly to each graduate program from the
homepage
3. Restructured Admissions nav
4. Created a new, central MBA “landing” page

19. Recommendations
5.
6.
7.
8.
9.
Decrease jargon
Make functional info clearer + easier to find
Social media more prominent
Reduce image file sizes
Delete homepage “touts”

20. All Aboard!

21. All Aboard!
Dear all,
You may have noticed some changes to the Kogod website recently. They are the
result of a usability study of the website we conducted this summer. The focus of our
testing was to determine how easy the website is for our primary audience—
prospective students—to use. Based on the study's results, we've implemented
several changes to the website's layout, navigation and terminology. A few more
changes lie ahead.
We expect that this process will make it easier for our prospects to find the
information they're looking for, and to take action. A follow-up study will be conducted
in 2013 to determine whether further improvements are needed.
If you have any questions, do let me know.

22. Relationships

23. Challenging.

24. The players
8 Faculty Program
Directors

25. Your BFFs <3
1. Data
2. Admissions
3. School Goals

26. Results @ 6 Weeks
Admissions Homepage: 7,637 pvs (+3%)
MS Finance: 3,819 pvs (+50%)
MS Accounting: 3,115 pvs (+60%)
MS Sustainability Mgmt : 1,325 pvs (+5%)
MS Taxation: 1,106 pvs (+84%)
MS Real Estate: 1,095 pvs (+63%)
Renamed “Tuition & Fees”: 2,147 pvs (+150%)
New MBA “landing” page: 2,617 pvs (n/a)
New MS Applicants page: 2,515 pvs, 02:19 TOP

27. …Spring 2013…
Redesigned All Degree Pages
• Clearer Calls to Action
• Consistency
Redesigned Graduate Programs Index Page

30. Remember:
Always send a status update to stakeholders.
Don’t forget SEO.

31. …Summer 2013…
No substantial change in visitors
Admissions homepage: +3.8%
Tuition & Fees: +190.7%
But we were DOWN:
• Pageviews -6.7%
• Avg visit duration down (slightly)
• Pages per visit -7.3%

32. …Summer 2013…
• Usability testing with same tasks
• 2-3 additional tasks
• Tested microsite for direct marketing
campaign
• Tested ad landing pages
• Tested on desktop + mobile devices

33. Follow-up Findings
•
•
•
•
3/5 liked the degrees listed on homepage
2/5 immediately clicked on “Admissions” tab
Users liked the MBA landing page
All 5 appreciated the simplicity, were not
overwhelmed by information

34. Task Completion Rate
Completion from user's perspective
100%
100%
100%
100%
Actual Completion
100%
100%
80%
100%
100%
80% 80%
100%
80%
60%
Task 1
Task 2
Task 3
Task 4
Task 5
Task 6
Task 7

35. Users' Final Impressions
5
My mistakes were easy to correct.
I always felt I knew what was possible to
do next.
5
3
5
Information is easy to read.
Yes
Maybe
No
5
Site was easy to navigate.
The overall site is attactive
4
1
0
1
2
3
4
5
6

36. Does this mean
we’re there?

37. Not quite.
•
•
•
•
Confusion about “part-time” vs. Professional MBA
Difficulty finding GMAT scores
Tuition confusion (PMBA)
Admissions events

38. Finally,
• Usability testing with same tasks
• 2-3 additional tasks
• Tested microsite for direct marketing
campaign
• Tested ad landing pages
• Tested on desktop + mobile devices

40. Photo by Cade Martin

41. Finally,
• Disconnect between what users say and
what they do
• Scrolling, task completion
• Career outcomes very important
• Photo reaction

42. Questions?
@jackiesauter