1. InsideNASA Usability - A
Case Study
Celeste J. Merryman
Knowledge Management and
Collaboration Technologies Office
NASA Jet Propulsion Laboratory
Computer Sciences Corporation
Pasadena, CA, USA
Celeste.Merryman@jpl.nasa.gov
NASA Science Web Federation Workshop
NASA Ames Research Center, Nov 2007

2. The Numbers
1 Set of Evolving 1 Intranet
Requirements
1 Redesign Team *62,000 + people
*(FAIR Inventory Data 2006)
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

3. Where to Start
NASA Science Web Federation Workshop
NASA Ames Research Center, Nov 2007

4. Approach
 Step 1: Bring something to
the NASA sponsor to critique
 Step 2: Ask the people what
they think
 Step 3: Incorporate what we
discovered into a working
Intranet Web site
 Step 4: Open access to the
beta version for comments
 Step 5: Go live
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

5. The Wireframe
 Known weaknesses
 The overall basic organizational structure of the Web site
 The visual aspects of usability frequently dubbed ‘look and feel’
 Search
 Basic research
 The ‘pictorial superiority effect’ says that people remember pictorial
representations of objects better than the textual representation of objects.
 The ‘category superiority effect’ says that people can remember words
significantly better if the words are organized into categories.
 Textbook usability and web design
 “Navigation isn’t just a feature of a Web site; it is the Web site.”
 “Clear; well-thought-out navigation is one of the best opportunities a site has
to create a good impression.” (Krug, 2006).
 Internet Web sites
 Looked at the way different Web Federation Workshop, NASA Ames Research Center, Nov 2007 site visitors -
Celeste Merryman, NASA JPL KM & Collaborations Office
NASA Science
Web sites presented information to
news sites, sites that would emulate a business Intranet

6. Take it to the People
NASA Science Web Federation Workshop
NASA Ames Research Center, Nov 2007

7. What We Wanted to Know
 Their reactions to our detailed wire-frame
 What was important, or valuable regarding
content, collaboration features, search
• Survey requests to 830+ InsideNASA
account holders / 104 responded
• From across 12 Centers, 5 other
contracting locations
• Roles they had in NASA
• engineers, program/project managers,
computer scientist, scientists, IT, human
resources, supervisors/line managers,
other
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

8. Usability Survey
By Survey Monkey
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

9. Usability Survey
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

10. Usability Survey
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

11. Usability Survey
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

12. Usability Survey
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

13. Participation Breakdown by NASA
Roles
• 103 people
Participation Breakdown by NASA Roles
answered this
question; one 40 36
person skipped 35
Computer Scientist
question. Engineer
28
• People could select 30
Human Resource/Human
more than one
Number of People
Capital
option. 25 21 IT/Infrastructure Support
20
• Most common role 20 Program/Project Management
was
“engineer” (n=36) 15 Scientist
11
followed by Supervisor/Line Manager
program/project 10 7
6
Other
management (21) 5 3
1 Skipped Question
0
NASA Roles
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

14. Participation Breakdown by NASA
Centers
• 103 people answered Participation Breakdown by NASA Centers
this question; one
person skipped 20 19
question. Headquarters
18
• JSC had the highest 16 Ames Research Center
participation (n=19). 16 Dryden Flight Research Center
• People indicating Glenn Research Center
14
Number of People
“Other:” 12 Goddard Space Flight
• NASA Johnson Space 12 Jet Propulsion Laboratory
Center White Sands
Test Facility 10 9 9 9 Johnson Space Center
• IV&V Facility 8 Kennedy Space Center
• JPL Contractor ITT for 8 6 Langley Research Center
DSN 5 5
6 Marshall Space Flight Center
• STSCI supported by
GSFC Stennis Space Center
4 3
• NSLD 2 Wallops Flight Facility
2 1
Other
0
0 Skipped
NASA Centers
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

15. Survey Reponses
• 98 people answered
this question; two How frequently do you visit InsideNASA?
people skipped
question.
• Most frequent 30
26
answer was “less 24
than once a 25
Number of Times
month” (n=26); Several times a day
followed by “once a 20 Once a day
week” (n=23). Once a week
13 14
• 14 people said they 15 12 Several times a week
have “never been.” Less than once a month
10 8 Once or twice a year
Never been
5 2 2 Skipped
0
Number of People
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

16. Survey Responses
• 93 people answered What are your reactions?
this question;
seven people 50 47
skipped question.
45
• In general people 40
seemed to have 40 37 35
Number of People
favorable reactions: 35
30
• Overall 30 27
• Navigation 25 26
25 Like It
• Organization of
20 19 Neutral
content 17
• Type of content Dislike It
15
11
• On “the way it 10 9 8 8
looks” responses 5
were almost even 5
for “like it” and 0
“neutral.” Overall Way it Looks Navigation Organization Type of
Reaction of Content Content
Compontents of Redesign Plan
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

17. Survey Responses
Features Related to Social Networking
45 43
40
35 35
35 33
31
Number of People
30 Very Important
25
25 Moderately Important
21
Not Important
20 17 17
Undecided
15
10 8
6
5 2
0
To Engage in Online Subscriptions to Use of NASA Secured
Social Networking Communities of Instant Messaging
with Collegues Practices/Interests
Features
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

18. Survey Responses
Features Related to Content
60
52
50
Number of People
40 38 37 Very Important
36 36
31 32 Moderately Important
30 29 29
Not Important
23 23 Undecided
19 20
20
16
12
10
10
5 5
0
0 0
Up-to-Date Org Access to Agency Able to View Access to Access to Up-to-
Charts of HQ and Management Content via RSS Timekeeping or Date News from
Centers Council Feeds Travel Systems Leaders at HQ
Information
Features
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

19. Survey Responses
Features Related to Search & Site
Personalization
60
55
53
50 48
Number of People
40
32 32
30 27 26 25 24
20
15
10 8 8
3 2 2
1
0
Ability to Search Across Find Information Ability to Ability to Create
Muliple NASA Data Pertaining to Doing My Personalize/Customize Personalized quot;My
Repositories Job Website (e.g. font, color) InsideNASAquot; Page
Features
Very Important Moderately Important Not Important Undecided
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

20. Summary
 Liked / Valued  Did not want / Was not
 Liked the overall wire-frame important
and direction
 Social networking features -
 News from NASA moderately to not important
Headquarters - very
important  Accessing information via
 Up-to-date organizational RSS feeds - moderately
charts - very important important
 Ability to search across  Access to timekeeping - not
multiple NASA data important at all
repositories - very important  Personalization of
 Find information pertaining to InsideNASA - not important
doing their job - very at all
important
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

21. Outcome
• Fewer portlets on front page
• Very busy front page • Improved look and navigation
• Navigation on top meant increased • Administrator Q & A
scrolling as pages were added
• Deputy Blog/ Journaling
• Search located on pages not in
header • Improved search
Before After
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

22. We Were Successful
 Worked closely with the NASA sponsor and understood
the expectation up front
 Great communication within distributed team - weekly
telecons, e-mail exchanges
 Kept our focus on the end consumer - the site visitors
 Able to meet the needs of the NASA Headquarter
customer
 Able to deliver a quality product within our technological
scope
 Kept our eye on the prize and avoided scope creep
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

23. Some InsideNASA Metrics
 07/2005 - 09/2005
 One-time visitors: 13,132
 Repeat visitors: 6,494
 Unique Visitors: 19,626
 Total Hits: 539,735
 Total page views: 1,895,142
 07/2007 - 09/2007
 One-time visitors: 10,658
 Repeat visitors: 5,961
 Unique Visitors: 16,619
 Total Hits: 4,180,626
 Total page views: 1,402,945
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

24. A big thank you for listening.
Questions ?
Contact: Celeste.Merryman@jpl.nasa.gov
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007

25. References
 Krug, Steve. Don’t make me think: A common sense approach to Web
Usability, 2nd Edition. Berkeley, California: New Riders Publishing, 2006.
 Nelson D. L., Reed V. S., & Walling J. R. “Pictorial superiority effect,”
Journal of Experimental Psychology, Human Learning, Vol. 2 , No. 5, pp.
523-8 (1976).
 Sharps M. J. , Wilson-Leff, C. A., & Price, J. L. “Relational and item-specific
information as determinants of category superiority effects,” Journal of
General Psychology, Vol. 122, No. 3, pp. 271-85 (1995).
 Toglia, M. P., Hinman, P. J., Dayton, B. S., & Catalano, J. F. “The blocked-
random effect in pictures and words,” Perception and Motor Skills, Vol. 84,
No. 3, Pt 1, pp. 976-8, (1997)
 2006 NASA FAIR (Federal Activities Inventory Reform) Act Inventory (http://
competitivesourcing.nasa.gov/2006fairindex.html)
Celeste Merryman, NASA JPL KM & Collaborations Office NASA Science Web Federation Workshop, NASA Ames Research Center, Nov 2007