This study evaluated two user interfaces - a traditional web form and a wizard form - for entering personal health information in a patient portal. Usability tests were conducted with 8 participants who used both interfaces and provided feedback. Analysis found slightly more usability issues with the traditional form regarding navigation and content. User satisfaction surveys showed no significant preference for either interface. The study aims to refine the interfaces based on these findings.

2. User-centered Design of a PHR: Traditional Web Forms vs. Wizard Forms Barbara Purin and Emiliano Ricci FBK, Fondazione Bruno Kessler, Trento, Italy Friday, September 5, 2008 e-Health Applied Research Unit

3. The Department of Health and the Department of Research and Innovation of the Autonomous Province of Trento (NE Italy) have funded a feasibility study of a Personal Health Information Management ( PHIM ) System for the citizens living in the Province. PHIM refers to activities that support consumers’ access, integration, organization, and use of their personal health information. PHIM activities rely on the collection and management of one’s own personal health information collected from each health care provider plus any health information you want to add. Considerable effort was dedicated to guide and support the management of health information in the System design phase. Friday, September 5, 2008 e-Health Applied Research Unit The Autonomous Province of Trento The Department of Health Andrea Civan, Meredith M. Skeels, Anna Stolyar and Wanda Pratt, Personal Health Information Management: Consumers’ Perspectives. AMIA Annu Symp Proc. 2006; 2006: 156-160

4. - Exploring the effectiveness of the traditional web forms vs. wizard step-by-step structure ; - gathering information about problems that users may encounter when interacting with such interfaces; - understanding which system, if any, improve more the data entry process and attracts more users. We focused on testing usability and user experience of two user interfaces designed and developed to support citizens to maintain drugs’ information themselves. Input data forms and navigation are crucial elements; they would be used consistently in order not to compromise the use of the System. REMARKS: Widespread and extensive use of traditional and wizard form layouts for data input and configuration in Web applications.  Friday, September 5, 2008 e-Health Applied Research Unit

7. Stories of common real life drug prescription: - recording a new therapy and specify drug dosage ; - recording a therapy evaluation . Task #1, Recording a new therapy Goals: entering new therapy data (drug name, confection, reason for consumption, therapy start date); the task is complete when the form is filled and saved (data are displayed in a not-editable format ). Inputs: --- Assumptions: --- Steps: the user has to (…) Friday, September 5, 2008 e-Health Applied Research Unit

12. 8 participants [1] ( women ) recruited among the administrative personnel of the Bruno Kessler Foundation (FBK). INCLUSION CRITERION: skill in using the web (use of Internet from more than 1 year and for at least 1 hour per week) [1] Nielsen, J. “Why You Only Need to Test with 5 Users”, http://www.useit.com Usability test participants’ characteristics COMPARISON TEST: - 4 of them used first the traditional UI and then the wizard UI; - the others used first the wizard UI and then the traditional UI. Friday, September 5, 2008 e-Health Applied Research Unit 0 4 4 Reason for using the Internet Only at home Only at work Both 2 4 2 Time spent using the internet Less then 3 hours per week 3-6 hours per week More than 6 hours per week 0 8 Internet experience Less than one year More than one year 5 3 Educational qualification High school Univesity 3 1 4 0 Age range (years) 19-29 30-39 40-49 More than 50

13. DATA GATHERING: reports were transcribed in a world processing file; a content analysis was performed. Users were encouraged to “think aloud” commenting on any difficulties they encountered while performing the tasks. ( Kushniruk, A.W., & Patel, V.L. (2004). Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of Buomedical Informatics, 37, 56-76 ) CODING CATEGORIES: navigation, graphics, layout/screen organization, color, resolution, meaning of labels, understanding of system instruction/error messages, consistency of operations, overall ease of use, response time, visibility of system status, data not displayed, data entry Friday, September 5, 2008 e-Health Applied Research Unit

14. Friday, September 5, 2008 Total usability problems by category e-Health Applied Research Unit

15. Friday, September 5, 2008 - Chart 1 - Usability problems funded by the subgroup of participants that used first the traditional UI and then the wizard UI. - Chart 2 - Usability problems funded by the subgroup of participants that used first the wizard UI and then the traditional UI. e-Health Applied Research Unit

16. At the end of the tasks, users were asked to fill a questionnaire for assessing user satisfaction . COLLECTED DATA: questions were based on the seven-point Linkert semantic scale; each response was converted to a numerical value in the range [-3,3]. FINDINGS: the user satisfaction is basically positive (1); no statistical differences between the two interfaces (2). Friday, September 5, 2008 e-Health Applied Research Unit 1,05 0,925 0,4 0,6 1,7 2,3 1 0,4 1,3 1,3 -0,3 0,4 2 0,8 1,8 1,8 0,2 0,1 MEAN (range [-3,3]) 1,625 1,125 2 2 2 2 1 1 2 2 -1 2 2 2 3 3 -1 -2 pleasant 0,75 1,25 0 0 1 3 2 0 -2 2 -2 1 2 2 2 3 1 1 I like 0,75 0,875 0 1 1 2 0 0 2 0 2 -1 2 2 2 2 -1 -1 satysfying 1,375 0,875 0 0 1 2 0 0 2 2 -2 1 2 0 2 2 3 3 engaging 1 0,75 0 1 2 3 0 0 2 1 0 0 2 1 2 2 -1 -1 comfortable 1 1,125 1 2 2 2 1 1 1 2 1 -1 2 1 2 2 -1 -1 clear 1,75 0,875 2 2 2 2 0 0 2 1 -2 2 2 0 2 2 2 2 interesting 0,625 0,75 -2 -2 2 2 2 -1 2 2 1 0 2 0 1 2 0 0 intuitive 1,5 1 2 2 2 3 2 2 1 1 -2 -1 2 -1 2 3 1 1 easy to use 0,125 0,625 -1 -2 2 2 2 1 1 0 2 1 2 1 0 -3 -1 -1 easy to understand MEAN w MEAN p p w w p p w w p p w w p p w w p p = traditional web form w = wizard web form User H User G User F User E User D User C User B User A USER & SYSTEM INTERFACE  QUESTION

17. User’s opinions about the preferred interface were gathered during the debriefing phase of the tests. FINDIGS: Friday, September 5, 2008 1 user was not able to state the preferred interface because in her opinion the two interfaces were similar. e-Health Applied Research Unit 4 3 # PREFERENCES “ it is easier to use ”; data are entered step-by-step; it calls your attention on the data you are entering; “ I see both data entered previously and information required by the form to be filled ” WIZARD WEB FORM more comfortable; (plenty of) data in a single page (“ in a wizard form I don’t know which information will be required in the following step ”); data are aggregated; TRADITIONAL WEB FORM REASON WEB FORM LAYOUT

18. No one user interface seems to prevail against the other. The most frequent usability issues encountered by the users were related to clearly-defined content and navigation problems. Friday, September 5, 2008 The incidental preference coming from the debriefing interview was founded on subjective impressions . e-Health Applied Research Unit

19. Repeated cycles of design-testing-measure-redesign allow pointing out wrong design assumption about a system that could cause usability problems later. Friday, September 5, 2008 Comparison tests ( different prototypes matched against each other ) avoid to commit too early to one design that could reveal its faults only later. The two prototypes are going to be refined according to the founded usability issues; then a usability test with participants ( both women and men ) less skilled in using the web will be performed. It could be interesting to evaluate the two interfaces after people have used one of them for a period of time. e-Health Applied Research Unit   

20. e-Health Applied Research Unit Thank you for your attention Friday, September 5, 2008  http://www.fbk.eu  http://ehealth.fbk.eu