This talk covers guidelines for writing plain language instructions on ballots and clear information in poll worker manuals. It also talks about common ballot design problems and wraps up with information about usability testing using the UPA LEO Usability Testing Kit.

1. 1
Plain language for voters and
election workers
Dana Chisnell, UPA | Usability in Civic Life
Drew Davies, AIGA | Design for Democracy
Kim Kizer, Elections Division - Texas
IACREOT - Spokane - 2009

2. 2
Many changes
lever punch card DRE optical scan
http://www.flickr.com/photos/plemeljr/61432544/

3. 3

4. Constraints exist
State law Machines Expense History

5. 5
Best practice
+ Incremental changes
+ Usability testing
lower residual vote rates + better security

6. 6
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing helps
Resources
The Texas story

7. 7
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

8. 8

9. 9
Key points
Poor ballot design frustrates voters,
undermines confidence, and contributes
to related Election Day problems
Dana Chisnell

10. 10
Key points
Thousands of votes are lost or miscast
All voters are affected
The risk is greater for some groups of
voters
All voting technologies are affected
Usability testing is the best way to make
sure that voters are successful
Dana Chisnell

11. 11
Design for Democracy - EAC
Best practice is a
great place to start to
redesign ballots

12. 12
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

14. Top 10 election design guidelines

15. 1 Use lowercase letters

18. Optical scan ballots [detail]

19. Rolling DRE (touchscreen) ballot [detail]

20. 2 Avoid centered type

23. Rolling DRE (touchscreen) ballot [detail]

25. 3 Pick one sans-serif font

26. Optical scan ballots [detail]

27. Rolling DRE (touchscreen) ballot [detail]

28. Voter information materials [detail]

29. 4 Use big enough type

31. Optical scan ballots [detail]

32. Rolling DRE (touchscreen) ballot [detail]

33. Voter information materials [detail]

34. 5 Support process
and navigation

36. Optical scan ballots [detail]

37. Rolling DRE (touchscreen) ballot [detail]

38. Voter information materials [wayfinding]

39. 6 Use clear, simple language

40. “A vote for the names
of a political partyʼs
candidates for president Vote for one
is a vote for the electors
of that party…”
“Vote for not more than
Vote for one pair
one set of candidates”

41. STATE REFERENDA 2A - CITY OF SPRINGFIELD
SHALL CITY OF SPRINGFIELD DEBT BE INCREASED BY AN AMOUNT NOT TO EXCEED $4,600,000, WITH A MAXIMUM REPAYMENT
COST OF $8,000,000, AND SHALL CITY OF SPRINGFIELD TAXES BE INCREASED $1,047,000 (FIRST FULL FISCAL YEAR DOLLAR
INCREASE) ANNUALLY; SUCH DEBT TO CONSIST OF SALES TAX REVENUE BONDS ISSUED SOLELY FOR THE FOLLOWING
PURPOSES:
· ACQUIRING, CONSTRUCTING AND EQUIPPING A COMMUNITY RECREATION CENTER WHICH SHALL INCLUDE THE FOLLOWING:
· AQUATICS CENTER/SWIMMING POOL
· FITNESS CENTER
· GYMNASIUM FOR BASKETBALL, VOLLEYBALL, AEROBICS AND OTHER ACTIVITIES
· FAMILY, YOUTH AND SENIOR MULTI-PURPOSE ROOMS
· PAYING ALL NECESSARY OR INCIDENTAL COSTS RELATED THERETO, INCLUDING THE FOLLOWING:
· OPERATION AND MAINTENANCE EXPENSES, WHICH MAY INCLUDE EXERCISE AND RECREATION ACTIVITIES FOR ALL AGES,
INTERESTS AND ABILITIES
· THE FUNDING OF A BOND RESERVE FUND AND COSTS OF ISSUING THE BONDS SUCH BONDS TO BE ISSUED, DATED AND SOLD AT
SUCH TIMES, AND AT SUCH PRICES (AT, ABOVE OR BELOW PAR) AND CONTAINING SUCH TERMS, NOT INCONSISTENT HEREWITH,
AS THE CITY COUNCIL MAY DETERMINE; SUCH TAX TO BE IMPLEMENTED BY AN AMENDMENT TO CHAPTER 5.06 OF THE
SPRINGFIELD MUNICIPAL CODE AND TO CONSIST OF A RATE INCREASE IN THE CITY-WIDE SALES TAX OF 0.70% (SEVENTY ONE-
HUNDREDTHS OF ONE PERCENT, WHICH REPRESENTS SEVEN CENTS ON EACH TEN DOLLAR PURCHASE AS SHOWN IN THE TAX
SCHEDULE AT THE END OF THIS BALLOT ISSUE) BEGINNING ON OR AFTER JANUARY 1, 2005, WITH A REDUCTION OF SUCH TAX TO
0.50% (FIFTY ONEHUNDREDTHS OF ONE PERCENT, WHICH REPRESENTS FIVE CENTS ON EACH TEN DOLLAR PURCHASE)
BEGINNING JANUARY 1, 2026 OR SUCH LOWER RATE AS THE CITY COUNCIL MAY DETERMINE AFTER SUCH DATE, TO BE USED
SOLELY TO PAY THE FOLLOWING:
· COSTS OF ACQUIRING, CONSTRUCTING AND EQUIPPING THE COMMUNITY RECREATION CENTER DESCRIBED ABOVE
· PAYING DEBT SERVICE ON THE SALES TAX REVENUE BONDS DESCRIBED ABOVE
· PAYING ALL NECESSARY OR INCIDENTAL COSTS RELATED THERETO, INCLUDING, WITHOUT LIMITATION, OPERATION AND
MAINTENANCE EXPENSES, WHICH MAY INCLUDE EXERCISE AND RECREATION ACTIVITIES FOR ALL AGES, INTERESTS AND
ABILITIES, AND REPAIRS, RENEWALS, REPLACEMENTS AND RENOVATIONS THEREOF, AND THE FUNDING OF RESERVES
THEREFOR; AND
· CONSTRUCTION OR UPGRADES TO CITY OUTDOOR ATHLETIC FACILITIES; AND SHALL ALL TAX REVENUES GENERATED FROM
THE SALES TAX AUTHORIZED HEREIN AND FROM ANY EARNINGS FROM THE INVESTMENT OF SUCH REVENUES AND THE
PROCEEDS OF SUCH BONDS CONSTITUTE A VOTER-APPROVED REVENUE CHANGE, AND AN EXCEPTION TO THE REVENUE AND
SPENDING LIMITS OF ARTICLE X, SECTION 20 OF THE COLORADO CONSTITUTION, OR ANY OTHER LAW?

43. 7
Use accurate instructional
illustrations

44. Optical scan ballots [detail]

45. Rolling DRE (touchscreen) ballot [detail]

46. Voter information materials

47. 8
Use informational
icons (only)

48. Optical scan ballots [detail]

49. Voter information materials [detail]

50. 9
Use contrast and
color functionally

52. Rolling DRE (touchscreen) ballot

53. Optical scan ballots [detail]

54. 10
Decide what’s
most important

56. Grid-style optical scan ballot

57. Optical scan ballot [detail]

58. Voter information material

59. Top 10 election design guidelines
• Use lowercase letters
• Avoid centered type
• Pick one sans-serif font
• Use big enough type
• Support process and navigation
• Use clear, simple language
• Use accurate instructional illustrations
• Use informational icons (only)
• Use contrast and color functionally
• Decide whatʼs most important

60. Resources
•
EAC report
•
Sample files and software
•
Get design help
•
Additional resources

63. Get design help
•
www.designfordemocracy.org
•
AIGA Designer Directory
•
AIGA Election Design Fellows now in
Oregon and Washington
•
designfordemocracy@aiga.org

64. Additional resources
Marcia Lausen: Ballot + Election Design
[University of Chicago Press]

65. Additional resources
Ballot design sample collection

66. Additional resources
AIGA | Design for Democracy
Get Out The Vote campaign

67. 67
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

68. 68
HAVA and NIST
✤ HAVA calls for improved standards for voting systems
✤ NIST provides technical support to develop
standards through EAC and TGDC

69. 69
Design for every voter
✤ Universal access and usability
✤ Design standards based on best practice and research
✤ Performance standards: user-based testing

70. 70
Groundbreaking
✤ First ever standards for usability and accessibility of voting systems
✤ Unique performance-based usability standards

71. 71
Wanted: research on plain language in elections
2005 2006-08 2007 2008
Review of ballot Test of ballot Review of Development
instructions instructions voting system of
and system documentation pass/fail test
messages of
voting system
documentation
for
poll workers

72. 72
Does plain language make a difference
when people vote?

73. 73
Does clear information make a difference in
election worker performance?

74. 74
Does plain language make a difference
when people vote?

75. 75
Empirical study of
language on ballots
Do voters vote more accurately
on a ballot with plain language
instructions?
Do voters recognize the difference in
language?
Do voters prefer one ballot over the
other?

76. 76
Education # of
Who achieved participants
participated? Less than high school 9
45 voters
High school or equivalent 15
Eligible - US citizens age 18+
Focused on lower education levels
Some college or associate 12
Balanced for
gender
ethnicity
Bachelor’s degree 8
age (18-61; median = 36)
Courses beyond college 1

77. 77
Where, what
3 locations
Georgia
Maryland
Michigan
suburban, city, small town
2 ballots
only the wording was different
based on NIST DRE “medium” ballot

78. 78
Ballot A (traditional language)

79. 79
Ballot B (plain language)

80. 80
“Retain” or “Keep”?

81. 81
Does plain language make a difference
when people vote?

82. 82
Yes.

83. 83
Participants voted more accurately
45 participants; 18 possible Ballot A Ballot B Total
correct votes on each ballot
Correct 698 726 1424
Marginally statistically Incorrect 112 84 196
significant, p<.071
Total 810 810 1620

84. 84
Voting on B first helped on A
16.700
Very little difference on B
whether it was first or second
12.525
Correct Votes
8.350
A first, ave. correct = 14.4
A second, ave. correct = 16.3 4.175
Statistically significant,
p<.001 0
A First B First
Correct Votes on A
Correct Votes on B

85. 85
Participants recognized and
overwhelmingly preferred plain language
Preference # of participants % of participants
Ballot A
4 9%
(traditional language)
Ballot B
37 82%
(plain language)
No preference 4 9%
Total 45 100%

86. 86
Education made a difference
Education level Mean # correct both ballots
Less than high school (n=9) 14.4
High school graduate or equivalent (n=15) 15.6
Some college or associate 16.0
Bachelor’s degree (n=8) 17.4
Some courses beyond college (n=1) 17.0
Statistically significant, p<.004

87. 87
1. At the beginning of the ballot, explain
how to vote, how to change a vote, and that
voters may write in a candidate.

88. 88
2. Put instructions where voters need them.

89. 89
3. Include information that will prevent
voters from making errors, such as a
caution to not write in someone already on
the ballot.

90. 90
4. Write short sentences.

91. 91
5. Use short, simple, everyday words.

92. 92
6. Write in active voice, where the person
doing the action comes before the verb.

93. 93
7. Write in the positive. Tell people what to
do rather than what not to do.

94. 94
8. When giving people instructions that are
more than one step, give each step as an
item in a numbered list. Do not number
other instructions.

95. 95
9. Keep paragraphs short. A one-sentence
paragraph is fine.

96. 96
10. Separate paragraphs by a space so
each paragraph stands out on the page.

97. 97
To see all 28 guidelines, go to
http://vote.nist.gov/NISTIR-7556.pdf
The guidelines are the last appendix in the
report, pages 189-190

98. 98
Does clear information make a difference
in election worker performance?

99. 99
Qualitative, exploratory
study of voting system
documentation for poll
workers
How should the documentation be
tested?
What are poll workers’ tasks?
How long do the tasks take?
How many participants will it take to
pass or fail the documentation?
What should the pass/fail criteria for the
documentation be?

100. 100
Testing the test
✤ Drafted protocol and checklists
✤ Recruited 4 pairs of participants
✤ 2 voting systems - one DRE and
one optical scan
✤ Each pair worked on both systems

101. 101
Observations

102. 102
Matching the
documentation to the
machine was difficult
Dana Chisnell

103. 103
Participants had
questions that the
documentation didn’t
answer
Dana Chisnell

104. 104
Information on
troubleshooting was
hard to use because
it was not related to
tasks
Dana Chisnell

105. 105
Documentation
covered too many
systems

106. 106
Met many best
practices but fails
because the
configuration is not
the same as
implementation

107. 107
Met many best
practices but fails
because the
configuration is not
the same as
implementation

108. 108
Does clear information make a difference
in election worker performance?

109. 109
Yes.

110. 110
Evidence
✤ Participants are able to use voting system
documentation to:
✤ Complete tasks without asking questions
✤ Find the information they need
✤ Match messages between system and
documentation
✤ Read, understand, and react
✤ Perform tasks without missing steps
✤ Perform steps to complete tasks

111. 111
Pass / fail criteria
✤ Have participants asked for help?
✤ Have they completed the tasks in the time
allotted?

112. 112
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

113. 113
What is a usability test?

114. 114
What is usable?
 You: A countable ballot
 Voter: Voting as intended
Dana Chisnell

115. 115
Sit next to someone.
Watch. Listen.
Dana Chisnell

116. 116
When
something
changes
Take
constraints into
account
Complement to Reveal
best practice remedies

117. 117
Why test?
Avoid residual votes and fall-off
Even best practice design can
introduce unpredictable problems
Make it more likely that voter intent
is carried out
More likely to have only good news
about an election
Dana Chisnell

118. 118
The Testing Kit
What you need to know
Session script
Report template

119. 119
The Testing Kit
What you need to know
Session script
Report template

120. 120
The Testing Kit
What you need to know
Session script
Report template

121. 121
The Testing Kit
What you need to know
Session script
Report template

122. 122
Redesign. Test. Improve.
Look at best practice Low risk, low cost Smoother voting
within your constraints
Reveal issues that are Smoother counting
Use checklists local, subtle
Better experience

123. 123
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

124. 124
Resources
Usability Professionals’ Association:
www.usabilityprofessionals.org/
LEO Usability Testing Kit:
http://www.usabilityprofessionals.org/
civiclife/voting/leo_testing.html
Ballot Usability and Accessibility Blog:
http://ballotusability.blogspot.com/
National Institute of Standards and
Technology:
http://vote.nist.gov
Dana Chisnell

125. 125
Resources
Brennan Center for Justice at NYU: Better
Ballots
http://www.brennancenter.org/
content/resource/better_ballots/
AIGA - Design for Democracy: Effective
Design for Federal Elections
http://www.aiga.org/content.cfm/
design-for-democracy-eac-reports
Handbook of Usability Testing, Second
Edition by Jeff Rubin and Dana
Chisnell
Dana Chisnell

126. 126
Announcing
LEOExchange
Online discussion group for local election
officials

127. 127
Dana
Dana Chisnell
dana@usabilityworks.net
www.usabilityworks.net
415.519.1148

128. 128
Drew
Drew Davies
drew@oxidedesign.com
www.oxidedesign.com

129. 129
Design problems cause voter errors
Design best practice helps
Plain language helps
Best practice + usability testing
Resources
The Texas story

130. Voter Registration Form (Before)

131. Voter Registration Form (After)
Version 1

132. Voter Registration Form (After)
Version 2
Back of this form looks exactly like Version 1 – horizontal layout versus vertical layout

133. Application Ballot by Mail (Before)
Front

134. Application Ballot by Mail (Before)
Back

135. Application Ballot by Mail (After)
Front
Layout went
from bifold to
trifold

136. Application Ballot by Mail (After)
Back
Layout went
from bifold to
trifold

137. 137
Questions?

138. 138
Thank you.