1. 9 Tips for Writing
Great Numeric
Questions
Annie Pettit
Chief Research Officer
Peanut Labs
2. Always Include Zero
• Even if people who should
choose zero have been skipped
or screened out
• People misread, misremember,
and change their minds
How many movies did you
watch last week?
o1
o2
o3
o4
o5
o6
o7
o8 or more
2
3. Always Include a Maximum
• Even if people who chose a
high number should have been
skipped or screened out
• People misread, misremember,
and change their minds
• Don’t forget 8 or more!
How many movies did you
watch last week?
o0
o1
o2
o3
o4
o5
o6
o7
3
4. Don’t Duplicate Numbers
• Where do I put my answer for
10?
• Be conscious of overlapping
answers
• Get into the habit of testing
out each possible answer
against the number ranges
How many movies did you
watch last month?
o0
o1 to 5
o5 to 10
o10 to 15
o15 or more
5. Don’t Skip Numbers
• Where is 15?
• It’s easy to miss when you see
both 14 and 15 in the answer
options
• Get into the habit of testing
out each possible answer
against the number ranges
How many movies did you
watch last month?
o0
o1 to 4
o5 to 9
o10 to 14
oMore than 15
6. Use Intuitive Breaks
• Use breaks that relate to how
people talk about the topic
How many times did you go
grocery shopping last month?
o0
o1 to 5
o6 to 10
o11 to 15
o16 or morePrecise Flexible
1 to 4 Once per week
5 to 8 Twice per week
9 to 12 Three times per week
13 or more Four times per week
7. Use Common Breaks
• People are very comfortable with
multiples of 2 and 5. Multiple of 3
and 4 are just strange.
• 0
• 1 to 4
• 5 to 9
• 10 to 14
• 15 or more
• 0
• 1
• 2 or 3
• 4 or 5
• 6 or 7
• 8 or more
How many books about
statistics do you have at
home?
o0
o1 to 3
o4 to 7
o8 to 11
o12 or more
8. Start With Numbers
• It’s easier to read if every
answer starts with a number
• It’s also easier to follow all the
rules if every answer starts
with a number
• 0
• 1 to 4
• 5 to 9
• 10 or more
How many times did you go
grocery shopping last month?
Less than 1
1 to 4
5 to 9
More than 9
9. OR does not equal TO
• Does the object being
measured happen in decimal
places?
How many children are in your
household?
o0
o1 to 2
o3 to 4
o5 to 6
o7 or more
1 or 2: 1 to 2:
Car Watched for half an hour
Movie Bought half a gallon
Child Drank half a cup of coffee
Website Used half a notebook
10. Is that a minus sign?
• What is clear to you may not
be clear to other people
• Reserve – for when it MUST be
used
o 0
o 1 to 4
o 5 to 8
o 9 to 12
o 13 or more
How many movies did you
watch last month?
o0
o1-4
o5-8
o9-12
o13 or more
11. Perfection Doesn’t
Exist
In an average week,
how many cups of
coffee do you buy at
coffee shops?
0
1 or 2
3 or 4
5 to 7
8 to 14
15 to 21
22 or more
In an average week,
how many cups of
coffee do you buy at
coffee shops?
0
1 to 4
5 to 9
10 to 14
15 to 19
20 or more
In an average week,
how many cups of
coffee do you buy at
coffee shops?
0
1 to 7
8 to 14
15 to 21
22 or more
12. How To Contact Us
Annie Pettit
Chief Research Officer
Peanut Labs
annie@peanutlabs.com
twitter.com/LoveStats
ca.linkedin.com/in/anniepettit/
Questions about our Sample Services?
Jonathan Cheriff
Director of Sales & Marketing
jonathan.cheriff@peanutlabs.com
twitter.com/paperbackdad
www.linkedin.com/in/jonathancheriff
Thank you for your participation!
We hope you learned a few tidbits of information!
Notas do Editor
Welcome to the PeanutLabs quick tips on survey design. Today we’re going to talk about how to create a great numeric question. It might seem simple, but I’ll show you a bunch of examples of bad question that I see all the time. Let’s make sure they don’t happen again!
Number 1. Always Include Zero. Make sure your question always includes a zero. Sometimes, you’ll think it’s not necessary. Maybe you screened out everyone who said they didn’t buy coffee last week, they don’t use pocket protectors, or they didn’t eat Chinese food last month. But, among the people who remain, someone might suddenly remember new information. Wait, actually, we usually go to the mandarin Chinese buffet every month but we didn’t last month because we were away. If we include the zero, we can prevent a second error from being made.
Number 2. Always include a maximum. Just as we saw with the last question, even if you think you weeded out all the high volume, frequent users, and heavy users, you need to account for the largest possible number. Maybe it doesn’t seem possible that someone could watch 15 movies in a week. Or that someone could spend more than $200 a week at Starbucks. But it could make perfect sense for other people. Make sure you always include an option that covers every possible large number. In this case, it would be simple to add 8 or more.
Number 3. Don’t duplicate numbers. This seems obvious but it happens all the time! It’s a smart move to test every number one at a time. Where would I put my answer if I said 0. if I said 1. if I said 2. if I said 3. And so on. In this case, I would have quickly found a problem with 5 and 10 and 15. They each have two possible answers!
Number four. Don’t Skip numbers. Again, this seems obvious but it happens all the time. It’s an easy oversight when you look at the question and see 14 and 15 in the options. It seems like you’ve got everything covered. But, in fact, there is nowhere to put your answer if your answer is 15. As before, if you go through every possible answer one at a time, you will catch this problem ahead of time.
Number five. Use intuitive breaks. This one is tricky. You have to think about the normal ways that people talk about things. Do they think about it in weekly terms? Like once per week, twice per month, three times per year? Then make sure you use breaks that reflect that. In this case, people might shop once per week which translates into 4 times per month. Where does that second answer of 5 times per month come from? It sounds strange.
Number six. Use common breaks. As always, we are striving for answer options that require as little brain work as possible. In this case, the answer options require us to think about numbers in ways we don’t normally we do. We’re all really good at counting by 2s and by 5s. If you ask people how much they spend on things, they’ll probably give you an answer like 2 bucks or 5 bucks or 20 bucks. They’ll rarely say 3 bucks or 12 bucks or 26 bucks. In this case, it’s just as easy, and much more intuitive, for us to increment each answer option by 5.
Number seven. Start with numbers. Sometimes, it makes sense for the first or last answer to be phrased as more than 10 or higher than 10 or greater than 10. But, you can easily switch that around to 10 or more, 10 or higher, 10 or greater. This means that every answer option now starts with a number. Your brain has one less task it needs to do to figure out an answer. It doesn’t have to translate a word into a number.
Number 8. OR does not equal TO. Many people don’t really think about this issue but think about this. Do you have half a child? Half a car? Half a website? No? Then you shouldn’t use the word TO. You have 1 or 2 children. 1 or 2 cars. 1 or 2 websites. On the other hand, you CAN watch TV for half an hour, buy half a gallon of milk, or use half of a note book.
Number 9. is that a minus sign? I’ll admit that this tip is a personal preference of mine. Using a dash instead of the word to might seem like no big deal but sometimes, it makes the numbers look strange and in rare cases, you will confuse people into thinking that math is required.. It requires no extra work on our part to be perfectly clear and use the word to.
Finally, remember that perfection does not exist. There is no perfect question. There are always multiple good ways to ask the same question. As you see here, each of these questions work. In the first case, it follows the rule of considering how people naturally view the topic. For instance, one coffee every day fits into 1 to 7. 2 coffees per day fits into 8 to 14. In the second case, each answer starts a common break that increments by five. In the third case, it accounts for the fact that many people will only buy 1 or 2 per week, and very few people will buy many. Your choice of answer options must reflect your research objective.
I hope you enjoyed today’s peanut labs tutorial. Feel free to visit our website at peanutlabs.com to learn more about survey design and sampling.