1. Open Textbook Student
Usability Research Results
#OpenEd16Justin Whiting
Indiana University
@jnwhiting juswhiti@indiana.edu
Barbara Illowsky, PhD
Foothill-De Anza Community College District
@DrBSI illowskybarbara@fhda.edu
John Hilton III, PhD
Brigham Young University
@johnhiltoniii johnhiltoniii@byu.edu
2. This study examined the perceptions that
students have of different versions of an
open statistics textbook used over
several years in one community college.
Survey results show that students
generally had at least as good of an
experience using the open textbook
compared to traditional textbooks.
3. Background
Study 1 – Collaborative Statistics
– Introductory statistics open textbook produced by
Connexions
– Black & White Only
– Could be viewed online (free) or printed ($30)
Study 2 – Introductory Statistics
– Introductory statistics open textbook produced by
OpenStax College
– Updated color and formatting
– Upgraded graphs and images
– More community developed with faculty narratives,
feedback, and examples
– Could be viewed online, via an app, in iBooks, or printed
4. Methods
• Two versions (Collaborative Statistics &
Introductory Statistics) of an open statistics
textbook
• Questionnaire data* collected during 7 terms
(2013-2015)
• Use of descriptive statistics to analyze the
quantitative data
• Emergent coding to identify themes from the
qualitative questions
*based in part on a survey utilized by Bliss, Hilton III, Wiley, & Thanos (2013)
5. Collaborative Statistics
Questionnaire sent to all De Anza College
students using the textbook during
Spring and Fall terms in 2013.
– N = 231 responses from 17 classes
– 54% female, 46% male
– 9% receive loans for education
– 37% receive Pell Grants or fee waivers
6. Collaborative Statistics
Student Perception of Cost
•48% of the students purchased the text (under
$30)
•61% reported printing some of the materials
•84 % of students that didn’t purchase textbook,
reasoned, “the text was available free of charge
online.”
It appears most students experienced significant
savings compared to traditional textbook
7. Collaborative Statistics
Student Perception
of Quality
•Students reported they used
this text about as much as they
would any other textbook (65%
said twice a week or more)
•When asked to rate quality of
this text compared to other
textbooks:
– 25% better
– 62% same
– 13% worse
10. Coding scheme examples
Positive general:
– “I think it was a good choice for this course. It had
a good variety of homework problems and labs
that appeal to most of the students.”
Positive clarity:
– “I appreciated it because the objectives were
clear and the examples followed the objectives.”
11. Coding scheme examples
Negative relevance:
– “It needs improvement as our instructor
substituted material for chapters that were not
very comprehensive.”
– “It was a good text but not good enough to rely
on for the sole purpose of learning, I prefer
teacher notes since it simplifies everything for
me.”
13. What made it better?
Online
– “Free…I have attended other institutions and this
course is the first one that offered a free textbook. I am
very appreciative both of the access to knowledge in a
form that is digital, online and hard copy. Also, cost is
prohibitive for textbooks sometimes, and not having to
pay for textbooks is a direction I'd like to see education
move towards.”
Price
– “It is free and it is just like regular textbooks.”
14. Introductory Statistics
(new version)
Questionnaire sent to all students using the
textbook during Spring semester in 2015
– N = 94 responses from 9 classes (however, the
majority came from a single class)
– 54% male, 46% female
– 13% received loans for education
– 41% received Pell grants or fee waivers
15. Introductory Statistics
Student Perception of Cost
•70% of students did not purchase textbook
•53% of students did print materials
– spent less than $20.00 on printing
•70 % of students that didn’t purchase textbook, reasoned, “the
text was available free of charge online.”
•Access
– 47% used the book online
– 23% downloaded a PDF
– 31% used a hard copy
It appears most students experienced significant savings compared
to traditional textbook
19. Discussion
• Overall, student responses to using both Collaborative
Statistics and the updated Introductory Statistics were
positive.
• For both open textbooks, students overwhelmingly
reported the text was the same as, or better than their
traditional texts.
• In response to open-ended questions, both groups of
students provided similar comments.
20. Limitations
• While the authors attempted to distinguish
between use of a hard copy textbook and use
of an open, online textbook in their research,
some students chose to purchase the hard
copy of the text, and thus may have given
responses that were based on the hard copy
version of the open textbook.
• Limited sample of one college population
21. Future Research
• Future research might focus on cost trade-
offs of open versus expensive textbooks, as
well as low cost (under $40) versus
traditionally priced, expensive textbooks.
• An expanded survey population of multiple
institutions and instructors in order to more
adequately generalize the findings of the
study.
22. Conclusion
• Students perceive the quality of the open
textbook to be as good or better than more
expensive commercial textbooks
• When it comes to textbook costs, students
might not, “get what they pay for.”
• Professors and educational administrators
should carefully consider adopting open
textbooks to reduce the high cost of
traditional texts.
23. Full article:
Open Praxis, Volume 8, No 3 (2016)
http://www.openpraxis.org/index.php/OpenPra
xis/article/view/304
Questions?
Please connect with us!
@jnwhiting juswhiti@indiana.edu
@DrBSI illowskybarbara@fhda.edu
@johnhiltoniii johnhiltoniii@byu.edu
Notas do Editor
Not sure if we need any information posted about DeAnza College. De Anza College is a large suburban community college in California, located in the region known as Silicon Valley. It operates on the quarter system. The college serves approximately 23,000 students (over 40% full-time). Locally, it has the highest graduation rate (associate degree) with over 60% of its full-time students earning the degree within three years. Asians (38%) and Latinos (24%) are the two largest ethnic groups at the college.
As mentioned previously, the text for this course was offered free of charge online; however, there was also a hard copy available for purchase for under $30. Forty-eight percent of students reported purchasing a text for this class. The rest of the students either used the online version or printed off a PDF copy of the text. Of the 56 students who reported purchasing texts, all but eleven reported spending $100 or less, and 57 percent reported spending under $40. Considering the cost of the hard copy, the “$100 or less” could refer to one instructor whose students purchased the optional hardcopy note pack from the campus bookstore instead of downloading and/or printing them.
Many students (61 percent) reported printing materials for the course. Of those who did, 82 percent reported spending $30 or less, with 46 percent spending less than $10. Many students reported that they did not purchase any texts for the course. When we asked them why not, the vast majority of them (84 percent) answered “the text was available free of charge online.” In an answer to our second research question of student perceptions about cost of OER, our findings indicate that whether students purchased a hard copy of the text or printed off pages, it appears that most students experienced significant savings relative to the average costs of college textbooks.
We can use either chart, but I actually think that using the pie chart is more compelling (I don’t normally say that), as we can show easily that 87% said same or better.
This figure shows students response in what ways was Collaborative Statistics better, the same or worse than traditional textbooks. It fared better in clarity, examples, cost and online material. It was the same in organization and relevance. CS was worse in specifics and color. There were also some general comments that were neutral.
A coding scheme was developed and used to code answers as, “positive, negative, and neutral.” The answers were then categorized as General, Organization, Clarity, Relevance, Specifics, Examples, Price, Online, & Color/Graphics. Some examples of negative relevance: “It needs improvement as our instructor substituted material for chapters that were not very comprehensive.” and “It was a good text but not good enough to rely on for the sole purpose of learning, I prefer teacher notes since it simplifies everything for me.”
A coding scheme was developed and used to code answers as, “positive, negative, and neutral.” The answers were then categorized as General, Organization, Clarity, Relevance, Specifics, Examples, Price, Online, & Color/Graphics. Some examples of positive
A coding scheme was developed and used to code answers as, “positive, negative, and neutral.” The answers were then categorized as General, Organization, Clarity, Relevance, Specifics, Examples, Price, Online, & Color/Graphics. Some examples of negative relevance: “It needs improvement as our instructor substituted material for chapters that were not very comprehensive.” and “It was a good text but not good enough to rely on for the sole purpose of learning, I prefer teacher notes since it simplifies everything for me.”
Two final open response questions were utilized in order to help students elaborate on their feelings regarding the textbook. First, in order to prompt students to record any negative impressions of the textbook, students were asked about their biggest complaint regarding the textbook.
In responding to the question “What do you like best about this text?” students’ answers fell mainly into three categories: examples, online benefits, and price. Statements regarding examples were similar to those discussed previously. Student comments with regard to online benefits are illustrated by the following comments made by the students:
As mentioned previously, the text for this course was offered free of charge online; however, there was also a hard copy available for purchase for under $30. Forty-eight percent of students reported purchasing a text for this class. The rest of the students either used the online version or printed off a PDF copy of the text. Of the 56 students who reported purchasing texts, all but eleven reported spending $100 or less, and 57 percent reported spending under $40. Considering the cost of the hard copy, the “$100 or less” could refer to one instructor whose students purchased the optional hardcopy note pack from the campus bookstore instead of downloading and/or printing them.
Many students (61 percent) reported printing materials for the course. Of those who did, 82 percent reported spending $30 or less, with 46 percent spending less than $10. Many students reported that they did not purchase any texts for the course. When we asked them why not, the vast majority of them (84 percent) answered “the text was available free of charge online.” In an answer to our second research question of student perceptions about cost of OER, our findings indicate that whether students purchased a hard copy of the text or printed off pages, it appears that most students experienced significant savings relative to the average costs of college textbooks.
Overall, 93% of students said that the book was as good or better than traditional textbooks. Only five students gave specific responses to the question, “Why did you rate the OER textbooks as being worse than traditional texts” and there was no observable pattern in their responses. Some felt there were errors; others felt it was too difficult to understand.
Participants were asked, “Overall, what do you think of the text used in this course?” All neutral answers given to this question were coded as general, and, as stated previously, were generic answers such as “It’s okay.” Figure 6 illustrates the answers to the question “Overall, what did you think of the text used in this course?”
This may not actually be that interesting. We can delete it if we need more time.
This may not actually be that interesting. We can delete it if we need more time.
Overall, student responses to using both Collaborative Statistics and the updated Introductory Statistics were positive. For both open textbooks, students overwhelmingly reported the text was the same as, or better than their traditional texts. In response to open-ended questions, both groups of students provided similar comments.