A Student Measure of the Quality of Open Digital Textbooks
1. A Student Measure of the Quality of
Open Digital Textbooks
Or
A Method For Getting Data-Backed
Student Opinions
TJ Bliss
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
All credited images fall under Fair Use, are intended for transformative purposes
only, and retain full copyright privileges.
2. John Dewey
Education is necessary to the
individual “to prepare him for the
future life means to give him
command of himself; it means so to
train him that he will have the full
and ready use of all his capacities.”
Image Credit: http://c250.columbia.edu/
3. “Serving society is only one of
higher education’s functions, but
it is surely among the most
important. At a time when the
nation has its full share of
difficulties...the question is not
Derek Bok
whether universities need to
concern themselves with society’s
problems but whether they are
discharging this responsibility as
well as they should.”
Image Credit: http://helenedison.ucsd.edu/
9. Dilbert By Scott Adams
Image credit: www.dilbert.com
10. It’s Alive!
The Birth of a Textbook Quality Measure
Hard data to back up student opinions about
the quality of their (open) digital textbooks
Image Credit: www.yankeeroundtable.com
11. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final” Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
12. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final”Questionnaire
Final Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
14. Project Kaleidoscope
Cross-institutional faculty teams
Aggregate OER-based textbook replacements (FWK)
11 courses, 9,000 students in 2011-2012
15. 490 Student Survey Responses
From your perspective, what are the
characteristics of a high quality digital textbook?
Overall, what did you think of the digital textbook
you used in your course?
How would you rate the quality of the digital
textbook you used in your course? Why?
18. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final”Questionnaire
Final Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
19. Thematic Analysis
7 main steps
Data prep, reading, sorting, writing, axial
coding, more sorting, finalizing
21. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final”Questionnaire
Final Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
24. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final”Questionnaire
Final Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
26. Pilot Questionnaire
6 Background Items
44 Main Items
8 Target Themes
October 2012 Administration
27.
28. student surveys
student interviews Qualitative Data
literature review
thematic analysis
item revision
Validated Model Target Themes
“Final”Questionnaire
Final Questionnaire
factor analysis mental modeling
Pilot Questionnaire Conceptual Model
item development
29. Next Steps
Model evaluation and validation (Nov. ‘12)
Item revision (Jan. ‘13)
“Final” Questionnaire 2.0 (Feb. ‘13)
Further evaluation, validation, revision (Apr. ‘13)
“Final” Questionnaire 3.0 (May ‘13)
Notas do Editor
Former President of Harvard University
Former President of Harvard University$17.5K up from $7.5KTextbooks – avg. student spends over $900/year
One of, if not the most important of motivations behind the open education movement is the desire to do a better job of discharging the great responsibilities of education by lowering costs and increasing access.
The power of the open textbook:Reduces costCustomizable
We want to give faculty hard data to back up their opinions about how to adapt their open digital textbooks from
Version 2.0 will be openly licensed and made publicly available in early 2013.Version 3.0 will be openly licensed and made publicly available in mid 2013