Building on the overview of the course redesign provided in our webinar in April entitled "Improving Outcomes and Reducing Costs: The Case for Redesign," Dr. Carolyn Jarmon of the National Center for Academic Transformation described the methods institutions have used to measure the impact of course redesign on student learning and instructional costs. Providing specific examples from institutions that have used these methods to prove their success, she also shared the tools that these institutions have used to demonstrate increased learning and reduced costs, so that other institutions seeking to validate their success with course redesign are equipped to do so.

1. Assessing Learning and Cost in Course
Redesign
with Dr. Carolyn Jarmon, Vice President for
The National Center for Academic Transformation

2. REDESIGNING STUDENT LEARNING
ENVIRONMENTS:
Measuring Cost and Quality

3. POLL #1
What stage is course redesign at your
institution?
See the poll on the right of your screen…

4. WHAT IS THE MOST IMPORTANT THING THAT
WE HAVE LEARNED ABOUT QUALITY AND
COST?
The factors that lead to
increased student
learning and increased
student retention are the
same as those that lead
to reduced instructional
costs!

5. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#1: Redesign the whole course
– Quality: Eliminate “course drift”; greater
course coherence and quality control
– Cost: Eliminate duplicate effort; create
opportunities for alternate staffing

6. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#1: Redesign the whole course
– Quality: Eliminate “course drift”; greater
course coherence and quality control
– Cost: Eliminate duplicate effort; create
opportunities for alternate staffing

7. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#2: Encourage active learning
– Quality: “Learning is not a spectator
sport.”
– Cost: Reduce faculty preparation and
presentation time; reduce grading time
(e.g., interactive software, peer learning
teams)

8. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#3: Provide students with individualized
assistance
– Quality: Students get help when they are
“stuck” and stay on task rather than giving
up: software tutorials, F2F in labs or help
rooms, “beep a tutor,” SMARTHINKING
– Cost: Apply the right level of human
intervention: peer tutors, course assistants

9. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#4: Build in ongoing assessment and prompt
(automated) feedback
– Quality: Enables practice, diagnostic
feedback, focused time on task
– Cost: Good pedagogy with large numbers of
students; individual and group assessment;
faculty spend time on what students don’t
understand

10. FIVE PRINCIPLES OF SUCCESSFUL COURSE
REDESIGN
#5: Ensure sufficient time on task and monitor
student progress
– Quality: Self-pacing vs. milestones for
completion; points for engagement
– Cost: Course management systems can
reduce costs while increasing oversight

11. READINESS CRITERIA
• What does it mean to be “ready” to do a major course redesign?
• Is your institution ready?
• Which courses are “ready”—i.e., are good candidates for a comprehensive
redesign?
• What decisions need
to be made?

12. READINESS CRITERION #1
Course Choice
• What impact would
redesigning the course
have on the
curriculum, on
students and on the
institution—i.e., why
do you want to
redesign this course?

13. POLL #2
Using the chat function, what course (or
courses) is your institution considering for
an upcoming redesign?
Write the course title in the chat box.

14. FACTORS TO CONSIDER WHEN THINKING
ABOUT HIGH IMPACT
• High drop-failure-withdrawal rates
• Student performance in subsequent courses
• Students on waiting lists
• Student complaints
• Other departmental complaints
• Lack of consistency in multiple sections
• Difficulty finding qualified adjuncts

15. READINESS CRITERION #2
Redesign Model
• Which redesign model do you think
would be most appropriate for your
redesign? Why?
• What aspects fit your particular
discipline and your particular
students?

16. REDESIGN MODELS
• Supplemental – Add to the current structure and/or change
the content
• Replacement – Blend face-to-face with online
activities
• Emporium – Move all classes to a lab setting
Fully online – Conduct all (most)
learning activities online
• Buffet – Mix and match according
to student preferences
• Linked Workshop – JIT workshops
linked to college level course

17. READINESS CRITERION #3
Assessment Plan
• Which assessment
model do you think
would be most
appropriate for your
redesign? Why?

18. ASSESSMENT GOAL
To establish the
degree to which
improved learning
has been achieved
as a result of the
course redesign.

19. ASSESSMENT PLANNING
Step 1. Establish the method of
obtaining data.
Step 2. Choose the measurement
method.

20. ESTABLISH THE METHOD OF
OBTAINING DATA
• Baseline “Before”
(traditional) and “After”
(redesign)
• Parallel Sections –
Compare traditional
sections and redesigned
sections

21. CHOOSE THE MEASUREMENT
METHOD: FOUR MODELS
A. Comparisons of Final Exams
B. Comparisons of Common Content
Items Selected from Exams
C. Comparisons of Pre- and Post- Tests
D. Comparisons of Student Work using
Common Rubrics

22. COMMON EXAMS
University of Massachusetts
In spite of more
difficult questions,
student scores on
exams in a redesigned
biology course
averaged 73% vs. 61%
in the traditional
course.

23. COMMON ITEMS
Carnegie Mellon University
• Redesign students showed
significant improvement based on
comparing exam questions.
• In solving problems, redesign
students made fewer than one
error per problem, whereas
traditional students made about
six errors per problem.

24. PRE/POST TESTS
University of Central Florida
Students in the
traditional format
posted a 1.6-point
improvement on a
content exam, whereas
at 2.9, the mean change
for students in the
redesigned course was
almost double that
amount.

25. COMMON RUBLICS
Tallahassee Community College
Students in a redesigned
English composition course
scored significantly higher
on final essays, with an
average score of 8.34
compared to 7.33 for
traditional students.

26. More Is Not Better!
• Performance in follow-on courses
• Student attitude toward subject matter
• Student interest in pursuing further
coursework in the discipline
• Differences in performance among student
subpopulations
• Deep vs. superficial learning

28. GRADES ARE NOT A SUFFICIENT MEASURE
OF STUDENT LEARNING
• Lack of consistency
• Different coverage
• Different tests and
exams
• Curving
• Inflation
Use only for course completion!

30. READINESS CRITERION #4
Cost Savings Plan
• Which cost savings
strategy do you think
would be most
appropriate for your
redesign? Why?

31. COST SAVINGS GOAL
Create cost savings that can
be used to sustain
ongoing redesign, to fund
future operations and to
free up resources for
program and/or
institutional priorities.

32. STEP 1: WHAT’S YOUR
ENROLLMENT SITUATION?
• Is your enrollment
growing or projected
to grow?
• Is your enrollment
stable or declining?

33. STEP 2: CHOOSE LABOR-SAVING
TACTICS
• Coordinated development and shared instructional
tasks for individual development and delivery.
• Software for face-to-face.
• Automated grading of homework, quizzes, exams
for hand grading.
• Course management software for human
monitoring and course administration.
• Interaction with other personnel for one-to-one
faculty/student interaction.

34. STEP 3: CHOOSE A COST
REDUCTION STRATEGY
• Each instructor carries more students
by
– Increasing section size.
– Increasing the number of sections carried.
• Change the mix of personnel teaching
the course.
• Do both simultaneously.

35. COST REDUCTION
EXAMPLE
Traditional
• Instructor load = 4
sections
• 150 minutes/week
3 50-minute or 2 75-
minute meetings
• Time spent = 600
minutes per week
Redesign
• Instructor load = 6
sections
• 150 minutes/week
2 50-minute meetings
• Time spent = 600 minutes
per week
• Option: Reduce # of sections
from 12 to 8 per semester,
saving an additional $32,000

36. COST REDUCTION
EXAMPLE
Traditional
• Each instructor
teaches 1 section
• Section size = 25
• Time spent = 200
hours
Redesign
• Time spent = 100
hours
• Options:
– Each instructor = 2
sections of 25
– Each instructor = 1
section of 50

37. GENERAL PSYCHOLOGY
Frostburg State University
Traditional
• 2 meetings/week
• Sections of 50
• 9 FT and 9 PT
faculty
• $89 cost-per-
student
Redesign
• 1 meeting/week
• Sections of 150
• 1 FT and 3 PT faculty;
ULAs in lab
• Required lab time
• $26 cost-per-student

38. ENGINEERING PHYSICS
Auburn University
Traditional
• 750 students annually
• Lecture, lab, recitation
structure
• 14 GTAs
• High DFW rates
• Passively unengaged
students
• $390 cost-per-student
• Redesign
• Online activities completed
before class
• Physics activity session
(combined lab and recitation)
• 9 GTAs
• Online: pre-tests, post-tests,
problems, conceptual
inventories
• $219 cost-per-student

39. STATICS
Mississippi State University
Traditional
• 3 hours of lecture
• Students do homework
and problem-solving
outside of class (the
toughest part!)
• 5 FT and 5 PT faculty
teach 11 sections of ~35
students
• $299 cost-per-student
Redesign
• 3-hour laptop lab &
hands-on experiments
• Peer-to-peer and GTA
assistance
• Lectures streamed for
students to watch and
review on their own
• One instructor teaches
all sections supported
by GTAs and ULAs
• $252 cost-per-student

40. FAIRFIELD UNIVERSITY
General Biology
Traditional
• 7 sections (~35)
• 7 faculty
• 100% wet labs
• $131,610
• $506 cost-per-student
Redesign
• 2 sections (~140)
• 4 faculty
• 50% wet, 50% virtual
• $98,033
• $350 cost-per-student

41. COURSE PLANNING TOOL
A decision-making tool
that enables
institutions to
compare the
“before” activities
and costs (the
traditional course)
and the “after”
activities and costs
(the redesigned
course)

42. Instructional Costs per Hour

43. TRADITIONAL COURSE
Traditional Course Tenure-Track Non-Tenure- Adjunct/
Faculty Track Faculty TAs/GAs PT Faculty
Hourly rate = Hourly rate = Hourly rate = Hourly rate =
# of Hours Total Cost# of Hours Total Cost# of HoursTotal Cost# of Hours Total Cost
I. Course Preparation
A. Curriculum Development $0 $0 $0 $0
B. Materials Acquisition $0 $0 $0 $0
C. Materials Development
1. Lectures/presentations $0 $0 $0 $0
2. Learning materials/software $0 $0 $0 $0
3. Diagnostic assessments $0 $0 $0 $0
4. Assignments $0 $0 $0 $0
5. Tests/evaluations $0 $0 $0 $0
Sub-Total 0 $0 0 $0 0 $0 0 $0
D. Faculty/TA Development/Training
1. Orientation $0 $0 $0 $0
2. Staff meetings $0 $0 $0 $0
3. Attend lectures $0 $0 $0 $0
Sub-Total 0 $0 0 $0 0 $0 0 $0
Total Preparation 0 0 0 0 0 0 0 0
II. Course Delivery
A. Instruction
1. Diagnose skill/knowledge $0 $0 $0 $0
2. Presentation $0 $0 $0 $0
3. Interaction $0 $0 $0 $0
4. Progress monitoring $0 $0 $0 $0
Sub-Total 0 $0 0 $0 0 $0 0 $0
B. Evaluation
1. Test proctoring $0 $0 $0 $0
2. Tests/evaluation $0 $0 $0 $0
Sub-Total 0 $0 0 $0 0 $0 0 $0
Total Delivery 0 $0 0 $0 0 $0 0 $0
TOTAL 0 $0 0 $0 0 $0 0 $0

44. CPT TASKS
• Calculations you have completed
– All personnel costs expressed as an hourly rate (page
1)
– The specific tasks associated with offering the
traditional course and how much time each person
spends on each of the tasks (page 2)
• Calculations you need to complete
– Tasks and hours in the redesign (page 3)
– Total instructional costs for the traditional course
(page 4)
– Total instructional costs for the redesign (page 4)

45. REDESIGNED COURSE
Redesigned Course Tenure-Track Non-Tenure- TAs/GAs Adjunct/
Faculty Track Faculty PT Faculty
# of Hours # of Hours # of Hours # of Hours
I. Course Preparation
A. Curriculum Development
B. Materials Acquisition
C. Materials Development
1. Lectures/presentations
2. Learning materials/software
3. Diagnostic assessments
4. Assignments
5. Tests/evaluations
Sub-Total 0 0 0 0
D. Faculty/TA Development/Training
1. Orientation
2. Staff meetings
3. Attend lectures
Sub-Total 0 0 0 0
Total Preparation 0 0 0 0
II. Course Delivery
A. Instruction
1. Diagnose skill/knowledge
2. Presentation
3. Interaction
4. Progress monitoring
Sub-Total 0 0 0 0
B. Evaluation
1. Test proctoring
2. Tests/evaluation
Sub-Total 0 0 0 0
Total Delivery 0 0 0 0
TOTAL 0

46. Annual Cost Comparison
Institution: NOTE If you insert or delete columns, you must adjust the formulas that calculate totals.
Course:
Tenure-Track Non-Tenure- Adjunct/
Faculty Track Faculty TAs/GAs PT Faculty
Sections Sections Sections Sections
ANNUAL COST OF THE TRADITIONAL COURSE
# of sections taught in fall and spring
Cost of one section
Cost of instruction by type #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! Total cost of direct instruction
Total cost of course coordination
GRAND TOTAL #VALUE! Total cost of other personnel
Total # of students
Cost per student #VALUE!
ANNUAL COST OF THE REDESIGNED COURSE
# of sections taught in fall and spring
Cost of one section
Cost of instruction by type #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! Total cost of direct instruction
Total cost of course coordination
GRAND TOTAL #VALUE! Total cost of other personnel
Total # of students
Cost per student #VALUE!

47. THE CPT – WHY DO IT?
• Provides a structure for the team
to think about activities and
costs
• Allows consideration of changes
in specific instructional tasks
• Permits visualization of
duplication and waste
• Enables cost/benefit analysis re:
type of personnel per task
TEAM!!!

48. READINESS CRITERION #5
Learning Materials
• Are the faculty able and
willing to incorporate
existing curricular
materials in order to
focus work on redesign
issues rather than
materials creation?

49. READINESS CRITERION #6
Active Learning
• Do the faculty members
have an understanding
of and some experience
with integrating
elements of computer-
based instruction into
existing courses?

50. READINESS CRITERION #7
Collective Commitment
• Describe the members of
your team, the skills they
bring to the project and
what their roles will be in
both the planning and
implementation phases
of the project.

51. WHY INSTITUTIONAL TEAMS?
• Faculty experts
• Administrators
• Technology
professionals
• Assessment experts

52. READINESS CRITERIA
1. Course Choice
2. Redesign Model
3. Assessment Plan
4. Cost Savings Plan
5. Learning Materials
6. Active Learning
7. Collective Commitment

53. NCAT’S REDESIGN METHODOLOGY
“A Menu of Redesign Options”
• Six Models for Course Redesign
• Five Principles of Successful Course
Redesign
• Four Models for Assessing Student
Learning
• Cost Reduction Strategies
• Readiness Criteria
• Course Planning Tool
• Five Critical Implementation Issues
• Planning Checklist

54. REDESIGNING STUDENT LEARNING
ENVIRONMENTS :
Measuring Cost and Quality
Carolyn Jarmon, Ph.D.
cjarmon@theNCAT.org
www.theNCAT.org

55. Consultation and Training
• Provide an orientation to course redesign
• Help you develop a set of learning outcomes
• Help you determine which method of course redesign will help you achieve your
desired outcomes
Faculty Development
• Help faculty develop a deeper understanding of the selected redesign model
• Train faculty on strategies for implementing the redesigned curriculum
• Work with faculty to determine the key assignments
• Help construct an assessment plan around both your desired outcomes and key
assignments
Ongoing Consulting
• Ongoing telephone and web-based consulting for the duration of the redesign

56. Additional Services
 Data Collection and Analysis
 Review and evaluation of existing course including
syllabus, course description and learning outcomes.
 Map Cengage Learning print and/or digital content to
your specific standards or learning outcomes.
 Development of supporting course documents
 Development of media assets and assessment tools