Session 7 in the Moving Learning Online series

1. Moving Learning On Line
- Evaluating & Reviewing Online courses
Phil Hardcastle
Email: Phil.Hardcastle@@rsc-em.ac.uk
Adam Blackwood
Email: A.Blackwood@kent.ac.uk
May 25, 2011 | slide 1

2. Plan for the session
 Ice Breaking & Introduction
 Observable Benefits of MLO
 Measuring Impact and Quality – Tools &
Approaches
 Review Session
 Further Resources
May 25, 2011 | slide 2

3. Who is here…
May 25, 2011 | slide 3

4. Who’s here
A Adult/Community Learning
B Work Based Learning
C FE College
D Specialist College
E 6th Form College
B. Williams RSCEM May 25, 2011 | slide 4

5. Online learning – a spectrum of use
A VLE is used to Used to deliver learning A fully online course
make a variety of activities and to requires the tutor to
documents encourage co-operation work online in the
available as a between students on same way as he/she
reference point for projects. Assignments would have done face
the learner when may be submitted and to face. The tutor will
he/she is outside the marked on-line and have to use digital
classroom. there will be means to promote
assessment exercises. engagement and a
sense of community.
B. Williams RSCEM May 25, 2011 | slide 5

6. Where are you on the scale?
Learning
activities,
assessment
B. Williams RSCEM May 25, 2011 | slide 6

7. Plan for the session
 Ice Breaking & Introduction
 Observable Benefits of MLO
 Measuring Impact and Quality – Tools &
Approaches
 Review Session
 Further Resources
May 25, 2011 | slide 7

8. A checklist for assessing courses 1
 Is it likely to work?
– Has information been provided to enable prospective
users to understand the course being offered whether it
will suit them?
– Have students had an introduction or orientation which
prepares them to be successful in the course?
– Do students have the necessary hardware & software to
complete the course?
– Have students the necessary study skills and IT skills to
complete the course – is there opportunity to practise
these skills?
May 25, 2011 | slide 8

9. A checklist for assessing courses 2
 Is it likely to work?
– Is the course designed to be interactive?
– Are the tutors trained in the techniques that are required
for online tutoring?
– Is there a system of support for students (and staff) –
when does it operate 24/7? 7/5? FAQ?
– Is data collected about the course so that
administrators, decision makers and course designers
can learn from the shared information?
This checklist is adapted from the Appleton Eschool’s online evaluation
programme.
http://www2.ed.gov/admins/lead/academic/evalonline/evalonline.pdf
May 25, 2011 | slide 9

10. Online materials presented to students
A basic course would contain the following:
 Course and module title  A reading list
 Tutor name, contact details and  A glossary of terms, if
photo appropriate
 Brief welcome/introduction to the  A course timetable (a list or
course calendar)
 Structured content (chronological  Links to specific
or topic order) resources/websites for research
 A brief introduction to each  A link to library services and
section search
 Lecture  Clearly label all resources
notes, presentations, hand-outs - indicating what they relate to
placed in context
 Assignments details including
 A clear indication of where and how to submit
how to get help May 25, 2011 | slide 10

11. Interactivity and “stickiness”
 The more engaging the course, the more likely the
students are to remain on it and complete
assignments.
 The course should therefore promote interactivity –
with the materials, with the tutor and with other
students.
 Courses can be assessed for the interactive elements
they provide.
 Varied media presentation will also help – video and
podcasts; short spoken introductions to materials; all
these help to give a varied experience to the student.
May 25, 2011 | slide 11

12. Plan for the session
 Ice Breaking & Introduction
 Observable Benefits of MLO
 Measuring Impact and Quality – Tools &
Approaches
 Review Session
 Further Resources
May 25, 2011 | slide 12

13.  Outline of the Evaluation Tool
 How it works.
 Criteria
 Download and use
 Complete survey for
May 25, 2011 | slide 13

14. Course Evaluation & Review
Tool
Students Data and
Quality
Assurance
Teaching ROI
Practitioners

15. May 25, 2011 | slide 15

16. MLOL Evaluation Tool
 The Process
Determine Evaluation
Criteria
Create a Survey Add to MLOL
Provide to Sample Download as Excel Evaluation Tool in the
Monkey Answer File
Groups… Grid Data Sheet Tab
Review Picture
May 25, 2011 | slide 16

17. May 25, 2011 | slide 17

18. May 25, 2011 | slide 18

19. May 25, 2011 | slide 19

20. MLOL Evaluation Tool
Poor
Evaluation
Picture
May 25, 2011 | slide 20

21. MLOL Evaluation Tool
Very Good
Evaluation
Picture
May 25, 2011 | slide 21

22. MLOL Evaluation Tool
 The Process
Determine Evaluation
Criteria
Create a Survey Add to MLOL
Provide to Sample Download as Excel Evaluation Tool in the
Monkey Answer File
Groups… Grid Data Sheet Tab
Review Picture
May 25, 2011 | slide 22

23. Survey
Monkey
Answer Grid
Option
May 25, 2011 | slide 23

24. MLOL Evaluation To
 TASK 1 Your Criteria evaluation
 http://tiny.cc/mlol7task1
May 25, 2011 | slide 24

25. TASKS
 TASKS
 Download the MLOL Evaluation Tool – 10 Minutes
– Evaluation Tool Download at
http://www.tiny.cc/mlol7tool
 Consider what Criteria you might want to add to
the different Sectors
 Feedback
May 25, 2011 | slide 25

26. Online Lesson Observation
 The main concern when observing a lesson is to
check whether learning is taking place.
 How could you demonstrate that learning was
taking place in an online course?
– Data from online interactions – forums etc
– Reports showing quiz answers, assignments completed
 How do learners feel about the quality of learning?
– Learner surveys by web or phone or email
– Use an online live forum of learners
 Feedback to tutors?
May 25, 2011 | slide 26

27. Formative and Summative
 Online systems and course design are in the very
early stages – formative assessment of the course
through user surveys, evaluation of online
interaction etc can be very useful to help re-focus
the course to meet learner needs.
 Summative assessment needs to be
disseminated around the organisation to make
sure that success and failure can be duplicated
and avoided.
May 25, 2011 | slide 27

28. Plan for the session
 Ice Breaking & Introduction
 Observable Benefits of MLO
 Measuring Impact and Quality – Tools &
Approaches
 Review Session
 Further Resources
May 25, 2011 | slide 28

29. Staff Development Programme and QA
Processes
 Staff Development Programme and QA Processes
May 25, 2011 | slide 29

30. Plan for the session
 Ice Breaking & Introduction
 Observable Benefits of MLO
 Measuring Impact and Quality – Tools &
Approaches
 Review Session
 Further Resources
May 25, 2011 | slide 30

31. References and Resources
 Google Docs List of references at:
 http://tiny.cc/mlol7
 Can be added to if you have other links and resources you
can suggest.
 Survey Monkey Example at http://www.tiny.cc/mlol7task1
 Evaluation Tool Download at http://www.tiny.cc/mlol7tool
May 25, 2011 | slide 31