This document discusses Opus Learning's approach to online open vocational qualifications and assessments. The key points are: 1) Opus Learning develops online open courses to SQA HND standards that can be taken directly by students or through other educational providers. 2) They provide the learning environment and content as well as assessments. Tracking tools are used to monitor student interactions and progress. 3) Lessons from MOOCs are discussed around managing costs, cohort sizes, and using peer support. Opus plans to run small private open courses of around 10 students with local tutoring rather than massive open online courses. 4) An experiment is outlined to offer two SQA units as MOO

1. Assessment and Credit in
Online Open Vocational
Qualifications
John Gordon
Opus Learning
digital by default, digital by design
30th January 2014

2. Opus Learning Ltd, Edinburgh
• We develop online open courses to SQA HND
standards
– A white label service for students enrolled with
other providers.
– Direct to students who are enrolled with Opus
• We provide the learning environment, and
deliver the learning content and assessments.

3. In this morning’s talk, I hope to indicate
how we have:
– Benefited by using technology to support and
manage vocational learning;
– Considered how to de-couple learning and
assessment
– Managed downwards the risks and costs of using
technology.

4. Origin of Opus
•
A spinout from CAPDM.com
•
Developing and delivering Masters
programmes for over 17 years
•
www.capdm.com
•
We deliver highly interactive courses,
delivered via an online platform such as
Moodle.
With tools to manage and monitor learning
with or without tutor support.
•

5. Structure of our programmes
• We build:
– integrated frameworks of Units for SQA HNDs and
University undergrad programmes
– within a framework of commercial discipline.
• Which are outward facing and globally
oriented

6. The Main Issues of 2014
•
•
•
•
•
•
Learning and Assessment
MOOCs and SPOCs
Financing and selling
Badges and Certificates
Initial training and Life Long Learning
RPL and de-coupled accredited assessment

7. The problems in
an online context
Online Learning does not change the problems of assessing
learning
–
–
–
–
Tracking learning
Authentication
Identifying problems in learning
Recognising need for intervention
MOOCs introduce new aspects
–
–
–
–
–
Learn without credit
Gain credit after learning
Building badges into credits
Peer support
Security issues

8. Interaction types
Student/Student
Student
Student/Tutor
Student/Content
Student/Environment
Content
Tutor
Tutor/Content
Tutor/Tutor
Content/Content
Content/Environment
Tutor/Environment
Environment
Based on diagram in
Anderson 2003
Environment/Environment

9. Where do the major costs lie?
Student/Student
Student
Support and Assessment
Student/Content
Student/Environment
Content
Review and Improvement
Academic Activity
Tutor
Community
Content/Content
Student and Institution
Management
Content/Environment
Environment
Based on diagram in
Anderson 2003
Environment/Environment

10. Interactions and Monitoring
Type of Interaction
Human to human
Student to Learning Platform
Student to content
Student and formative
assessments
Students and personal
reflections
Tracking Function
Within Forums
Via a ticket system for formal
student to tutor interactions
Via built in tracking tools on
Platform
Our in-built tracking and
monitoring tool

11. Interacting with Content Objects
Student interacts with content
Learner
interacting
with
content, leavin
g traces
Makes notes, responds to
quiz, send messages, etc
Store in MLE
Capture notes
Print off
record of
learning
The Digital Workbook

12. The Digital Workbook is
• A portfolio embedded in
the content
• A dashboard for learning
• Rich seam of data and
behaviours for mining
• Ongoing formative
assessment, and potential
summative assessment
• A pdf for the student

13. MOOC Value Proposition
• Education Access
– On a massive, international scale. Currently, most students who are not
enrolled college students.
– May change as colleges integrate MOOC content to traditional teaching
• Experimentation
– New business model
– New resources for learners
– New learning technologies based on mining
• Brand Extension
– MOOCs can extend the institution’s reach and reputation internationally.
– Particularly among elite research institutions, MOOCs have become a way of
enhancing the institution’s brand and signalling innovation.
– Successful professors can gain a global following, building their own
reputation
http://net.educause.edu/ir/library/pdf/PUB4005.pdf

14. How does this affect Learning
and Assessment?

15. Assessment In the
SQA environment
All units are stand alone, they can be delivered
in a very flexible manner, but there are very
clear pointers to how assessment should be
carried out.
It is no longer appropriate to base the
assessment of competence on a final 3 hour
written examination.

16. Candidate
Requests
Assessment
Instrument
from VLE
Assessment
sent to
Candidate –
time limited
PDF
Assessment and Verification
what happens
Assessment
pdf
Candidate
responds to
Assessment
Candidate
Attempt
Assessor
reviews
Assessment
Attempt Makes
comments
Verifier
consultation
yes
Satisfactory
No
Sign
assessment
record detailing
Satisfactory
Sign
assessment
record detailing
re-assessment
No
Resubmit?
yes
Send assessment
record to
candidate –
complete record
in VLE/SQA
Connect
Candidate
Assessment
Record
Verifier
consultation
Assessor sends
comments to
candidate
Assessor
comments
Candidate
attempt
Candidate
Resubmits

17. Open Distance Learning

18. The Tensions in Technology Based
Open and Distance Learning
• Who is actually at the end of the screen?
• What is the level of Granularity for assessment?
– We want freedom for student to learn what they
want when they want
– We want to integrate assessment across a group
of units
• What is the cohort size?
– We want individualised learning
– We want efficiency in the use of tutor support

19. Who is at the end of the screen?
Opus Policy is:
Digital Authentication
+
Physical Authentication

20. Managing Learning
with Technology
• At Opus we track student behaviour.
• We use 2 levels of tracking
– Moodle Based (the VLE) and
– Content Based.

21. Tracking through the
learning content
• Opus content is massively connected to the
students workbook
• Every reflection, exercise, quiz, short response
and so on are logged within the learning
content and in the workbook
• The tutor can view, track and compare
workbooks

22. DWB Examples – Source Page

24. DWB Dashboard –
Managing groups of students

25. Some conclusions
from tracking
• Tracking appears to:
– Provide the potential of personalised learning
within mechanical presentation of study
– Assist formative assessment in the content driving
tracking and the experience of learning progress
– Provide feedback on Tutor effectiveness and the
effectiveness of the pedagogy underpinning the
content

26. The Commercial Imperative
We wish to
– drive down cost of assessment
– Drive up the quality and efficiency of assessment
within the context of a well resourced learner.
We have to manage costs, and the use of human
resource

27. Assessor/Tutor Activity Model
in Distance Learning
1. Monitor VLE activity, identify any candidate with lack of
progress and contact course leader for action.
2. Monitor Digital Workbook (if used) and identify any
outliers in progress, contact course leader with details.
3. Respond to direct tickets from candidate, making use of
FAQs for responses, if new response is required, copy
response to course leader.
4. Responding to assessments; remediation advice

28. Assessor/Tutor Activity Model
in MOOCs
1. Monitor VLE activity automatically.
2. Monitor Digital Workbook but ignore any outliers in
progress, contact course leader with details.
3. Send any queries to student peer group for support
4. Responding to machine assessed assessments; only
automatic remediation advice
Take out the tutor!

29. Managing Resource
• We can time and cost elements in traditional online courses
• Times and costs vary with cohort size in traditional courses
• For the MOOC approach, times and costs must be
independent of cohort size.

30. Timing Model vs Cohort size
HND Business Timings
Assumptions
Students per
assessor
Average timing per
unit for HND
Business - minutes
1
5
10
15
20
93
85
77
70
64

31. Learning Centre Cost Model
SPOC
• Based on the previous models
– Time per tutor/unit/student can be estimated
– Payment to Tutor/assessor is known
– Viability of the Distance Course can be calculated.
• We need to know these numbers and to
watch them

32. Potential Cost Model
Possible costs/charges per student
awarding body charges = £250;
tutoring = £1000;
tech + online content licence = £300;
agent fees = £250
Centre overheads per student = £500
Total cost per student = £2300

33. Potential Centre Surplus
• In UK Charge to student £3500-£4500
• In India Charge to student £1500-£2500
We cannot serve one of largest markets in the
world, with this model.
We have to be able to manage costs.

34. MOOC Costs models
• Charge to student – Zero
• Cost for development of module not well
reported
– EdX charge $250k to produce a MOOC**
– University of North Carolina quoted $150k for inhouse production*
• Revenue shares between platform provider
and university

35. Learning from MOOCS
•
•
•
•
Massive size of cohort
Highly programmed week by week
Clearly signposted weekly targets
Using the student body as the main tutor
toolset
• Building a community of activists and lurkers
• Managing and controlling costs
• Generate revenue by conversions?

36. Learning from MOOCS
• Do not do assessments
– Until students join full programme
• Keep the granularity of the course fairly small
– not full degrees
• Set up an honour system – make the students
police themselves and even support each
others’ learning.

37. Applying all these lessons
• Manage the cohort size
• Make sure investment reflects assessment and
tutor support provided – the added value of a
non-free course
• Make assessments social, encourage group
help. Track this activity!

38. Opus Strategy
Small Private Open Courses
• Organise cohorts in groups of 10s, not groups of
10k’s or 100k’s
• Localise tutors and assessors
• Build communities including assessors
• Track activity and behaviour at keyboard level and at
cognitive level
• Record activity – perhaps award a badge for partially
completed activity
• Guide the learners towards completion, intervene as
necessary

39. Opus MOOC Experiment
• From the 1st April 2014, there will at least 2 Opus
Units following SQA curriculum
–
–
–
–
–
With no registration at SQA
No access to Assessment Support Pack items
With formative assessment – peer assessed
With summative assessment with peer commenting
Open badges for course completion/attainment
• With the potential to register post mooc, and fast
track to assessment.

40. The Opus Open Courses
• Management: Developing Self Management
Skills based on course H1F1 34
• Project Management: An Introduction based
on course F1NH 34

41. Developing Self Management Skills
Summative Assessment
• Students are encouraged to provide evidence
in a format which suits their particular
situation.
• It is particularly suited to an electronic
portfolio method of assessment.

42. Project Management
Summative Assessment
• Outcomes can be assessed by the
development of project management
documentation, for a workplace project for
which the candidate has responsibility

43. Our to do list
• Design the courses
• Design the peer assessment elements
• Design the learning pathway (based on
current open online pedagogy)
• Design the Badge structure
• Work with SQA to provide pathway to
certification!!!!

44. The End
John Gordon
Opus Learning
jg@opuslearning.com
Thank You