This document discusses online education quality assurance and benchmarking. It provides background on the presenter, Professor Michael Sankey, and the organizations he represents, including Griffith University in Australia and the Australasian Council on Open, Distance and eLearning (ACODE). It outlines different levels of technology-enabled learning and the importance of quality frameworks and standards for online education. It introduces a benchmarking toolkit from the Commonwealth of Learning to help institutions evaluate and improve their technology-enabled learning practices by comparing them to good practices and other institutions. Benchmarking is presented as an important quality assurance method that can identify areas for improvement and facilitate collaboration.

1. Online Education Quality Assurance
Professor Michael Sankey
Director, Learning Transformations
President, Australasian Council on Open,
Distance and eLearning (ACODE)

2. Griffith University
• Large comprehensive metropolitan
university
• 50,000+ students
• 6 campuses (5+1)
• 15,000 students taking online courses
• Although young a strong history of
blended learning and use of
Technology enhanced learning (TEL)
michael_sankey

3. ACODE
• Australasian Council on Open,
Distance and e-Learning
• 49 Universities + 8 Private providers
• Workshops
• Leadership Institute
• Benchmarking
• National TEL Awards
• Develop sector White Papers
• Community Mentoring
• Forums
michael_sankey

4. Levels within TEL
There are levels of TEL seen within
the global sector, dependent largely
on the capacity of the:
• Educational jurisdiction
• National technology
infrastructure
• Geographical constraints
• Level of staff training
Technology-Enabled Learning
Technology Enhanced Learning
Technology Intensive Learning
michael_sankey

5. 1. The boundaries are quite blurred
2. It does not help to put it in binary terms like this
3. Institutions need to consider TEL at the Meta level
4. Then provide clear guidance for staff and students
5. Each have nuanced standards
6. Digitally-enabled Learning
michael_sankey
Online, Blended, Flipped, Hybrid, etc.
TEL
Blended
Flipped
Hybrid
Online

6. The Importance of a quality
framework and standards
• Australia's quality agency TEQSA now
paying a particular interest in TEL
• Especially in relation to fully online courses
• But now we have moved fully online we cant
rest on our laurels
https://www.teqsa.gov.au/latest-news/publications/guidance-note-technology-enhanced-learning
michael_sankey

7. Online Learning: Good practice
https://www.teqsa.gov.au/online-learning-good-practice
michael_sankey
Professor Nick Saunders

8. Key
Elements
of TEL
Quality
TEL Policy and
Procedures
Institutional
TEL Framework
Baseline
Standards for
All Programs &
Units/Courses
Standards for
Fully Online
Units/Courses
Governance is essential to ensure each
School/Faculty are working from the
same baseline. Policy for TEL needs to
be aligned & be consistent with other
policies, such as the learning &
teaching policy.
A framework unpacks what is in policy
& procedure & aligns this with the
systems & practices across all
departments. It nominates which
department is responsible for each
element of TEL practice.
TEL sits within the full spectrum of
Program & Course delivery, whether it
be face to face, online, or a mix
(blended). Students require a level of
consistency across their Program/s
For fully online courses, where there
are less physical cues on how staff &
students should operate in this
environment, an extra level of
consistency is required to help support
& scaffold their practice.

9. https://www.teqsa.gov.au/latest-news/publications/guidance-note-external-referencing-including-benchmarking

10. michael_sankey
Monitoring, review and improvement processes can and should
encompass review against comparators, both internal to the provider and
external.
A number of approaches and techniques can be used for external
referencing, such as benchmarking, peer review and moderation.
Benchmarking is perhaps the most elaborate form of external referencing
and typically consists of focused improvement through relationships with a
benchmarking partner or partners, but can also include comparing course
design against publicly-available information and market intelligence.
Further detail on benchmarking practice is given in the Appendices A and B
below.

11. 3 Quality tools for TEL
michael_sankey

12. Sharing practice

13. • This Commonwealth of Learning (COL)
Benchmarking Toolkit
• To help institutions see their TEL practice in the
light of what is considered good practice
• Then to compare their analysis with others
• http://oasis.col.org/handle/11599/3217
Benchmarking
michael_sankey

14. michael_sankey
The 10 Benchmarks for TEL
1. Policy 6. Documentation
2. Strategic Plan 7. Organisational Culture
3. IT Support 8. Leadership
4. Technology Applications 9. Human Resource Training
5. Content Development 10. Technology-enabled Learning Champions

15. michael_sankey
Benchmarking as a quality instrument
Almost all … quality management [activities] emphasise evaluation, and broadly this
can only be undertaken in four ways:
1. against defined objectives or standards (set internally or by external bodies);
2. against measures of customer satisfaction;
3. against expert and professional judgement; and
4. against comparator organisations;
with analysis in all four approaches being undertaken over a defined time scale.
(Schofield, 1998, p. 6).
• This is Benchmarking

16. michael_sankey
Benchmarking is…
• A journey that starts with a self-assessment based on evidence, not
opinion.
• Identifying successes ‘to-date’, providing signals on how to adapt to
future events.
• Generating information needed for improvement and a realistic
appreciation of how well the organisation is moving towards its goals
• A structured method of identifying ideas to improve processes to meet
institutional expectations

17. michael_sankey
Benchmarking is not…
• a numbers-only exercise, as simply capturing metrics
does not necessarily lead to an understanding of how
underlying processes have enabled a given score.

18. michael_sankey
The major benefits of Benchmarking
• Areas for improvement are identified
• Participants end up with a shared understanding of capacity
and capability
• Helps you identify plan of action
• The collaboration experience provides an invaluable form of
professional development for staff

19. michael_sankey
Q26 I learned a number of strategies from other institutions that
I would like to see implemented at my institution
97.5%

20. michael_sankey
Q30 This benchmarking self-assessment activity has provided an opportunity
to stimulate a more in-depth discussion about TEL at my institution
90.0% agree that this has provided opportunity for more in-depth
discussion within their institutions
90%

21. • This Commonwealth of Learning (COL)
Benchmarking Toolkit
• To help institutions see their TEL practice in the
light of what is considered good practice
• Then to compare their analysis with others
• http://oasis.col.org/handle/11599/3217
Benchmarking
michael_sankey

22. michael_sankey
Each benchmark contains the following elements:
• A description of the Benchmark;
• Between 4-6 Performance Indicators;
• Self-review rating for each Performance Indicator – on a 5-point scale;
• A place to provide a rationale and evidence to support your assessment;
• A place to consolidate and calculate your combined score (on the radar
chart);
• An area to note an initial recommendation which may be useful for future
improvement.

23. michael_sankey
The 10 Benchmarks for TEL
1. Policy 6. Documentation
2. Strategic Plan 7. Organisational Culture
3. IT Support 8. Leadership
4. Technology Applications 9. Human Resource Training
5. Content Development 10. Technology-enabled Learning Champions

24. michael_sankey
Example

25. Example
• blaa

26. The institution only briefly mentions technology-enabled learning in its strategic plan, but it does link
this with the learning and teaching plan, where more time is given to this. The Learning and
Teaching Plan is strongly aligned to the overall Strategic Plan. So, although the link is vicarious (not
explicit) it is possible to make this alignment.
The Operational Resource Management Plan (ORMP) states that it will ‘establish related standards and guidelines, including,
assessment hurdle and delivery mode definition standards. This will be in conjunction with Academic Services and Academic
Board’ (p 23). But it does not explicitly mention online learning environments.
In the ORMP – in the Academic Services Division section it does state:
‘Improvements in online learning and teaching capacity and experience including optimised: Staff dev opportunities; Systems
for students studying online; and for staff teaching online’ (p x).
• Enterprise ICT Plan 2016-2020
• Learning and Teaching Systems Roadmap – https://lor.xxx.edu.au/LandT_Roadmap.pdf
• Plans and reports – http://www.xxx.edu.au/about-xxx/about-us/plans-reports

27. michael_sankey
4 3 2 5 14
3.5

28. michael_sankey

29. michael_sankey
It was found that there was only a moderate level of
alignment between the institutions Strategic Plan and
the plans articulated at a faculty/school/department
level in relation to using technology to enable learning
and teaching. It would be helpful if future iterations of
the Strategic Plan could be more inclusive and explicit
on the use of technology to assist the institution in its
mission to enable student learning by way of the
technologies at our disposal.
Given we are two years into the five-year plan, special
note should be made for when this plan is reviewed.
It is recommended that the TEL Working Party make
representation to the Management Committee to
ensure that this is considered.

30. http://hdl.handle.net/11599/3217

31. Course/Unit/Subject Quality Tools
• ASCILITE TELAS
• ACODE Threshold Standards
• Quality Matters (QM)
• OLC quality score card and toolkit
• Commonwealth of Learning
• eLearning Guidelines (New Zealand)
• JISC - eLearning Quality Standards
• European set associated with eExcellence
• E-learning Quality Model (ELQ) out of Sweden
michael_sankey

32. 1. Partnership-Based Learning
2. Engaging and
Empowering Pedagogies
3. Scholarly-Inspired Curriculum
4. Locally and Globally
Connected
5. Learner-Enabling Design
6. Digitally-Enabled Learning
michael_sankey
https://www.griffith.edu.au/learning-futures/our-practice/program-course-design-development-review/course-design-standards

33. michael_sankey
Conclusion
• It is by institution’s undertaking regular quality activities, such as
benchmarking, that they facilitate a level of design consistency, across
their many evolving practices associated with TEL.
• The methodology has spawned a heightened willingness to share
information across the sector, particularly relating to a range of quality
services to students and staff.
• It is the methodology that is the big story here, as it could quite easily be
applied across other contexts within Higher Education
• The key is to use something and get started

34. Thankyou!
धन्यवाद
dhanyavaad