Assessment and recognition in technical massive open on-line courses with and without on-line laboratories - Manuel Castro

1. Assessment and recognition in technical
massive open on-line courses with and
without on-line laboratories
Manuel Castro
Universidad Nacional de Educación a Distancia (UNED)
mcastro@ieec.uned.es

2. Context
THE "INDUSTRY 4.0" PARADIGM IS
GENERATING NEW OPPORTUNITIES AND
NEW CHALLENGES FOR COMPANIES.
ESTIMATES CONSIDER THAT INDUSTRY 4.0
CAN GENERATE ESTIMATED ANNUAL
PROFITS IN MANUFACTURING OF BETWEEN
6% AND 8%.
ACCELERATING THE DIGITAL TRANSITION
TO INDUSTRY 4.0 WILL HAVE SUCH A BIG
IMPACT.

3. Challenges
Train the current 2.76
million European
engineering students.
Retrain employees in a
sector that involves more
than 2 million companies
and 33 million jobs at a
European level, most of
which will prefer online
education.

4. 4
The acquisition of practical skills is key in engineering
education...

5. 5
…then…
How to teach practical competences on-line?

6. Online labs types
Remote labs
Web-based but connected to real
equipment
Virtual labs
Web-based software simulations
Deferred labs
Prerecoredd remote labs
pregrabados
All possible situations recorded

7. Bibliometric analysis
Analysis of publications
education and Industry 4.0
1265 papers (until 2020) => High interest
Only 18 papers from remote laboratories Industry 4.0
Shows low implementation, possibly due to development
difficulties
A project on an open platform for remote laboratories in Industry
4.0 can be a great boost to the sector

8. In4Labs project
Open platform to facilitate the development of Industry 4.0 remote labs
Strategic Projects Oriented to the Ecological
Transition and the Digital Transition 2022

9. Remote Labs for Industry 4.0
Powered by
Internet of
Things Lab
Big Data Lab
Cloud
Computing Lab
Artificial
Intelligence
Lab
Systems
Integration
Lab
Robotics Lab
Cibersecurity
Lab
Perception Lab

10. In4Labs stages
1. Development of an open software platform to support and facilitate the
development of remote laboratories
2. Development of remote Industry 4.0 laboratories
3. Pilot test of these remote laboratories with the developed infrastructure
4. Dissemination and international cooperation with interested technical
associations, companies and institutions
The focus of this paper

11. MOOC creation in In4Labs
• MOOC 1 – Industry 4.0
• MOOC 2 - Internet of Things
• MOOC 3 - Cloud computing and Big Data in Industry 4.0
• MOOC 4 – Artificial Intelligence in Industry 4.0

12. Research questions
• RQ1: Do MOOCs including remote laboratories gets lower drop-out rates
than those without them?
• RQ2: Do MOOCs including remote laboratories gets more paid
certification request?
• The results of this research may be of interest for those looking for
design new MOOCs about technical disciplines
• especially those thinking about including on-line laboratories as a resource in
the course

13. MOOCs to study
• Introduction to Digital Electronics
• 5 modules
• bool algebra, function simplification, logic
gates, combinational circuits, flip flops, and
sequential circuits (e.g., counters)
• includes 10-20 minutes video-lectures and
numerous interactive virtual labs

14. MOOCs to study
• Enjoying FPGAs: Foundations, simulation
and practice
• architecture of these devices to the use of
commercial tools and real practice with remote
laboratories
• based on short and entertaining videos
• practice-oriented design thanks to the use of real
devices through remote labs (LabsLand)
• Therefore, it is not necessary to have an FPGA to be
able to take the course and enjoy designing with
real FPGAs

15. MOOCs to study
• Educational Robotics for all: gender,
diversity, and inclusion in STEAM
• presentation of a combination of teaching
methodologies, educational robotics tools, and a
student learning management methodology
• All of these topics are discussed in a way that
focuses on strengthening the presence of women
in engineering
• Includes 10-20 minutes video-lectures

16. MOOCs to study
• Foundations to Open Education
and OERs repositories
• Designed to provide a comprehensive
understanding of the ways in which open
education and open educational resources are
changing the rules of education
• Purpose of the course is to help educators to
understand the advantages of open education
and how they can apply it to their everyday
teaching
• This course includes 8-15 minutes video-
lectures and numerous reading materials
• All developed materials (e.g., video-lectures,
PPTs, reading documents) were provided for
download as OERs

17. MOOC assessment and interaction definition
• Introduction to Digital Electronics
• Students in this course were divided into two groups:
• experimental group, with access to video-lectures and virtual laboratories;
• control group, with access to video-lectures and static images, instead of virtual
laboratories
• Assessment by module
• Pre-test and post-test
• Satisfaction survey
• Interaction through forums

18. MOOC assessment and interaction definition
• Enjoying FPGAs: Foundations, simulation and practice
• Assessment by modules.
• 2 peer to peer review tasks (modules 3 and 4), where students have to
grade the VHDL designs of other students
• Interaction through forums

19. MOOC assessment and interaction definition
• Educational Robotics for all: gender, diversity, and inclusion in
STEAM
• Assessment through a pre-test at the beginning of each module and a post-
test at the end, to allow the evaluation of the learning improvement in
each module
• Interaction through forums but also a social network activity was proposed.
• Students were encouraged to Tweet with the #STEAM4ALL hashtag about their
advances in the course and some collaborative activities

20. MOOC assessment and interaction definition
• Foundations to Open Education and OERs repositories
• Aassessment through a pre-test at the beginning of each module and a post-
test at the end,
• Interaction through forums

21. Learning analytics of key
performance indicators
• The following indicators were analyzed in each
course in the last edition of each one of them:
• number of students enrolled in the course
• number of students that carried out any test or
task
• number of students approving the course
• number of recognition certificates requested

22. Results
MOOCs Enrolled students
Students doing
some task or quiz
Students passing
the course
Recognition
certificates
requested
Introduction to Digital
Electronics
339
60
(17.7%)
21
(6.2%)
12
(3.5%)
Enjoying FPGAs: Foundations,
simulation and practice
191
44
(23.1%)
14
(7.3%)
14
(7.2%)
Educational Robotics for all:
gender, diversity, and inclusion
in STEAM
172
15
(8.7%)
9
(5.2%)
1
(0.6%)
Foundations to Open Education
and OERs repositories
106
19
(17.9%)
15
(14.1%)
6
(5.6%)

23. Discussion
A study found that only a small
percentage of students request
recognition certificates in MOOC courses
The use of remote labs did not increase
motivation to pass the course.
Only around 6% of enrolled students
pass the course in three of the four
courses
Around 80% of students did not carry out
any tasks or quizzes

24. • Researchers of the In4Labs project will
use these results to specify the
performance indicators of the MOOC
that has to be designed in the In4Labs
project
• Limitations:
• low number of students analyzed
• Future work:
• More editions are needed to get more
sound results
• Analyze student satisfaction.
Conclusions

25. Thanks!
Manuel Castro
Universidad Nacional de Educación a Distancia (UNED)
mcastro@ieec.uned.es