The document summarizes a study that evaluated the effect of a robotics laboratory on computational thinking skills in primary school children in Italy. The study (1) involved robotics laboratories using Lego kits for 71 intervention students and regular curriculum for 65 comparison students, (2) assessed computational thinking using Bebras tasks, finding intervention students scored higher, and (3) found no difference in academic achievement between groups. The study provides evidence that educational robotics can effectively enhance computational thinking skills in primary students.

1. Exploring the Effect of a Robotics
Laboratory on Computational
Thinking Skills in Primary School
Children
USING THE BEBRAS TASKS
Giuseppe Chiazzese, Marco Arrigo, Antonella Chifari, Violetta
Lonati, Crispino Tosto

2. Summary
The “PROMISE”
Project
The
Current Research

3. The «PROMISE»
Project
Project funded by Italian Ministry of
Education, University and Research
(MIUR)
Network of Italian Partners also
including research institutes,
universities, non-profit organizations
http://www.promiseproject.it/

4. Aim of the project
to activate an Italian national network for
enhancing technological and scientific skills in
school and extra-school settings
1. Training of 105 teachers in educational
robotics in North, Central and Southern
Italy
2. Allowing teachers and educators to
design and conduct educational
robotics laboratories in primary and
secondary schools

5. The Current
Study
Introduction
and
Significance
▪ can be effective in enhancing students’
learning of concepts related to the STEM
area in K-12 education1,2
▪ Can support a Computational Thinking
(CT) learning progression from sequencing
ability to system knowledge3
▪ Literature highlights more qualitative and
few quantitative studies on the effect of
educational robotics on the development
of CT skills among primary school students
Why Educational Robotics in
STEM teaching

6. The Robotics Laboratories
are
• part of the second phase of the “PROMISE”
project.
• carried out by school teachers (who have
attended the first phase training courses),
researches of the Institute for Educational
Technologies (CNR)
• organized in 4 sessions (2 hours each) for 3
parallel laboratories
• supported by students of the Computer
Science course of the High School “Istituto
Superiore Majorana” Palermo that have
coached the younger students in building
and programming educational robot

7. Action plan
• Introduction of Lego® Education WeDo 2.0 [18] kit and
exploration of hardware components kit and the visual
programming environment to students.
• Teaching of basic programming concepts, designing of
simple programs, and use of the sensors and actuators
• peer-coaching in construction and programming of
new robots solving STEM challenges.

8. The Current Study
Aim, research design, and
Participants
Aim
Evaluate the usefulness of a basic robotics laboratory, supported by the use of
Lego® Education WeDo 2.0 robotics kit [18], for the development of early CT
skills in children attending 3th and 4th grade of primary school.
Quasi-experimental posttest-only research design4
Intervention
Children attending 3 different classrooms1 (2 third, 1 fourth grade)
participated to the robotics laboratories (Intervention)
71 students attended the robotics laboratories; 51 students (age
range 8-10; 18 female) completed the outcome measures
Comparison
Children of 3 classrooms1 (2 third, 1 fourth grade classrooms) attended the
regular school curriculum (Comparison)
65 students from the regular curriculum classrooms; 32 students (age
range 8-10; 12 female) completed the outcome measures
1Classrooms of a primary public school in Palermo (Italy), Direzione Didattica Statale “Moni Iblei”

9. The Current Study
Measures – Computational
Thinking
The Bebras Tasks.
Set of activities designed for the “International Challenge on
Informatics and Computational Thinking”a
The 10 tasks of n the “KiloBebras” 2017/2018 Italian
challenge were proposed.
Total scores on the Bebras tasks used as measure of
computational thinking skills.
The Bebras tasks already used as a tool for the assessment of
children’s ability to transfer their computational thinking skills
in the solution of “real-life” problems6
https://bebras.it Italian platform

10. For pupils of grades until 8th, the main skills
subsumed/promoted by the Bebras tasks are:
▪ logically analyzing and organizing data
▪ representing data through abstraction and formal
encodings
▪ using algorithmic thinking as a way to automatize
solution
▪ implementing simple algorithmic procedures
(coding)
Skills they subsumed/promoted are transversal and
shared with other (not only STEM) disciplines
1https://bebras.it/, Italian Platform
The Current Study
Measures – Computational Thinking

11. The Current
Study
Measures –
Additional
Academic Achievement
Teachers’ assessment of achievement in all scholastic
subjects
Average academic performance and average
performance on STEM subjects derived to compare the
intervention and comparison groups.
Experience with the Bebras Tasks
Self-report questionnaire to assess students’ experience
with the Bebras tasks:
a) satisfaction with the tasks,
b) tasks perceived difficulty, and
c) tasks as learning tools
Satisfaction with the Laboratory
Self-report questionnaire to evaluate students’
satisfaction with the intervention

12. The Current Study
Statistical Analyses
Two Mann-Whitney U tests
to evaluate differences between the intervention and
comparison group on average academic performance
and average performance on STEM subjects.
Independent t-tests
to assess the difference between intervention and
comparison conditions
▪ in computational thinking (Bebras total scores)
▪ in time to complete the Bebras tasks, and
▪ satisfaction with the Bebras Tasks

13. The Current Study - Results
0
5
10
15
20
25
Intervention Comparison
Bebras CT evaluation
Bebras CT evaluation
Academic Performance
no significant differences between the intervention and
comparison groups
Mann-Whitney U tests
• overall (intervention median=5, comparison median=4.5,
n=51, Mann–Whitney U=650, p=.10), and
• STEM subjects (intervention median=5, comparison
median=4.5, n=32, Mann–Whitney U=655, p=.11).
Computational Thinking
students participating to the robotics laboratory reported
significantly higher total scores on the Bebras challenge
than did those attending only the regular school curriculum.
Independent t-test, t(81)=-4.45, p<0.001.
Students in the intervention group spent more time in minutes to complete the Bebras challenge
Students in both the groups showed comparable levels of satisfaction with the Bebras tasks
(moderately to high satisfaction) and perceived the tasks as a useful learning tool

14. The Current
Study
Brief
Discussion
Our results adds to the existing
literature on the effectiveness of
robotics education in fostering
the acquisition of CT skills2, 7, 8
Our results confirm the relevance
of using robotics kits to promote
the development of CT abilities in
the solution of “real-life”
problems6
The enhancement of CT may
explain why children in the
intervention group spent more
time in completing the Bebras
challenge while showing a better
performance.
Students in the comparison
group could have found some
tasks more difficult to complete
and renounced completing the
challenge faster than their
intervention counterpart.

15. The Current Study
Implications and future directions
1
Educational robotics
facilitate the early
development of CT-
related cognitive
abilities that children
may transfer for
adequately coping
with realistic
problematic
situations.
2
The study encourages
the systematic
adoption of
educational robotics
in K-12 education as
an engaging and
effective tool
3
Bebras Tasks captured
differences in
performance between
the two condition
4
Limited sample size
5
Future research
should assess CT skills
acquisition within
different domains of
application6

16. References
1. Benitti, F. B. V., & Spolaôr, N. (2017). How have robots supported stem
teaching?. In Robotics in STEM Education (pp. 103-129). Springer,
Cham.
2. Benitti, F. B. V. (2012). Exploring the educational potential of robotics in
schools: A systematic review. Computers & Education, 58(3), 978-988.
3. Sullivan, F. R., & Heffernan, J. (2016). Robotic construction kits as
computational manipulatives for learning in the STEM
disciplines. Journal of Research on Technology in Education, 48(2), 105-
128.
4. Montero I., and Orfelio G. León O. G. (2007). A guide for naming
research studies in Psychology. International Journal of Clinical and
Health Psychology, 7(3), 847-862
5. LEGO Education WeDo 2.0, https://education.lego.com/en-
us/elementary/shop/wedo-2
6. Román-González M., Moreno-León J., and Robles G. (2017).
Complementary tools for computational thinking assessment. In
Proceedings of International Conference on Computational Thinking
Education (CTE 2017), S.C. Kong, J. Sheldon, and K.Y. Li (Eds.). The
Education University of Hong Kong, 154-159.
7. G. Chiazzese, G. Fulantelli, V. Pipitone, and D. Taibi. 2018. Engaging
Primary School Children in Computational Thinking: Designing and
Developing Videogames. Education in the Knowledge Society, 19(2),
63-81. DOI:https://doi.org/10.14201/eks20181926381.
8. Guanhua Chen, Ji Shen, Lauren Barth-Cohen, Shiyan Jiang, Xiaoting
Huang, and Moataz Eltoukhy. 2017. Assessing elementary students’
computational thinking in everyday reasoning and robotics
programming. Computers & Education, 109, 162-175.

17. Thank you!
Giuseppe Chiazzese
Istituto per le Tecnologie Didattiche
Consiglio Nazionale delle Ricerche
Palermo Italy
giuseppe.chiazzese@itd.cnr.it
Marco Arrigo
Istituto per le Tecnologie Didattiche
Consiglio Nazionale delle Ricerche Palermo Italy
marco.arrigo@itd.cnr.it
Antonella Chifari
Istituto per le Tecnologie Didattiche
Consiglio Nazionale delle Ricerche Palermo Italy
antonella.chifari@itd.cnr.it
Violetta Lonati
Dipartimento di Informatica Università degli studi di Milano
Italy
lonati@di.unimi.it
Crispino Tosto
Istituto per le Tecnologie Didattiche
Consiglio Nazionale delle Ricerche
Palermo Italy
crispino.tosto@itd.cnr.it