1. Some Ways to Conduct
SoTL Research in
Augmented Reality (AR) for
Teaching and Learning
Shalin Hai-Jew
Kansas State University

2. Presentation
• One of the extant questions about augmented reality (AR) is how
(in)effective it is for the teaching and learning in various formal,
nonformal, and informal contexts. The research literature shows
mixed findings, which are often highly context-based (and not
generalizable). There are some non-trivial costs to the
design/development/deployment of AR for teaching and learning.
For the users, there is cognitive load on the working memory [(1)
extraneous/poor design, (2) intrinsic/inherent difficulty in topic, and
(3) germane/forming schemas]. For teachers, there are additional
knowledge, skills, and abilities / attitudes (KSAs) that need to be
brought to bear.
2

3. Presentation(cont.)
• This work explores some basic research approaches that may be
applied for the Scholarship of Teaching and Learning (SoTL) research
to assess the efficacy of AR applied to teaching and learning. The
work is not comprehensive and offers a way to get started.
3

4. About Augmented Reality (AR)
4

5. About Augmented Reality
• “Augmented Reality” (AR) refers to the overlay of 2d and 3d and 4d
objects over a physical space. This may involve head-mounted
displays (HMDs), mobile devices, projectors, and others.
• 2d exists on the x and y axes. 3d exists on the x, y, and z axes. 4d involves
motion and / or changes over time, in either 2d or 3d.
• AR is not only visual. It may include sound, live data streams, tastes,
and smells. It may include interactivity.
• AR is not only for singleton experiences. It may be social and
collaborative, with the presence of others in real space and in virtual
space (such as through telepresence).
5

6. AR in Teaching and Learning
• In concept, AR is thought to be beneficial in the following ways:
• It enables depictions of imaginary worlds that might be inaccessible to the
physical senses otherwise.
• It enables the experiencing and re-experiencing of experiences that may be
risky and / or expensive and / or impossible to provide otherwise.
• It enables virtual interactivity with (virtual and physical) objects and with
people.
• It enables geographical location-based experiences.
• It enables full-sensory immersion and engagement.
• It enables collaboration and intercommunications.
• It enables creativity, design, and innovations.
6

7. AR in Teaching and Learning(cont.)
• AR may enhance human memory.
• AR may serve as distributed cognition (with digital and smart physical devices
taking on some cognitive load for the learner).
7

8. 8

9. 9

10. Highlights of the Scholarship of
Teaching and Learning (SoTL)
10

11. Scholarship of Teaching and Learning (SoTL)
• The Scholarship of Teaching and Learning (SoTL) refers to “action
research” applied to what goes on in formal, nonformal, and informal
learning contexts. The focus is on benefitting teaching and learning
practices.
11

12. Scholarship of Teaching and Learning (SoTL)
(cont.)
• A research team describes SoTL the following way (with a variety of
requirements):
• “the systematic study of teaching and learning, using established or validated
criteria of scholarship, to understand how teaching (beliefs, behaviours,
attitudes, and values) can maximize learning, and / or develop a more
accurate understanding of learning, resulting in products that are publicly
shared for critique and use by an appropriate community” (Potter & Kustra,
2011, p. 2).
12

13. Scholarship of Teaching and Learning (SoTL)
(cont.)
Another source asks four questions:
1. What is there to know? (scholarship of discovery)
2. What does it mean? (scholarship of integration)
3. How could it be useful? (scholarship of application)
4. How to build students’ understanding of it? (scholarship of
teaching) (Hoessler, Britnell, & Stockley, Summer 2010, p. 82)
Students are clear stakeholders in the SoTL process along with teaching
practitioners, researchers, administrators, and others.
13

14. Known Standards for AR for
Teaching and Learning
14

15. Known Standards
• Instructional design
• (for teaching and learning, for controlling against negative learning and
misapprehensions, for low extraneous cognitive load, and others)
• Accessibility
• Intellectual property protections (copyright, and others)
• Privacy protections (legal releases for “the talent”)
15

16. Some General Educational Research
Approaches for Assessing AR
Teaching and Learning (with SoTL)
16

17. Assessing AR using SoTL: General Educational
Research Approaches
• Spatial Usage via Mapping (qualitative, quantitative, mixed methods)
• Educational Artifacts (qualitative, content analysis)
• User Experience (qualitative, quantitative, mixed methods)
• Instructor Experience (qualitative, quantitative, mixed methods)
• Experimental Research (quantitative)
17

18. Spatial Usage via Mapping and
Observational Notetaking
SOTL
18

19. Spatial Usage via Mapping and Observational
Notetaking
• Spatial mapping involves the observations of how the physical space
(interior, exterior) is used in an AR learning experience (tour,
exploration, simulation, gameplay, collaboration, co-design, design,
and others).
• Observational notetaking involves observations of how the learners
engage in various stations…and / or with others, and so on. What is
said? What is done?
• Data are collected and coded.
19

20. Educational Artifacts
SOTL
20

21. Educational Artifacts
• What is the quality of the (recorded) discussions?
• What is the quality of the collaborative work?
• What is the quality of the individual work?
• What is the performance on the assessments?
• What is the quality of the created or innovated objects?
• What is the quality of the shared ideas?
21

22. Post-AR Experience Performance
• How long does the learning continue after the experience?
• Is there long-term learning? Is it accurate?
22

23. User Experience
SOTL
23

24. User Experience
• Interviews (singular or group)
• Focus group (group)
• Surveys (individuals and groups)
• Others
• The AR experience, interest
level, pacing
• The cognitive load
• The clarity of the learning
• The ability to communicate with
others
• What sorts of technical
challenges
• The sense of learning support
• Others
24

25. Instructor Experience
SOTL
25

26. Instructor Experience
• Interviews (singular or group)
• Focus group (group)
• Surveys (individuals and groups)
• Others
• Effectiveness of approach for
teaching and learning
• Ability to teach with originality
and insight
• Ability to identify learner needs
(and to meet their needs), incl.
those on both ends of a normal
curve and the big middle
• Flexibility of the learning objects
• Others
26

27. Instructor Experience(cont.)
• Ability to coordinate across technologies
• Ability to provide effective feedback
• Formative assessment; summative assessments
• Alternate teaching and learning to the AR
• And others
27

28. Limits to Self-Reportage
• There are limits to self-reportage.
• There are subjectivities and biases.
28

29. Experimental Research
SOTL
29

30. A Classic Experimental – Control Group
Approach
Pre-test Post-test
Control group
Experimental group
30

31. Seating the Research Participants
• Were the two groups assigned in randomized ways?
• Were there sufficient numbers of participants in the research study to
ensure power in the statistics?
• Were human subjects research standards applied?
31

32. Gold Standard
• What is the empirical difference between the average pre-test and
post-test scores?
• Can the null hypothesis be rejected?
• Were the findings statistically significant (with a p-value of < .05? < .01?)
• What is the empirical difference between the performances of the
control and experimental groups of learners?
• What is the control group experience vs. the experimental group experience?
How are the respective experiences controlled for, so that the essential
comparison is between the two learning experiences?
• What pre- and post-test instruments were used, and how were they
validated?
32

33. Critical Questions
33

34. Critical Questions
• What sorts of teaching and learning benefit from augmented reality
builds?
• What are some optimized ways to build augmented reality
experiences for teaching and learning?
• With prototypes? With learner participatory inputs on designs?
• What style elements matter in such AR experiences? Design tones?
Color palettes? Digital scripted agents?
• What sorts of post-learning study and practice resources would be helpful?
• What are ways to assess whether to go with AR or with some other
approaches? Other media?
34

35. Critical Questions(cont.)
• When does the benefit outweigh the cost for using augmented reality
for teaching and learning? What are some straightforward ways to
calculate the cost-benefit trade-off?
• How much of the design should come from people and how much
from generative AI and computer programs and pre-mades and stock
imagery and set color palettes and others? Why?
35

36. Open-Source Sharing?
• How transferable is the AR experience? The digital artifacts?
• How usable are such AR objects by others? How heritable are they?
• Who are the stakeholders to these resources?
• What sort of training will those who acquire such resources need in
order to best use them? Why?
36

37. References
37

38. References
• Hoessler, C., Britnell, J., & Stockley, D. (2010). Assessing the impact of
educational development through the lens of the scholarship of
teaching and learning. New Directions for Teaching and Learning,
2010(122), 81-89.
• Potter, M.K., & Kustra, E. D.H. (2011). The relationship between
scholarly teaching and SoTL: Models, distinctions, and clarifications.
International Journal for the Scholarship of Teaching and Learning,
5(1), 1 – 18.
38

39. 39

40. Conclusion and Contact
• Dr. Shalin Hai-Jew
• Instructional Design
• ITS
• Kansas State University
• 785-532-5262
• shalin@ksu.edu
• The visuals in this slideshow were created using the Deep Dream
Generator.
40