My pre-defense presentation on my EdD dissertation research.

1. A Framework for Promoting Teacher
Self-Efficacy with Mobile Reusable
Learning Objects
Pathways and Progress
EDDE 806 Presentation (Fall 2014)
Robert Power
Instructional Developer, College of the North
Atlantic-Qatar
EdD Student, Athabasca University

2. The Question:
“What is the single greatest barrier to the widespread
adoption of mobile learning strategies in K12 and
higher education institutions?”
The Response:
“Teachers’ confidence in the technology and their
ability to use mobile learning in their own practice.”
Paraphrased exchange between Robert Power (moderator) and Dr. Mohamed Ally (panelist) at the
Panel Discussion on Tablet Deployment Initiatives at the 12thWorld Conference on Mobile and
Contextual Learning (mLearn 2013), Doha, Qatar

3. The Problem with Teacher Training
The current educational model is outdated because it
was developed before the advent of information and
communication technologies. The current model,
based on classroom-based face-to-face delivery, is
geared towards educating a certain segment of the
population. Also, teachers are being trained for the
current model of education, and will therefore
continue using the model when they become
teachers. Teacher training must be re-invented to
prepare teachers for the technology-enhanced
educational system.
(Ally & Prieto-Blazquez, 2014)

4. The Problem with Our Understanding
of Teacher Efficacy
Lack of training in the pedagogical considerations for the
integration of a specific type of technology can have a
negative impact upon teachers’ perceptions of self-efficacy
(Kenny, et al, 2010). However, Kenny et al. (2010) note
that:
While a significant body of research exists on learners’ feelings of
self-efficacy concerning computer technology, online learning, and
even podcasting… this concept does not yet appear to have been
examined in any detail in a mobile learning context (p. 2).

5. The Essential Intervention in this Study
• Remove the “new technology” element from
the equation in so much as is possible.
• Put the focus on pedagogical decision-making.
• Determine if that approach has an impact on
teachers’ perceptions of self-efficacy with
mobile learning.

6. Background

7. QR Cache (2012)
Power (2012 b, c)

9. Developing the CSAM Concept
Power (2012a, 2013b)

10. Power (2013 a, c)

11. C =
Collaborative
S =
Situated
M =
Mobile
A =
Active
Effective
Collaboration
via Mobile
RLOs

12. What is CSAM?
• CSAM is:
– A summarization of the key pedagogical elements
present in recent case studies of the use of mobile RLOs
to facilitate collaborative learning.
– A framework to guide instructional design decision-making.
– Consistent with Activity Theory, the zone of proximal
development, Transactional Distance Theory, and FLOW
Theory.
• CSAM is not:
– A new learning theory.
– A new model of instructional design.

13. Finding My Way

15. Conceptual Framework

16. Research Questions
1. Does the Collaborative Situated Active Mobile (CSAM)
learning design framework provide teachers with an
increased sense of self-efficacy in the use of mobile
reusable learning objects (RLOs) to facilitate or
enhance collaborative learner interactions?
a) Do teachers perceive greater self-efficacy when
using the CSAM framework?
b) Do teachers perceive their use of mobile RLOs be
more effective when using the CSAM framework?

17. Supporting CSAM
1. Qualitative Literature
Review
2. Qualitative Meta-Analysis
3. Supporting Theories, Models
& Frameworks

18. Qualitative Meta-Analysis
Detailed meta-analysis of 403 chapters, journal
articles, and conference proceedings papers between
2009-2014

20. Attention
ARCS
(Learner
Motivation)
Relevance
Confidence
Satisfaction
The ARCS Model
(Keller, 1987, 2013)
The FRAME Model
(Koole, 2009)
The TPACK Framework
(Koehler & Mishra,
2006, 2008)
Activity Theory and ZPD
Relationships
Between CSAM,
Learning Theory, and
Supporting Models

21. Methodology

22. How to Measure Impact on
Perceptions of Self-Efficacy?
• Use the CSAM Learning Design Framework as the focus
of pedagogical decision making and self-reflective
practice in a short, online professional development
course on creating mobile reusable learning objects.
• Measure the impact of that training on participants’
perceptions of self-efficacy.
• Get feedback from participants on their perceptions of
self-efficacy and the use of CSAM to help make
instructional design decisions.

23. Research Methodology
How will I measure effects on perceptions of self-efficacy?
• Mixed-methods research:
– Mix of quantitative survey data and qualitative feedback
from follow-up interviews
• Design-Based Research:
– This proposed study would constitute the first phase of a
longer-term DBR project. Subsequent phases would build
upon this research to inform iterative improvements to
the professional development course, and the eventual
development of an OER RLO (Anderson & Shattuck, 2012;
Cohen et al., 2011; Design-Based Research Collective [DBRC],
2003)

24. Research Instruments
• Combined Teacher’s Sense of Efficacy Scale (TSES)
and Mobile Teacher’s Sense of Efficacy Scale
(mTSES) survey instruments (Benton-Borghi, 2006;
Tschannen-Moran & Woolfolk Hoy, 2001a, 2001b)
– Pre and post-intervention surveys
– General sense of self-efficacy vs self-efficacy with
using mobile RLOs before and after the training
• Follow-up Interviews
– Qualitative feedback regarding the training, and
participants’ perceptions of self-efficacy with mobile
RLOs

25. Research Design:
A Framework for Promoting
Teacher Self-Efficacy with Mobile
Reusable Learning Objects

27. Participants

28. A Snapshot of the Intervention
• Online Professional Development course called
Creating Mobile Reusable Learning Objects
Using Collaborative Situated Active Mobile
(CSAM) Learning Strategies.
• Hosted on the Canvas open LMS
• Can be accessed via computer or mobile
device.
• Five modules, run over ten days (two days per
module).
• Research survey instruments were embedded
as learning activities (to reduce extra time
commitments for participants).

30. Quality Assurance
Quality Matters
(2012, 2013a, b)

31. Data Analysis and Results

32. mTSES Analysis (Quantitative)

33. Construct Validity
TSES, I-TSES and mTSES reliabilities
Cronbach's alpha
SCALES TSES I-TSES 1st TSES 2nd TSES 1st mTSES 2nd mTSES
Efficacy for Student Engagement 0.85 0.86 0.86 0.91 0.88 0.9
Efficacy for Instructional Strategies 0.89 0.89 0.87 0.87 0.84 0.89
Efficacy for Classroom Management 0.91 0.88 0.78 0.93 0.77 0.91
Total Scale Efficacy 0.93 0.93 0.93 0.95 0.92 0.96

34. Overall Trends
Changes in TSES and mTSES subdomain scores between 1st and 2nd administrations
SCALES 1st Admin 2nd Admin MChange
TSES Scoring MmTSES1 MmTSES2 MChange
Efficacy in Student Engagement: 6.04 6.23 0.19
Efficacy in Instructional Strategies: 6.94 7.25 0.31
Efficacy in Classroom Management: 6.86 6.87 0.01
mTSES Scoring MmTSES1 MmTSES2 MChange
Efficacy in Student Engagement with mLearning: 5.9 6.48 0.57
Efficacy in Instructional Strategies with mLearning: 6.59 7.27 0.68
Efficacy in Classroom Management with mLearning: 6.78 6.89 0.11

35. Accounting for Maturation
(mTSES2 – mTSES1) – (TSES2 – TSES1) = Net Change(Intervention Effect)
Net change (intervention effect)
Domain
Net Change
(mTSES2 – mTSES1) – (TSES2 – TSES1)
Student Engagement 0.38
Instructional Strategies 0.37
Classroom Management 0.11
(Kirk, 2004)

36. CSAM Feedback Survey Analysis
Analysis of responses to CSAM feedback questions in second mTSES survey
Item # Question M SD
1
How much has the CSAM learning strategies framework had an influence on your interest in
using mobile reusable learning objects to facilitate collaborative learning for your students?
6.14 1.83
2
How much does the CSAM learning strategies framework influence your decision-making
about pedagogical design for collaborative learning activities involving mobile reusable
learning objects?
5.86 1.81
3
How much does the CSAM learning strategies framework influence your reflection on
collaborative learning activities involving mobile reusable learning objects?
6.41 1.76
4
How well can you identify appropriate collaborative learning activities for your students that
could be facilitated through the use of a mobile RLO?
6.41 1.59
5
How well can you identify opportunities to use mobile RLOs to situate a collaborative learning
activity in a realistic context?
6.23 1.69
6
How well can you develop a plan for students to actively interact with content to produce new
knowledge or aretefacts (evidence of learning)?
6.55 1.34
7
How well can you identify opportunities to use mobile RLOs to allow students to actively
explore alternative learning environments?
6.18 1.47
8
Do you find the Collaborative Situated Active Mobile (CSAM) learning strategies framework
useful in helping you to plan for the integration of mobile resuable learning objects (RLOs) into
your lesson planning?
6.5 1.77
9
Do you feel more comfortable with planning to integrate mobile devices and mobile RLOs into
your lesson planning when using the CSAM learning strategies framework to guide
instructional design decisions?
6.59 2.11
10
Has use of the CSAM learning strategies framework influenced your plans to use mobile RLOs
with your students?
6.68 2.17
TOTALS 6.35 1.75

37. Survey and Interview Analysis
(Qualitative)

38. Inter Rater Reliability
Inter-rater reliabilities for the interview transcript coding
Percent
Agreement
Scott's Pi Cohen's Kappa
Krippendorff's
Alpha
#
Agree
# Disagree
Primary Coding
Sample 1 96.72 0.96 0.96 0.96 59 2
Sample 2 96 0.95 0.95 0.95 24 1
Simultaneous Coding
Sample 1 91.8 0.9 0.9 0.9 56 5
Sample 2 84 0.81 0.81 0.81 21 4
Notes: nSample1 = 61 units of analysis; nSample2 = 25 units of analysis
Freelon (2011), http://dfreelon.org/utils/recalfront/recal2/

39. Qualitative Coding
Frequency counts of primary comment codes
Primary Codes Descriptions nSurvey nInterviews nTotal
0 Not Coded 7 37 44
100 Framework Strengths 21 35 56
200 Framework Weaknesses 1 12 13
300 Course Strengths 1 50 51
400 Course Weaknesses 6 12 18
500 Self-Efficacy 0 0 0
600 Interest 11 31 42
700 Other Barriers 1 22 23
800 Other Supports 6 25 31
Totals 54 224 278

40. Trends from the Qualitative Coding
Frequency counts of primary comment codes
Primary Codes Descriptions nSurvey nInterviews nTotal Most Common Sub-Theme
100 Framework Strengths 21 35 56 Guidance (n = 26)
200 Framework Weaknesses 1 12 13 Too narrow in scope (n = 6)
300 Course Strengths 1 50 51
Interaction / Feedback (n = 19),
Multiple learning resources (n = 11)
400 Course Weaknesses 6 12 18 Development tools (i.e Winksite™) (n = 8)
500 Self-Efficacy 0 0 0
600 Interest 11 31 42 May use mRLOs if appropriate opportunity arises (n = 18)
700 Other Barriers 1 22 23 Lack of institutional interest (n = 5)
800 Other Supports 6 25 31 Informal community of practitioners (n = 8)

41. Summary of Findings
• Participants enjoyed the professional development course.
• Participants expressed a perception that the CSAM framework
and the professional development course had increased their
understanding of, and confidence with, designing and using
mobile RLOs in their teaching practice.
• The mTSES results demonstrate increases in participants’
perceptions of self efficacy with the design and use of mobile
RLOs.
• Participants expressed increased interest in integrating mobile
RLOs into their teaching practice.

42. Limitations
• Voluntary self-enrollment:
• Results may be limited to individuals with a pre-existing
interest in educational technology, mobile learning, and
professional development.
• Four partner institutions:
• Results may not be generalizable beyond North American
teacher populations.
• Participant demographics:
• Undergraduate education students and unemployed
teachers were not included in this study.
• Interview participation:
• Random or stratified-random sampling was not possible.
• Results may not be generalizable to the entire participant
population

43. Recommendations for the CSAM Course
• Longer duration.
• Incorporate a practicum.
• Incorporate a module or resource section on classroom
management considerations for mobile learning.
• Alternative development tools.
• Standalone mTSES tool.
• Multimedia tutorials.
• Community of practitioners.

44. Recommendations for Future Research
• First phase of a DBR project.
• Future phases should:
• Develop and seek feedback on recommended course refinements.
• Verify the findings from this study, and the applicability of those
findings to wider subsets of the teacher population.
• Redevelop survey instruments and interview questions to include
questions pertaining the reasons why participants perceived
changes in their perceptions of self-efficacy.
• Provide future offerings of the PD course in partnership with the
four collaborating institutions.
• Seek PD partnerships with additional institutions, and with a wider
range of participants.

45. Significance of This Research
• Demonstrated the utility of the CSAM framework.
• Explored the potential for CSAM-focused PD to increase teachers’ perceptions
of self-efficacy.
• Development of an instrument to gauge teachers’ perceptions of self-efficacy
with respect to designing and using mobile RLOs.
• mTSES can be used to compare effects of other training interventions on perceptions
of self-efficacy with mobile learning.
• Explored issues of instructional design competency and perceptions of self-efficacy
with the use of educational technologies that are becoming increasingly
pervasive in all sectors of education (F2F, blended, distance…).
• Contributed to the body of knowledge about how to better prepare teachers to
integrate mobile learning strategies and resources into instructional design at
any level of the education system.

46. Try the Course Yourself!
https://canvas.instructure.com/enroll/XG8XW6

48. References
Ally, M., Farias, G., Gitsaki, C., Jones, V., MacLeod, C., Power, R., & Stein, A. (2013). Tablet deployment in
higher education: Lessons learned and best practices. Panel discussion at the 12th World Conference on
Mobile and Contextual Learning (mLearn 2013), 22-24 October, 2013, Doha, Qatar
Ally, M. & Prieto-Blázquez, J. (2014). What is the future of mobile learning in education? Mobile Learning
Applications in Higher Education [Special Section]. Revista de Universidad y Sociedad del Conocimiento
(RUSC), 11(1), 142-151. doi http://doi.dx.org/10.7238/rusc.v11i1.2033
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research?
Educational Researcher, 41(1), 16-25. DOI: 10.3102/0013189X11428813. Retrieved from
http://edr.sagepub.com/content/41/1/16.full
Benton-Borghi, B. (2006). Teaching every student in the 21st century: Teacher efficacy and technology
(Doctoral dissertation, Ohio State University). Retrieved from
http://www.pucrs.br/famat/viali/tic_literatura/teses/BentonBorghi%20Beatrice%20Hope.pdf
Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed). New York:
Routledge.
The Design-Based Research Collective (2003). Design-based research: An emerging paradigm for
educational inquiry. Educational Researcher, 32(1), 5-8. Retrieved from
http://www.designbasedresearch.org/reppubs/DBRC2003.pdf
Freelon, D. (2011). ReCal2: Reliability for 2 coders. Retrieved from
http://dfreelon.org/utils/recalfront/recal2/

49. References
Keller, J. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Design,
10(3), 2-10. Retrieved from
http://download.springer.com/static/pdf/67/art%253A10.1007%252FBF02905780.pdf?auth66=1395208839_f16a62
cb46b48a70cc08b9166706ffce&ext=.pdf
Keller, J. (2013, September 17). ARCS explained. Retrieved from http://www.arcsmodel.com
Kenny, R.F., Park, C.L., Van Neste-Kenny, J.M.C., & Burton, P.A. (2010). Mobile self-efficacy in Canadian nursing
education programs. In M. Montebello, V. Camilleri and A. Dingli (Eds.), Proceedings of mLearn 2010, the 9th World
Conference on Mobile Learning, Valletta, Malta.
Kirk, R. (2004). Maturation effect. In M. Lewis-Black, A. Bryman, & T. Liao (Eds.), The Sage encyclopedia of social
science research methods. Research Methods. DOI: 10.4135/9781412950589. Retrieved from
http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-research-methods/n534.xml
Koehler, M., & Mishra, P. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge.
Teachers College Record, 109(6), 1017-1054. Retrieved from
http://punya.educ.msu.edu/publications/journal_articles/mishra-koehler-tcr2006.pdf
Koehler, M., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), The
handbook of technological pedagogical content knowledge (TPCK) for educators (pp. 3-29). American Association of
Colleges of Teacher Education and Routledge, NY, New York.
Koole, M. L., (2009). A model for framing mobile learning. In M. Ally (Ed.), Mobile learning: Transforming the delivery
of education and training, 25-47. Edmonton, AB: AU Press. Retrieved from
http://www.aupress.ca/index.php/books/120155

50. References
Power, R. (2012a). Effective learning strategies with mobile devices: Collaborative situated active mobile
learning. Unpublished manuscript, Center for Distance Education, Athabasca University, Athabasca,
Canada.
Power, R. (2012b). QR Cache: Connecting mLearning practice with theory. In M. Specht, M. Sharples, & J.
Multisilta (Eds.), Proceedings of the 11th Annual World Conference on Mobile and Contextual Learning
(mLearn 2012) held in Helsinki, Finland, 16-18 October 2012 (pp. 346-349). Retrieved from http://ceur-ws.
org/Vol-955/
Power, R. (2012c, October). QR Cache: Linking mLearning theory to practice in Qatar. Qatar Foundation
Annual Research Forum Proceedings, 2012(CSP31). DOI: 10.5339/qfarf.2012.CPS31. Retrieved from
http://www.qscience.com/doi/abs/10.5339/qfarf.2012.CSP31
Power, R. (2013a). Collaborative situated active mobile (CSAM) learning strategies: A new perspective on
effective mobile learning. Learning and Teaching in Higher Education: Gulf Perspectives, 10(2). Retrieved
from http://lthe.zu.ac.ae/index.php/lthehome/article/view/137
Power, R. (2013b, April). Collaborative Situated Active Mobile (CSAM) learning strategies: A new
perspective on effective mobile learning. Presentation at the Mobile Learning: Gulf Perspectives Research
Symposium, Abu Dhabi, United Arab Emirates, 25 April 2013.
Power, R. (2013c, April). Create your own mobile RLOs (reusable learning objects) for situated active
learning.Workshop presentation at Technology in Higher Education 2013, 16-17 April, 2013, Doha,
Qatar.
Power, R. (2013d). Create your own mobile RLOs RLO. Retrieved from
http://winksite.mobi/robpower/mrlos

51. References
Quality Matters (2012). Quality matters self-review form. Retrieved from http://www.qualitymatters.org
Quality Matters (2013a). About us. Retrieved from https://www.qualitymatters.org/about
Quality Matters (2013b). Introduction to the Quality Matters program. Retrieved from
https://www.qualitymatters.org/sites/default/files/Introduction%20to%20the%20Quality%20Matters%2
0Program%20HyperlinkedFinal2014.pdf
tpack.org (2012). The TPACK image. Retrieved from http://www.tpck.org/
Tschannen-Moran, M., & Woolfolk Hoy, A. (2001a). Teacher efficacy: Capturing and elusive construct.
Teaching and Teacher Education, 17(7), 783-805.
Tschannen-Moran, M., & Woolfolk Hoy, A. (2001b). Teacher’s sense of efficacy scale. Retrieved from
http://people.ehe.osu.edu/ahoy/files/2009/02/tses.pdf