This study builds on and contributes to work in teacher education and educational technology, in international development contexts. Recent reviews, funded by the UK Department for International Development (DFID) have examined the characteristics of teacher education programmes (Westbrook et al. 2013) and educational technology programmes (Power et al. 2014), that show evidence of impact on teaching practice or learning outcomes. These both illustrate the importance of a strong focus on improving the quality of classroom practice in programme design, and both indicate some of the key characteristics of effective programme support for teachers. But in both reviews, the studies reviewed present problems of evidence. Such evidential problems arise in relation to reporting changes in: attitudes and understanding; teaching and learning practices; and learning outcomes. In this article, we draw particular attention to evidence of classroom practice: in terms of extensiveness, of methodology, and of understanding the relationships between the variables considered. As such, the purpose of this article is to provide insight into three inter-related issues: the methodological challenges - of rigour, systematic observation, and extensiveness; the practical challenges - of human capacity for research activity, geographical remoteness, and cost; and the evidence requirements of different audiences - donors, policy makers, practitioners and the academic and research communities. This is done by considering these three issues, through a case study of English in Action, a large scale teacher education programme in Bangladesh, in which Educational Technology plays a central role in supporting both teacher professional development, and new classroom practices. There are several implications from the recent reviews and the case study, that lead us to argue for greater development of evaluation approaches for classroom practice, based upon rigorous, systematic observation (using standardised observations, of objective behaviors). Such approaches must be capable of deployment at scale, and reliable implementation through relatively inexperienced field researchers, available and affordable in country. This may suggest certain kinds of large scale quantitative observation, that are rare in the global north. Is there an opportunity, for a collective accumulation of data, to deepen our basic understanding of classrooms and the actors within them?

1. Evaluating classroom practice: a critical analysis
Approaches to evaluation in large scale teacher education
or education technology programmes,
in international development contexts
Tom Power: tom.power@open.ac.uk
Bob McCormick: r.mccormick@open.ac.uk
#Edtech4dev

2. DFID / EPPI Literature Review:
…Teacher Education in Developing Countries
• pedagogy, curriculum, teaching practices and
teacher edu- cation in developing countries
• 2000 articles > 489 through appraisal > 54 on TE >
11 provide evidence
• “With few exceptions [studies did not holistically
look at] changes in teachers’ attitudes and beliefs,
their knowledge (both content and PCK) and their
practices…
• …Even here, students’ learning outcomes as a
result of ITE or CPD were often not obtained for
reasons of scale and feasibility. Thus, evidence on
the impact of training was only partially captured in
most studies, highlighting a need for more holistic
and robust evaluations of teacher education
initiatives” (p31)
Westbrook et al., 2013
Westbrook, J., Durrani, N., Brown, R., Orr, D., Pryor, J., Boddy,
J., and Salvi, F. (2013). Pedagogy, curriculum, teaching practices
and teacher education in developing countries: final report.
Education rigorous literature review, EPPI-Centre, Social Science
Research Unit, Institute of Education, University of London.
https://eppi.ioe.ac.uk/cms/LinkClick.aspx?fileticket=E93CRI7ONw
A%3D&tabid=3433

3. • Tatto:
Poor outcome measures; no educational theory of
change; economist driven evaluations
“field lacked a well-developed research
infrastructure to adequately study teacher education
trajectories” (p3)
• Cordingley:
• 14 studies;11 used; 6 had data on student
outcomes.
• Most had qualitative observational data; variety
of approaches limit comparability.
• evidence is mostly from the USA; one study
(Namibia) from LEDC.
Tatto, 2013
Tatto, M. (2013). The role of research in
international policy and practice in
teacher education. Research and teacher
education: the BERA-RSA inquiry,
BERA.
http://www.bera.ac.uk/wp-
content/uploads/2014/02/BERA-Paper-2-
International-Policy-and-Practice-in-
Teacher-Education.pdf
Cordingley, 2013
Cordingley, P. (2013). The contribution of
research to teachers’ professional
learning and development. Research and
teacher education: the BERA-RSA
inquiry, BERA, London.
http://www.bera.ac.uk/wp-
content/uploads/2013/12/BERA-RSA-
Research-Teaching-Profession-FULL-
REPORT-for-web.pdf
BERA Reviews:
The role of research in international policy and practice:
in initial teacher education; in CPD

4. DFID review: Educational Technology
in low to lower-middle income countries
• Review of over 80 studies (45 research
documents; 20 literature reviews; 18 grey literature
reports) of Educational Technology use in schools,
for teaching and learning, in low to lower-middle
income countries.
• Many studies present no evidence of classroom
practice or learning outcomes
• Only 3 studies present quantitative evidence on
classroom practice
• IRI, Mali: (Ho & Thukral 2009, p.32)
• Bridge IT, India: (Wennerstan et al, 2012)
• English in Action, Bangladesh: (EIA 2011, 2012,
2014)
Power et al., 2014
Power, T., Gater, R., Grant, C., and Winters, N. (2014). Educational
technology. Topic guide, Health and Education Advice and Resource
Team, Department for International Development, London, UK
#Edtech4dev

5. Common problems of evidence
relating to classroom practice
1. Locus of Evidence
2. Problems of Scale
3. Weak or absent theories of change
4. Limited focus on teaching or learning practices
5. Predominance of qualitative methods
6. Limited accumulation of data

6. Common forms of Teacher Development
6
e.g. context: UP 2012:
* RMSA introduction; NCERT maths & Science exams
* 449 new schools; 402 schools ‘uplifted’; 26,000 teachers to train through INSET

7. 2. English in Action:
Upscaling a school-based, mobile enhanced,
Teacher Development Programme
See: Power et al (2012):
The Curriculum Journal, 23(4):503–529
EIA Pilot approach
mobile
materials
Conceptual framework
• teachers identity and
expertise is developed in
context of local practice
• the school is the main
setting of professional
learning
• cultural artefacts and tools
mediate learning
• support mechanisms
integrated school-based
peer supportSocial Practice Theory. See for example:
Chaiklin and Lave, 1993; Vygotsky, 1962;
Bruner, 1996; Sen, 1999

8. …emphasis on local peer support,
with practice expertise via
(offline) mobile technology
Pilot
mobile
materials mobile
materials
Upscale
local
peer
support

9. Westbrook et al., 2013
Westbrook, J., Durrani, N., Brown, R., Orr, D., Pryor, J., Boddy,
J., and Salvi, F. (2013). Pedagogy, curriculum, teaching practices
and teacher education in developing countries: final report.
Education rigorous literature review, EPPI-Centre, Social Science
Research Unit, Institute of Education, University of London.
https://eppi.ioe.ac.uk/cms/LinkClick.aspx?fileticket=E93CRI7ONw
A%3D&tabid=3433
DFID / EPPI Literature Review:
…Teacher Education in Developing Countries
How can teacher education and guidance materials best support
effective pedagogy?
Westbrook et al EIA School Based
Teacher Development
1. teacher peer support pairs of teachers in
school; local clusters
2. aligned with T needs;
follow-on support
school-based, classroom
practice focus; 1 year
3. support for T from
head teachers
explicit role for head
teacher
4. alignment with
curriculum & assessment
linked to national
curriculum and textbook
5. mobile technology +
high quality TD & curriculum
materials

10. Teacher & Student
Perceptions
Classroom Practices Learning Outcomes
Teachers report improved
competence (96%-99%) and
confidence (88-89%) in using
English.
student talk time increased
substantially (to 27%
primary; 24% secondary).
34% more Primary students
achieve Grade 1 or above
Most teachers (63-66%)
strongly agree has impacted
teaching practice.
student talk in target
language increased
substantially (to 91%
primary; 87% secondary).
20% more Primary students
achieve Grade 2 or above
89% of all teachers say focus
of lessons is now on student
communication and
interaction
substantial increases in
observations of student talk
in pairs or groups (18%
primary; 28% secondary).
11% more Secondary
students achieve Grade 1 or
above
students report regularly
speaking in English (79%-
80%)
teachers talked less (45-
48%) but used target
language more (76-87%).
14% more Secondary
students achieve Grade 2 or
above
EIA Research Findings 2014
www.eaibd.com > publications > research reports

11. Conclusion
• The illustration addresses common problems of evidence
relating to classroom practice, at least partially.
• Limitations (to be addressed by quasi-experimental studies
2015):
• absence of control groups;
• of directly comparable baseline of practice;
• of standardised outcome measures (e.g. effect sizes).
• Although methodological issues partially addressed, substantive
issues remain: how do key components of teacher development
inter-relate, and what is their relative effect?