د. جوناثان ألان بلكر مركز سياسة التقويم والتعليم, أستاذ علم النفس التربوي والعلوم المعرفية في جامعة إنديانا الولايات المتحدة الأمريكية

1. Jonathan A. Plucker
University of Connecticut
IEFE 2013 – February 18, 2013

2. } Historical Background
} Strategic Versus Tactical Evaluation
◦ A Model of Intervention Success
} The Role of Evaluation Technical Assistance

3. } For many decades, the prevailing
education intervention model was to:
◦ Design promising interventions
◦ Implement them as best as possible
◦ Collect anecdotal evidence of success
◦ Revise the interventions or, more likely,
design new interventions

4. } It’s hard to tell, because so few interventions
were evaluated, and many were not evaluated
adequately.
} This made it hard to build on past success, as
“success” was defined by personal opinion
and not rigorous, 3rd-party evidence.

5. } In recent years, many countries have
acknowledged the importance of program
evaluation.
} Implementations of evaluation have been less
widespread.
◦ Many of those efforts have struggled.
} But why?

6. } U.S. Department of Education (ED) began to
emphasize evaluation roughly 10 years ago:
◦ All recipients of ED funds required to evaluate.
◦ Creation of several new programs to train people to
do high quality evaluations.
} After 5 years, not much to show for it:
◦ Required evaluations were either poor quality or
inconclusive
◦ No evidence of longer-term effects
} Their conclusion: They were being tactical but
not strategic.

7. } Strategic Level:
◦ Longer-term, systems-level thinking
◦ “How can I use evaluation to reach my goals for this
program?”
} Tactical Level:
◦ Shorter-term, task-level thinking
◦ “How can I best answer this specific evaluation
question?”
} Common framework in many fields (business,
technology, military), but not education

8. } BOTH strategic and tactical evaluation are
necessary for a high quality, high impact
evaluation program.
} In most countries, we are seeing the quality
of tactical evaluation increase, often sharply.
◦ U.S., China, The Netherlands, several Middle Eastern
countries
} Strategic evaluation is generally not
increasing.
} Charter school example: Poor tactics, but
poor strategy, too?

9. Desired
change
Interventions
Policies
Politics
Public
Opinion
Capacity
& Talent
Data
Systems

10. Managing &
Communicat
-ing Change
Sustaining
Change Through
Effective
Institutions &
Policies
Creating Capacity for
Change
Creating a Model for Change
Four Macro Strategies for Achieving Outcomes
in the National Education Plan
How has leadership preparation
changed to foster change
management?
How have institutions and
policies changed?
How much capacity was
created? Was it of high
quality?
Was the model
created? Does it look
promising?
(not really evaluation)
Tactical Evaluation Examples

11. Managing &
Communicat
-ing Change
Sustaining
Change Through
Effective
Institutions &
Policies
Creating Capacity for
Change
Creating a Model for Change
Strategic Evaluation
Examples
Was model designed in a
way that increased
capacity was achievable?
Have educator-
policymaker feedback
loops been created?
Are they working?
Do some policies foster
certain outcomes but
unintentionally discourage
other outcomes?

12. Managing &
Communicat
-ing Change
Sustaining
Change Through
Effective
Institutions &
Policies
Creating Capacity for
Change
Creating a Model for Change
Put it all together for a nice
mix of …
… Strategic
… and
Tactical
evaluation
strategies

13. } An often overlooked aspect of systemic
evaluation is technical assistance.
} Essentially, to what extent are systems in
place to promote high quality, successful
evaluations?

14. } They were doing good tactical work …
} … but had forgotten to put the necessary
systems in place to support evaluation.
} My team designed and implemented the
Evaluation Technical Assistance Continuum
that now serves as the guide for most ED
evaluation efforts.

15. 1. Program-
level capacity
building
2. Guidance to
implementers
3. Field-based
technical
assistance
4. Evaluator
Follow-up
5. Evaluation
synthesis

16. } Tuesday workshop on problem-based
learning, 10:00-11:30, Mecca Hall
} Wednesday workshop on creativity,
10:30-12:00, Dammam & Arar Halls
} I’ll also be at the University of Connecticut
Exhibit Hall booth Tuesday and Wednesday
afternoons.

17. University of
Connecticut
Neag School
of Education
Jonathan A. Plucker, Ph.D.
Endowed Professor of
Education
jonathan.plucker@uconn.edu
neag.uconn.edu/jonathan.plucker