Hélène Giacobino (Abdul Latif Jameel Poverty Action Lab) előadása a Haza és Haladás Alapítvány szociálpolitikai konferenciáján - 2012. március 29.

1. Impact Evaluations
for
Social Inclusion
Hélène Giacobino
Director
J-PAL Europe
Jbjb;kjbkjbkb

2. The Need for Evaluation
• We have little hard evidence on key questions
– What is the most cost-effective program to reduce
unemployment? What is the real impact of microcredit?
• Evidence is important:
– for maximizing the impact of limited resources
– to give donors and policy makers evidence to select better
programs
• Evidence provides an objective platform for debate
• Evaluations sometimes demonstrate that
conventional wisdom needs to be rethought

3. Why Are Programs Not Evaluated?
• Partisans want to show their program had an
impact
– Further funding, re-election, promotion, depend on it
• Evaluation is not that easy
– Participants are often very different from non-
participants
• Often programs are implemented in specific areas, at
a specific moment for a specific reason
• Volunteers are often more motivated or better
informed or…
– But we cannot observe the same person both:
• exposed and
• not exposed to the program

4. J-PAL: Jameel Poverty Action Lab
• A network of researchers
at universities around the
world
• Focused on randomized
evaluations
• Founded in 2003 by Esther
Duflo and Abhijit
Banerjee, MIT Professors
of Economics

5. Build
capacity
J-PAL
Improve
Evaluate
social the lives
programs of the poor
Disseminate
the results

6. J-PAL today: 5 Offices & 64 Affiliates

7. J-PAL Evaluations around the World
• 318 randomized evaluations in 52 countries
• About 162 are completed

8. J-PAL Europe
• Started in 2008 with 9 affiliates
• Today: 18 affiliates (Belgium, Denmark, Germany,
France, United Kingdom, Sweden)
• About 50 projects:
– Many in developed countries
– Mostly ongoing

9. How to evaluate the impact of an idea?
• Implement it on the ground in the form of a real
program, and see if it works
• Common approaches:
– Before and after (But many things happen over time?)
– Participants vs. Non-participants (But are they different?
More motivated? Live in a different region?)
• Need an adequate comparison group
– individuals who, except for the fact that they were not
beneficiaries of the program, are similar to those who
received the program

10. Randomized Evaluation
• Randomized evaluations (or RCTs) allow us to
identify impact rigorously
– People are randomly assigned to a treatment group, who
receives intervention, or to a control group
– Program beneficiaries are not more
motivated, richer, educated, than non-beneficiaries, so
changes in outcomes are purely because of program
– Gives nice clean results everyone can understand—no
fancy econometrics: rigor and transparency

11. Basic set up of a randomized evaluation
Not in
evaluation
Target
Population
Treatment
Group
Sample Random
Population Assignment
Control
Group
Based on Orr (1999)
11

12. Randomized assignment
• This method works because of the law of large
numbers
• Both groups (treatment and control) have the same
characteristics, except for the program
• Differences in the outcomes can confidently be
attributed to the program
• We can use this knowledge to set policy
– Identify programs that actually solve social problems
– Make government policies and programs more effective

13. The question of ethic
• Resources are very limited any way
• This random assignment is usually seen as very fair
• Inefficient programs are not ethical!
• Every project has to go through an ethic committee
• J-PAL never works on projects if the cost of the
evaluation means less beneficiaries. After the
evaluation is over, if necessary, the control group will
also get the program

14. La Mallette des Parents

15. La Mallette des Parents (the parent'stoolkit)
Questions:
• Is it really possible to improve parents’
involvement ?
• Has increased parental involvement any effect on
children?
• Does the effect on program participants spread
out on other families?

16. The program
• Children and their parents in first year after primary
school in low-income schools
• A toolkit:
– a booklet with thematic guidelines
– 3 meetings with the parents
– A DVD in 10 different languages
• Cost: 1000€ per school (7€/child)

17. Messages to the parents
• All parents can help their children
• Work outside of school is extremely important for
success
• Parents should be involved in their children's
homework
• Children need to feel that their parents understand
how school functions and care that they adhere to
the demands of teachers and administration

18. Protocol: Four groups
Test Classes Control classes
Volonteer Volonteer
Compare
Non Non
volonteers Compare volonteers

19. Parents' involvement
• More interactions with school:
– Meetings with teachers
– Participation in parents' organizations
• Helped more their children at home
• No more difference in involvement between blue-
collar and white-collar families

20. Childrens' behavior
• Large improvement, even for classmates whose
parents were not volunteers
• 34% less likely to be sanctioned for disciplinary
reasons
• Similar results in reduction of absenteeism
• Cognitive achievement:
– Limited but significant impact in test scores in
French
– No impact in Math

21. Policy implication
• Simple and inexpensive program
• Rigorous evaluation: can convince schools or
governments that such action is worth taking
• Ongoing generalization in France
• New program launched for children in the last class
of compulsory school

22. Policy lessons
• Directed parent discussion groups are an effective
policy tool for increasing parental involvement, even
in underprivileged area
• Increasing parental involvement and awareness of
school structure improved student behavior and
positively impacted learning
• Even though only a small fraction of parents choose
to participate, the benefits of their involvement were
felt by all children in the classroom.

23. Other Programs…
• Counseling the unemployed
• Counseling and job placement for young graduate job
seekers
• Counseling welfare recipients
• Discrimination in hiring and anonymous CVs
• Facilitating youth's access to apprenticeships and
encouraging them to complete them
• Small business training and loans for aspiring
entrepreneurs in disadvantaged neighborhoods
• Supporting 18-25 year-olds through long-term mentoring
plus financial assistance
• …

24. Scale-ups: million of lives improved

25. More information
www.povertyactionlab.org
www.povertyaction.org
www.worldbank.org/dime
www.worldbank.org/impactevaluation
www.3ieimpact.org
www.usaid.gov/div

26. • Many thanks !