ReSAKSS-AfricaLead Workshop on Strengthening Capacity for Strategic Agricultural Policy and Investment Planning and Implementation in Africa Safari Park Hotel, Nairobi, June 25th‐ 26th 2012

1. Result Based Monitoring and
Evaluation for Agriculture and Rural
Development
Safari Park Hotel, Nairobi on 25th- 26th June 2012
Stella Clara Massawe
ReSAKSS-ECA, ILRI

2. Session One

3. Learning Objectives Session 1
o Discuss the concept of Result Based M&E
o To discuss the role of M&E in agriculture and rural
development interventions
o Developing a theory of change in M&E context
o Practical Assignment

4. Monitoring
• Tracks what is happening within an intervention.
• Continuous process of observation, interpretation and institutional
learning
• Is an internal management tool whose purpose is to:
• to take timely corrective action in the day-to-day
management decisions
• Track progress, or lack thereof, in the achievement of
results.
• Facilitate subsequent evaluation

5. Evaluation 1
• Periodic
• Systematic assessment of ongoing or completed
interventions
• Looks at their design, implementation and results
• Deal with strategic issues such as: Performance, quality,
relevance, effectiveness, efficiency, and sustainability.
• Addresses various questions:

6. Evaluation 2
• Addresses various questions:
– Why: What caused the changes
– How: What was the sequence or processes that led to
successful (or not) outcomes
– Compliance/ Accountability: Did the promised activities actually
take place and as they were planned?
– Implementation process: Was the implementation process
followed as anticipated, and with what consequences?

7. Why M&E System for Agriculture?
• Accountability & Transparency: development effectiveness….national
SHs, donors, beneficiaries …..
• Learning: Decision making, learning……
– Key SHs: accountability & development effectiveness
– Management: evidence decision making, right strategy?
– Implementers: delivering on time? Budget? and target?
• Adaptive Management/ Managing for impact: Respond to changing
circumstances, increased understanding from knowledge generated
by M&E
• If done well contributes ++ to team morale and foster an atmosphere
of transparency and professionalism.

8. What to Monitor and Evaluate
• Best to develop at the beginning of the design process
• Theory of change indicates intervention-to-impact pathways
• Need to define key indicators along the pathways, including
their measurements, methods, and data requirements
• Laying out the theory of change for an intervention is
important when deciding what to monitor and evaluate

9. Underlaying Logic or Theory of change
• Theory of how an initiative leads to the desired results,
• Describes the causal logic of how and why a particular
intervention will reach its intended outcomes
• Often developed during project conceptualization.
• Involvement of stakeholders to put together a TOC
improve programme design

10. Thinking about impact pathways
• Example from the regional framework document
• Channels of impact on poverty and food security
• A key step in reaching the goals (reducing poverty and
increasing food security) is by improving the productivity of
existing assets and/or by accumulating more assets

11. Channels of impact: Examples
INTERVENTION AREA OUTCOMES / IMPACT CHANNELS
•Outcomes: (i) lower transactions cost, (ii) higher
marketing margins, (iii) lower postharvest losses, (iv)
Market Access and greater volume/value of trade
Trade Facilitation •IP: (i) direct via higher income, and (ii) indirect via higher
output-input price ratios and/or greater access to production
technologies
•Outcomes: (i) greater adoption of profitable technologies
and enterprises, (ii) higher productivity, output, and
NRM & Agricultural marketable surplus, and (iii) more sustainable production
Production activities
•IP: (i) direct via higher food supply and income, and (ii)
indirect via higher real income (lower food prices)
•Outcomes: greater protected productive assets and
Early Warning and investments in times of disaster
Disaster Preparedness •IP: (i) direct via reduced negative effect of disaster on
production, consumption, marketable surplus, and trade

12. Components of a Theory of Change Model
Should include the basic elements of theory of
change:
causal chain,
outside conditions and influences
key assumptions

13. A theory of change must
• Depict a sequence of inputs, the project or policy will use,
activities, outputs, outcomes and impacts expected
• Identify events or conditions that may affect obtaining the
outcomes
• Identify the assumptions the programme is making about
cause and effects
• Identify critical assumptions that, based on the policy and
environmental context and review of literature, the evaluation
needs to examine

14. Components of TOC
Your Planned work (Implementation)
• Describes what resources you think you need to implement
your program and what you intend to do.
– Resources/ Inputs: include the human, financial,
organizational, and community resources a program has
available to direct toward doing the work.
– Activities: What we do. Actions taken to convert inputs to
outputs. Stated with a verb (“market”, “train”, “deliver”).
– Outputs: What we produce. Are the direct products (tangible
goods and services) resulting from program activities. These
are usually expressed as nouns.

15. Components of TOC
Your Intended Results
• All of the program’s desired results
– Outcomes: Why we do it. Are the specific changes in
program participants’ behavior, knowledge, skills, status and
level of functioning.
– Results likely to be achieved once the beneficiaries use
project outputs (usually achieved in short-to medium term)
– Final outcomes/ Impact: Long-term changes that result from
accumulation of outcomes.
– Is the fundamental intended or unintended change occurring
in organizations, communities or systems as a result of
program activities. Can be similar to strategic objectives.

16. Modeling Theory of change
• Gertler et al., 2010 indicates that theory of change can be
modeled in various ways e.g.
– Using theoretical models,
– Logic models,
– Logical frameworks and outcome models,
– Outcome models,
– Result chain

17. A result Chain
Goal • Long-term, widespread improvement
(Impacts) in society
Supply Results
+
Supply • Intermediate effects of outputs on
Outcomes clients
• Products and services produced
Outputs
Implementation
Supply • Tasks personnel undertake to
Side Activities transform inputs to outputs
• Financial, human, and material
Inputs resources

18. Results-Based Monitoring:
Agriculture Extension Programme
Goal • Higher yields, higher income levels;
(Impacts) increase access to food
Outcomes
• Increased better farming skills; increased
adoption of new technologies
Outputs
• Number of farmers visited, number trained in
field days
Activities
• Farm visits, farmer field days, distribute
handouts
Inputs • Extension workers, materials

19. Theory of Change Template
Strategies (5) Assumptions (6)
Influential factors (4) Problem Issue (1) Desired results (3)
• Foreseen budget for Low productivity, poor maize (Outputs, outcomes and
fertilizer subsidies is harvest, inadequate food, need impacts)
available in time for food aid
• Fertilizer is more
• Climatic conditions accessible
• Government • Fertilizer is more
policies affordable
• Macro-economic Community Needs/ Assets (2) • use of fertilizer increases
conditions • Almost 38% of the population
needed emergency food aid. • soil able to support
• Programme increase food availability
implementation • Fertilizer is not affordable • Reduce hunger
capacity
• Increased income (from
sales)

20. Session two

21. Session 2: M&E framework
• To discuss the meaning of M&E framework
• To discuss the relationship between TOC and
M&E framework
• Discuss examples of M&E frameworks
• Developing an overarching FW

22. Participatory Discussion
• What is the M&E system?
• Using the case of Agriculture M&E framework in your
country

23. What is a M &E Framework?
• Builds on the Theory of change
• Lists each indicator from the program logic model
• Presents how indicators are defined and calculated
• Defines Who, What, When, How by:
• Identifying who is responsible for data collection
• What the data source is ( the methods we use to collect
data, report and disseminate)
• When and how often an indicator is measured

24. M&E Frameworks (1)
• M&E Framework outlines the plan for monitoring in concrete steps
providing the who, what, where and when
• Define relationships among inputs, activities, outputs, outcomes,
impacts and external factors.
• Assist with thinking through programmatic strategies, objectives
and planned activities, and whether they are indeed the most
appropriate ones to implement.
• Plan about implementation of M&E activities

25. M&E Frameworks (2)
• Developing a corresponding M&E plan that acts as a monitoring tool
by defining how information from the programme will be tracked.
• Develop the framework and plan before activities are implemented.
• Different kinds of interventions will need different kinds of
frameworks, tools and indicators.
• If M&E activities and tools are considered and built into
programmatic work from the start, the resource and time burden is
minimized.

26. Designing, Building and Sustaining
agriculture M&E system
•Borrow from the model: Ten steps to Designing, Building &
Sustaining a Result-Based M&E System –Kustel and Rist,
2004

27. Step 1: Readiness assessment should be
conducted to understand:
• Willingness of the stakeholders
• Incentives and Demand for M&E, what disincentives may hinder
• Organizational Capacity for M&E (technical, financial,
managerial)
• key players that will be responsible for program implementation.
• Roles and responsibilities
• Barriers
M&E technical staff, project managers and senior managers
involved in strategy development should be involved

28. Step 2: Agreeing on Specific Outcomes to
Monitor and Evaluate
• Generate interest in assessing the outcomes and impacts in
addition to the implementation issues
• Need to be derived from strategic priorities of the country
• Involves all relevant stakeholders to arrive at a mutually agreed
set of goals and objectives for the program.
• Linkages: Programme strategy-- theory of change—translate
into key set of indicators
• Clearly setting outcomes-and deriving indicators based on
them-essential to designing and building result-based M&E
system
M&E technical staff, project managers and senior managers
involved in strategy development should be involved

29. Step 3: Select Key indicators to monitor
outcomes
• Numerical measurement that tracks progress (or not) toward
achieving an outcome
• SMART---Specific, Measurable, Achievable, Relevant and Time-
bound
• Decide how trends in agreed outcomes will be measured.
– For example, if children’s schooling were an important outcome for a
program: use the proportion of children enrolled in school, test scores,
school attendance, or another metric?
• Indicator development drives: data collection, analysis and reporting
• The costs of measurement need to be considered
• Where possible reduce costs by using already available indicators

30. Minimum set of indicators (2)
• Selecting indicators
• All indicators should be neutrally stated, not as “increase in” or
“decrease in”
• Feasibility of deploying each indicators: Completing each cell in the
matrix below
Matrix for indicators
indicator Data Data Who will Frequency Cost to Difficulty Who will Who will
source collectio collect? of collect to analyze use the
n collection data collect and data
method data report
data

31. Step 4: Baseline data, where are we today
• Measurement of initial conditions, preprogram data
• Very helpful in assessing the program’s impact
• Provided qualitative and quantitative information
• Sources of baseline data can be primary or secondary
data
• Decide who is going to collect data and how, develop
data collection instruments

32. Step 5: Planning for Improvements
Establish Realistic Result Targets:
• These targets can also be used to monitor results.
• This effort includes setting periodic targets over time (for
example, annually or every two years).
• Consider the duration of the likely effects of the program
• Important to consider other factors (political..)

33. Step 6: Monitoring results
• Involves the collection of good-quality data.
• Data sources: Primary,& Secondary data sources, Project
data, Spatial data
• Establish data collection system ---------Appropriate
forms for record & data management systems …Staffing:
enumerators, data entry
• Leverage Partnerships in data collection management,
analysis and dissemination

34. Step 7: Using Evaluations
• Evaluation important in supplementing information
require to gauge whether there are achievements with
regard to impacts
• Evaluation helps in in indicating whether:
– We are doing right things (strategy)
– We are doing things right (operation)
– There are better ways of doing it (learning)

35. Data Collection Methods
Panel
Key informant Surveys
interviews
Conversation
with Focus
One-Time
concerned Group
Survey
individuals Interviews
Participant
Community Observation Direct
Census
Interviews observation
Reviews of
official records
(MIS and admin Field
Field data) experiments
visits Questionnaires
Informal/Less Structured Methods More Structured/Formal Methods

36. Steps 8-10
• Step 8: Reporting: Feedback sessions & strategic
communication activities, appropriately targeted to
different audiences--------- M&E, COM, KM, PRO, MA Reports,
newsletters, issue briefs, website, CDs
• Step 9 & 10: Using findings and sustaining the system

37. Session Three
Case Study: CAADP M&E
Framework

38. Rationale for common M&E framework:
The need to assess impact more broadly, adding up to more than of
progress and performance associated with individual pillars:
(i) to comprehensively assess whether and how investments and
policies surrounding the CAADP implementation are having their
desired impact on targeted growth and poverty reduction goals
(i) individual pillar M&E frameworks focus on specific targets
without considering how they interact with those of other pillars
(e.g. through price effects) to affect the overall CAADP goals and
objectives

39. Idea behind M&E framework

40. Rationale for common M&E framework
• A mechanism by which processes put in place, commitments etc can be
regularly and transparently measured against stated targets and if
necessary can lead to the revision of the processes
– Are countries achieving the targeted growth rates? (6%)
– Are countries investing at the targeted level? (10%)
– Are these investments having their intended impacts on poverty and hunger?
• To bring cohesion across the different systems being developed to track
specific components of CAADP, including the individual CAADP pillar
M&E systems, APRM and MAF
• To inform the review processes established by CAADP-PP (mutual, peer
and progress reviews)
• To further inform policy-making and dialogue

41. http://www.resakss.org/

42. THANK YOU

43. References
• World Bank (2004). Designing and Building a Results-Based Monitoring and Evaluation System:
A Tool for Public Sector Management, A Workshop for Government Officials and Their
Development Partners, Presentation made on 20th October, 2004
• Gertler,PJ., Martinez, S.,Premand,P., Rawlings,LB., Vermeersch, CMJ. (2011) ImpactEvaluation
in Practice, The International Bank for Reconstruction and Development / The World Bank,
Washington DC., USA
• Kusek JZ. and Rist RC (2004).Ten steps to a results-based monitoring and evaluation system : a
handbook for development practitioners, The International Bank for Reconstruction and
Development / The World Bank, Washington DC., USA
• Benin, S.; Johnson, M.; Omilola, B.; Beintema, N.; Bekele, H.; Chilonda, P.; Davis, K.; Edeme,
J.;Elmekass, A.; Govereh, J.; Kakuba, T.; Karugia, J.; Makunike, R.; Massawe, S.; Mpyisi, E.;
Nwafor, M.;Olubode-Awosola, F.; Sanyang, S.; Taye, B.; Wanzala, M.; Yade, M.; Zewdie, Y.
Monitoring and Evaluation (M&E) System for the Comprehensive Africa Agriculture
Development Programme (CAADP). ReSAKSS Working Paper No. 6. International Food Policy
Research Institute (IFPRI). Washington, D.C.:

44. Exercise 1
Constructing and Using a Theory of Change
Task objective: to enable the participants practice how to develop a theory of
change for their programme
Estimated time: 30 minutes

45. Task 1
• Consider a programme you are currently working
with or one which you are familiar but not directly
involved. Develop a graphic showing of the theory
of change for this programme or project.
• Be sure to identify the assumptions underlying the
programme or project
• Use simple theory of change template (please only
include a few outputs and activities)
• Reference pages 150-171