WASH basins workshop on IWRM_ME&L

Monitoring, Evaluation and Learning
अनुवीक्षण, मूल्यांकन एवां शिक्षण
Importance in Project Management
कार्यक्रम प्रबंधन मे महत्त्व
22 November 2019 1
WASH Basins
Program
- Yatin RS Diwakar
PhD Scholar
Knowledge Sharing Workshop on
Integrated Water Resource Management
एकककृ त जल सांसयधन प्रबांधन
. ME&L Measuring Change Indicators Evaluation

ME&L
Linkage to SDGs
Definitions
MEL in project cycle
22 November 2019 2

Monitoring, Evaluation & Learning
and Sustainable Development Goals
• Achieving SDGs needs collaboration, learning from past
experiences, regular updating of situation for course
correction.
• 17 goals – 232 indicators- need to measure and keep up
• Understand what worked, how and why?
• Also understand what did not work and why?
• To improve program delivery
• Showcases effectiveness to community and funders
• Improves frontline practice with community
• Allows critical self-assessment and improvement
• Can build knowledge for activities out of program ambit.
22 November 2019 3
• For assured success, learn from each success and mistake and improve

Monitoring
• Monitoring is
• Regular collection and analysis and distribution of
information for the surveillance of progress of the
project’s implementation.
• Project monitoring is the collection of data prior to, and
during, the project. These data, when analysed, pinpoint
progress or constraints as early as possible, allowing
project managers to adjust project activities as needed.
Monitoring is a continuing process throughout project
implementation and often extends beyond project
completion.
22 November 2019 4
• United Nations Environment Program, 2008

Evaluation
• Evaluation is
• Systematic and Impartial assessment of an activity, project,
programme, strategy, policy, topic, theme, sector, operational
area, or institutional performance.
• Analyse achievement of expected and unexpected results by
examining the results chain, processes, contextual factors,
and causality using appropriate criteria such as relevance,
effectiveness, efficiency, impact, and sustainability.
• An evaluation should provide credible, useful evidence-based
information that enables the timely incorporation of its
findings, recommendations, and lessons into the decision-
making processes of organizations and stakeholders
22 November 2019 5
• United Nations Evaluation Group guidelines, 2016

22 November 2019 6
Char. Monitoring Evaluation
Subject: Address operational issues Focused on strategic aspects
Character: Continuous, regular, systematic Incidental, flexible subject & methods
Client: Program managers/ implementors Implementers, donors and beneficiaries;
interested public and research community
Aim: Improve efficiency & effectiveness;
transform activities into results/ outputs
Explore intended & unintended results;
formulate recommendations
Focus: Check progress of planned activities,
improve implementation efficiency &
effectiveness
Assess achievement of results, relevance,
effectiveness, impact and sustainability of
activities and their contribution to results.
Objective: Identify & resolve problems, assess
progress towards objective
Verify developmental hypothesis, document
successes & lessons learned
Time Continuous during implementation of a
project, programme, plan, or strategy
during an activity (mid-term evaluation) or
upon completion (ex-post evaluation)
Content Progress attained measured against the
set indicators
Fulfilment (degree of and quality) of the
indicators

ME&L in Project Cycle
22 November 2019 7
http://homepages.iitb.ac.in/~yatindestel/reports/TD%20694%20report%20Develo
pment%20Evaluation.pdf
Study and Goal Setting
(Planning)
Initial Assessment of:
Community, processes,
problems, and
opportunities
Identify solutions and
Design
Appraisal and program
design,
Stakeholder analysis,
Identify program
objectives/ expected
results
Establish M&E
Framework
Identify performance
indicators at diff. levels
Establish M&E schedule
Draw risk management
plan
Conduct baseline
Set up monitoring systems
Implementation
Continuous monitoring:
Ongoing data collection
Progress reports, reviews
Mid-term evaluation
Modify program design
and activities in response
to feedback
Evaluation
End-term evaluation:
Assess relevance,
efficiency, unintended
consequences,
sustainability
Distil key lessons learned
Operation and Follow
up
Review and respond to
evaluation findings
Distribute findings widely
Apply lessons to current
and future programs

Measuring change
Importance
Logical Framework Approach
Output and Outcome
22 November 2019 8

Importance on Measuring Change
• “Measurement is the first step that leads to control
and eventually to improvement. If you can’t
measure something, you can’t understand it. If you
can’t understand it, you can’t control it. If you can’t
control it, you can’t improve it.”
• Activities – What to do? Why to do it? How to do it?
• Measurement – What to measure? How to measure?
• Planning before any implementation
22 November 2019 9
H. James Harington (performance and quality improvement guru)

Logical Framework Approach
• Standard tool for activity planning
• Links objectives of a program to activities
• Establishes connection between input – activities-
outputs
• Documents expected outcomes
• Defines indicators to measure each activity,
outputs, outcomes
22 November 2019 10

http://archive.ssw
m.info/category/pla
nning-process-
tools/implementati
on/implementation
-support-
tools/project-
design/logical-f
22 November 2019 11

Output and Outcome
• Every activity gives some immediate output
• Output are solely dependent on input and activities
• Outcomes are long term; multiple activities affect
each outcome
• Outcomes are influenced by social context, other
programs; require behaviour change
22 November 2019 12

22 November 2019 13
• Table 4 -Outputs vs. Outcomes
https://www.researchgate.net/publication/281489950_Impact_of_Project_Governance_on_Benefits_Realization_Mana
gement_and_Project_Success_Towards_a_Framework_for_Supporting_Organizational_Strategy

Example
• Input – planning, fund and labour
• Activity – construction of water harvesting structures
• Immediate Output – No. of bunds, ponds, etc. formed
• Intermediate Output – volume water stored, recharged
• Immediate Outcome – increased water availability
• Intermediate Outcome – improved irrigation, assured
drinking water availability
• Final Outcome – same as broader objectives of
program
• Impact – Climate resilience, income increase, health
improved
22 November 2019 14

Theory of Change
22 November 2019 15
• ADB’s ToC for Drinking Water interventions in West Bengal

Literature Review
• We are standing on shoulder of Giants
• 30 years an little improvement – making same
mistakes? Reinventing the wheel? Loss of rigour?
• Whatever we are doing, someone has done and
published before
• Learning – learn from others and document your
work to help others learn
• Read before writing
22 November 2019 16

Indicators
What are Indicators
Good indicators
Monitoring plan
22 November 2019 17

Indicators
• A parameter, or a value derived from parameters,
which points to/ provides information about/ describes
the state of a phenomenon/ environment/ area with a
significance extending beyond that directly associated
with a parameter value
• Indicators perform many functions
• Lead to better decisions and effective actions by simplifying,
clarifying and making aggregated information available
• Help incorporate physical and social science knowledge into
decision-making; measure and calibrate progress toward
sustainable development goals.
• Provide an early warning to prevent economic, social and
environmental setbacks.
22 November 2019 18
• Lecture notes, Prof. Bakul Rao

Good Indicator
• Measurable
• Representative of the issue/ system being assessed
• Able to detect human induced change from natural variations
• Reproducible & repeatable – based on critical attributes of the
issue/ system
• Responsive/ sensitive to environmental change
• Helpful in relating causes, effects and responses
• Developed within a consistent analytical framework
• Scientifically credible and robust
• Consistent in standards for data collection, analyses and data
management
• Have statistical integrity
22 November 2019 19
• Lecture notes, Prof. Bakul Rao

Monitoring Plan
• A monitoring plan in should be prepared to ensure:
• All parties understand what is monitored and why
• Monitoring is seen as tool for learning and
improving work, not as a demand from authority
• Project indicators defined are relevant
• Information is collected regularly throughout the
project
• Operational environment is considered in the plan
• Corrective measures are planned
22 November 2019 20
• https://itseopiskelu.kepa.fi/en/node/501

22 November 2019 21
• https://itseopiskelu.kepa.fi/sites/default/files/5.3.3%20Monitoring.PNG

Tracking sheets
• Process monitoring – activities – updates
• Monthly physical – financial progress – basic
tracking sheet
• Micro-planning, useful for implementors only
• Examples
• HH survey – master list of all HH, date of survey, name of
surveyor, remarks
• DPR submission – list of all DPRs to be submitted with
date of submission, current status, total area/ amount
and other important common parameters
22 November 2019 22

Evaluation
Evaluation Criteria
Summary
22 November 2019 23

Evaluation Criteria
• Minimum set of questions that every evaluation
should answer
• Relevance
• Effectiveness
• Efficiency
• Impact
• Sustainability
• Equity
• Coherence
22 November 2019 24

Relevance
“The extent to which the aid activity is suited to the
priorities and policies of the target group, recipient
and donor.”
• To what extent are the objectives of the programme still
valid?
• Are the activities and outputs of the programme
consistent with the overall goal and the attainment of its
objectives?
• Are the activities and outputs of the programme
consistent with the intended impacts and effects?
22 November 2019 25
• https://www.oecd.org/dac/evaluation/49756382.pdf

Effectiveness
“A measure of the extent to which an aid activity
attains its objectives.”
• To what extent were the objectives achieved/ are likely
to be achieved?
• What were the major factors influencing the
achievement/ non-achievement of objectives?
22 November 2019 26

Efficiency
“Efficiency measures outputs - qualitative and
quantitative - in relation to inputs. It is an economic term
which signifies that aid uses least costly resources
possible in order to achieve desired results. This
generally requires comparing alternative approaches to
achieving same outputs, to see whether the most
efficient process has been adopted.”
• Were activities cost-efficient?
• Were objectives achieved on time?
• Was the programme or project implemented in most efficient
way compared to alternatives?
22 November 2019 27

Impact
“Positive and negative changes produced by a
development intervention, directly or indirectly, intended
or unintended. This involves main impacts and effects
resulting from the activity on local social, economic,
environmental and other development indicators. The
examination should be concerned with both intended
and unintended results and must include positive and
negative impact of external factors, such as changes in
terms of trade and financial conditions.”
• What has happened as a result of the programme or project?
• What real difference has the activity made to the
beneficiaries?
• How many people have been affected?
22 November 2019 28

Sustainability
“Sustainability is concerned with measuring whether
the benefits of an activity are likely to continue after
donor funding has been withdrawn. Projects need to
be environmentally as well as financially sustainable.”
• To what extent did the benefits of a programme or
project continue after funding ceased?
• What were the major factors which influenced
achievement/ non-achievement of sustainability of
programme or project?
22 November 2019 29

Equity
• Equity refers to provision of assistance/ benefits,
and participation of all groups of stakeholders,
especially the vulnerable ones, such as women,
children, elderly, tribals, other vulnerable
communities.
• Who benefits?
• Why a group received/ excluded?
• Disaggregate data by sex, socio-economic categories,
ethnicity, etc.
•
22 November 2019 30

Coherence
• “Coherence refers to consistency/complementarity and
freedom of contradiction of guiding general principles
of different topics, such as development, trade,
military, humanitarian aid, analysing whether all
aligned or not.”
• How was coordination (coherence) achieved, and/or why was
there a lack in coherence?
• What political factors were specifically responsible for the
coordination of assistances or relief items or what made the
latter more difficult?
• Is coherence necessary or feasible in the present situation at
all?
22 November 2019 31
• https://www.oecd.org/development/evaluation/dcdndep/47069197.pdf

22 November 2019 32
• Prof. Bakul Rao
and Yatin
Diwakar, work
in progress

Thank You
Mail – yatindestel@iitb.ac.in
Whatsapp – 9823578400
http://homepages.iitb.ac.in/~yatindestel/
22 November 2019 33

WASH basins workshop on IWRM_ME&L

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to WASH basins workshop on IWRM_ME&L

Similar to WASH basins workshop on IWRM_ME&L (20)

Recently uploaded

Recently uploaded (20)

WASH basins workshop on IWRM_ME&L