1. VALUE CHAIN MONITORING
AND EVALUATION GUIDE
MODULE
8
Integrating Other Design
Features into the M&E System

2. GETTING
STARTED
1
May 22, 2012

3. GETTING
STARTED
2
May 22, 2012

4. MATERIALS / INPUTS
RECOMMENDED
All M&E system design
pieces developed to this
point.
Senior M&E staff and/or
country program M&E staff
working on related
3
May 22, 2012
initiatives

5. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
4

6. STEP-BY-STEP GUIDE:
STEP 1
Data Collection Instruments:
•Must be developed for every level - observation & measurement
•Must be developed in a participatory way, in collaboration with future
users:
•Front-line staff members using them routinely
•Project evaluators designing baseline, midterm and endline
•Important that these tools are developed in local languages and tested for
accuracy
•Must be accessible to project participants, staff and partners – whoever
will be using them
5
May 22, 2012

7. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
6

8. STEP-BY-STEP GUIDE:
STEP 2
Pilot Test
•Should be done prior to full rollout
•Identifies inappropriate, poorly worded,
or mistranslated questions
•Validates the existing coding
scheme
•Identifies omitted questions
•Trains field researchers in tool
implementation
7 • Varies in importance
May 22, 2012

9. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
8

10. STEP-BY-STEP GUIDE:
STEP 3
Ensure all necessary logistical arrangements, to
include:
•Selecting research team members
•Transportation to and from the research site
•Meals and lodging
•Security arrangements
•Communication with respondents or potential respondents
•Means of communication between research teams and with
the head office
•Protocols for addressing problems or questions as they arise
•Safeguarding of blank and completed information gathering
instruments
•Storage and transfer of research findings
9
May 22, 2012

11. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
10

12. STEP-BY-STEP GUIDE:
STEP 4
Establish data quality procedures to
prevent:
Security lapses
Data collection
errors
Data tampering
11
May 22, 2012

13. STEP-BY-STEP GUIDE:
STEP 4
Quality control procedures that mitigate threats:
Quality M&E System
Training
Collaboration with data
collectors
Routine Refreshers
In-field Supervision
Reviews of Data
12
Collection Forms Random Quality Checks
May 22, 2012

14. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
13
• Assign clearly defined roles
9 May 22, 2012

15. STEP-BY-STEP GUIDE:
STEP 5
Electronic databases for survey/questionnaire
results:
•Common format is to list questions across columns,
and respondents down rows
•Data management programs most often used:
-Microsoft Excel -SPSS -Microsoft
Access
•Should capture info in the order it was asked
14
•Should include built in data controls
May 22, 2012

16. STEP-BY-STEP GUIDE:
STEP 5
Consolidation of Qualitative Data:
•Should be compiled and managed in a central space
•Carefully consider data collection tools
Challenges Lessons
learned
Easy Access
for the
Reader
Anecdotes Success
15 stories
May 22, 2012

17. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
16

18. STEP-BY-STEP GUIDE:
STEP 6
Critical components of data
management:
Data Storage
Data Backup
Data Entry
Data Access
Data Revision
17
May 22, 2012

19. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
18

20. STEP-BY-STEP GUIDE:
STEP 7
Useful
Raw Data Informatio
n
FINAL M&E
EVALUATION
AWESOME PROJECT
KENYA
19
May 22, 2012

21. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
20

22. STEP-BY-STEP GUIDE:
STEP 8
Disseminating Data to Internal/External
Users:
•Procedures include timing, content and format
of information
•Externally, determined by
donor requirements
•Internally, co-developed with
staff who will use the
21
information
May 22, 2012

23. STEP-BY-STEP GUIDE
1 • Develop data collection instruments
2 • Pilot test data collection instruments
3 • Arrange data collection logistics
4 • Establish data quality procedures
5 • Develop Project Database and Data Entry Procedures
6 • Establish data management procedures
7 • Establish data analysis procedures
8 • Establish information dissemination procedures
9 May 22,•2012
Assign clearly defined roles
22

24. STEP-BY-STEP GUIDE:
STEP 9
Effective operation requires that all
involved understand roles and
responsibilities clearly, which should be
included in:
•Staff job descriptions
•MOUs with implementing partners
•Contracts with external researcher
23
May 22, 2012

25. COMMON
PITFALLS
•Failure to consider all the features necessary for a
functioning and effective M&E system.
•Projects do not pilot test data collection tools and
instruments.
•Projects do not establish data quality procedures or
conduct periodic assessments of data gathering
activities and data quality.
•Projects do not establish data management
procedures or plan for staff turnover
•Projects do not plan to and/or invest in conducting
sufficient staff training
•Including follow up mentoring
•Too much focus on donor needs, not enough on other
M&E system clients.
24
May 22, 2012

26. QUESTIONS
?
COMMENTS
?
25
May 22, 2012

27. Want to Learn More?
Multiple ways to continue the discussion
and continue learning:
• Initiate a monthly session on the M&E guide and
case studies from across CARE. Contact
cpennotti@care.org
• Join the Market Engagement Community of
Practice on LinkedIn.
• Join a task force to review and refine the universal
26
indicators. Contact nardi@careinternational.org
May 22, 2012