1. Data Quality in Remote
Monitoring
A comparative analysis of
experiences in Somalia and
Eastern Burma
Mona Fetouh (Co-Author and Presenter), Christian Balslev-Olesen (Co-Author), and Volker Hüls (Co-
Author)

2. Remote Monitoring
 Humanitarian space restricted in many situations; risks
have increased
 Both delivery and monitoring of programmes affected
 Remote monitoring and cross-border approaches
required in a number of countries in recent years
 Increased reliance on local actors
 Reduced ability to collect and verify information
 Often results in compromises on data quantity & quality
 Although remote management is increasingly common,
evidence on best practice is still emerging

3. Comparable Contexts
 Somalia
 Classic remote management situation
 Difficulty of access due to insecurity, heavy reliance on
national civil society (NGOs, communities) for aid delivery
(still largely managed from neighbouring Kenya)
 Risk to external monitors; monitoring relies heavily on local
partners – risk of bias
 Eastern Burma
 Hard to impossible to access from capital due to
government restrictions and long-running conflicts
between ethnic groups in the East and the government
 Heavy reliance on civil society (LNGOs, communities) for
aid delivery that is managed from neighbouring Thailand
 Little to no access for INGO staff, including national staff
 Risk to external monitors, monitoring relies heavily on local
partners – risk of bias

4. Different Data Environments (1)
 Somalia:
 Main concern is availability of data.
 Little opportunity to collect data regularly,
even for local partners
 Local partners have varied capacity to
produce quality data, severe education gap
results in overall low staff capacity
 Focus of strengthening objective monitoring
was through investment in independent
systems (―third party verification‖).

5. Different Data Environments (2)
 Eastern Burma:
 Main concern is management and quality of data
 Data are abundant, both monitoring data and surveys
 Local partner capacity is varied but good – strong
ethnic and professional exchange with Thai border
area; better access to education
 Information remains in technical silos; local NGOs are
largely confined to ethnic areas; access opportunities
are often limited to a particular sector.
 Few mechanisms for independent verification/
triangulation of data.
 Focus of strengthening objective monitoring was on
the quality and verification of information
 Access is improving due to ceasefire agreements

6. Comparative Analysis
 Similar contexts require different
approaches
 Presentation with details on
 Third party verification in Somalia
 Quality assurance of monitoring information in
Eastern Burma
 Experiences presented are based on of
work of UNICEF (Somalia) and International
Rescue Committee and the Border
Consortium (Thailand/Burma)

7. Somalia – Third Party Verification (1)
 First Level: Information from partners and
networks
 Implementing Partner Reports are main source of
primary performance data
 Reviewed against:
 Previous track record of partner / confidence level
 Specific concerns about partner performance
 Reporting complete and realistic?
 Specific issues flagged in or apparent from
requiring follow-up
 Comparison to occasional information from staff
contacts in the filed (email, telephone)

8. Somalia – Third Party Verification (2)
 Second Level: Third Party Verification
 Flagged issues are scheduled for third party
monitoring
 3rd Party Systems use field monitors that are
 not affiliated with any implementing partner
 often outside of the aid business
 And therefore
 can move more freely with less risk
 are not as qualified to judge details of implementation
 used mostly for verification of easily obtainable information
 Third party monitors, are ‗blind‘ tasked to avoid
fabrication of reports
 Were successfully tasked to track leakage of relief
goods into markets, including quantities and pricing.

9. Somalia – Third Party Verification (3)
 Third Level: Follow up on concerns from Level 1 and 2
 Third party information is kept confidential and assessed for
the risk level of a particular performance issue.
 Depending on risk level, issues are taken up with the
partner:
 without revealing source
 e.g. dedicated open monitoring mission at next opportunity
 Main reasons for staggering:
 First level flags issues, but is in itself not sufficient for
reliable data
 Second level is costly, and can be targeted to only flagged
issues
 Third level is costly and not timely, and should only be used
with knowledge of problems
 Low-level third party networks have worked well in
Somalia for other purposes, e.g. for food price monitoring

10. Eastern Burma – Strengthening
Monitoring Quality (1)
 Correlating data in geographical information
systems
 Main limitation to data correlation / triangulation is
sector-based systems (Health, Education, Relief
information management systems)
 Sector IMS are basis for cross-sectoral GIS solution
 Platform maps service delivery data of all sectors to
village location
 Allows analysis of performance data from all sectors
per location
 Allows sectors to engage in cross-monitoring and
data sharing

11. Eastern Burma – Strengthening
Monitoring Quality (2)
 Regular surveys are expanded in scope
and feed into the information system
 Key strength of the Eastern Burma
programmes is history of conducting regular
(sector) surveys.
 All partners are now supporting the expansion
of the scope and the coverage of these
surveys.
 Example: Annual Poverty Survey

12. Eastern Burma – Strengthening
Monitoring Quality (3)
 Increased linkages between implementing
partners, and improved M&E capacity
 Effortsin recent years to connect local
organizations across sector and ethnic group
 Discussion on cross-monitoring
 Comprehensive M&E training for local groups
 Improved community feedback/village
monitoring (also used by Oxfam in Somalia
and Tearfund in Afghanistan)

13. Eastern Burma – Strengthening
Monitoring Quality (4)
 Post-facto review of health centre
logbooks and patient files
 Simple but innovative example:
 Can be done remotely and is not time
sensitive
 Reveals substantial information about quality
of support and services
 Initiated by the IRC, and now being expanded
to whole sector
 Variations conceivable for other sectors

14. Eastern Burma – Strengthening
Monitoring Quality (5)
 Increased use of photographic and video
evidence of implementation
 Where possible, local NGOs use
 photography
 video
to document their work.
 Use of video started with success by one local
NGO
 Provides better representation e.g. of
trainings and public awareness activities.

15. Lessons learned, and looking towards
the future
 Similar contexts, different data environments
 Good examples of when similar contexts warrant
different approaches
 Both contexts are changing rapidly – more access in
Somalia, political reforms in Burma
 Improved monitoring systems instill long-term effects
to adapt to these changes—stronger information,
increased local capacity
 Changes in Somalia may make Eastern Burma
lessons applicable in near future
 Experience in Somalia valuable for similar situations
elsewhere, e.g. Syria

16. Thank you!
For further questions:
Mona Fetouh - fetouh@un.org
Volker Hüls - volker@makingaidwork.com
Christian Balslev Olesen -
christianbalslev@gmail.com