Beyond Reporting: Monitoring and Evaluation as a Health Systems Strengthening Intervention
•
10 gostaram•2,277 visualizações
Presented at the 2012 Global Symposium on Health Systems Research.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (15)
Semelhante a Beyond Reporting: Monitoring and Evaluation as a Health Systems Strengthening Intervention
Semelhante a Beyond Reporting: Monitoring and Evaluation as a Health Systems Strengthening Intervention (20)
Mais de MEASURE Evaluation
Mais de MEASURE Evaluation (20)
Beyond Reporting: Monitoring and Evaluation as a Health Systems Strengthening Intervention
- 1. Beyond Reporting: Monitoring & Evaluation as a Health Systems Strengthening Intervention Jason B. Smith,1 Siân Curtis,1 and Tara Nutley2 MEASURE Evaluation 1UNC Chapel Hill, 2Futures Group Second Global Symposium on Health Systems Research Beijing, China, 31 October – 3 November, 2012
- 2. Presentation Overview ■ M&E ■ Health systems ■ HSS and M&E ■ Strengthening M&E ■ Challenges and Opportunities
- 3. What is M&E? Adapted from Organizing Framework for a Functional National HIV Monitoring and Evaluation System (UNAIDS, 2008).
- 4. Why Should We Improve M&E? ■ Accountability and Funding ■ Program Management, Performance Improvement ■ Strengthen the Health System
- 5. Historic Look at M&E in Global Health ■ 1970-80s: Development of Logic Models ■ 1980-90s: Health for All ■ 1990s: Results Frameworks ■ 2000: MDG targets ■ 2000s: PEPFAR & GFATM ■ 2010-ish: HMN Framework; GHI
- 6. The WHO Health System Framework Everybody’s Business. Strengthening Health Systems to Improve Health Outcomes. WHO’s Framework for Action. WHO, 2007.
- 7. M&E and HIS
- 8. The WHO Health System Framework Everybody’s Business. Strengthening Health Systems to Improve Health Outcomes. WHO’s Framework for Action. WHO, 2007.
- 9. WHO Framework: Alternate view Everybody’s Business. Strengthening Health Systems to Improve Health Outcomes. WHO’s Framework for Action. WHO, 2007.
- 10. Unpacking the Building Blocks Global learning Workforce Leadership Service Information & Governance Delivery Products and Technology Finance
- 11. Ways to Improve M&E Systems ■ Supply data better focused on priority questions ■ Improve the quality of the data supplied ■ Improve capacity to convert data into usable information ■ Improve dissemination of the information produced ■ Improve use of the information that is disseminated ■ Demand data better focused on priority questions Creating a culture of data use!
- 12. Opportunities ■ Large shift in awareness/culture of M&E ■ Key system elements increasingly in place ■ More and better data ■ Creative solutions at country level ■ Increasing flexibility at global level
- 13. Challenges (1) ■ Beginning with the end in mind ■ Shifting towards adequacy
- 14. Challenges (2) ■ Transitioning ■ More and better data ■ Maintaining momentum
- 16. References Abou-Zahr C, and T Boerma. 2005. Health Information Systems: the foundations of public health. Bulletin of the WHO, 83(8):578-583 Boerma J.T., and S.K. Stansfield. 2007. Health statistics now: are we making the right investments? Lancet, 369:779-86. http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTH EALTHNUTRITIONANDPOPULATION/EXTHSD/0,,contentMD K:22239824~menuPK:376811~pagePK:148956~piPK:216618 ~theSitePK:376793~isCURL:Y,00.html Nutley T. 2012. Improving Data Use in Decision-Making. MEASURE Evaluation, Special Report.
- 17. References Porter LE, et al. 2012. Beyond Indicators: Advances in Global HIV Monitoring and Evaluation During the PEPFAR Era. J Acquir Immune Defic Syndr, 60(S3):120-126. Schwarrtlander B, et al. 2011. Towards an Improved investment approach for an effective response to HIV/AIDS, Lancet, 377:2031- 2041. UNAIDS. 2008. Organizing Framework for a Functional National HIV Monitoring and Evaluation System. WHO/Health Metrics Network. 2008. Framework and Standards for Country Information Systems (2nd ed.). Geneva. WHO. 2007. Everybody’s Business. Strengthening Health Systems to Improve Health Outcomes: WHO’s Framework for Action.
- 18. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) through cooperative agreement GPO-A-00-03-00003-00 and is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. The opinions expressed in this presentation are those of the authors and do not necessarily reflect the views of USAID or the U.S. government. Visit us online at http://www.cpc.unc.edu/measure
Notas do Editor
- I’m going to start by talking a little about Monitoring and EvaluationThen I’m going to talk a little bit about Health Systems, especially about the Health Information SystemThen I’m going to talk about the relationships among Monitoring & Evaluation, the Health Information System and the Health System as a wholeWhen I get through with that, I’ll make some recommendations for how to improve Monitoring and Evaluation from a systems perspective Finally I’ll offer opinions about some of the opportunities and challenges we face going forward.
- So why should we improve M&E? First, we should improve Monitoring and Evaluation for reasons of Accountability and Funding. If you measure what you do, it makes it easier to sustain and improve the activity. Similarly, if you don’t measure what you do, it is difficult to justifycontinued funding for the activity.Secondly, we should improve Monitoring and Evaluation for reasons of program management, and performance improvement. If we improve M&E, we can better manage our programs, learn from our experiences, and use that knowledge to make our programs more effective and more efficient. Lastly we should Improve Monitoring and Evaluation to strengthen the health system as a whole and that will be the subject of the remainder of this talk.
- Before I leave this part of the presentation, I just wanted to show a selected but evolutionary look at the development of M&E in Global Health:The 1970s and 80s gave us the development of logic models for project design and evaluation. I’m sure there are many people here who remember being first exposed to the concept of a Log Frame during this period Building on the previous Alma Ata principles, the 1980s and 90s continued with global target setting under rubrics like the Health for All by the Year 2000 initiativeDuring the 1990s we saw an evolution from use of logic models for project design to use of results frameworks to improve the measurement and accountability of efforts at a program level.Around the turn of the century of course we saw the promulgation of the Millennium Development Goals and their associated targets, a paradigm that still has relevance for us today.During the first decade of the new century we saw considerable investment in Monitoring and Evaluation, both in terms of infrastructure and intellectual capital, as a result of large programs like PEPFAR and the Global Fund.And lastly, there have been a number of developments in recent years including the Health Metrics Network Framework and Standards for Country Health Information Systems, and encouragement for Monitoring and Evaluation under USAID’s Global Health Initiative
- Okay now switching gears a little I want talk about the health system.Again, many people in this room will be familiar with this graphic from the Everybody’s Business document showing the WHO Health System Framework and the logic that ties improvements in the health system building blocks to improvements in systemic functioning and in achievement of our aspirational goals for health.The Framework uses seven building blocks, including a block that represents the Health Information System.
- This Venn diagram shows our notion of the relationship between the Health Information block and the Monitoring and Evaluation system. While the two are not identical, we think it’s fair to say that there is substantial overlap and that the M&E system is a major component of the Health Information blockUsing the transitive property of logic, we can then assert that by strengthening the Monitoring and Evaluation system, we strengthen that Health Information block and by strengthening the Health Information block, we strengthen the Health System as a whole. Ergo, it is logical to conceive of M&E as a health system strengthening intervention. Okay now that I’ve made that point , I want to go back to the WHO Framework for a bit.What’s in the brown? At least parts of the system that have to do with strategic information and forecasting.
- Now, please understand that I’m not trying to be overly critical here – When it was published the WHO Framework advanced our understanding of health systems by articulating the building blocks and their place in the chain of logic leading to health improvement. However, we do think that this Framework has some shortcomings. Specifically, in this depiction of the Framework the building blocks look both monotonic and static, that is, all of the building blocks look at if they are equal and there is no sense of sequencing of the interactions among the various blocks.
- Now this slide shows an alternate view of the WHO Framework also from the Everybody’s Business document. This view gives a sense that the authors thought the Leadership and Governance block had an importance greater than the other blocks. It also shows what we think is a confusing, and circular set of relationships among the remaining blocks.
- This slide shows our attempt to articulate the relationships among the building blocks in terms of sequencing. One thing that emerges from this view is the special role of the health information block. Please notice that improvements in the health information block have the potential to improve the performance of every other block in the system, to capture the experience of the other blocks, and also to act as in a bi-directionalway to insure that global experience gets factored into understanding at country level and that country experience informs the Global Learning Agenda. Said another way, if you were to remove or severely hamper the functioning of the health information block, all of the other blocks would still be able to function, but each would function less well, experiential learning would prove difficult, and there would be a disconnect between global learning and the ability of national and sub-national systems to contribute to, or learn from, the experiences of others. Another thing that emerges from this depiction confirms what we saw in the alternate view of the WHO Framework I showed earlier, which is that the Leadership and Governance block does indeed occupy a place of special importance in the chain of events. By virtue of its positioning, the Leadership and Governance block plays a key role in interpreting the information flow from the Information block to the other system blocks.In this view of a well functioning system, each component block feeds back to the information block, often through the M&E system. This information then cycles back through the entire system in an iterative, performance improvement loop.
- So having hypothesized that strengthening Monitoring and Evaluation strengthens the Health Information building block and also that the Health Information building block plays a special role in the sequence of Health System Strengthening, that leaves us with the question of “What can be done to improve M&E systems?”. Many people including Boerma and Stansfield, Schwartlander et al , AbouZahr and Boerma, and most recently Porter et al have made suggestions for how to answer this question. This slide shows a brief summary of some of the main points. [Read points and comment if there is time]Over time, repeated iterations of this virtuous cycle results in the creation a culture of data use, where people become accustomed to asking for the data they think will help them and accustomed to getting the data they need to make decisions.
- There are number of opportunities that exist now for continuing to use Monitoring and Evaluation as a means to strengthen the functioning of health systems.First ,there has been ,over the course of the last 20 years or so, a large shift in the awareness of the need for Monitoring and Evaluation and in the culture of implementing Monitoring and Evaluation as an integral , foundational health system function.Secondly, driven in large measure by the global response to the HIV epidemic, investments in Monitoring and Evaluation systems have resulted in a situation where key system elements are increasingly in place, making it easier to improve systemic functioning by filling gaps and improving efficiency, as opposed to having to install basic capabilities. Third, again, largely due to recent investments and improving systems, we have more and better data at our disposal now than we have ever had before.Fourth, with the recent emphasis on country ownership, we are starting to see more creative, collaborative solutions at country level andLastly, there is a growing recognition among global actors of the need to coordinate efforts for the sake of efficiency. As a consequence we are seeing increasing flexibility at global level, both in terms of streamlining collective needs (i.e. indicator harmonization) and also in the willingness to support experimentation with the creative and collaborative country level solutions that I just mentioned .
- However, despite the progress that has been made and despite the wind at our back provided by the Opportunities, the field still faces a number of Challenges.The first one is a mental challenge that I’m going to characterize as Beginning with the End in Mind. By this I mean moving the M&E mindset away from the basics of indicators and reporting and towards this notion of planning and implementingM&E as a foundational health systems strengthening intervention.The second challenge is what I will call shifting to an adequacy model and by this I am talking about changing the way we think about capacity building for M&E. Going forward we should move away from a traditional approach that says “What have we got now and what can we do to make it incrementally better?” Instead we should move forward towards a model that begins with a vision of what it will take to produce an adequate functioning M&E system and works backwards from that vision to determine what needs to be done to realize the vision.
- The third challenge I’ve called Transitioning and by this I mean that we will need to pay careful attention to the balance between meeting the immediate needs for Monitoring and Evaluation in support of programs in progress, and support for the longer term system strengthening goals implied in the shift to a “Begin with the end in mind” mentality.You might recognize the fourth challenge from the Opportunities slide – more and better data. This challenge implies both reducing the collection and processing of data we don’t use, as well as building out our systems, human and technological, to make maximum use of the data we decide to collect. The last challenge I’m going to mention is “maintaining momentum”. We’ve come a long way in the last 20 years and but past performance is no guarantee of future success. Right now, we are at an inflection point in the history of Monitoring and Evaluation in global health, where systemic improvement or decay are both possibilities. It is critical at this juncture that we make sustained efforts to capitalize on our recent achievements and realize the potential systemic improvements that are finally within our grasp.
- In conclusion:Despite all the challenges along the way, the field of monitoring and evaluation has long served the interests of global health according to the needs of each point in time. Looked at in an evolutionary sense, it is possible to demonstrate great success. If nothing else, it is a fact that we know more about health now that at any other time in our history. And that is progress. Now is the time to embrace a new view of M&E, a view for this time, a view of M&E as a Health Systems Strengthening intervention. As experience has shown, it will take time and sustained effort to produce success. But, as experience has also shown, we are on the right track and poisedto improve the inclusiveness, build the capacity, create the linkages, reduce the waste, and improve efficiency in a way that will create the stronger health systems of the futureThank you.
- alp