NDWC Chennai 2013 - Evaluating ABC - Dr Elly Hiby and Lex Hiby
•
1 gostou•975 visualizações
Dr Elly Hiby and Lex Hiby's presentation on Evaluating Animal Birth Control Programme
Recomendados
Mais conteúdo relacionado
Semelhante a NDWC Chennai 2013 - Evaluating ABC - Dr Elly Hiby and Lex Hiby
Semelhante a NDWC Chennai 2013 - Evaluating ABC - Dr Elly Hiby and Lex Hiby (20)
Mais de Dogs Trust
Mais de Dogs Trust (20)
Último
Último (20)
NDWC Chennai 2013 - Evaluating ABC - Dr Elly Hiby and Lex Hiby
- 1. Evaluating ABC Elly Hiby (elly.hiby@dogstrust.org.uk) Dogs Trust & Lex Hiby Conservation Research
- 2. Evaluating ABC • Why evaluate • What indicators can we measure • Methods for measuring indicators – Dog population survey • Novel tools for surveys (developed with funding from HSI) – Combining clinic database with survey data collected on a smart phone
- 3. • Are you involved in running an ABC programme? • Do you evaluate your programme? • Does this evaluation go beyond reporting number of dogs through ABC?
- 4. What is evaluation? • Evaluation “the assessing and judging the value of a piece of work, an organisation or a service. Its main purpose is to help an organisation reflect on what it is trying to achieve, assessing how far it is succeeding, and identify required improvements” Evaluation Trust
- 5. What is evaluation? • Evaluation “the assessing and judging the value of a piece of work, an organisation or a service. Its main purpose is to help an organisation reflect on what it is trying to achieve, assessing how far it is succeeding, and identify required improvements” Evaluation Trust
- 6. Why evaluate? • Donors – reward their generosity and build commitment • Government and animal welfare organisations – inspire them to replicate • Your organisation – pride and motivation • Your end beneficiary – the animals that matter to us Learn and improve
- 7. Indicators • Evaluation involves measuring a range of indicators to expose important changes • When selecting indicators consider the aim of ABC programme and opinions of stakeholders – Government including health department – Local community both dog owning/caring and not – Donors – Your own organisation (encompassed in the aim) What do they want to see change?
- 8. Indicators • Animal welfare: – Body condition score – Skin condition – Lactating females (more visible and consistent to score than litters of puppies) – Disease incidence (dog rabies cases) – Euthanasia rates – Dog population size • Community satisfaction – Nuisance complaints – Attitudes towards dogs • Human health – Dog bites – Human rabies cases
- 9. Methods of measurement • Official records • Community questionnaires/focus groups • Dog population surveys
- 10. Methods of measurement • Official records • Municipality records of nuisance complaints • Health centre records of dog bites and human rabies cases – Samples are sufficient, find a friendly source – Note locations (e.g. postcodes) if this is possible, can you find data from within the ABC area and outside for comparison over time (incl pre-ABC) and between treatment and control – Note human population increase – e.g. don’t quote just dog bite data, quote bites per 100,000 people – Right to Information Act (RTIA)
- 11. Colombo dog rabies cases Dog rabies cases in CMC against elimination, vaccination and sterilisation from 1990 to end 2010 0 2000 4000 6000 8000 10000 12000 14000 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Year Numberofdogseliminated, vaccinatedorsterilised 0 10 20 30 40 50 60 70 Numberofdograbiescases Elimination Vaccination CMC and BPT Number of sterilisations Dog Rabies Start of project
- 12. Methods of measurement • Community questionnaires/focus groups – WHO (2004) Guidelines for conducting community surveys on injuries and violence – Questionnaires can reveal changes in quantitative data such as dogs bites (e.g. has anyone in your household been bitten in the last 12 months) – Focus groups can reveal attitudes and practices (e.g. do you think stray dogs are dangerous or safe? What do you do if you are bitten by a dog?)
- 14. Methods of measurement • Dog population surveys – Index of abundance (aka ‘indicator survey’) • Quicker/cheaper • But only indicates change in density and not total population size – Estimate of total population size • Requires more time investment ahead of ABC, so only invest in this if necessary • Can be relatively simple once ABC had begun
- 15. Jaipur Pink City counts
- 16. Example – surveying for a new ABC project • Baseline 1. Dog population estimate based on exhaustive search in small number of sample areas extrapolated to whole area (assuming you need a total dog population estimate) • Refine after approximately 1 year of ABC 2. Mark-resight of dogs notched through ABC (novel tool)
- 17. 1) Baseline population estimate • Establish smallest authority area for which information on covariates is available (e.g. human population, road length) • Select small sample of these areas (as many as can be realistically managed, suggest minimum 5%) • Complete count of all dogs visible on public areas at peak roaming time (usually early morning) • Extrapolate to entire city: – Using sampling fraction (multiply number of dogs counted by total number of areas/number of samples areas) or – Find reliable covariate (e.g. dog:human ratio) and extrapolate to entire area
- 21. 1) Baseline population estimate • Extrapolate to entire city: – Using sampling fraction (multiply number of dogs counted by total number of areas/number of samples areas) • E.g. counted 462 dogs in 7 wards out of a total of 47 Estimated total roaming dog population = 463 X (47/7) = 3,102 or – Find reliable covariate (e.g. dog:human ratio) and extrapolate to entire area • E.g. counted 462 dogs in 7 wards in which live 16,000 people = 1 dog:35 people. 100,000 people live in the city hence estimated total roaming dog population = 100,000/35 = 2,857
- 22. 1) Baseline population estimate • This is a ‘rough’ minimum roaming dog population estimate: – Small sample size – Dogs roaming and visible in the early morning – not total roaming dog population as some will be missed – But can be a basis for lobbying and budgeting with these clear limitations – Be clear this will be refined and is not the final figure
- 23. 2) Mark-resight of notched dogs • Dogs are notched during the ABC programme providing a sample of ‘marked’ dogs in the city. • Taking into account a survival estimate (some ABC dogs will have died since the operation) the number of notched dogs in the city can be calculated • Combining that with proportion of notched dogs can be used to measure the population size
- 24. 2) Mark-resight
- 25. 2) Mark-resight of notched dogs • ABC example – ABC programme sterilises 3,500 dogs in first year, with an annual survival estimate of 0.7 the number of dogs notched and alive in the city is estimated to be 2,800 – The percentage of notched dogs on the street is estimated to be 46% – Total dog population estimate is 2,800 / 0.46 = 6,087 Hiby et al (2011) A mark-resight survey method to estimate the roaming dog population in three cities in Rajasthan, India • Novel tool combining database for recording clinic data with a smart phone app
- 26. 2) Mark-resight of notched dogs • Mark-resight estimates the total number of roaming dogs rather than the number roaming at the time of the initial baseline survey • Relies on a survival estimate – Can use the estimate in Reece et al (2008) Fecundity and longevity of roaming dogs in Jaipur, India; annual survival of 0.7 – Can estimate a survival for your own area if using individual tattoos
- 27. 3) Index of abundance routes • Several standard routes of known length • Representative areas • Confounding factors – Time, person, weather, protocol… • Quick so can repeat – Seasonal breeding, politically important time… • Evaluate ‘like with like’
- 28. What to record • Information for each dog: – Age • Adult • Puppy – agree maximum age (e.g. 4 months) – Sex • Male • Female • Unknown – Reproductive status • Lactating • Sterilised (notched) – Welfare state • Body condition score • Skin condition
- 33. Welfare measures • Body condition score (5 point BCS) – Reduction % of dogs in score 1 BCS over time and between notched and unsterilised dogs • Skin condition (Presence/absence) – Reduction in % of dogs with skin conditions visible • TVTs? • Behaviour? 1 – runs away immediately 2 – keeps a distance but remains in area 3 – approaches 4 – in your pocket!
- 34. Using an Android phone to record data • An offline database plus some free smartphone apps can help you to monitor your intervention
- 38. % Jaipur females pregnant 0 5 10 15 20 25 30 O ct-08N ov-08D ec-08Jan-09Feb-09M ar-09A pr-09M ay-09Jun-09Jul-09A ug-09S ep-09O ct-09N ov-09D ec-09Jan-10Feb-10M ar-10A pr-10M ay-10Jun-10Jul-10A ug-10S ep-10O ct-10Nov-10D ec-10Jan-11Feb-11M ar-11A pr-11M ay-11Jun-11Jul-11A ug-11S ep-11O ct-11N ov-11
- 73. ADMIN: download and analyze city data FUNDING AGENCY: view by NGO and time period NGO 1: upload by month, view NGO 2: upload by month, view NGO 3: upload by month, view
- 74. Discussion point • Which indicator is best for dog abundance? • Total population for the city? – But cities are expanding in size with increasing human population in expanding areas = increasing carrying capacity = increasing total dog population – Average city dweller has no concept of total dog population size, but they do perceive the density and behaviour of dogs – Takes a lot of time/resources to establish a really reliable dog population estimate ahead of ABC – although easy once ABC has started using phone/database tool • In comparison the number of dogs per km of street is: – Quick to estimate from the standard routes used in index of abundance surveys – Relates more closely to the dog problem
- 75. Conclusion • Evaluation provides a measure of change in important indicators and an opportunity to improve performance
- 76. Conclusion • Animal welfare: – Body condition score – Skin condition – Lactating females (more visible and consistent to score than litters of puppies) – Disease incidence (dog rabies cases) – Euthanasia rates – Dog population size • Community satisfaction – Nuisance complaints – Attitudes towards dogs • Human health – Dog bites – Human rabies cases
- 77. Conclusion • Methods of measurement – Official records – Community questionnaires/focus groups – Dog population surveys • Database and phone app
- 78. Whose responsibility? “ An important job had to be done and Everybody was sure that Somebody would do it. Anybody could have done it, but Nobody did it. Somebody got angry about that because it was Everybody’s job. Everybody thought that Anybody could do it, but Nobody realized that Everybody wouldn’t do it. It ended up that Everybody blamed Somebody when Nobody did what Anybody could have done” Make someone responsible and provide time and budget
- 79. Thank you for listening Any questions?
Notas do Editor
- Good morning and thank you for joining us today. We’ll be talking through some aspects of evaluating ABC today – its quite a broad topic, so apologies if we do not cover everything. You’ll notice that we have a double act today – I will be presenting for the first part of the workshop and then will be handing over to my father – Lex Hiby. Please note there are handouts available with a list of resources relevant to this topic here – if there are not enough copies, or indeed if you would like an electronic copy, please do email me afterwards and I’ll send this on. I also want to encourage you to ask questions throughout this wrkshop – I will also be stopping at times to ask for your input to certain key questions – so please spk up, I hope this can truly be a 2-way conversation
- So today we plan to cover the following. By way of an intro looking at the definition of evaluation and why we would ever bother delaying our normal work to play with data What indicators are available that can tell us something about the aspects of the world we are hoping to change I’ll then go through some methods of collecting data that will help us measure these indicators, in particular different methods of dog population surveying Before handing over to Lex to talk through a novel tool for conducting ongoing dog surveys during the running of an ABC programme – combining a mobile phone app and database of clinic data from the ABC operations to track dog population size and coverage of ABC. I should be quick to point out that this tool has been developed with funding support from HSI.
- I wanted to start by just checking who in the audience is involved with running an ABC programme? If you can put your hands up if this is true for you. Can you keep you hands up if you evaluate that programme in anyway
- This is one of many definitions of the word evaluation. This time from the Evaluation Trust. But from my perspective the most important reason to define and measure your goal is so we can learn to do our job better
- This is one of many definitions of the word evaluation. This time from the Evaluation Trust. But from my perspective the most important reason to define and measure your goal is so we can learn to do our job better
- You may ask – why evaluate? It takes up time I could be using to ABC more dogs. Who cares? First audience is our donors – who may be government, AWBI or municipalities or state gov, or may be an animal welfare donor - raise your hand if you are responsible for raising money for your organisation. Effective evaluation can prove to your donor that their money is making a difference and secure further commitment to donate – important when we are running ABC projects that have to sustained for the long-term Government or animal welfare organisations that have yet to launch ABC – good evaluation can inspire them to start and replicate the benefits Yourself and your staff, your volunteers, your CEO. – pride in your work and motivates the team to keep dedicated to ABC And finally your end beneficiary – the animals themselves, who I am sure if they could would ask – would value seeing the benefit of ABC, they are after all the ones that have to be caught, held, anaesthetised and undergo surgery. What difference will this make to their life and the lives of other animals But I think that the most important reason is to be from my perspective the most important reason to define and measure your goal is so we can learn to do our job better
- This is an example from Colombo – you can see on this blue line the number of dog rabies cases, measured and reported by the municipal vet office – we could plot this against the years of culling and then against the years of mass vaccination – showing neatly what worked best
- - Explored attitude towards dogs ‘now’ and ‘in the past’ (>4 years ago prior to when intervention started) Participants were asked open questions about roaming dogs in their area, both currently and in the past – facilitators then followed up with additional questions and participatory exercises to draw from each group a list of problems presented by roaming dogs – both in 2011 and before the intervention started. On this spider web diagram you can see the 9 most commonly mentioned problems caused or experienced by roaming dogs around the outside – the blue line indicates problems in the past and the red those mentioned in 2011. We can see that all 9 focus groups mentioned rabies and breeding or the presence of puppies as a problem in the past – however only 1 group mentioned this as a problem in 2011. We can also see that the total number of problems mentioned in the past – as indicated by the area delineated by the blue line - also decreased significantly by 2011 – as indicated by the smaller area outlined in red
- We conducted a population survey like this in Colombo Sri Lanka using ‘wards’ the Colombo city municipality split into 47 wards and we counted dogs in 7 of those. Importantly there were maps available for these wards that allowed to ensure we were counting all the streets in the city
- Need maps available for these official boundaries
- Use Google Earth where maps are poor
- Jack Jack to explain why they chose the Pink City and the time it takes to complete the counts
- 3
- 1
- 4
- 2
- Never tried it but what about behaviour? Does this describe your dogs?
- xxx
- So I just want to share something with you that I learnt the hard way. Have a read of this quote. Ever experienced this? I most certainly have and specifically on this issue of evaluation. I have been part of workshops were over a period of days and using consultation with government representatives, community groups and local vets we developed goals and indicators that lit each of our respective fires. But it became a paper exercise because we didn’t follow this trough to make sure we had the funds, time and clearly identified person responsible to make these indicators work for us. So we weren’t able to do good evaluations and hence learn and improve – we continued to stumble on in the dark.