This document summarizes findings from a baseline study evaluating a USAID-funded Community-based Forest Management Program (CFP) in Zambia. The study used mixed qualitative and quantitative methods, including household surveys, focus groups, and participatory mapping exercises. Key findings include: 1) Respondents observed increases in deforestation and changes in rainfall patterns and seasonal cycles. Most viewed these environmental changes as severe problems. 2) Perceptions of the causes of environmental change were mixed and included deforestation, population changes, poverty, and biblical explanations. 3) Participatory mapping revealed tensions between communities and "outsiders" over land and forest resource use and control.

1. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
Perceptions of Environmental Change:
Using Qualitative Methods to Ground
Culture in the Evaluation of
Environmental Programs
Cynthia Caron, Clark University and The Cloudburst Group
Stephanie Fenner, The Cloudburst Group
November 13, 2015

2. 2
Outline
• Background: Community-Based Forest Management Program (CFP)
• Impact Evaluation of CFP: Objectives and methodology
• Illustrative Findings: Perceptions of deforestation and climate
change
• Conclusions and Lessons Learned

3. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
Background

4. 4
The USAID-funded Community-based
Forest Management Program (CFP)
• CFP aims to establish the largest REDD+ program in Zambia.
Four primary objectives:
1. Empower and equip communities to lessen the drivers of
deforestation;
2. Establish and improve forest and natural resource
management plans;
3. Promote alternative livelihoods to unsustainable charcoal and
timber production;
4. Implement pay-for-performance and/or revenue-sharing
programs for forest conservation and carbon sequestration.

5. 5
Project Area: Eastern Province and
Program Districts in Zambia

6. 6
Community-Based Forest Management
Program: Intervention
CFP initiates local livelihood and community development
projects to promote the adoption of alternative livelihoods and
energy sources. Projects designed to:
• Provide tangible benefits that replace the income and/or
livelihood benefits received from deforestation or forest
degradation activities, such as charcoaling and timber
harvesting
• Promote business minded approach to support development of
sustainable enterprises.

7. 7
Community-Based Forest Management
Program: Intervention
Potential livelihood initiatives to promote forest conservation
include:
• Eco-charcoaling
• Conservation farming (maize)
• Eco-tourism
• Non-timber forest product (NTFP) extraction / small business
development

8. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
CFP Impact Evaluation

9. 9
CFP Impact Evaluation (IE):
Purpose
USAID’s primary learning objectives for the CFP IE:
1. Understand how REDD+ programs impact land tenure and
property rights (LTPR) and related livelihoods, either positively
or negatively.
2. Learn what aspects of REDD+ programming are most effective in
incentivizing long-term carbon sequestration and reduced GHG
emissions from forests and landscapes.

10. 10
CFP Impact Evaluation:
Research Questions
Example research questions include:
• Has CFP resulted in increased knowledge and awareness of
deforestation and climate change?
• Which CFP benefits do stakeholders cite as effective incentives
for the adoption of behaviors that reduce deforestation,
degradation and GHG emissions?

11. 11
CFP IE: Mixed Methods Design
• Quasi-experimental Difference-in-Differences (DD) approach,
complemented with qualitative component.
• Survey data collected at baseline (Spring 2015), midline, and end
line.
• This presentation draws on baseline data from 4395 household
surveys; 80 focus group discussions (FGDs) with local
stakeholders, including women and youth; 40 participatory
mapping exercises with separate groups of men and women.

12. 12
Culture and the Evaluation of
Environmental Programs:
Drawing from the literature:
• Krause et al. (2015) propose using an “assessment framework that
focuses on subjective perspectives on adaptation (i.e.,
perceptions and judgments of individual decision-makers) and
links them with objective perspectives” (38).
• Samuels et al. (2011) “evaluations characterized by Western
culture and ways of thinking overlook indigenous knowledge(s),
threatening the cultural relevance and validity of evaluation
results”(184).

13. 13
Grounding Culture: Participatory
Mapping Excercises
• Kawakami et al. (2008) note that participatory maps spatially
reflect the cultural views of the participant mapmakers’ forest
environment, allowing stakeholders to participate in an
evaluation through a practice that is driven by and with them.
• Sletto (2009) proposes participatory mapping can highlight
contests surrounding rights, identities, and authenticities, while
simultaneously facilitating the unveiling of the multiple, complex
relations of power that shape landscapes in the Global South.

14. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
CFP IE Illustrative Findings:
Perceptions of Deforestation
and Climate Change

15. 15
Observed changes in
environment: Deforestation
• HH survey respondents state that the majority of forests in
project area are in ‘good’ or ‘very good’ condition (62%, 2482).
• Despite generally good rating, the overall condition of most
forests (42%, 1688) was noted to have worsened in the past 3
years; the overall condition of 33% (1305) of forests was noted to
have remained the same.
• Respondents also reported that 35% (1405) of forests have
decreased slightly in area, and 38% (1540) of forests have
decreased in thickness and forest cover.

16. 16
Observed changes in environment:
Rainfall and seasonal patterns
• 82% of HHs noted a reduction in rainfall in the past 3 years; 28%
noted changes in the intensity/concentration of rain
• 44% of HHs noted changes in seasonal patterns
• “In the past it used to rain and the rains used to start early but that has
changed. Now you plant in November and just after 2 months the rains go.
That’s changing of the world.”
• “The rains are not starting in their usual month...Sometimes it will only rain
twice and stops until December. So the seeds you planted will not grow well.
We see that there is change in the distribution of the rain.”

17. 17
Perceptions of environmental
change: Is it a problem?
HH survey respondents asked to rank top 5 development problems
faced by their community (from list of 10 potential problems)
• Forest degradation: 41% of HHs ranked forest degradation among
top five problems; Ranked as #1 problem by 11% of HHs.
• Changes in rainfall and temperature: 50% (2229) of HHs ranked
either changes in rainfall patterns or changes in temperature as
#1 problem.

18. 18
Perceptions of environmental
change: Is it a problem?
HHs asked to rate severity of problems on the development of the
community (on a scale from 1 to 10):
• Changes in rainfall patterns or changes in temperature: Average
rating was 8/10
• Deforestation or forest degradation: Average rating was 5.6/10

19. 19
Perceptions of environmental
change: Is it a problem?
• “It is a big problem because it has brought poverty, food is not
enough, fruit trees are no longer so fruitful and the grass we use
for our houses is not growing well.”
• “It is a problem, it will be a problem because the next generation
will not be able to find forest products nearby. There will be
challenges in the villages. Instead of acquiring forest products
they need they will be failing because of distances.”

20. 20
What’s causing environmental
change?
• Deforestation contributes to climate change: “(M): You said that
you haven’t had good rains in the last three years, what do you
think has caused that? (R):That time there were still a lot of trees
in the forest but this time they are all cleared so you find that
there is poor rainfall in this area now.”
• Conflicting views: “Sometime back some people came and told
us lies that the oxygen from the trees has finished so we need to
stop cutting down the trees carelessly.”

21. 21
What’s causing environmental
change?
• Biblical explanations: “Like these days our thinking, the world
has changed, we are in the end times now… Like us who read the
bible, it says in that in the last days there will be drought, rain
will not come in its season. Now when we see them like this, we
say ah, this is what is written in the bible, it has been fulfilled.”
• More conflicting views: “We can say it is God yes, but when we
look carefully again, we hear that trees help to have rainfall, but
with this modernization, modernization is too much, we can be
seated here, your friend would just pass with a motor bike, it is
just problems, because they leave the smoke behind.”

22. 22
What’s causing environmental
change?
More reasons:
• Changes in population: “Tress have reduced…Trees have reduced
due to Chewa migrants who are given settlements and fertile land
for agriculture.”
• Poverty and Hunger: “What makes us go into the forest is not our
fault, it is because of poverty... Here there is nothing to do and
there are no sources of income, that’s why people don’t follow
the rules”

23. 23
Environmental change: What
can be done?
• “(M): You said rain pattern has reduced, what have done about it?
(R1): We have done nothing. That’s God’s plan. (R2): Mumm… it’s
God’s power. (R3): Mumm….its God’s plan. That’s a difficult
question to answer (laughed).”
• “There is nothing that can be done about this climate change. We
will continue experiencing hard times.”
• “It is the government that comes to stop us from doing these
activities, but there is nothing they give us so, now like this,
there is nothing to do and in the end we go in the forest to
collect whatever we want”

24. 24
Participatory Mapping
Excercises
Women in Nyimba District

25. 25
Unique Findings: Participatory
Mapping Data
Spatialization of resource use:
• Agricultural fields are “outside the village”
• Charcoaling is done “in the mountains”
• Overlaps in resource use without neighboring villages and
“outsiders”
“(M): Are there charcoal burners nearby? (R): Yes they are there,
those from the mountains (M): Oh, but these are your mountains?
(R): Yes, they burn charcoal, destroying our trees. (M): But it is your
land? (R): It is our land but then it was sold. How can we protect it if
the chief sold it?”

26. 26
Unique Findings: Participatory
Mapping Data
The research agenda, power relations and politics of control:
• “(R): What is the purpose for asking us all these questions? (F):
we are looking at how people live and use the forest (R): The
research is about those living in the forest how they are
destroying the forest? This is just about restricting charcoal
burning, not that. But we have already drawn the map! If they
stop us from burning charcoal how are we going to survive? The
thing is you can not reveal to us the truth [laughter]. So now this
forest is going to have rules.”

27. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
Conclusions & Lessons
Learned

28. 28
Does the CFP IE design
integrate culture?
• Documents local understandings of causes of climate change
(understandings that are not grounded in Western science)
• Reveals tensions with migrants are also ethnic tensions,
exacerbated by resource competition
• Illuminates power relations controlling access to forests and
forest resources
• Highlights mistrust of outside agencies as doing working for those
who wish to restrict access (local politics)

29. 29
Moving forward, Lessons
Learned:
Conceptual:
• Acknowledge that more than basic economic logics frame behavior
• Use baseline study findings to inform CFP design and implementation
Methodological:
• Method Triangulation
• Training (Probing needs to improve, avoid leading questions)
• Sample size versus quality of data, important implications for qualitative
component
• Mapping needs to document the entire debate and process of creating of
the map

30. 30
Questions and
Acknowledgements
Thank you.
• Questions?
• ccaron@clarku.edu
• stephanie.fenner@cloudburstgroup.com
Acknowledgements:
The Cloudburst Group wishes to acknowledge USAID’s
funding of CFP and the CFP IE.