This article discusses approaches to setting standards for evaluating community-based health promotion programs. It presents a typology of different types of standards, including arbitrary, experiential, utility, historical, scientific, normative, propriety, and feasibility standards. The article argues that evaluations should adopt a "salutogenic orientation" focused on health, well-being and empowerment. It also stresses the importance of collaborative and participatory evaluation approaches that incorporate multiple stakeholders' perspectives. Overall, the article aims to provide a framework for setting standards that can make the evaluation process more transparent and mutually beneficial for communities and funders/policymakers.

1. HEALTH PROMOTION INTERNATIONAL Vol. 16, No. 4
© Oxford University Press 2001. All rights reserved Printed in Great Britain
Setting standards in the evaluation of
community-based health promotion programmes—
a unifying approach
JENNI JUDD, C. JAMES FRANKISH1 and GLEN MOULTON1
Territory Health Services, Darwin, and Deakin University, Australia and 1Institute of Health Promotion
Research, University of British Columbia, Vancouver, Canada
SUMMARY
Community-based health promotion often emphasizes A typology of standards is presented. Arbitrary, experiential
elements of empowerment, participation, multidisciplinary and utility standards are based on perceived needs and
collaboration, capacity building, equity and sustainable priorities of practitioners, lay participants or professional
development. Such an emphasis may be viewed as being in decision-makers. Historical, scientific and normative
opposition to equally powerful notions of evidence-based standards are driven by empirical, objective data. Propriety
decision making and accountability, and with funders’ and and feasibility standards are those wherein the primary con-
government decision-makers’ preoccupation with measur- cern is for consideration of resources, policies, legislation
ing outcomes. These tensions may be fuelled when com- and administrative factors. The ‘model’ standards approach
munity practitioners and lay participants feel evaluations is presented as an exemplar of a combined approach that
are imposed upon them in a manner that fails to appreciate incorporates elements of each of the other standards. We
the uniqueness of their community, its programme, and argue that the ‘optimal’ standard for community-based health
practitioners’ skills and experience. This paper attempts to promotion depends on the setting and the circumstances.
provide an approach that depicts evaluation as being mutu- There is no ‘magic bullet’, ‘one-size-fits-all’ or ‘best’ stand-
ally beneficial to both funders/government and practitioners. ard. Further, we argue that standards should be set from an
First, a values stance for health promotion, termed a ‘salu- inclusive, salutogenic orientation. This approach offers a
togenic’ orientation, is proposed as a foundation for the means of creating a situation in which policy-makers and
evaluation of community-based health promotion. Secondly, funders are more supportive of evaluation designs that
we discuss possible objects of interest, the first component fit with community realities, and community stakeholders
of an evaluation. We then discuss the spirit of the times and are more capable and consistent in rigorously evaluating
its implications for community-based health promotion. community-based health promotion programmes and
Finally, we address the key question of setting standards. policies.
Key words: community-based health promotion; evaluation; salutogenic; standards
INTRODUCTION
Community-based health promotion programmes ing and control over personal, social, economic
often emphasize empowerment, participation, and political forces in order to take action to
social and sustainable development, multidis- improve their life situations (Israel et al., 1994),
ciplinary collaboration, capacity building and and capacity building refers to the problem-
equity. Empowerment, in its most general sense, solving capability among individuals, organizations,
refers to the ability of people to gain understand- neighbourhoods and communities (Hawe, 1994);
367

2. 368 J. Judd et al.
in the context of health promotion workers, it The issues raised in this paper are closely related
refers to their ability to enhance the capacity of a to the sustainability and improvement of pro-
system to prolong and multiply health effects, grammes, and the health of the communities
which represents a ‘value added’ dimension to served. Practitioners are often concerned that
health outcomes offered by any particular health their programmes will not be continued due to a
promotion programme (Hawe et al., 1998). This perceived lack of success by decision makers.
emphasis is often juxtaposed with equally power- We assert that standards for the evaluation
ful notions of evidence-based decision making of community-based health promotion are, for
and accountability, in that funders and govern- the most part, implicitly defined or assumed.
ment decision-makers are frequently more con- Secondly, we assert that standards employed in
cerned with measuring outcomes and defining the evaluation of community-based health pro-
success. Community practitioners and lay partici- motion that are not expressed may succumb
pants often feel that evaluations are imposed to the same fate as health promotion indicators;
upon them, and that the evaluation process does i.e. they ‘are often predetermined and shaped by
not appreciate the uniqueness of their community, those in political, administrative and economic
its programme, and its resources and skills fields’ [(St Leger, 1999), p. 194]. More import-
(Labonte and Robertson, 1996; Trussler and antly, they are often ignored or forgotten as a
Marchand, 1998). programme proceeds over time. We endorse
Portraying these viewpoints in a dichotomized making the use of such standards more explicit
manner may appear overly simplistic. We present and transparent in a collaborative process. We
them in this manner to assert that the issues raised recognize that the word ‘standard’ may have a
by both sides represent legitimate concerns pejorative connotation for some stakeholders in
within the practice of community-based health community-based health promotion; however,
promotion. It is essential to recognize that all we argue that all stakeholders consciously or
parties involved are seeking to provide the most unconsciously employ their own standards in
worthwhile programmes or policies to a designated assessing programme quality.
community. The progression from programme The first section of the paper underscores the
objectives, to strategies employed, to data collected, values and philosophy of health promotion as it
to definitions of programme success is often not relates to the evaluation process. Next, we discuss
operationally articulated in a transparent, meas- possible objects of interest, the first component
urable fashion. The focus of this paper is on the of an evaluation, in community-based health
latter stage (i.e. setting of standards), which we promotion. Then, we further explicate the spirit
argue is the least developed in health promotion. of the times and its implications for community-
While concerns for accountability and outcomes based health promotion. Finally, we present a
are part of our current zeitgeist (spirit of the taxonomy of ‘standards’ for evaluating community-
times), evaluation should not be a disempower- based health promotion against which such
ing process. Rather, it can contribute to ‘the pro- objects of interest can be measured.
cess of enabling people to increase control over, For our present purposes, we endorse the
and to improve their health’ (World Health definition of evaluation proposed by Green and
Organization, 1986). If practitioners are provided Kreuter (Green and Kreuter, 1999), namely that
with adequate support for conducting an evalu- evaluation involves the comparison of an object
ation, they are highly motivated in knowing if of interest against a standard of acceptability.
they are making a difference, and how they can We believe all stakeholders have a role in articu-
improve their programme. lating the objects of interest (e.g. changes in health
This paper provides an approach that depicts status, community development, intersectoral
evaluation as being mutually beneficial to all collaboration) and standards of acceptability for
stakeholders. Our aim is to make the evaluation a given programme or policy, not just the person
process more transparent and collaborative so who commissions the evaluation. We endorse the
that all parties will be satisfied, and gain from the use of a comprehensive, diverse set of standards
outcomes of community-based health promotion that reflects different concerns and forms of
evaluations. It will assist practitioners and evidence related to the evaluation of health
decision-makers in defining programme success promotion programmes. This approach offers a
at the outset of a programme and/or its evalu- means of creating a situation in which policy-
ation, rather than being the last issue discussed. makers and funders can be more supportive of

3. Setting standards in health promotion evaluation 369
evaluation designs that fit with community real- Social capital can be used to measure the capacity
ities, and community stakeholders can become of the social linkages and their resilience or fragility.
more capable and consistent in evaluating their Social capital is iterative and experientially developed,
health promotion programmes and policies. and requires both levels of trust and competence in
social interaction’ [(Cox, 1997), p. 2] [see also (Putnam,
Evaluation should facilitate understanding by all
1993; Kawachi et al., 1997; Lomas, 1998)].
stakeholders. Although we refer to community-
based health promotion, the issues raised may
be relevant to health promotion interventions Using this approach, evaluation standards should
at other levels (i.e. national) in other settings maximize human health, quality-of-life and well-
(i.e. workplace- or school-based health promotion) being. This view also recognizes that health has
and other disciplines. an instrumental value rather than being an end in
itself.
We propose that, from a salutogenic orientation,
VALUES AND RELATED ISSUES IN evaluation standards in community-based health
THE EVALUATION OF COMMUNITY- promotion must consider the values and pertin-
BASED HEALTH PROMOTION ent issues of health promotion in appraising the
success or failure of a given initiative (Labonte,
Community-based health promotion is explicitly 1996; Hancock et al., 1998). While the purpose of
concerned with a vision of a preferred future this paper is not to reiterate all such values and
(Labonte, 1996). This vision includes a viable issues, we highlight those that we deem to be
natural environment, a sustainable economic critical.
environment, a sufficient economy, an equitable First, power is central to practice; accordingly,
social environment, a convivial community and the proposed salutogenic view is consistent with
a liveable environment (Labonte, 1993). This an egalitarian approach that rejects professional
‘vision’ can be encapsulated in what Antonovsky dominance in the decision making surrounding
termed as a ‘salutogenic’ orientation to health programme evaluations. Programmes are evalu-
(Antonovsky, 1979; Antonovsky, 1996). We ated ‘by real people in complex organizations that
propose salutogenesis as the core or foundational are marked by historically developed and struc-
value underlying the development, articulation turally organized power relations and human
and implementation of standards for community- wants and interests’ [(Cervero and Wilson, 1994),
based health promotion programmes or policies. p. 249]. Multiple stakeholders (i.e. consumers/
The word salutogenic derives from a com- citizens, practitioners, managers and decision-
bination of ‘salus’ meaning health, and ‘genesis’ makers) have a role to play in evaluation. Health
meaning to give birth. Salutogenesis literally promotion demands coordinated action and
means ‘that which gives birth to health’. In trad- collaboration among governments, health, social
itional public health and community medicine and economic sectors, non-governmental and
approaches, a ‘pathogenic’ perspective, in which voluntary organizations, local authorities, in-
the focus is on disease or illness and its preven- dustry and the media to promote individual and
tion or treatment, most often dominates inter- community health. Scientific and local indigen-
ventions. Adoption of a salutogenic perspective ous knowledge each have a contribution to make
highlights the importance of starting from a towards the evaluation of programmes and
consideration of how health is created and main- policies.
tained through community-based health promotion Secondly, the salutogenic view recognizes that
(Cowley and Billings, 1999). Salutogenesis sug- health promotion is people-centred and collect-
gests a link to notions of ‘social capital’, capacity ivist (Raeburn and Rootman, 1998). It is at odds
building and citizen engagement in that it focuses with a strong emphasis on individual responsi-
on activities that seek to maximize the health bility for health that ignores the impact of social,
and quality-of-life of individuals, families and cultural, economic and environmental determin-
communities. Social capital is defined here as: ants of health. Health promotion seeks to maximize
the inclusion or involvement of individuals or
‘the factor that allows collective action in the public groups who have been historically marginalized,
sphere and for the common good. It is social cohesion, such as Aboriginal peoples and the poor—those
and comprises attention, engagement and trust of both with the poorest health status. Participation and
non-familiar people and the institutions of governance. ownership of the programme by the community

4. 370 J. Judd et al.
facilitates problem solving, builds community made at an individual level can underestimate
competence, and creates successful, sustainable the gains that an intervention might make.
programmes, rather than programmes that are
imposed by outsiders (Eisen, 1994; Camiletti,
1996). Stakeholders of programmes and evalu- OBJECTS OF INTEREST IN THE
ations should recognize that communities are EVALUATION OF COMMUNITY-
dynamic, and socially, culturally and economic- BASED HEALTH PROMOTION
ally heterogeneous. Therefore, strategies need to
be adapted to local needs and possibilities. Before one can set standards for a community-
Thirdly, the proposed salutogenic approach to based health programme there is the need to
community-based health promotion clearly ques- articulate relevant ‘objects of interest’ (i.e. those
tions the dominance of economic rationalism and factors or variables that will be tracked and
market ideology in public policy (Labonte, 1996). assessed in a given evaluation). The objects of
It is explicitly concerned with the aforemen- interest for community-based health promotion
tioned vision of a preferred future that includes a programmes or policies can be quite diverse.
viable natural environment and a sustainable Below we discuss several issues related to the
economic environment. Its emphasis is on equity nature of the community-based health pro-
rather than productivity; and health rather than gramme as it relates to the question of ‘objects of
wealth. interest’ for evaluation.
Underpinning each of these values and the The diversity of potential objects of interest
adoption of a salutogenic approach to setting derives first from the fact that health promotion
standards is the notion of empowerment— interventions can occur at multiple levels
increasingly recognized as a key element in the (biomedical, lifestyle or behavioural, and socio-
evaluation of community-based health pro- environmental) and in diverse settings within
motion (Fetterman et al., 1996). Empowerment a community. Internationally, community-based
is usually described as a process, but may be health promotion has become a major strategy
considered an outcome variable (i.e. an object of that has received prominence through major
interest) when capacity building is a major studies [e.g. Stanford Five City Project, PATCH
activity of a community intervention. Empower- (Planned Approach Towards Community Health),
ment encompasses participation, multidisciplin- and various Healthy Cities approaches]. Many
ary collaboration, equity, capacity building, and such projects have faced major challenges and
social and sustainable development (Hawe, 1994). many have not achieved great changes in health
Approaches such as empowerment evaluation (Syme, 1997; Green and Kreuter, 1999; Potvin
(Fetterman et al., 1996), participatory research and Richard, 2001). Programmes have not always
(Green et al., 1995), participatory evaluation been relevant to those being targeted and inter-
(Health Canada, 1996) and ‘responsive construct- vention methods are not always appropriate to
ive evaluation’ (Guba and Lincoln, 1989) foster those involved. The focus on communities does
the systematic generation of new knowledge and not always take into account the diverse sub-
social capital. This is done through a process that groups and the social context in which people live
builds upon the skills and experiences of all parties and work (Syme, 1997). Nutbeam and his col-
involved, and contributes to quality-of-life and leagues (Nutbeam et al., 1993) concluded that the
well-being. Such evaluations move toward saluto- information gained in large-scale programmes
genesis in that they offer viable possibilities for can disseminate quickly to surrounding jurisdic-
the evaluation of community-based health pro- tions and interfere with classic intervention and
motion programmes. They are commensurate evaluation designs through contamination.
with the unifying approach to setting standards Secondly, health promotion often employs
as detailed in this paper. multiple strategies, including creating healthy
With these principles in mind, evaluation is public policy and supportive environments,
necessarily a collaborative group activity, funda- fostering individual or group skills and capacities,
mentally democratic, participatory, and must strengthening community action and reorienting
examine issues of concern to the community in health services (World Health Organization,
an open forum. Evaluations of community-based 1986). It may try to modify the social context
health promotion programmes limited to aggre- that influences health behaviours as a means to
gates of changes in health behaviour or attitudes achieve improved quality-of-life and well-being.

5. Setting standards in health promotion evaluation 371
In this regard, health promotion is wholly con- in their evaluations. They should attempt to
sistent with what Hamilton and Bhatti termed integrate ‘process’ evaluation (i.e. intervention
‘population health promotion’ (Hamilton and activities, staff performance, etc.) with ‘impact’
Bhatti, 1996), and with notions of ‘population evaluation (i.e. proximal, intermediate changes
health’ and the determinants of health that have in behaviour, lifestyle and the environment) and
come to dominate the health discourse in Canada with ‘outcome’ evaluation (i.e. distal, longer-term
and elsewhere (Frankish et al., 1999). changes in policy, health status, etc.). While one
Community-based health promotion program- need not address every level of evaluation in a
mes are often large in scope, have extended time single project, it is useful to consider each level
frames and require many resources. We argue from a conceptual and planning perspective. The
that health promotion programmes are most recent shift toward a ‘population health’ ap-
likely to be beneficial when they are flexible and proach in which the objects of interest are distal,
responsive to changing realities. Health promotion non-medical determinants of health further
programmes and associated evaluations must complicates the evaluation process.
also accommodate diverse definitions of what the The preceding section highlights some of the
term ‘community’ means. Community has all of complexities associated with defining ‘objects
the following elements—identity, geography and of interest’ for health promotion. Such objects of
politics. Finally, they must struggle with issues of interest to be included in an evaluation need
representativeness and who can speak for a given to be clearly delineated at the outset of the
community (Wiesenfeld, 1996). process. However, in keeping with our definition
Evaluations of community-based health pro- of evaluation, the identification of objects of
motion programmes may be quantitatively and interest is only the first step. Next, programme
qualitatively distinguished from typical experi- planners and decision-makers must articulate
mental studies to the degree that they embrace ‘standards of acceptability’ for each object of in-
a multi-level, multi-strategy vision of individual terest. That is, they must choose where they will
and environmental change. In this regard, a ‘set the bar’ and how they will define the success
wide spectrum of evaluation approaches has been of each element of a given programme or policy.
used in community health promotion. These As Patton states:
incorporate highly structured, methodologically
driven evaluations, including randomized con- ‘Objectives are often set a long time before the
trol trials (RCTs) through to much less rigidly programme is under way or well before an actual
structured, highly participatory forms of research evaluation has been designed. Reviewing objectives
and establishing precise standards of … [acceptability]
and evaluation as discussed in the previous sec-
just before data collection increases the likelihood that
tion (Nutbeam, 1998). Little consensus has been judgement criteria will be up to date, realistic, and
reached about the most appropriate method meaningful’ [(Patton, 1997), p. 304].
of evaluating community-based programmes.
Nutbeam encourages an ‘expansionist’ approach If they are unwilling or unable to set standards
to evaluation that would consider the range of before data collection in a calm and deliberate
strategies employed, the different outcomes from manner, there is no reason to believe they can
those strategies and provide a wide range of do so afterward (Patton, 1997). One means of
potential indicators of success (Nutbeam, 1998). facilitating the setting of standards is to create
Consideration of a broad range of measures of speculative or dummy data for the objects of
success or standards of acceptability fits much interest; this makes the process more concrete.
more comfortably with modern concepts of The explicit articulation and linking of standards
health promotion. The development of indicators and objects of interest will facilitate a worthwhile
and instruments that measure these changes is evaluation that in turn will improve the pro-
equally important [see (Nutbeam, 1998)]. gramme, and the health of communities.
Based upon the complexity of health pro-
motion programmes delineated in the preceding
paragraphs, it is evident that the possible objects THE SPIRIT OF THE TIMES
of interest in an evaluation are vast. Ideally,
community-based health promotion programmes In recent times, many health promotion program-
have a balanced emphasis on processes, impacts mes, their evaluations, and standards of accept-
and outcomes that serve as the objects of interest ability seem to be driven more by a concern for

6. 372 J. Judd et al.
the electoral cycle than by scientific evidence result in drawing inappropriate conclusions
or community relevance. This drive towards regarding health promotion practice.
‘accountability’ stems from a public demanding In summary, the evaluation of community-
greater responsiveness of health professionals based health promotion programmes differs in
and policy-makers, and concerns regarding allo- substantive ways from controlled experimental
cations of economically pressed health resources studies (see next section). Many of the traditional
by governments, health care providers and organ- assumptions of positivist research either do not
izations (Alexander et al., 1995; Solberg et al., hold or are very difficult to apply in a community
1997; Morfitt, 1998; Zakus, 1998). This drive is in setting. Having recognized these difficulties does
tension with a parallel, increased interest in social not mean that community stakeholders are free
capital and the role of community-level factors in from responsibility for evaluation and/or account-
generating healthy communities (Minkoff, 1997; ability. Rather, there is a need for a balanced
Rose et al., 1997; Eastis, 1998; Lomas, 1998). It is approach to evaluation that accommodates both
often in conflict with the idea that programmes community realities and decision-makers’ concerns
may be more effective if they emerge from local for evidence and accountability.
consensus and priorities (Health and Welfare The next section presents an integrative
Canada, 1990; Tabrizi, 1995; Zakus and Lysack, approach to setting standards in community-
1998). based health promotion. Our purpose is to
Within the current economic and political frame evaluation as a win-win, collaborative and
climate there is also strong pressure to incorpor- capacity-building exercise.
ate what is termed ‘evidence-based practice’—a
borrowed medical paradigm that has been applied
mainly to clinical decision making. Evidence- THE USE OF STANDARDS IN
based practice is a framework in medicine for EVALUATING COMMUNITY-BASED
asking questions, tracking new types of strong HEALTH PROMOTION INITIATIVES
and useful evidence, distinguishing it from weak,
irrelevant or useless evidence, and putting it into We recognize that community-based health
practice. The concern with this approach is the promotion (and its evaluation) is a multi-stage
undue emphasis that is placed on RCTs and process involving the setting of objectives, the
meta-analysis (Rada et al., 1999). Not all inter- execution of strategies, the collection of data, and
ventions can be investigated by these methods, an assessment or appraisal of the relative success
nor can they, in the case of health promotion prac- or failure of a given intervention. Glasgow, Vogt
tice, be economically justified. While approaches and Boles proposed a similar comprehensive
that consider clustering issues [see (Simpson evaluation framework, where they argued that
et al., 1995; Thompson et al., 1997; Hayes and multifaceted interventions incorporating policy,
Bennett, 1999)] can come closer to addressing the environmental and individual components should
realities of conducting community-based evalu- be evaluated with measurements suited to their
ations, they are not wholly satisfactory. settings, goals and purpose (Glasgow et al., 1999).
Most stakeholders support the need for a con- They proposed the RE-AIM model for evalu-
ceptually sound evidence base for interventions ating public health interventions that assessed
that aim to promote health. However, the cur- five dimensions: reach, efficacy, adoption, imple-
rent search for evidence using methods and mentation and maintenance.
strategies that do not fit with community It is the process of stating objectives and asso-
realities is unlikely to succeed. Health prom- ciated standards that is of interest here. Three
otion programmes may be at risk of the appli- elements are central to our proposed approach.
cation of inappropriate methods of assessing The first is our strong endorsement and adoption
evidence, an over-emphasis on health status of a salutogenic stance and values base. The
outcomes and individual behaviour change, and second is our recommendation for the use of a
an increased pressure on precious resources comprehensive, diverse set of standards that
(Speller et al., 1997b). These emphases may be reflect different concerns and forms of evidence.
to the detriment of important considerations The third is for the use of an inclusive, empower-
and evidence relating to the building of com- ing process of dialogue that engages all relevant
munity capacity and addressing the broader, stakeholders in the setting of standards for a
non-medical determinants of health. It may also given initiative.

7. Setting standards in health promotion evaluation 373
A ‘standard’, as defined by the Webster’s New The following section identifies eight ap-
Collegiate Dictionary (1979) is something estab- proaches to setting standards that we argue
lished by authority, custom or general consent should be considered in community health
as a model or example. In the health promotion promotion. We discuss the relative strengths,
context, standards of acceptability serve to weaknesses and applicability of each approach.
identify the desired level of outcome and allow The different approaches to setting standards are
all parties to agree on how much change should organized according to what Green and Kreuter
be achieved in return for a given investment term the ‘three world views of population needs
of resources. They serve as targets, which, when and planning’ (Green and Kreuter, 1999) (see
met or exceeded, signal success, improvement or Figure 1).
growth. Standards can be technically, procedur- Arbitrary, experiential and utility standards
ally, system- or outcome-oriented. fall into the upper left circle, in which planning
Many fields and disciplines have utilized (and evaluation) is primarily driven by the
standards of acceptability (McKenzie and Jurs, perceived needs, values and expectations of
1993; McKenzie and Pinger, 1997; Green and practitioners, lay participants or professional
Kreuter, 1999). Similarly, Patton uses the phrase decision-makers. Historical, scientific and norm-
‘standards of desirability’ to evaluate program- ative standards fall into the upper right circle,
mes (Patton, 1997). The use of standards in where planning and evaluation are driven by
community-based health promotion is in keeping empirical, objective data. Finally, propriety and
with the parallel movement toward use of a broad feasibility standards fall into the bottom circle,
range of community health indicators identified wherein the primary concern is for consideration
through a collaborative process (Hancock et al., of available resources, existing policies, legis-
1998) [see also a special issue of Health lation and administrative factors. Objective and
Promotion International (1988), 3 (1)]. For health policy-related standards (which are like scientific
promotion programmes, the standards will be standards) are often given greater weight by
the expected level of improvement in the social, external decision-makers than those in the upper
economic, health, environmental, behavioural, left circle. The intersection of the three circles
educational, organizational or policy conditions represents what has been termed ‘model’ stand-
stated in the programmes’ objectives and repre- ards (American Public Health Association, 1991).
sented in the associated objects of interest for Our position is that there is nothing inherently
evaluation. superior about any one of the eight types of
Fig. 1: Setting standards for evaluation. Adapted from Green and Kreuter (Green and Kreuter, 1999).

8. 374 J. Judd et al.
standards. Judgment and discretion are unavoid- evaluation processes helped to document the need
able, and to some degree desirable, in decision for, and the effectiveness of their programme.
making, which operates within a paradigm or
environment that shapes the process and the Utility standards
outcome(s). Decision making is a social process Utility standards are intended to ensure that a
and methods are social constructions that are community-based health promotion programme
historically determined and situated, and build will serve the needs of programme recipients,
only on existing knowledge (Potvin et al., 1994; community stakeholders, practitioners and
Potvin, 1996). The more important question is: government decision-makers (Joint Committee
which type of standard fits, with which questions, on Standards for Educational Evaluation, 1994).
in what circumstances? This approach may include a priori identification
of stakeholders and their needs, and the selection
of pertinent evaluation questions. Although needs-
Standards based on perceived needs
based or utility standards have the potential ad-
and priorities
vantage of relevance to local circumstances, they
Arbitrary standards may be limited in their representativeness.
Arbitrary standards are a simply declared or
expected level of change, and are most often put
Standards based on objective data
forward by individuals or groups in a position of
authority. An example of an arbitrary standard is Historical standards
one in which a decision-maker sets the standard Historical standards are based on previous
for a given initiative without sufficient consulta- performance and data. Generally, this method
tion with important stakeholders and/or consid- applies to outcome objectives that can be easily
eration of available relevant information. measured such as attendance at clinics, and birth
An advantage of arbitrary standards is the or mortality rates. They are incremental in nature,
efficient way in which they are created. Some and are most useful in situations in which data
disadvantages include that such standards may are routinely accessible.
be biased in favour of their creator’s point of view The use of historical standards has several
and the process may be perceived as dictatorial potential advantages. Practitioners may be more
and non-inclusive. For communities, arbitrary comfortable with these standards because they
standards are often not realistic, and often little have been previously involved in devising and/or
ownership or motivation to meet such standards carrying out these standards. Their skills have
exists. Arbitrary standards are not capacity build- been developed, and can build on previous suc-
ing, and thus practitioners and the communities cesses. A practitioner’s role in the development
they serve are likely to have little commitment of historical standards may be of a technical
to facilitating or participating in such a health- nature, such as collecting and interpreting data.
promoting project. Historical standards are not necessarily a
single point but may represent several points
Experiential standards across time, as in trend analysis. The benchmarks
Experiential standards involve a community’s may be transparent and repeatedly collected in
perceived needs and priorities. They recognize a consistent manner. For example, Serxner and
the value and utility of local, indigenous know- Chung conducted a trend analysis of social and
ledge and are community-specific. Their use is in economic indicators of mammography use in
tension with other types of standards (i.e. norm- Hawaii (Serxner and Chung, 1992). Systems like
ative, scientific) that are based on external data or the Behavioral Risk Factor Surveillance System
information drawn from other jurisdictions. Some offer the necessary longitudinal data. Similarly,
communities may want to emphasize process and/ Hughes and Cox examined breastfeeding initi-
or unanticipated outcomes. Some decision makers ation in Tasmania by demographic and socioeco-
may perceive these issues as a disadvantage in nomic factors for the period 1981–1995 (Hughes
the use of experiential standards. Rodney et al. and Cox, 1999). They noted that trend data is
offer an example of the use of indigenous an important component of infant health and
knowledge as it relates to the evaluation of a nutrition monitoring and surveillance systems. It
community health advocate programme (Rodney is also an important basis for identifying breast-
et al., 1998). Their use of three interrelated feeding promotion needs, prioritizing target

9. Setting standards in health promotion evaluation 375
groups and strategies, and in evaluating the successfully elsewhere as a standard for evalu-
effectiveness of breastfeeding promotion efforts. ation, they may allow for comparative interven-
There are several potential limitations or tions across jurisdictions. Qualities of credibility,
disadvantages to the use of historical standards. efficiency and feasibility are often associated
They may be skewed and data may not be attrib- with normative standards and may enhance the
utable to a health promotion programme when probability that health promotion planners will
unique phenomena occur. Such phenomena may endorse this type of standard.
be the result of new policies and/or media cam- There are several potential limitations to the
paigns within an altered socio-political context. use of normative standards. For communities
These standards may be flawed if they are based or states, normative standards set in relation to
upon inaccurate or biased data. In such incidences, other jurisdictions may be unrealistic and/or
historical standards only serve to replicate an in- unachievable, and may not represent a priority
herent error. By their nature, historical standards focus for a specific jurisdiction. A further prac-
are not appropriate for new programmes since tical difficulty is that of finding an appropriate
there is no pre-existing data. comparison community or jurisdiction. In some
cases, the appropriateness of using one com-
Normative standards munity’s achievements for another community’s
Normative standards, as with historical stand- standard can be questioned. Some communities
ards, are those wherein data such as the state or (i.e. Aboriginal or low socioeconomic groups)
national average for a given health behaviour is have become frustrated with evaluation reports
routinely collected. Normative standards are continually positioning them at the bottom.
usually based on what other programmes or Others question the feasibility of generating
organizations in similar settings have achieved, community-specific data in order to demonstrate
with the advantage that these may be used as a normative comparison.
benchmarks. In this case, the benchmark is a Finally, there are ethical concerns related to
level, and may or may not represent a point in the question ‘what makes a fair comparison across
time. To use this method, documentation must be communities?’. Fair is a relative term, and is
available to practitioners. dependent upon the resources at one’s disposal.
In Canada, the British Columbia Ministry of In this regard, it is important to make a distinction
Health (BCMH, 1994) has produced a frame- between responsibility and reliance. Communities
work and process for screening for local area and health promotion practitioners can only be
benchmarks that involves selective causes of death expected to meet standards that are consistent
(eight indicators), lifestyle characteristics (five with available resources and capacities. Ideally,
indicators) and birth factors (four indicators). an evaluation process can assist communities to
More recently, many governments have adopted be more self-reliant and responsible with the
a ‘report card’ approach that reports on the resources they do have or to acquire additional
health status of a given population, usually on a resources.
year-to-year basis. Associated with such report
cards is the parallel proliferation of a host of Scientific standards
national, provincial or state databases. Each is Scientific standards may be empirically and/or
intended to provide the requisite data for making theoretically based, and are developed from
normative comparisons and planning program- outcomes achieved in controlled studies and gen-
mes or policies. It is important to note, however, erally based on systematic reviews of available
that most of these databases are not oriented literature. Such standards place emphasis on
toward health promotion. Furthermore, indicators RCTs and meta-analysis (Rada et al., 1999).
of ‘community health’ or community-level indi- Recent examples include the movement toward
cators are often excluded (Frankish and Bishop, ‘best practices’ (Sherman, 1999), the development
1999). of ‘preventive practice guidelines’ (US Prevent-
Normative standards may provide a clear point ive Services Task Force, 1996) and systematic
of reference for health promotion planners research syntheses of the type associated with the
and are most likely to be based on ‘objective’ Cochrane collaborations and databases.
(quantitative) data. These provide a measure The major advantage of scientific standards is
of efficiency because practitioners are ‘not that they are viewed as objective, empirical and
re-inventing the wheel’. If they have been used unbiased. They align with a dominant view of

10. 376 J. Judd et al.
‘evidence’, which suggests that such standards consider practical issues such as existing policies,
are more credible and trustworthy than data or regulations and legislation, logistical factors and
evidence generated by other means (i.e. quali- the availability of resources.
tative methods). From a positivist perspective,
this ‘gold’ standard is only achievable through Propriety standards
empirical science of the type associated with Propriety standards are intended to ensure that
RCTs. community-based health promotion programmes
Several disadvantages exist in trying to apply are conducted legally, ethically and with regard
‘scientific’ standards to community-based health to the welfare of community participants (Joint
promotion programmes or policies. Such settings Committee on Standards for Educational Evalu-
make it impossible to randomly assign individ- ation, 1994). Issues such as formal agreements,
uals or groups to a particular community, and it is fiscal responsibility and conflict of interest are
sometimes difficult to identify appropriate com- relevant in consideration of propriety standards
parison or control communities. When the unit [see (Roman and Blum, 1987; Jacob, 1994;
of analysis is an entire community (rather than Starzomski, 1995; Jenkins and Emmett, 1997)].
an individual) it is difficult to manifest the level Brown provides an example of propriety standards
of ‘control’ desired in a typical scientific study. in relation to environmental health issues and
In fact, the complexity of factors associated the US Congress debate over a ‘polluter-pay’
with community life is a key to the dynamics of approach to dealing with violations of existing
community-based health promotion. Attempting legislation (Brown, 1997).
to isolate single variables is contrary to notions of
holism, reciprocal interactions and interdepend- Feasibility standards
ence associated with communities. Feasibility standards are intended to ensure that
The use of scientific standards in community- the programme will be realistic, prudent and frugal
based health promotion may be perceived as (Joint Committee on Standards for Educational
arbitrary, and their ‘goodness of fit’ to the cir- Evaluation, 1994). Feasibility involves consider-
cumstances or needs and expectations of a given ations of cost effectiveness, political viability and
community is questionable. There are also ethical practical procedures. One advantage of including
questions inherent in the notion of ‘control’ feasibility standards is that they may serve as a
communities. Holding some components of a ‘reality check’ with respect to available resources.
community’s capacities constant is contradictory They may also act as a catalyst for securing
to the empowering, skill-developing process of additional resources. One potential disadvantage
community-based health promotion. is that a ‘bottom-line’ mentality may undermine
Scientific standards, when imposed by external innovation and creativity. Richardson questioned
decision-makers (e.g. government or funders) are the common belief that economic evaluation is
a source of tension for most practitioners and hostile to health promotion and that the require-
many health promotion theorists. Randomized ment for health programmes to be cost effective
control trials are time-consuming, expensive, will result in a biased allocation of funds in favour
and require a skill level many practitioners do not of programmes that can demonstrate short-term
possess. Community practitioners may not have benefits as defined by inadequate outcome meas-
access to relevant data, such as the latest pub- ures (Richardson, 1998). He notes the potential
lished evaluations, which are most often contained for economic evaluation to be counter-productive
in academic journals. From a policy perspective, if applied to ‘immature’ projects, and the prac-
government decision-makers may not be able or tical problems inherent in the measurement
want to wait for ‘scientific’ data to be generated. of outcomes in health promotion programmes.
He proposes a four-fold classification based on a
distinction between disease cure, individual health
Standards based on available resources promotion, community welfare and systemic
and existing policies change designed to promote either individual
When it comes to setting standards for community- health or social well-being.
based health promotion programmes, planners, Van der Weijden and her colleagues analysed
practitioners and government decision-makers the feasibility of using national cholesterol
must consider different options with respect to guidelines in general medical practices (Van der
data, evidence and benchmarks. They may also Weijden, 1999). Their programme was developed

11. Setting standards in health promotion evaluation 377
after barriers to working according to the approach, a lead agency, such as the local health
guideline had been investigated. The quality of department, drives the process of articulating
targeting of cholesterol testing did not improve ‘model’ standards by organizing the effort and
following the intervention. This research demon- providing the needed technical expertise in
strated that neither simple dissemination nor an relevant public health practice. The use of a lead
intensive programme had a measurable impact agency approach may, however, raise issues of
on performance of work according to the chol- control, questions about roles and responsibil-
esterol guideline. Stephenson et al. assessed the ities, and has the potential for disempowerment
feasibility of conducting a large RCT of peer-led of the community members.
intervention in schools to reduce the risk of HIV/ Model standards have also been used else-
STDs and promote sexual health (Stephenson where (Speller et al., 1997a). A project to develop
et al., 1998). Questionnaire completion rates of a framework for quality assurance in health
90% indicated considerable enthusiasm for peer- promotion practice in England has recently been
led education among educators and pupils. developed. Six key functions of health promotion
Evaluation of the behavioural intervention was (strategic planning, programme management,
shown to be acceptable to schools, pupils and monitoring and evaluation, education and training,
parents, and feasible in practice. resources and information, and advice and con-
sultancy) were identified. Model standards and
criteria were drawn up for each function, together
A composite approach to setting standards: with guidance on implementation processes.
model standards Model standards may be expressed as pro-
The section above highlights a variety of gramme processes, risk factors or objectives
approaches to setting standards that are relevant related to a specific health outcome. These
to the practice and evaluation of community- standards need to be flexible to accommodate
based health promotion. The presentation of differences in the mix of contexts and services
the eight types of standards recognizes that the available. Stakeholders can therefore participate
various approaches are not mutually exclusive, in determining their own public health priorities
nor are they independent. The diversity of ap- that are compatible with national objectives and
proaches does beg the question of how different targets.
approaches might be combined. These standards represent a form of comprom-
One method of combining a variety of stand- ise or consensus standards. They are generally
ards is the so-called ‘model standards’ approach. established from a consensus of informed opinions
This approach is an amalgam and incorporates by professionals and experienced others, and
elements of each of the other types of standards. may also have the endorsement of professional
The term ‘model’ standard is associated with a organizations. A disadvantage of these standards
specific approach developed in the United States may be the time taken to generate them.
in response to Healthy People 2000 and Healthy Compromise standards may be political in
Communities 2000 through the cooperation nature and depend on the quality of the people
of communities, local health agencies and the involved. Individuals or specific stakeholder
private sector (APHA, 1991). Similar to its groups may come to the table with diverse and
original usage, our use of the term ‘model’ is not sometimes competing/hidden agendas. While
intended to connote that, in and of itself, this ap- ‘model’ standards suggest an optimal mix of
proach represents the ‘optimal’ or best approach standard setting approaches, for some these
to setting standards in community-based health standards may be settling for the lowest common
promotion. denominator.
With the US approach, model standards were
developed to plan programmes and to allocate
resources. As a companion to the Healthy People MOVING TOWARD ‘OPTIMAL’
2000 report, these standards offer community STANDARDS FOR COMMUNITY-
implementation strategies for putting objectives BASED HEALTH PROMOTION
into practice by establishing achievable community
health targets. This method adapts national This paper addresses issues related to evaluation
targets for local relevance and suggests an array and the use of standards in community-based
of activity-based objectives. In the American health promotion. These issues include the

12. 378 J. Judd et al.
definition and measurement of relevant out- views of evidence and definitions of success be
comes and the use of participatory, empowering examined. In the end, ‘optimal’ standards for
evaluation methodologies that assess both the community-based health promotion will be those
outcomes achieved and the processes by which that engage diverse stakeholders in a process of
they are accomplished. collaborative dialogue and decision making. They
We recognize that considerable progress has will maximize the fit of the evaluation process
been made in understanding the complexity of and targets with community capacities, perspect-
undertaking evaluations in community settings. ives and resources. Finally, optimal standards will
We acknowledge the corresponding need for tools, help to yield new knowledge that will contribute
measures and evaluation designs that accommodate to health, well-being and quality of life of indi-
this complexity. viduals, families and communities.
Finally, we recognize two realities. First, that Our hope is that collaborative evaluations will
good science poorly applied will not advance the take into account the varying nature of com-
quality and utility of community-based evaluations. munities while building social capital, community
There is little benefit to be gained from forcing capacity, economic viability and well-being. Well
RCT-type designs to be used in circumstances formulated evaluations can assist funders, policy-
where they do not fit. Both the process and makers, practitioners and communities in linking
outcomes of community-based evaluations must the success of specific programmes or policies to
be relevant to community stakeholders, policy- broader contextual economic, environmental or
makers and/or funders. Secondly, the ‘balloons social issues.
and t-shirts’ approach to community-based health
promotion programmes, in which there is little or Address for correspondence:
Ms J. Judd
no attention paid to evaluation, is equally inap- Territory Health Services
propriate. Policy-makers, funders and taxpayers PO Box 40596
have a right to demand accountability and some Casuarina 0812 NT
measure of the success of health promotion Australia
initiatives.
Our taxonomy of standards, grounded in a
salutogenic values stance, is offered as a potential ACKNOWLEDGEMENTS
means of bridging these ‘two solitudes’. The hope
is to create a win-win situation in which policy- This work was completed while the first author
makers and funders are more supportive of was a Visiting Student at the Institute of Health
evaluation designs (i.e. processes and outcome Promotion Research (IHPR), University of
measures) that fit with community realities, and British Columbia and a Doctoral student in
community stakeholders are more capable and Health Science at Deakin University, Australia.
consistent in evaluating community-based health The authors wish to acknowledge the support of
promotion programmes and policies. Territory Health Services—Long Service Leave
We advocate a shift away from a view of (Darwin, Australia) and Health Canada. They also
evaluation that is dominated by a pathogenic, wish to recognize the support of their colleagues
risk factor and outcomes-oriented perspective in the Institute of Health Promotion Research.
toward a more balanced menu of possible targets From Australia, Penny Hawe, Lawry St Leger
for change and accompanying standards for and Sandy Gifford provided valuable comments
defining success. This suggestion is not at odds on an earlier version of this manuscript.
with standards that are systematic and supportive
of accountability. We conclude by recommending
that each of our eight types of standards [arbitrary,
REFERENCES
experiential (community), utility, historical,
scientific, normative, propriety and feasibility] Alexander, J., Zuckerman, H. and Pointer, D. (1995) The
be considered in planning the evaluation of challenges of governing integrated health care systems.
community-based health promotion programmes Health Care Management Review, 20, 69–92.
or policies. Explicit consideration of this diverse American Public Health Association (APHA) (1991)
Healthy Communities 2000: Model Standards. American
set of standards may be used to engage all stake- Public Health Association, Washington, DC.
holders in inclusive, empowering dialogue. It Antonovsky, A. (1979) Health, Stress and Coping. Jossey-
demands that stakeholders’ respective concerns, Bass, San Francisco.

13. Setting standards in health promotion evaluation 379
Antonovsky, A. (1996) The salutogenic model as a theory to Hancock, T., Labonte, R. and Edwards, R. (1998) Indicators
guide health promotion. Health Promotion International, that Count!—Measuring Population Health at the Com-
11, 11–18. munity Level. Health Canada, Ottawa, ON.
British Columbia Ministry of Health and Ministry Hawe, P. (1994) Capturing the meaning of community in
Responsible for Seniors (1994) Processes, Benchmarks community intervention evaluation: some contributions
and Responsibilities for Developing Community Health from community psychology. Health Promotion Inter-
Councils and Regional Health Boards: Meeting the Chal- national, 9, 199–210.
lenge, Action for a Healthy Society. BC Ministry of Health Hawe, P., King, L., Noort, M., Gifford, S. M. and Lloyd, B.
and Ministry Responsible for Seniors, Victoria, BC. (1998) Working invisibly: health workers talk about
Brown, K. S. (1997) Off the hook: what Olin might mean. capacity building in health promotion. Health Promotion
Environmental Health Perspectives, 105, 44–47. International, 13, 285–294.
Camiletti, Y. A. (1996) A simplified guide to practicing Hayes, R. J. and Bennett, S. (1999) Simple sample size
community-based/community development initiatives. calculation for cluster-randomized trials. International
Canadian Journal of Public Health, 87, 244–247. Journal of Epidemiology, 28, 319–326.
Cervero, R. M. and Wilson, A. L. (1994) The politics of Health and Welfare Canada (1990) Prevention through
responsibility: A theory of program planning practice Partnership: Collaborating for Change. National Strategy
for adult education. Adult Education Quarterly, 45, for Enhancing Preventive Practices of Health Professionals.
249–268. Report of a National Workshop, October 28–31, 1990,
Cowley, S. and Billings, J. R. (1999) Resources revisited: Ottawa. Health and Welfare Canada, Ottawa, ON.
salutogenesis from a lay perspective. Journal of Advanced Health Canada Population Health Directorate (1996)
Nursing, 29, 994–1004. Guide to Project Evaluation: a Participatory Approach.
Cox, E. (1997) Building social capital. Health Promotion Available at: http://www.hc-sc.gc.ca/hppb/phdd/guide/
Matters, 4, 1–4. introduction.htm. Accessed on October 21, 1999.
Eastis, C. (1998) Organizational diversity and the produc- Hughes, R. and Cox, S. (1999) An analysis of breastfeeding
tion of social capital. American Behavioral Scientist, 42, initiation in Tasmania by demographic and socioeconomic
66–77. factors for the period 1981–1995. Breastfeeding Review, 7,
Eisen, A. (1994) Survey of neighbourhood-based compre- 19–23.
hensive community empowerment initiatives. Health Israel, B., Checkoway, B., Schulz, A. and Zimmerman, M.
Education Quarterly, 21, 235–252. (1994) Health education and community empowerment:
Fetterman, D. M., Kaftarian, S. J. and Wandersman, A. (eds) conceptualising and measuring perceptions of individual,
(1996) Empowerment Evaluation: Knowledge and Tools organisational, and community control. Health Education
for Self-Assessment and Accountability. Sage Publications, Quarterly, 21, 149–170.
Thousand Oaks, CA. Jacob, F. (1994) Ethics in health promotion: freedom or
Frankish, C. J. and Bishop, A. (1999) Background Paper and determinism? British Journal of Nursing, 3, 299–302.
Plan for Inclusion of Community Health Indicators in the Jenkins, D. and Emmett, S. (1997) The ethical dilemma of
Canadian Community Health Survey. Prepared for the health education. Professional Nurse, 12, 426–428.
Canadian Consortium of Health Promotion Research Joint Committee on Standards for Educational Evaluation
Centres and the Advisory Committee on the Canadian (1994) The Program Evaluation Standards: How to Assess
Community Health Survey, Ottawa, ON. Evaluations of Educational Programs. Sage Publications,
Frankish, C. J., Veenstra, G. and Moulton, G. (1999) Popu- Thousand Oaks, CA.
lation Health in Canada: a Working Paper. Prepared for Kawachi, I., Kennedy, B., Lochner, K. and Prothrow-Stith, D.
the National Conference on Shared Responsibility for (1997) Social Capital, Income inequality and mortality.
Health and Social Impact Assessments: Advancing the American Journal of Public Health, 87, 1491–1498.
Population Health Agenda. Institute of Health Promotion Labonte, R. (1993) Health Promotion and Empowerment:
Research, University of British Columbia, Vancouver, BC. Practice Frameworks. Participaction Series. Centre
Glasgow, R. E., Vogt, T. M. and Boles, S. M. (1999) for Health Promotion, University of Toronto, Toronto,
Evaluating the public health impact of health promotion ON.
interventions: the RE-AIM framework. American Labonte, R. (1996) Health promotion and population
Journal of Public Health, 89, 1322–1327. health: what do they have to say to each other? Canadian
Green, L. W. and Kreuter, M. W. (1999) Health Promotion Journal of Public Health, 86, 165–167.
Planning: an Educational and Ecological Approach, 3rd Labonte, R. and Robertson, A. (1996) Delivering the goods,
edn. Mayfield Publishing Company, Mountain View, CA. showing our stuff: the case for a constructivist paradigm
Green, L. W., George, M. A., Daniel, M., Frankish, C. J., for health promotion research and practice. Health
Herbert, C. J., Bowie, W. R. and O’Neill, M. (1995) Study Education Quarterly, 23, 431–447.
of Participatory Research in Health Promotion: Review Lomas, J. (1998) Social capital and health: implications
and Recommendations for the Development of Partici- for public health and epidemiology. Social Science and
patory Research in Health Promotion in Canada. Institute Medicine, 47, 1181–1188.
of Health Promotion Research, University of British McKenzie, J. F. and Jurs, J. L. (1993) Planning, Implementing
Columbia and the BC Consortium for Health Promotion and Evaluating Health Promotion Programs: a Primer.
Research for The Royal Society of Canada, Vancouver, Macmillan Publishing Company, New York.
BC. McKenzie, J. F. and Pinger, R. R. (1997) An Introduction
Guba, E. G. and Lincoln, Y. S. (1989) Fourth generation to Community Health. Jones and Bartlett Publishers,
evaluation. Sage, Newbury Park, CA. Sudbury, MA.
Hamilton, N. and Bhatti, T. (1996) Population Health Pro- Minkoff, D. (1997) Producing social capital—national social
motion: an Integrated Model of Population Health and movements and civil society. American Behavioral
Health Promotion. Health Canada, Ottawa, ON. Scientist, 40, 606–619.

14. 380 J. Judd et al.
Morfitt, G. (1998) Report of the Auditor General on trials, 1900 through 1993. American Journal of Public
Regionalization, Accountability and Governance. Auditor Health, 85, 1378–1383.
General’s Office, Victoria, BC. Solberg, L., Mosser, G. and Mcdonald, S. (1997). The
Nutbeam, D. (1998) Evaluating health promotion—progress three faces of performance measurement—improvement,
problems and solutions. Health Promotion International, accountability and research. Joint Commission Journal on
13, 27–44. Quality Improvement, 23, 135–147.
Nutbeam, D., Smith, C., Murphy, S. and Catford, J. (1993) Speller, V., Evans, D. and Head, M. (1997a) Developing
Maintaining evaluation designs in long term community quality assurance standards for health promotion practice
based health promotion programmes: Heartbeat Wales in the UK. Health Promotion International, 12, 215–224.
case study. Journal of Epidemiology and Community Speller, V., Learmouth, A. and Harrison, D. (1997b) The
Health, 47, 127–133. search for evidence of effective health promotion. British
Patton, M. Q. (1997) Utilization-Focused Evaluation: The Medical Journal, 315, 361–363.
New Century Text, 3rd edn. Sage, Thousand Oaks, CA. Starzomski, R. (1995) What do ethics have to do with
Potvin, L. (1996) Methodological challenges in evaluation lifestyle change? Canadian Journal of Cardiology,
of dissemination programs. Canadian Journal of Public 11[Suppl. A], 4A–7A.
Health, 87, S79–S83. Stephenson, J., Oakley, A., Charleston, S., Brodala, A.,
Potvin, L. and Richard, L. (2001) The evaluation of Fenton, K., Petruckevitch, A. and Johnson, A. M. (1998)
community health promotion programs. In Rootman, I., Behavioural intervention trials for HIV/STD prevention
Goodstadt, M., Hyndman, B., McQueen, D. V., Potvin, L., in schools: are they feasible? Sexually Transmitted
Springett, J. and Ziglio, E. (eds) Evaluation in Health Infections, 74, 405–408.
Promotion: Principles and Perspectives. World Health St Leger, L. (1999) Health promotion indicators. Coming
Organization, Copenhagen. out of the maze with a purpose. Health Promotion Inter-
Potvin, L., Paradis, G. and Lessard, R. (1994) Le paradoxe national, 14, 193–195.
de l’évaluation des programmes communautaires multiples Syme, S. L. (1997) Individual vs community interventions
de promotion de la santé. Ruptures, 1, 45–57. in public health practice: Some thoughts about a new
Putnam, R. (1993) Making Democracy Work: Civic approach. Health Promotion Matters, 2, 2–9.
Traditions in Modern Italy. Princeton University Press, Tabrizi, S. (1995) Effective Public Participation in Health
Princeton, NJ. Decision Making: Vancouver Health Board’s Population
Rada, J., Ratima, M. and Howden-Chapman, P. H. Health Advisory Committees. Unpublished manuscript,
(1999) Evidence based purchasing of health promotion: University of British Columbia, Vancouver, BC.
methodology for reviewing evidence. Health Promotion Thompson, S. G., Pyke, S. D. and Hardy, R. J. (1997) The
International, 14, 177–187. design and analysis of paired cluster randomized trials:
Raeburn, J. and Rootman, I. (1998) People-Centred Health an application of meta-analysis techniques. Statistics in
Promotion. John Wiley & Sons, Toronto, ON. Medicine, 16, 2063–2079.
Richardson, J. (1998) Economic evaluation of health Trussler, T. and Marchand, R. (1998) Knowledge from
promotion: friend or foe? Australian and New Zealand Action: Community-based Research in Canada’s HIV
Journal of Public Health, 22, 247–253. Strategy. AIDS Vancouver/Health Canada, Ottawa, ON.
Rodney, M., Clasen, C., Goldman, G., Markert, R. and US Preventive Services Task Force (1996) Guide to Clinical
Deane, D. (1998) Three evaluation methods of a com- Preventive Services, 2nd edn. Williams & Wilkins,
munity health advocate program. Journal of Community Baltimore.
Health, 23, 371–381. van der Weijden, T., Grol, R. and Knottnerus, J. (1999)
Roman, P. M. and Blum, T. C. (1987) Ethics in worksite Feasibility of a national cholesterol guideline in daily
health programming: who is served? Health Education practice. A randomized controlled trial in 20 general
Quarterly, 14, 57–70. practices. International Journal for Quality in Health
Rose, R., Mishler, W. and Haerpfer, C. ( 1997) Social capital Care, 11, 131–137.
in civic and stressful societies. Studies in Comparative Wiesenfeld, E. (1996) The concept of we—a community
International Development, 32, 85–111. social psychology myth. Journal of Community Psychology,
Serxner, S. and Chung, C. S. (1992) Trend analysis of social 24, 337–346.
and economic indicators of mammography use in Hawaii. World Health Organization (WHO) (1986) Ottawa Charter
American Journal of Preventive Medicine, 8, 303–308. for Health Promotion. Health Promotion, 1, iii–v.
Sherman, V. C. (1999) Raising Standards in American Zakus, J. (1998) Resource dependency and community
Health Care: Best People, Best Practices, Best Results. participation in primary health care. Social Science and
Jossey-Bass, San Francisco, CA. Medicine, 46, 475–494.
Simpson, J. M., Klar, N. and Donnor, A. (1995) Accounting Zakus, J. and Lysack, C. (1998) Revisiting community
for cluster randomization: a review of primary prevention participation. Health Policy and Planning, 13, 1–12.