Chapter 23
Developing Interventions When There Is Little Science
Carol W. Runyan and Kimberley E. Freire
23.1. INTRODUCTION
Injury control practice would be easy if only there were packaged programs that were sure to work in any setting and that all one had to do was open them up, implement them and—Voilá—watch the rates of injury decline. It would be like having a simple cake mix that works anywhere with only the addition of water. But this is not the case. Few highly successful packaged programs exist. And even when evidence of success exists, it can be hard to know which program components were the active “ingredients” responsible for the outcomes. There is often no clear recipe for exactly replicating the intervention. Plus, no two settings are completely alike, requiring careful adaptation to the particular environmental and cultural charac- teristics of different populations. Program development is not as straightforward as taking a new medical procedure from one hospital to the next. Injury and the prevention of injury, as with other complex public health problems, result from the interplay of complicated social dynamics that must be accommodated.
Consequently, successful injury interventions rely on sound planning to deter- mine which strategies to implement and which injury risks to change. Interven- tions can include one or more strategies that address injury risks at different levels and different approaches to program and policy. For example intervention strategies may include policies intended to change environments (e.g., changes in playground design), policies intended to modify behaviors (e.g., seat belt laws), and efforts at the organizational level (e.g., changes in the safety climate within a workplace or screening procedures within a health care facility). Interventions may also include strategies directed at modifying behaviors of individuals at risk (e.g., pedestrian education), caregivers (e.g., appropriate discipline of children), and those who make or enforce policy decisions (e.g., legislator’s consideration of smoke alarm legislation, school coaching practices or law enforcement of speed limits).
411
This chapter is designed to help practitioners develop strategies for interven- tion development that uses available and appropriate evidence when it does exist and facilitates progress even when scientific evidence has not been well developed.
It addresses what planners need to consider in assessing an injury problem and structuring an intervention by applying a systematic approach to planning that relies on critical thinking skills. In addition, this chapter includes questions for prac- titioners to consider and suggests important areas for professional development.
23.2. PRINCIPLES UNDERLYING INTERVENTION PLANNING In an analysis of successful interventions for substance abuse, risky sexual behav- iors, delinquency and violence, and school failure, published in the 1990s, Nation et al. (2003) reviewed 35 interventions and cataloged the characteristics of effective programs. This analysis identified program elements critical to success. First, the more comprehensive programs had greater effect. This included both programs that employ multiple interventions (e.g., information and awareness interventions and interventions focused on skill development) and interventions that engaged with multiple settings (e.g., clinical and school settings). In addition, the authors noted that programs that used varied teaching methods were more effective as were those that incorporated a sufficient dosage of intervention, were theory driven, promoted positive relationships, were appropriately timed, were tailored to the sociocultural norms of the community, included outcome evaluation, and were conducted by well-trained staff.
Although Nation et al. (2003) focused on interventions that addressed educa- tional strategies and a limited range of behavioral problems, the list of character- istics associated with successful programs highlights the kinds of issues that injury planners need to consider—for example, the importance of tackling problems at multiple levels and relying on more than one method, with adequate dose or inten- sity over sufficient time periods for the intervention to effect change. Extending from this is the essential process of carefully considering what other interventionists have done and learning from their successes and failures through the use of sound evidence-based approaches.
As described in other chapters, evidence-based planning increasingly is recom- mended for public health, health promotion, and injury prevention interventions, drawing on the lessons from evidence-based medicine (Brownson, Baker, Leet, &
Gillespie, 2003; Glasgow, Lichtenstein, & Marcus, 2003; Rychetnik & Wise, 2004) An evidence-based approach emphasizes the importance of applying sound sci- entific principles to understanding problems and designing solutions. However, public health interventions, including those designed to address injury problems, can be complex and require components addressing multiple interacting levels within a social-ecological framework (Bronfenbrenner, 1979; Stokols, 1992). A social-ecological perspective to injury prevention argues that practitioners should pay attention to risk and protective factors at the level of the individual who is at risk (i.e., the intrapersonal level) as well as factors at the interpersonal level (i.e., the interactions among the individual at risk and other individuals with whom he or she associates, such as friends, colleagues, and family members) (Runyan, 2003). Another level addresses organizational issues, including the various institu- tions with which the person interacts, such as schools, workplaces, and religious organizations; and finally the sociocultural level made up of the social customs
and norms as well as governmental policies, laws, and regulations that affect all the other levels.
Varied types of intervention strategies are available. One relatively simple typology can help the practitioner think through some of the possible approaches along two continua. One continuum, from mandatory to voluntary, addresses the extent to which interventions rely on mandatory (e.g., passing and enforcing laws that require certain behaviors) to coercive (e.g., using insurance incentives), to persuasive strategies (e.g., education encouraging voluntary behavior change). The other dimension refers to the extent to which the strategy is active (i.e., requiring individuals to take steps to effect change) vs. passive (i.e., strategies that are engi- neered into the environment). Examples are given in Table 23.1 (Runyan, Fischer, Moore, Waller, & Hooten, 1993).
Deciding among intervention strategies should, ideally, rely on sound evidence so as to enhance the probability of program success. Unfortunately, in a young field like injury prevention, there is not an extensive base of evidence on which to build.
Furthermore, programs that have been effective for one group may not work for other groups—for example, different cultural groups, adults vs. children or boys vs. girls. Consequently, the practitioner faces a major challenge of deciding what evidence is good enough when making decisions about how to most judiciously use scarce resources to design and deliver the best interventions in a manner that is socially acceptable and culturally appropriate. Practitioners can partner with univer- sity and other institutional collaborators to learn where to find scientific evidence and how to evaluate its quality. In addition, they can partner with other stakehold- ers to collect, interpret, and apply lessons from informal data sources, such as intended audiences, community leaders, and practitioner colleagues. A challenge for all practitioners is making the transition from the established evidence to an intervention that works in their own settings and then contributing back to the field by critically examining these efforts through careful observation and evaluation and by making the results of the work accessible to others. Even more difficult is the need to build a sound approach to addressing a particular injury problem when there is little scientific evidence about what does and does not work. This chapter helps with these challenges.
Table 23.1. Continuum of Intervention Approaches
Approach Active Passive
Voluntary Interventions that rely on Interventions that rely on voluntary
repeated voluntary actions action to put in place a passive intervention (wearing bike helmets, driving safely, (requesting plumbers to set hot water providing adequate supervision heaters at a safe level on installation, of children, storing guns safely) encouraging families to install a fence
around swimming pools, encouraging families to buy only personalized guns) Mandatory Interventions that mandate individual Passive interventions that are required by law
action (requiring seat belt or (requiring air bags in all cars, requiring child restraint use, requiring use pool fencing, requiring that only of motorcycle helmets, requiring personalized firearms be sold) safe storage of firearms in homes
with children)
23.3. A SYSTEMATIC APPROACH TO INTERVENTION PLANNING Sound interventions require careful planning. Figure 23.1 provides a model for planning developed by Freire and Runyan (2006) that represents a combina- tion of a standard program-planning approach, the PRECEDE-PROCEED model
Planning preparation and problem assessment—stage 1
Selection of intervention approach—stage 2
Intervention development—stage 3
Intervention and evaluation implementation—stage 4 Organize planning team
with diverse expertise
Define injury focus and health outcomes
Identify and prioritize injury risk factors (e.g., intended
audience members, engineer, epidemiologist, behavioral scientist, evaluator, policymaker)
• Type of injury event (e.g., falls at home)
• Injury-related health problems (e.g., hip fractures)
• Quality of life issues (e.g., mobility loss)
• Intended audience (e.g., older adults)
• Preevent, event, postevent
• Host, agent/vehicle, physical environment, social environment
• Preevent, event, postevent strategies
• Host, vehicle, vector, physical environment, social environment Haddon Matrix categories
Prioritize behavioral risk factors
Apply criteria to choose evidence-based intervention approaches
Develop goals and objectives
Specify program activities and evaluation procedures
Specify intervention strategies and tailoring needs
Specify evaluation strategies and procedures
Implement and evaluate
Identify potential program strategies to address injury risk factors (using Haddon’s Matrix)
• Predisposing (e.g., age, knowledge)
• Reinforcing (e.g., social support)
• Enabling (e.g., skills)
Figure 23.1.Planning process. (Reprinted with permission from Freire & Runyan [2006]).
(Green & Kreuter, 2005), with the Haddon Matrix (Haddon, 1972, 1980; Runyan, 1998). Throughout the planning process multiple types of evidence are necessary.
Problem assessment requires both surveillance data and other information about the affected population or community. Other types of evidence address what risk factors can be modified through interventions, what interventions work in what situations, and what evaluation approaches have been successful. At each stage, practitioners need to assess what evidence already is available and what is possible to collect to inform planning. The chapter describes four stages of a process, with emphasis on selecting an intervention approach.
23.3.1. Stage 1—Planning Preparation and Problem Assessment
In stage 1 (Fig. 23.1), planners should begin program development with a clear understanding of the injury problem they will address—it’s nature, scope, and magnitude in the given population. Once the problem is clarified, planners can judge whether the problem is important and how it fits within their organization’s mission and scope of work. Planners must make choices among priorities within the context of scarce resources while balancing professional judgments, commu- nity interests, and community will. Some problems have high incidence but lower severity (e.g., minor sprains/strains encountered during sports), whereas others are of lower incidence but high severity (e.g., concussions or amputations). Though difficult to balance these two factors, especially when resources are limited, the planner must make such judgments. As part of this process, one also needs to consider the extent to which the problem is preventable (Green & Kreuter, 2005).
Although progress will never be made if hard problems are avoided, there may be advantages to starting with important (by incidence and severity) problems that can be addressed successfully, then tackling harder problems building on a history of success.
This process also requires considering both one’s professional wisdom and that of the community. Without community buy-in, even the best intervention may have limited success. On the other hand, relying solely on community perspectives and not applying what trained professionals know is equally unwise. Learning to balance these influences in helping a community arrive at its own decisions about what topics and issues to tackle and how to tackle them requires finesse and experience.
There is a growing literature on community participatory action research that pro- vides guidance about these issues (Leung, Yen, & Minkler, 2004; Minkler, 2000).
It is also helpful to affirm that tackling a given problem is within the mission of the organization and to consider whether and how affiliating with other groups would strengthen the organization’s capabilities, recognizing that no one organiza- tion can do everything (Box 23.1).
A practitioner cannot accurately assess how a program affected an injury problem unless the problem can be clearly defined. Therefore, if it is not possible to clearly define the problem or if the problem is of limited importance, one should consider moving on to other issues. If the problem is important but not fully within the scope of the organization, carefully assess partnership opportunities. And, if the problem is important, but the evidence is of limited availability or quality, one should develop the foundation more fully before proceeding. This argues for a team of people having expertise in epidemiology and surveillance as well as other forms of needs assessment who can help interpret data. Consider the issues iden- tified with stage 1 in Figure 23.1 as part of defining the injury focus and health outcome of interest and prioritizing the risk factors to address.
The Haddon Matrix can be used here as a means of helping identify risk factors (an example is given in Table 23.2). The matrix combines the time element inher- ent in the process of an injury (the rows) with elements that are either modifiable risk factors or protective factors consistent with the social-ecological framework (the columns). The rows define the preevent phase (before the person comes in contact with the injury producing energy), the event phase (during the time when the energy is causing injury), and the postevent phase (after an injury has occurred but while it is still possible to influence the severity of injury or the outcome).
The risk factors are classified in terms of individual (host) factors particular to the biology or behavior of the at-risk person; factors associated with the mechanism by which the energy is delivered (inanimate vehicles such as particular products or animate vectors such as assailants), and the characteristics of the physical environ- ment (workplace design, home construction) and social environment (laws, poli- cies, social climate, norms) that influence and are influenced by all other risk and protective factors. Information from Chapters 1–20 about specific injury problems and cross-cutting issues helps with this process.
23.3.2. Stage 2—Selecting an Intervention Approach
Once the risk and protective factors are understood, the process requires selecting an intervention approach. The majority of this chapter focuses on this aspect.
Three elements are included in stage 2 (Fig. 23.1). One can start in different places within this step and work through each element and will likely need to cycle through the process more than once. Ultimately, the goal at this stage is to decide who will be the intended audience(s) of the intervention and what specific strate- gies or combination of strategies will be applied.
Box 23.1. Questions to Consider at Stage 1
• Have you clearly defined the problem?
• What additional information do you need to define the problem?
• Are the incidence and severity of the problem sufficient to warrant attention?
• How important is the problem relative to other priorities from the perspective of the community (i.e., those affected) and the professionals responsible for addressing the problem in the community?
• Does tackling this problem fall within the scope of your organization, collaborating groups, and resources?
• What additional resources can you obtain to tackle the most important problems?
• What evidence exists about the problem in peer-reviewed journals to help you under- stand and prioritize what risk or protective factors to address with an intervention?
Specifically, consider the following:
How much evidence is there?
How good is the quality of the evidence?
How applicable is it to your situation?
Table 23.2.Haddon Matrix Used to Identify Risk Factors: Poisoning Risk Factors That Influence Injury Process Agent/Vehicle/VectorSocialEnvironment (energy source and means of (interpersonal, Hosttransmitting—e.g., chemical Physical Environment organizational, Phasea(child and caregiver) products and packaging) (general environment) cultural) Pre-event• Child’s developmental factors • Ease with which the package • Absence of locking devices • Lack of regulations (e.g., (before ingestion) (e.g., curiosity, lack of judgment) can be opened on cabinets marketing of substances • Parent’s characteristics (e.g., • Attractiveness of the substance • Cupboards where poisons in containers that look distractedness, lack of • Lack of visibility and/or stored in places easy for young like food) knowledge of poison hazards, understandability of warning child to reach poison storage behaviors, labels (e.g., poorly placed, not literacy)visible, not clear, not in language of the reader) Event• Child’s secrecy about ingesting • Chemical composition of the • Presence of hiding places • Inaccessible health care (during ingestion) (e.g., hiding) poison where young child could system that deters parents • Parent’s ability to notice unusual • Substance easy to swallow in ingest substances and not from seeking guidance behavior on part of child who is sufficient quantities to do harm be noticed about potential ingestion ingesting Postevent• Child’s ability to communicate • Chemical agent that does not • Lack of accessibility for • Lack of a poison-control (after ingestion) with adults about what was have antidote emergency vehicles facility or emergency ingested medicalcare • Parent’s ability to access and • Poor publicity communicate with poison- of poison-control resources control resources and how to access them aAt which the risk factor has infl uence.
23.3.2.1. Consider Behavioral Elements
During this stage, behavior or behavior-related components are considered, includ- ing predisposing, enabling, and reinforcing factors (Green & Kreuter, 2005). Some- times these will focus on the behavior of the individual affected by the injury risk;
other times it may be a caregiver (e.g., parent, teacher, health care provider) and other times people who make decisions about design of the physical environment or products (e.g., engineers, architects) or about policies (e.g., legislators, corpo- rate executives). Predisposing factors are individual characteristics such as develop- mental stage, knowledge, attitudes, and values, all of which affect how individuals perceive an intervention and the intended behavior change. Enabling factors are the skills and resources necessary to facilitate behavioral and environmental change (e.g., access to funding, appropriately trained staff, political will). Reinforcing factors provide negative and positive feedback that affect behavior maintenance.
They include such elements as social relationships and the structural factors (e.g., workplace safety climate) that facilitate adherence to worker safety policies (Barling
& Frone, 2004; Hofman & Tetrick, 2003). To learn more about these concepts see Green and Kreuter (2005) and Freire and Runyan (2006).
23.3.2.2. Develop Intervention Ideas
At this stage of the planning process, it is useful to apply the Haddon Matrix in the more traditional way it is used—that is, to generate ideas about interven- tions. The model (Table 23.3.) facilitates the consideration of a range of strategies to address interventions within the social-ecological framework, helping planners identify strategies directed not only at the person at risk but also at the mechanisms through inanimate vehicles (e.g., consumer products) or animate vectors (e.g., assailants) that transmit injury-producing energy. This could include redesigning potentially dangerous products such as toys, cars, household appliances, and fire- arms that serve as the vehicles by which the energy is transferred. In addition, one can address behavioral changes in the vector that transmits energy (e.g., interven- ing with perpetrators to prevent violence-related events). In addition, developing interventions aimed at changing the physical environment might include strategies related to architectural or engineering designs of the home, school, workplace, playground, or highway, such as installation of fencing on bridges to prevent sui- cides, speed humps on roads, smoke alarms in homes, and improved playground surfacing. The model also highlights the social environment, which may include strategies directed at altering systems, policies, and norms within the institutional or organizational context (e.g., work hours and staffing patterns in a factory, how protocols for diagnosing and reporting child maltreatment are applied in a given health care facility, and how schools monitor playgrounds and crosswalks) as well as broader social and cultural norms. This could be policies that are designed explicitly to affect injuries (e.g., controls on the sale of hazardous consumer prod- ucts, bike helmet ordinances, required fencing around swimming pools) and social policies that may be designed for broader purposes but have an influence on injury (e.g., policies that affect availability of jobs, welfare programs, or taxes on alcoholic beverages). Social norms also are powerful influences on behavior and on injury prevention. Examples are norms about parenting, child discipline, and supervision;
the availability or storage of firearms; the consumption of alcohol; and the use of designated drivers.
Table 23.3.Haddon Matrix Applied to The Problem of Preventing Childhood Poison Ingestion Factors to Be Altered by the Intervention Agent/Vehicle/Vector (energy source and means of Social Environments (interpersonal, Host transmitting—e.g., chemical Physical Environment organizational, Phasesa(child and caregiver) products and packaging) (general environment) cultural) Pre-event• Teach child to avoid • Childproof packaging of • Ensure that homes are built • Counseling by pharmacists and (before ingestion) poisonous substances medicines and toxins with locked cabinets for storing physicians about toxicity of • Clear labeling of toxic medicines and other toxins chemicals and safe storage products• Encourage stocking and • Alter pharmacist dispensing to marketing of locking cabinets nonlethal doses in package by home-building stores • Regulate availability of toxic substances that are easily mistaken for edible items Event• Improve parent–child • Ensure that drugs are • Household sound or video- • Ready access to poison-control (during ingestion) communication to help in packaged by manufacturer in monitoring systems to assist in centers responding to questions and amounts too small to cause monitoring child behavior describing ingested material serious consequences even when in another room • Teach parents how to if all are ingested identify unusual behavior on part of child Postevent• Teach parents how to • Package in containers that • Clear directions, marking of • Good-quality, affordable emergency (after ingestion) contact and communicate are easily stored to reduce residence to facilitate access by medical care available effectively with temptation to repackage in emergency personnel poison-control facility ways in which substance is not easily identifi ed aAt which the intervention has its effect.
The other dimension of the matrix facilitates consideration of interventions at various phases of the injury process. Examples of interventions having their effects at different phases are those that prevent a potentially injury-producing event from occurring (e.g., improved brakes to reduce likelihood of a crash), those that reduce the likelihood of injury during an event (e.g., seat belts protecting against injury during a crash), and those that facilitate rescue and recovery (e.g., in-car global positioning devices to facilitate location of wrecks by rescue personnel).
23.3.2.3. Choosing Interventions
The other part of stage 2 (Fig. 23.1) is the application of the third dimension of the Haddon Matrix which provides planners with assistance in weighing decisions about which program, among multiple options, to select (Runyan, 1998). This dimen- sion, derived from literature on policy analysis (Haskins & Gallagher, 1981; McRae
& Wilde, 1979; Patton & Sawicki, 1993), suggests a variety of value criteria that a planner might want to consider in judging the relative merits of different interven- tion ideas. Each of these types of criteria relies on different types of evidence either gathered by the planner or derived from other sources. In some cases, the evidence will be directly relevant to the injury problem at hand. In other cases, one may have to rely on information collected for other purposes and make judgments about the applicability to the injury intervention being considered (e.g., parental acceptance of guidance from pediatricians about diet and exercise may provide clues to the extent to which they will be receptive to guidance about discipline or use of child car seats). However, assuming that responses to all problems or solutions are similar should be done with caution, given the complexities of human behavior.
First and foremost is the issue of program effectiveness, which relies on prior intervention evaluations and documentation of the extent to which an intervention worked in other settings. In considering effectiveness, all the issues associated with finding, assessing, and critiquing available scientific evidence and its applicability to the given situation are essential, as discussed more extensively in Chapter 21.
Another important factor to consider is the extent to which a given interven- tion option is compatible with the preferences, wishes, and cultural beliefs of the affected population. The extent to which the affected community’s views and cul- tural perspective can be addressed will often greatly affect the likelihood of inter- vention effectiveness. This is important not only to determine which intervention to apply, as it closely relates to the political feasibility of the intervention as well as to issues of cultural sensitivity, but also to ensure that the implementation strategies put into place are culturally appropriate, as discussed below.
In addition, planners also need to consider other criteria such as the costs of doing the intervention (e.g., implementation costs) balanced against the costs of not doing it (e.g., medical care expenses, lost wages). Another consideration is equity. There are two types of equity. Vertical equity concerns equalizing risk through unequal treatment of certain groups so as to make the groups more equal (e.g., car seat giveaway programs for low-income families). Horizontal equity is often addressed by providing universal interventions or services that affect everyone equally (e.g., development and enforcement of building codes for all types of dwell- ings). For many types of interventions, particularly those that involve mandated changes, there are issues involving restriction of personal freedoms. In this case, it is important to consider how much and what type of restriction on freedom is associated with the intervention and whose freedom is affected. An intervention to
protect one group (e.g., freedom of children to walk safely to school) may come into conflict with the freedoms of another group (e.g., drivers wanting to drive faster on certain roadways). Freedom issues often create intense public discourse as in the case of efforts to limit access to certain types of firearms or to require motorcycle helmets. Variations in culture often relate to acceptance of specific safety measures that may restrict freedom.
Another delicate issue is that of stigmatization, a value sometimes considered in intervention planning from the standpoint of avoiding stigmatizing individuals who have some distinguishing characteristic (e.g., poverty, disability, and mental health problems) but is sometimes deliberate (e.g., identifying prior sex offenders in a community). Often stigmatization issues affect the most vulnerable popula- tions, so it requires careful attention.
Finally, once other features of the intervention have been considered, it is important to examine technological and political feasibility. These two factors are often related to effectiveness and equity and freedom issues. As noted above, politi- cal feasibility relates to the preferences of the affected community and to many political forces that affect how decisions are made. Again, input from intended audience members can help planners determine how politically feasible selected strategies will be within a given community. To be applied, an intervention must also be technologically feasible (e.g., the technology for intelligent vehicle design has enabled new safety approaches not available a decade ago). Though feasibility issues may present obstacles, they often can be overcome if other criteria all point toward adopting a particular intervention. In addition, any given situation may require consideration of additional or different factors. These should be identified at the outset, with evidence about how much each option does or doesn’t achieve the desired criterion considered along with the other criteria suggested here (Box 23.2).
Runyan (1998) describes a process for using this framework to guide plan- ning that includes defining which criteria are important and identifying evidence to address the concerns. The planner needs to decide, with guidance from other stakeholders and community members, how much to emphasize each of these or other criteria in making choices among interventions.
23.3.2.4. Finding Alternative Sources of Evidence
In stage 2, it is important to search widely for available evidence about prior inter- ventions to understand effectiveness and value criteria (e.g., cost, political feasibil-
Box 23.2. Criteria to Consider at Stage 2
• Effectiveness
• Preferences
• Cost
• Equity
• Freedom
• Stigmatization
• Feasibility
• Others suited to the situation
ity, community views). The first step is to look for articles published in peer-reviewed journals. The purpose of the peer-review process it to allow critical reviewers to consider a manuscript with a skeptical eye and ask the author to respond to detailed questions before it is determined that the paper is of high enough quality to merit publication. Information indicating if a given journal is peer reviewed is usually found in the section of the journal or journal Web site providing information for authors or by asking a librarian.
Another consideration at this stage is to think about the consistency of results across multiple studies. It is important to examine the whole body of evidence, not just one or two reports. If there are multiple evaluations of the same interven- tion all pointing in the same direction, even though none is perfect, the weight of the evidence as a whole can be used to help make a judgment. The more there is consistency in the success or failure of trying a given intervention implemented in multiple places, the more likely the result can be successfully generalized to other settings. This does not mean that an intervention tried in only one place is not helpful. But an intervention having been tried in multiple places with con- sistent findings can strengthen one’s confidence in replicating it. On the other hand, taking this too literally may squelch innovation when, in fact, new ideas are needed.
Practitioners must be skeptical but recognize that there may well be successful interventions for which no published report exists. This could be for a variety of reasons. Some who conduct interventions do not do thorough evaluations or do evaluations but never publish their results. It is also important to remember that getting published takes time, so information about newer interventions may take longer than a year to be published. This emphasizes the value of maintaining net- works of practitioners who can provide direct information about their experiences and indicate when new findings are being released. In addition, useful information does appear in places other than the peer-reviewed literature.
23.3.2.5. Other Sources
Gray literature describes project reports, technical reports, and monographs. These are written documents that are often filed with funding agencies but have not undergone review by a peer-reviewed journal. Sometimes technical reports can be an excellent source of data on injury incidence and trends as well as local and state injury prevention efforts. Agency collaborators, such as program officers and evaluators, can help practitioners discern the quality of program data by providing a context for data collection and reporting limitations.
There is an increasing amount of information available on the Internet but little or no quality control. Good information can be found on the Web sites of relevant federal agencies (Centers for Disease Control and Prevention, National Highway Traffic Safety Administration, Department of Labor) and many reputable nongovernmental organizations (Insurance Institute for Highway Safety, State and Territorial Injury Prevention Directors Association, the Society for the Advance- ment of Violence and Injury Research) or via electronic versions of peer-reviewed journals. Lists of research studies in the peer-reviewed literature are updated regu- larly online at www.safetylit.org.
Starting with searches on reputable Web sites is a good way to get a feel for who is working on an injury problem and which strategies have been implemented.
However, it is important to remember that just because it looks slick does not mean
it is of high quality. In addition, groups use terms such as “best ” and “promising ” practice in different ways and may not apply the same standards of evidence review discussed in this book. Another aspect of this search is to identify whether there are interventions that are considered strong by reputation—that is, known by practi- tioners to work even if the evidence is not yet available. The use of gray literature or reliance on reputationally strong programs should never substitute for trying to find peer-reviewed literature but is an appropriate addition to the search process and sometimes is the only option available.
23.3.2.6. Judging the Evidence
There are numerous factors to consider in assessing evidence. It is important to consider whether the studies match the problem at hand. This is where having a clear understanding of the problem, derived from careful surveillance and needs assessment is useful. It is also important to assess whether the evidence available can be generalized to the particular setting in which it will be adopted. What
“plays in Peoria” may not be equally applicable in Miami or in rural New Mexico.
In doing this, consider the specific social and cultural elements of the problem being addressed and the aspects of the interventions as they have been previously applied. Imagine applying this exact intervention in your setting. Does it seem like it would be culturally appropriate and effective or is something off about it? Behavior is complex to understand and to change, and no two situations are exactly alike. Hence it is best to rely on information with some degree of similar- ity to the situation at hand and make educated guesses about the transferability of the findings. Although this approach is not as good as evidence from a highly similar setting, with a very similar population to address the same problem, useful information often can be gleaned and is far better than starting without benefit of others’ experiences. Often this requires subtle judgments that can benefit from discussion among several people knowledgeable about the problem, the setting, and program planning. This is another argument for a team approach that brings varied expertise to the planning process.
Depending on one’s success in finding evidence to help structure an inter- vention, practitioners may be faced with the decision of starting fresh vs. deciding whether to wait until more of a scientific base is developed before tackling the issue.
Another consideration is that of thinking through the potential that, by addressing the problem without tested methods, one could actually do harm. For example, screening women to identify domestic violence without a plan for what to do when abuse is discovered may be more harmful than helpful if revealing the abuse puts women at risk with their partners or endangers their children.
23.3.2.7. Developing an Intervention Strategy
As with any public health issue, injury problems can be approached through a wide range of intervention options. There are two key elements to consider. One is to determine what needs to be changed and how that change will result in the desired outcome. For example, the desired change may alter the injury-causing product, the physical environment, social norms, or the behavior of individuals or some combination of these. Two, there needs to be thought given to what is going to be done to bring about this change. For example, if individual behavior change is the key, then understanding issues of knowledge, attitudes, beliefs and behaviors
is critical, and interventions might be devised to address any and all of these. If, on the other hand, the focus of the intervention is determined to require changes in the physical environment (e.g., the design of a new playground), then it is critical to determine what is required (e.g., hire a playground designer, change a playground maintenance policy, find funds to purchase new equipment, educate the school board or county commissioners about the need to make the playground safer, and/or raise the awareness of parents about the lack of safety in the playground).
In both these examples, the construction of a logic model, as described below, is helpful to clarify the relationships among the factors that need to change to achieve the desired effect (Gielen & McDonald, 2002). As a consequence, one can then disentangle the elements of the problem and the solution and decide where and how to target the intervention—for example, a mass-media campaign to change community awareness, an educational campaign to change individual behaviors, an advocacy strategy to alter policy or raise resources, or an engineering approach to redesign the equipment.
23.3.3. Stage 3—Intervention Development 23.3.3.1. Adopting and Adapting Interventions
If sound evidence demonstrates that a given intervention is effective, or at least promising, then it is wise to consider either adopting the intervention or adapting it to your setting rather than starting over. A number of questions are suggested below to help you think about this. Do not underestimate the value of learning from others who have attempted to address the same problem and/or implemented the intervention before. Be sure to ask people to share their experiences of program failures and successes.
If the evidence does not fit the circumstances in which new planning is taking place, no matter how high the quality, it may be helpful to look at other literatures on related problems and/or similar settings so as to consider whether to adapt information from other domains to the problem at hand. For example, if one is thinking of an initiative to increase use of carbon monoxide detectors, it could be helpful to look at smoke alarm distribution efforts as a close parallel. Likewise, an effort to improve use of fall-protection devices (i.e., harnesses) by young construc- tion workers might benefit from literature on efforts to increase use of protective devices among similarly aged athletes (e.g., football helmets or knee braces). Inter- ventions for the same population aimed at changing other behaviors may also be informative (e.g., efforts to increase construction worker’s use of steel-toed boots or adherence to protocols for operating equipment near power lines). Finally, it may be useful to consider the transferability of intervention strategies that have been successful with a different health or injury problem. For example, examining how media campaigns have been successful in reducing tobacco initiation among teens (Hopkins et al., 2001) or alcohol-impaired driving (Elder, Shults, & Sleet, 2004) may be helpful in thinking about the applicability of using media strategies to intervene on specific types of injury or violence problems. Given that public health approaches to injury control are not as longstanding as attempts to address other problems, there is much to be learned from other areas of public health (e.g., infectious or chronic diseases, reproductive issues). However, any of these efforts to transfer experiences from one population, topic, setting, or strategy to another must be done with both caution and careful judgment. Even with a peer-
reviewed paper, it is often wise to contact those who have done the intervention before to learn about their experiences and how it may or may not be transferable to a new locale (Box 23.3).
23.3.3.2. Developing New Interventions
In developing new interventions, a critical stage of planning involves the develop- ment of clear goals and objectives. A goal articulates the overall purpose of the intervention including what change will occur and who or what will experience the change. Two examples are to reduce mortality from drowning among children age 3 years and younger (injury goal) and to increase use of fall-protection devices by construction workers (behavioral goal). Objectives are more specific, defining more concretely what will be accomplished by whom within a specific time frame.
Box 23.3. Questions to Consider in Adopting or Adapting an Existing Intervention
• What types of evidence are available (peer reviewed, gray literature)?
• Are there interventions that have a reputation of being good (reputationally strong programs)? What can you learn from these programs?
• What is the quality of the evidence from these sources?
• How much evidence is there?
• How consistent are the results across studies and settings?
• How well did the intervention(s) work?
• Do the results generalize to your setting?
How similar to the current population is the population from which the evidence was derived (age distribution, gender distribution, ethnic variations, cultural traditions, injury risk factors, or receptivity)?
To what extent might these differences between the populations used in other intervention studies make a difference in how the intervention would work in the new setting?
How similar is the setting to your setting?
Who carried out the successful intervention (specific types of professionals vs. com- munity members or volunteers)?
Can the same kind of person or group carry out the intervention in your setting?
Would it be culturally appropriate to do so?
How will the setting affect program implementation?
What changes would need to be made in the way the intervention is delivered (sequence and timing issues or the general approach)
What other things about the approach would need to vary (did the other program have a well-known community leader recruit the participants, but you have no such leader)?
How likely are any of these kinds of factors to alter how the intervention works in the new setting?
What, if any, elements of the original intervention are not feasible to implement in your setting and how much difference will this make?
What, specifically, do you need to do to adapt this to your setting?
• Are there interventions from parallel literatures (about similar problems in the same or similar populations) from which you can extrapolate?
Ideally, practitioners should be able to link program objectives to specific evalua- tion measures to demonstrate their program’s affect. Therefore, objectives should be specific, measurable, achievable, relevant, and time bound (i.e., “SMART”) (Green & Kreuter, 2005). Examples of specific objectives are to decrease the rate of drowning among residents of North Carolina by 10% within the first 3 years of the program (injury objective) and during the first year of the program, to increase safety harness use nationally among construction workers by 30% from baseline.
It should be very clear from an objective what the criteria of success are and what sources of information are needed to measure success.
In addition to developing goals and objectives, planners can create a logic model to illustrate how program activities relate to changes in risk and protec- tive factors and injury outcomes (W. K. Kellogg Foundation, 2004). Logic models can help planners consider the causal pathway through which an injury process may occur and where in the chain of events an intervention serves to interrupt the process and create new pathways to the desired end. They can also help plan- ners identify obvious gaps in resources, activities or risk factors that will influence program implementation, and injury events. Hence a good logic model helps plan- ners articulate the “theory of change” behind the program.
23.3.4. Stage 4—Intervention and Evaluation Implementation
As discussed in Chapters 26 and 28, fidelity to the original intervention is criti- cally important when replicating someone else’s intervention. Unfortunately, many reports of effective programs often have brief descriptions of program implementa- tion and little discussion of lessons learned. Practitioners can usually contact the original program developers to learn more about the process and potential pitfalls and to get advice about how to adhere to the protocol. It can be useful to ask for copies of forms that have been used to monitor program delivery, participation, or other attributes of process because they can facilitate more precise replication. The interview questions suggested in Box 23.4 can be helpful here as well. Practitioners should understand how any changes or omissions to the original program might
Box 23.4. Suggested Interview Questions
• What worked well in carrying out this intervention and what didn’t go as planned?
• Did some elements of the intervention work better than others? Which ones and why?
• What positive or negative unanticipated consequences did you discover?
• If you could do the intervention over again, what would you do differently this time around? Be specific about any and all elements that might need revision or refinement.
• What aspects do you think are absolutely critical to keep the way you did them? Why is that?
• What challenges can I expect if I try to carry this out in my setting (describe your setting)? Can you give me advice about overcoming these challenges?
• Do you know others who have tried this same intervention, or one very similar to it, from whom I might learn about their experiences?
effect desired changes in program outcomes. Some adaptation is likely. The trick is to figure out how much adaptation is appropriate without changing the interven- tion so much that the prior evaluation results are invalid. This is a judgment call, best considered by a multidisciplinary team experienced in program design. Once adapted, fidelity to the plan established for the new setting is critical.
The purpose of process evaluation is to learn about how the program was carried out. This is particularly important for programs that are not as successful as hoped. It allows the planner to consider whether the program was inherently limited in design or if the problem was in the area of program implementation. If someone has conducted a new intervention, it is hoped that they have documented exactly what constituted the intervention, who carried it out, what they did, and how it may have been modified along the way for any of a variety of reasons. For example if one designs an intervention in which coaches are to encourage their soccer players to voluntarily use shin guards, it is important to know exactly what the coaches did (e.g., made a rule, created incentives, told the players they would lose playing time if they didn’t wear them, gave a handout to players about the benefits of shin guards, asked parents to buy shin guards, reminded players to use the shin guards before every game). Perhaps it is clear that the intervention was a reminder to all players before every game. Did this actually get done or did the coach forget to remind players at away games? Did the coach do the reminding at some games and the trainer other times? Was the reminder done in such a way that the players were sure to hear it or was it simply written on the board in the locker room? Any of these variations could potentially affect the outcome of the intervention. If the intervention fails, it is critical to know what was actually done so proper corrective action can be taken rather than to assume the intervention has no worth. Likewise, if the intervention succeeded, it is important to know what was done so that others who want to replicate it will know exactly what the intervention contained.
Unfortunately, process information often is not included in reports of program evaluations. In the absence of process data, the intervention has little meaning and is merely a black box. This would be akin to a surgeon telling another doctor how to do a new type of open-heart surgery by merely saying, “take the patient to the operating room and perform surgery” instead of describing the procedure in detail.
In addition to contacting the person who carried out the program and interviewing him or her about process issues, one might consider inviting someone from the prior project’s operational team to serve as a consultant.
The extent of adaptation required often is determined by responses to the kinds of questions listed above. If extensive change is envisioned, it may be wise to plan for and conduct a formative evaluation as part of the adaptation process.
Issues to think about in adapting an intervention include similarity of the settings in which the intervention has been applied before and the situation in which new application is planned, considering what adjustments for culture, language, or his- torical changes might be in order. In addition, one should always consider carefully whether certain types of changes may alter the intervention too much. As always, effort should be made to measure any changes made via careful process documenta- tion and evaluation. This will not only help keep the adaptation on target but also permit interpretation of differences in effect compared to the setting in which the interventions were originally implemented. Finally, careful conduct and publica- tion of the process and outcome of your own evaluation is critical to the further development of the field.
23.4. MAINTAINING SKILLS IN INJURY PREVENTION PRACTICE Developing, conducting, and evaluating interventions, regardless of the presence of prior evidence, require skills that improve with experience. To help identify the basic competencies for injury control practitioners, the National Training Initiative for Injury and Violence Prevention (NTI) has developed a set of core competencies, drawing on the expertise and input of a national panel of experts. (NTI, 2005) Also of importance are skills in critical thinking, getting help from others, and contribut- ing to the knowledge base in the field.
23.4.1. Developing Basic Competency in Injury Prevention Practice Nine core competencies have been defined by the NTI (Table 23.4). For each there are multiple specific objectives outlined for achieving the competency (available at:
www.injuryed.org) (NTI, 2005). These core competencies may be achieved by any individual at any level of proficiency, depending on his or her role and the setting in which he or she works. Many of these skill areas are discussed elsewhere in this book. These are intended to help guide individuals’ professional development as well as help organizations assemble teams with complementary expertise.
23.4.2. Critical Thinking
Also important to good science and good practice is asking questions with a skep- tical eye, making careful observations, systematically approaching problems, and carefully recording information about what is observed and learned, and sharing that information with others. One needs a certain degree of skepticism when con- sidering what others have done, not trusting information at face value but rather carefully examining its merits and applicability (as discussed above). Also important is a willingness to identify one’s own assumptions and biases about how things work and be sure they are based on evidence and not just beliefs without basis in fact.
It is often useful to ask: How do I know this? Am I sure I know this? Do I need to
Table 23.4. Core Competencies for Injury and Violence Preventiona
• Ability to describe and explain injury and/or violence as a major social and health problem
• Ability to access, interpret, use, and present injury and/or violence data
• Ability to design and implement injury and/or violence prevention activities
• Ability to evaluate injury and/or violence prevention activities
• Ability to build and manage an injury and/or violence prevention program
• Ability to disseminate information related to injury and/or violence prevention to the community, other professionals, key policy makers, and leaders through diverse communication networks
• Ability to stimulate change related to injury and/or violence prevention through policy, enforcement, advocacy, and education
• Ability to maintain and further develop competency as an injury and/or violence prevention professional
• Demonstrate the knowledge, skills, and best practices necessary to address at least one specific injury and/or violence topic (e.g., motor-vehicle occupant injury, intimate partner violence, fire and burns, suicide, drowning, child injury) and be able to serve as a resource regarding that area
aDeveloped by the SAVIR-STIPDA Joint Committee on Infrastructure Development (2005). (SAVIR was formerly the National Association of Injury Control Research Centers.)
gather more information to be confident that my assumptions are true? Likewise, be sure you question other people’s assumptions—don’t take anything as gospel just because someone told you or it is printed in a report. As a general principle:
Always question assumptions, especially your own.
23.4.3. Getting Help
A critical skill is knowing when and how to get extra help from outsiders. Depend- ing on the circumstances, one solution is to find a collaborator and/or seek con- sultation and technical assistance on specific aspects of the program. In so doing, it is important to consider what aspects of program development, implementation, and evaluation are suited to an individual’s own skills and setting and what needs supplementation from outsiders. If ongoing help is needed, it may be helpful to add a collaborator who can invest in the project in the long term. For example, it may be wise to add a collaborator with skills in instrument design, data collection, or program evaluation—elements required at different points in the project. On the other hand, if the need is more short term, requiring someone to help make sense of existing data or to conduct a specific set of focus groups, this might be easily handled by employing a consultant.
Once it is clear what is needed, it is prudent to get advice from more than one person about where to find help. Most university faculty and agency staff are receptive to email or telephone communication from people seeking help, provided the question is focused and that the person asking has done his or her homework and has prepared the request based on careful thought and review of existing resources. Calling with a very broad question such as “I want to develop a program to reduce child injury in my community; where should I start?” is too vague. In contrast, a question such as “I am developing a smoke alarm giveaway program and need help identifying a good instrument for evaluating its effects” is a more easily handled request.
When engaging with a consultant or new collaborator, it is always a good idea to be clear about the scope of help required as well as the timing of work and any deadlines that may exist. For example, calling someone for help with developing an evaluation instrument that needs to be in the field in a week is very different from contacting them as one works toward a deadline in 6 months. Be sure expec- tations are both realistic and clear. In negotiating this strategy, it may be helpful to explore with the individual what types of barriers or facilitators impinge on his or her ability to assist. For example, faculty often have restricted time schedules associated with the academic calendar but may also be very receptive to consulting work during the summer months and/or be more eager if there are opportunities to collaborate on publications or involve their students.
In addition, be sure to understand any payment (consulting fees, travel) or payback (authorship on publications, access to data) that a consultant may require or expect. It is always best to clarify these kinds of obligations at the outset.
23.4.4. Sharing Results
Often it is assumed that publishing results of interventions is reserved for academ- ics, yet most interventions do not take place in academic settings. Consequently, it is critical to advancing the science of injury control for practitioners to engage in the process of sharing evidence with the field at large. This requires not only skill
development but also a commitment to participating as a member of a commu- nity of scholars exchanging experiences and ideas and participating in a scholarly approach to injury control. Though beyond the scope of this chapter, there are resources for learning to prepare and publish papers about practice experience in the peer-reviewed literature (Rivara & Cummings, 2001).
23.5. CONCLUSION
This chapter has explained a systematic process to program planning, includ- ing integration of evidence of various types and at various stages in the planning process. The chapter introduces readers to a range of issues related to finding and evaluating various types of evidence and possible intervention approaches. Many of these topics are addressed more fully in other chapters of this volume and in other sources.
Systematically developing and implementing interventions are critical com- ponents of the quest to reduce injury. The science base to support this work is multidisciplinary and multifaceted, requiring clear and collaborative thinking to be successful. More important than any of the specific guidance offered is taking a perspective that injuries are preventable, that no one approach is likely to solve any problem, and that change takes a long time. But change is possible with careful thinking and perseverance.
Acknowledgments. This chapter has benefited from critical review and helpful suggestions from reviewers of earlier drafts: Dr. Corinne Peek-Asa, Dr. Lynda Doll, Dr. Renee Johnson, Ms. Janet Place, and Dr. Desmond Runyan. This work was sup- ported, in part, by funding from the National Center for Injury Prevention and Control to the University of North Carolina Injury Prevention Research Center (Grant number 1R49/CE000196-01) and by a R. J. Reynolds Faculty Leave from the University of North Carolina at Chapel Hill to Dr. Runyan.
REFERENCES
Barling, J., & Frone, M. (2004). The psychology of workplace safety. Washington, DC: American Psychologi- cal Association.
Bronfenbrenner, U. (1979). The ecology of human development. Cambridge, MA: Harvard University Press.
Brownson, R. C., Baker, E. A., Leet, T. L., & Gillespie, K. N. (2003). Evidence Based Public Health. New York: Oxford Press.
Elder, R. W., Shults, R. A., & Sleet, D. A. (2004). Effectiveness of mass media campaigns in reducing alcohol-impaired driving: A systematic review. American Journal of Preventive Medicine, 25, 57–65.
Freire, K., & Runyan, C. W. (2006). Planning models: PRECEDE/PROCEED and Haddon Matrix. In A. C. Gielen, D. Sleet, & R. J. DiClemente (Eds.), Injury prevention: Behavior change theories, methods and applications. San Francisco: Jossey-Bass.
Gielen, A. C., & McDonald, E. M. (2002). Using the PRECEDE-PROCEED planning model to apply health behavior theories. In K. Glanz, B. K. Rimer, & F. M. Lewis (Eds.), Health behavior and health education: theory, research and practice (pp. 409–436, 3rd ed.). San Francisco: Jossey-Bass.
Glasgow, R. E., Lichtenstein, E., Marcus, A. C. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 93 (8), 1261–1267.
Green, L. W., & Kreuter, M. W. (2005). Health program planning: An educational and ecological approach (4th ed.). New York: McGraw-Hill.
Haddon, W. (1972). A logical framework for categorizing highway safety phenomena and activity. Journal of Trauma, 12, 193–207.
Haddon, W. (1980). Advances in the epidemiology of injuries as a basis for public policy. Public Health Reports, 95, 411–421.
Haskins, R., & Gallagher, J. (1981). Models for social policy analysis: An introduction. Norwood, NJ: Ablex Press.
Hofman, D. A., & Tetrick, L. E. (2003). Health and safety in organizations: A multilevel perspective. San Francisco: Jossey-Bass.
Hopkins, D. P., Briss, P. A., Ricard, C. J., Husten, C. G., Carande-Kulis, V. G., Fielding, J. E., Alao, M.
O., McKenna, J. W., Sharp, D. J., Harris, J. R., Woollery, T. A., Harris, K. W., & Task Force on Community Preventive Services. (2001). Reviews of evaluations regarding interventions to reduce tobacco use and environmental exposure to tobacco smoke. American Journal of Preventive Medicine, 20 (2S), 16–66.
Leung, M. W., Yen, I. H., & Minkler, M. (2004) Community based participatory research: A promising approach to increasing epidemiology’s relevance in the 21st century. International Journal of Epide- miology, 33 (3), 499–506.
McRae, D., & Wilde, J. (1979). Policy analysis for public decisions. Belmont, CA: Duxbury Press.
Minkler, M. (2000). Using participatory action research to build health communities. Public Health Reports, 115 (2–3), 191–197.
Nation, M., Crusto, C., Wandersman, A., Kumpfer, K. L., Seybolt, D., Morissey-Kane, E., & Davino, K.
(2003). What works in prevention. Principles of effective prevention programs. American Psycholo- gist, 58 (6–7), 449–456.
National Training Initiative for Injury and Violence Prevention (NTI) (2005). Retrieved March 1, 2006, from www.injuryed.org.
Patton, C. V., & Sawicki, D. S. (1993). Basic models of policy analysis and planning. Englewood Cliffs, NJ:
Prentice Hall.
Rivara, F. P., & Cummings, P. (2001). Writing for publication in Archives of Pediatrics and Adolescent Medicine. Archives of Pediatric and Adolescent Medicine, 155, 1090–1092.
Runyan, C. W. (1998). Using the Haddon Matrix: Introducing the third dimension. Injury Prevention, 4, 302–307.
Runyan, C. W. (2003). Introduction: Back to the future—Revisiting Haddon’s conceptualization of injury epidemiology and prevention. Epidemiologic Reviews, 25, 60–64.
Runyan, C. W., Fischer, P., Moore, J., Waller, P., & Hooten, E. (1993). Attempting to change local injury policy. Family & Community Health, 5 (4), 6–74.
Rychetnik, L., & Wise, M. (2004). Advocating evidence-based health promotion: reflections and a way forward. Health Promotion International, 19 (2), 247–257.
Stokols, D. (1992). Establishing and maintaining healthy environments: Toward a social ecology of health promotion. American Psychologist, 47, 6–22.
W. K. Kellogg Foundation. (2004). Logic model development guide: Using logic models to bring together planning, evaluation and action. Retrieved August 5, 2005, from www.wkkf.org.
