PACE EH Online Course Module 4

Using the profiles, the team ranks according to health impact only.  Of the 10 to 20 issue profiles, which ones have the most detrimental impact on local human health.  This task tends to unite and focus the team.  Also, it can start frustrating the single issue advocates.  Team members by now have developed "pet" issues and ranking can ruffle their feathers.  

Keep in mind, PACE EH offers one pretty simple ranking methodology.  Other ranking methodologies exist and may be more useful for for the needs of a specific team.  Ranked profiles are now “prioritized” according to factors outside of health impact (political support, economic feasibility, legal ramifications, etc.) 

Similar to Task 10, this task can lead to tense meetings and, other priority setting exercises may be worth considering.  Elements of an effective environmental health action plan can be found in PACE EH on pages 60-63. We suggest no more than two to four action plans be developed and implemented based on highest prioritized issue profiles.  The resulting action plans need to balance long and short term goals, make sure they identify some “low hanging fruit.”  Keep in mind, the local health agency need not be the primary agency responsible for developing and implementing the action plans.

To set the stage for the future, the team needs to establish a process for evaluating their PACE EH work.  The guidebook includes some sample considerations on page 64 of the guidebook. It can be hard to maintain momentum and new members can help. The authors of PACE EH envision the methodology as an ongoing process, completed every three to five years.  Completed once, PACE EH it is a snapshot.  Only through multiple iterations can PACE EH demonstrate change over time.

At the completion of this task, the assessment team should have:

  • A list of environmental health issues, ranked according to locally defined criteria.

At this point, the team uses the profiles developed in Task 9 to compare issues according to the relative importance of each issue against all other environmental health issues identified by the community. This section describes a technique for undertaking a standardized ranking process. The technique can be adapted to suit the needs and goals of the team. Explicit discussions about why one issue is more important than another will contribute to both a collective appreciation for community values and a greater understanding of the issues themselves. The following steps focus on developing criteria, selecting a ranking method, and carrying out the ranking process.

Determine the Purpose of Ranking

The first step in designing a ranking process that is tailored to the community is deciding what the team hopes to achieve through the process and how the results will be used. Two questions to consider are: 

  1. What outcomes are expected from the process? and 
  2. How will the results of the ranking be used?

If the desired outcome is to build community support, then the assessment team might choose to use a tool that emphasizes community values over technical information. One way to do this is to calculate additional weight for criteria that the community has identified as especially important, such as impact on children. Other purposes might include educating the public, changing departmental priorities, and cataloguing technical information for a range of future uses. This decision has implications for who should be engaged in the process, what information should be considered, and how the information should be packaged and presented.

Decide on Ranking Criteria

Although individual judgment plays a role in the ranking process, the use of clear and agreed-upon criteria will ensure that the participants view the process as fair and valid. The process can be facilitated by use of a worksheet that standardizes the criteria, summarizes the team’s knowledge and attitudes about a given issue, and alerts members to additional data needs. The information to complete the worksheet is derived from the corresponding issue profile developed in Task 9.

Use the sample worksheet [Environmental Health Issue worksheet] as a starting point in the discussion of criteria. In the sample worksheet, each environmental health issue is characterized according to magnitude of risk, distribution of risk in the community, and severity of risk. Discuss these criteria. Do they make sense for the community being assessed? Will they help in the ranking process? Are there other criteria that would help in discriminating among the issues? The set of criteria chosen by the team may be similar to those on the sample worksheet or completely different. What matters is that the participants agree on a set of criteria and consider these criteria as they evaluate each issue.

Select a Method for Ranking

After deciding on the criteria, adapt/revise the sample worksheet, or design another. Fill out the worksheets as a group or individually, using information from the issue profiles created in Task 9. Develop a composite of individual scores through discussion and consensus building and/or use of group decision-making techniques.

If team members work on this task independently and then wish to generate a group score, the results can be added (or averaged). Participants can also vote to reach group consensus. If there is discrepancy in individual evaluations, the team would listen to the arguments of those in support of each option and then vote for the more compelling “answer.” Unresolved debate or controversy may indicate a need to collect more information.

Rank the Issues

At the end of the worksheet, participants are asked to summarize the importance of the issue (high, medium, or low concern). This step involves judgment and is therefore an expression of personal values. When translated into concrete and explicit criteria, these expressions of values provide fertile ground for developing mutually acceptable plans and defining a community’s priorities. This step therefore forms the foundation for the priority-setting task presented next. A potential pitfall at this point is assuming that all participants understand the professional judgments and public health “value” systems used to rank environmental health issues. Explicit articulation of these values will facilitate the process and minimize frustration, confusion, and nonproductive debate.

Participants should continually ask themselves: Why did I score this risk as I did? For example, is the risk of environmental lead exposure scored “high” because:

  • The risk is unfairly distributed according to race? 
  • It affects the development of children rather than adults? 
  • It affects lower income persons who may not be able to choose lead-safe housing and thus may be more exposed to a health risk? 
  • It is particularly dangerous to pregnant women and their babies? 
  • All of the above?

Participants may not base their decisions on similar reasons or values. The underlying value systems on which health policies are based are generally unstated. By explicitly stating reasons for their rankings, participants at least can have a common basis for discussing values and policies and an opportunity to gain consensus on community health actions.

Whether summarized quantitatively or qualitatively, the final scores will allow the issues to be ranked relative to one another, with regard to the agreed-upon criteria. Issues found to be of “high concern” (or meeting some other agreed-upon cutoff point) will be evaluated in terms of overall community priorities in Task 11.

The Value of Local Process

Although the tools included in PACE EH have been found to be useful and mostPACE EH communities did not alter them significantly, many coordinators reported that their teams needed to put aside the supplied tools and think through locally appropriate ranking and prioritizing processes.

Ranking and prioritizing are complex and difficult processes, made more so because they require assessment team members to examine their own values, opinions, and judgments. Thus, teams must understand the rationale behind any system for ranking and prioritizing the issues before participating in the exercise. Users ofPACE EH may find the tools included here useful for beginning a discussion about their own ranking and prioritizing processes. This approach reflects the philosophical intent of the authors: users are encouraged to adopt included tools where helpful, alter them where necessary, or jettison them altogether if the team advocates a more locally appropriate approach.

Question to Consider

Can you describe “ranking” in regards to a PACE EH process?

At the completion of this task, the assessment team should have:

  • A statement about the community’s environmental health priorities for action

Once issues have been ranked, it still may be necessary to determine which are most important for action. Again, locally specific criteria that reflect community values can be used. Prioritizing issues allows the community to direct resources, time, and energy to those issues that are deemed most critical and practical to address.

Determine Local Priority-Setting Criteria

This sample priority setting worksheet is designed to guide the process by using criteria other than magnitude, severity, and distribution (considered in the ranking process) to evaluate the issues deemed most important in Task 10. The goal of the process is to decide the feasibility of addressing each issue. These criteria should be reviewed by the assessment team and revised as needed to reflect the values of the community.

Select a Method for Prioritizing

Use the worksheet provided (or one developed by the assessment team) to evaluate issues determined through the ranking process to be of “high concern.” As in the previous task, the worksheet can be filled out as a group or individually. Discussion and consensus building and/or use of group decision-making techniques can be used to develop a composite of individual evaluations. Assign numerical weights to the criteria and to the qualitative column headings as appropriate to reflect the relative importance of each. Then quantify the results of the exercise by multiplying the weight assigned to the criteria by the numerical equivalent of the column heading. Each person’s opinion can then be represented with an overall numerical score. The resulting scores are easily comparable and can be averaged or mathematically manipulated in any way the team considers appropriate.

The process may be done mathematically or more qualitatively, by simply engaging in discussion about the role of community factors in determining environmental health priorities. Use the above method, or devise an alternate one more appropriate to the community’s needs.

Determine Priorities

The method used should result in the identification of a manageable number of priority issues. For example, one means of determining the top priorities is to select the three issues receiving the highest scores. In Task 12, action plans will be developed for these top issues only. Alternately, the team may choose to develop action plans for all issues ranked highly in Task 10 and to use the priority-setting process solely as a means of deciding the order in which the issues will be addressed.

Questions to Consider

Describe "Prioritizing" in regards to a PACE EH process.  What is the difference between "ranking" and "prioritizing?"

At the completion of this task, the assessment team should have:

  • A community-specific environmental health action plan.

The outcome of the ranking and prioritizing processes will guide the development of strategies to address the community’s most pressing environmental health concerns. For issues considered high priority, strategies should be developed to address the problem or ensure ongoing maintenance of the asset. The collection of strategies for all priority issues constitutes a community action plan for environmental health.  

Moving to action planning requires another round of information gathering. More detailed information about the issue, available resources and related current activities, and the effectiveness of various potential interventions is needed. For each issue, the assessment team should engage in a strategic planning process, which could include the following steps.

Develop a Goals and Objectives

Evaluate each objective to ensure that it is SMART (Specific, Measurable, Agreed upon, Realistic, and Time-based):

  • Specific—An objective must be specific to be measurable. For instance, instead of defining an objective as “Educate the community,” a more specific objective would be “Train the community’s parents regarding the health effects of environmental lead and appropriate exposure prevention measures.”   
  • Measurable—It is easier to demonstrate progress toward objectives that are quantified. For example: “Train 80% of the community’s parents regarding the health effects of environmental lead and appropriate exposure prevention measures.”   
  • Agreed upon—Objectives should be developed through full involvement of the assessment team if the intent is to assure community commitment to accomplishing them.   
  • Realistic—“Realistic” is different from simply “feasible.” An objective is feasible if it is capable of being accomplished; it is realistic if it is feasible given time, resource, and technical considerations. For example, “Train 80% of the community’s parents regarding the health effects of environmental lead and appropriate exposure prevention measures” might be technically possible. But if it is not likely to happen in a timely fashion given current staffing limitations, it would not be considered realistic. 
  • Time-based—Target dates increase motivation, commitment, and action. “Within 12 months, train 80% of the community’s parents regarding the health effects of environmental lead and appropriate exposure prevention measures” is more likely to be achieved than an objective without a target date. *adapted from NACCHO’s Partnerships for Environmental Education.

Identify Contributing Factors

Refer to the framework developed in Task 7 in which the team identified exposure factors, environmental agents/conditions, contributing factors and behaviors, and public health protection factors for the issues of concern.

Identify Possible Interventions

Identify applicable potential interventions as well as those already in place that should be maintained or enhanced. There are three main types of interventions (Kansas Department of Health and Environment, 1995):

  • Individual-based interventions—These lead to changes in individuals, typically through direct service to clients or residents. 
  • Community-based interventions—These create changes in populations (e.g., immunizing all children in the community). 
  • System-based interventions—These create changes in organizations, policies, laws, and structures.

In the context of a community-based environmental health assessment, community-based or system-based interventions are likely the most appropriate options.

Identify Community Assets

These are needed to help implement each proposed intervention. Examples include educational organizations and schools that can disseminate relevant information to parents. Other assets include resources available at the state and federal levels, such as educational materials or grant funds available through the state health agency, or resources provided from national organizations. Review the community asset work completed in Task 1 for ideas.

Asset mapping materials

http://outreach.missouri.edu/about/fy00-03/assetmapping.htm

http://www.ipr.northwestern.edu/publications/community/mcc.html

Identify Potential Barriers

Review work completed in Task 2 (identifying and characterizing the community) to identify conditions or aspects of the community that may have implications for implementation of the intervention. Language barriers, for example, may require the dissemination of educational materials in more than one language. Constraints associated with the broad scientific, legal, economic, social, and political systems in the community, as identified in Tasks 9 and 11, may also present barriers to implementing specific interventions or activities.

Identify Potential Partners

The work completed in Tasks 1 and 2 (determine community capacity and identify and characterize the community) on compiling assets within the community should help in identifying appropriate parties to assume/share responsibility for undertaking or enhancing the activity. Consider health agency staff, other local agencies, community members, academic institutions, and other community organizations.

Determine Measures of Success

Ultimately, achievement of the objectives and goals will verify “success.” Additional events or data points to indicate that the issue is being effectively addressed can also be identified.

Action Planning

Local variability makes it difficult to provide guidance on action planning. It is therefore the least predictable task in the PACE EH process. Nevertheless, PACE EH site experiences suggest two factors that may affect the success of an action plan: (1) the relative duration of the plan and (2) the stage(s) at which outcomes will be measured. The most successful plans offer both long- and short-term activities and measure outcomes periodically throughout the life span of the plan.

One coordinator noted that the ideal action plan would provide the assessment team with a method for long-range environmental health planning and assessment, but provide enough positive feedback to ensure the team can celebrate small successes along the way. By way of illustration, one PACE EH site assessment team set up an action plan goal to “reduce asthma hospitalization by 10% by the end of 2003.” The long-range goal, however, was achieved by specific smaller-scale activities beginning in the summer of 1999, such as convening a task force and conducting relevant local workshops. These activities provide concrete action that can be individually lauded and help monitor progress toward the ultimate goal.

Questions to Consider

What are the elements of a successful PACE EH action plan? Why is it important to create action plans that balance short- and long-term goals in PACE EH action planning?

At the completion of this task, the assessment team should have:

  • A plan for ongoing evaluation, both of the progress achieved on the action plan and of the assessment process

The completion of the first assessment process should be celebrated and the hard work and dedication of the team members acknowledged. Over time, the success of the actions to address priority issues should be evaluated. An evaluation measures and documents the degree to which activities and outcomes are being achieved, within the designated timeframe.

Agree on the Questions to Be Answered by the Evaluation

Discuss the definition of “success” for the assessment. It may be based on process (e.g., the quality of interaction among community members), outcomes (e.g., improved health status), or a combination of the two. Discussing the questions the team hopes to answer through the evaluation contributes to the development of an effective evaluation process. These questions may include:  

  • Has the goal been achieved completely?
  • Was it achieved in an effective and efficient manner?
  • Did the process raise new issues or concerns?
  • Are there ongoing measures that should be taken to ensure long-term success?

Evaluate the Success of the Assessment Process

Indicators are often used as one tool to evaluate progress. The team may choose to continue reporting on the original indicators or create new ones that describe success in terms of the questions above. These may be mostly indicators of health status, or they may include other pieces such as protection factors to describe what the community is doing to make progress or identify where actions have been inefficient.

Information gathered by tracking indicators can be used to communicate progress to the community and/or to identify the need for additional intervention. Building on success, a community may choose to take on additional issues as priorities for action.

Begin preparations for ongoing community-based environmental health assessment activities  

PACE EH is designed to offer a process for ongoing assessment and evaluation and not as a one-time project. Much of the value lies in tracking key environmental health indicators over time, in continuing the relationships developed through the process, and in evaluating the success of the community in addressing selected priorities. The process can be reinvigorated as changes in the community suggest the need for more information, additional community involvement, or a shift in concerns and priorities.

Questions to Consider

Describe the successful elements on an evaluation plan for an ongoing PACE EH process.