Roadmap for State Program Planning: Evaluate the Program

• Assess Capacities
• Assess Interventions
• Report Results

Overview

Program evaluation is a systematic collection of information and data about the activities and outcomes of programs and interventions. It can improve effectiveness and inform decisions about future activities and resources. As you conduct your state HDSP program evaluation, you will need to evaluate both capacity building activities and interventions.

Expected Outcomes

After completing the "Evaluate Program" component of the Roadmap, you will be able to—

• Assess the success of capacity building activities and interventions.
• Reassess the program's activities.
• Report program results for the past year.

Skills and Competencies

The skills needed to address this program area are adapted from the National Association of Chronic Disease Directors (NACDD) Competencies for Chronic Disease Practice.

• Evaluating data integrity and comparability.
• Explaining relevant inferences from quantitative and qualitative data.
• Applying ethical principles to the collection, maintenance, use, and dissemination of data and information.
• Presenting accurate demographic, statistical, programmatic, and scientific information for professional and lay audiences.
• Recognizing how the data illuminates ethical, political, scientific, economic, and overall public health issues.
• Identifying the role of medical, genetic, environment, social, and behavioral factors in determining the underlying causes and management of chronic disease.
• Identifying relevant and appropriate data and information sources for chronic disease.
• Selecting and using appropriate data collection methods.
• Working collaboratively with partners on data collection and interpretation.

The state HDSP program supports the CDC Evaluation Framework, [PDF–1.3M] a practical tool designed to organize the essential elements of any program evaluation. The framework provides a six-step process to help think through evaluation planning—

• Engage stakeholders.
• Describe the intervention or activity.
• Focus the evaluation design.
• Gather credible evidence.
• Justify conclusions.
• Share results and lessons learned.

In conducting evaluation, you may also consider—

• Partnering with a university or another state health department program.
• Contracting services from Prevention Research Centers (PRC), a network of different types of researchers conducting applied research.
• Locating contract evaluators through a professional association such as the American Evaluation Association,* a professional association of evaluators.

What to Do

Ideally, evaluation will focus on both the processes and outcomes of your program. The CDC encourages states to report both types of evaluation results when possible. However, as noted in the "Develop an Evaluation Plan" component of the Roadmap, you may not have the resources to conduct both process and outcome evaluation for all of your objectives. For some activities, it may be appropriate to report on the achievement of your activities.

Evaluation for CDC's National HDSP Program

The evaluation expectations for funded state HDSP programs are to—

• Develop an evaluation plan.
• Document changes in state capacity to address HDSP. States are encouraged to evaluate the quality aspects of capacity areas, such as the functioning, effectiveness, or achievements of your state partnership.
• Systematically document HDS burden using surveillance data.
• Document changes in HDSP policies and systems change factors that support HDSP.
• Conduct evaluation of some interventions.

How to Do It

Step 1. Review your HDSP Evaluation Plan

Review your HDSP Evaluation Plan, Logic Model, and the documentation requirements identified in the Performance Measures for Capacity Building and Basic Implementation Programs for needed changes. The Evaluation Plan should agree with the annual Work Plan and program logic model, and should have a feasible timeline.

Step 2. Evaluate program achievements

To evaluate your program's achievements, you can—

• Compare your starting capacities with new or enhanced capacities and review effectiveness of the processes used to build them.
• Assess the activities and interventions carried out in support of your objectives to determine how well they worked.

Your evaluation should—

• Evaluate progress toward completion of work plan objectives, including overall measures of effectiveness and a more detailed plan for evaluation of at least one policy or systems change intervention. (Basic Implementation)
• Identify lessons learned.
• Evaluate your state partnership. (All funded states)
• Monitor state level data.

A process evaluation—

• Looks at quality and quantity of products and activities.
• Assesses whether an activity or intervention was implemented as planned.
• Identifies areas that need to be modified or improved.
• Reveals strengths and weaknesses.
• Investigates why an intervention is successful.
• Assesses the quality and quantity of inputs, activities, and products.

An outcome evaluation—

• Measures the overall effect of an intervention or activity.
• Measures the extent to which supporting and long-term objectives were achieved.
• Measures the impact on the target population.

Evaluation Guides

The CDC Division for HDSP developed a series of evaluation guides to provide guidance for a number of evaluation areas. Each guide provides an overview of issues to consider, including sample evaluation questions for varying levels of evaluation as well as suggested tools. New guides will be accessible through the Roadmap as they become available. These are currently available—

• Developing an HDSP Evaluation Plan
• Writing SMART Objectives
• Developing Logic Models