Assessing the Effectiveness of Disease and Injury Prevention Programs: Costs and Consequences

Summary

Because resources are finite, public health decision makers need to consider the costs and effectiveness of alternative prevention strategies. A simplified approach to performing marginal cost-effectiveness analyses requires a) a description of the program, b) a description of the health outcomes averted and the timing of those events, c) the rates of the health outcome, d) the preventable fraction of the health outcomes averted, e) the costs per unit of the intervention, and f) the direct medical costs of the health outcome prevented and the side effects incurred. With this information, the marginal cost-effectiveness of an intervention can be determined and applied for decision making.

INTRODUCTION

The assessment of the cost and effectiveness of prevention programs is crucial to public health activities. The comparison of different preventive and curative strategies and decisions concerning allocation of finite health resources both depend on obtaining reliable and consistent cost and effectiveness data. This information is used to document which programs and activities provide the greatest benefit for the funds expended.

Federal, state, and local health decision makers need to know the costs of prevention programs and the effects of such programs on the health of the populations targeted for the intervention. To decrease the burden of illness and improve the health status of the population, resources should be allocated to the extent possible according to scientifically determined priorities.

Public health practitioners should provide effective health promotion and protection programs and return justifiable benefits to their constituents for the resources consumed. To accomplish these objectives, two questions must be answered: "What works?" and "How much does it cost?"

This report explains how prevention effectiveness methods can be applied in public health practice and highlights CDC's mission as the nation's prevention agency. Previously published CDC reports examine a basic approach to evaluating available data to determine the costs and consequences of prevention programs (1-10). In these reports, several prevention strategies were discussed, including water fluoridation (3), mammography (4), control of nosocomial infections (5), tractor rollover protection systems (6), smoking cessation (7), screening for diabetic-associated eye disease (8), use of bicycle helmets (9), and increased physical activity (10). Each report included a) the program description; b) the health outcome averted through the program; c) the current rates and burden of the health outcome; d) the proportion of this health outcome potentially averted as a result of the program; e) the program cost; and f) the direct medical treatment cost per prevented health outcome.

This report provides a basic methodology for calculating and comparing the costs and consequences of prevention programs. This methodology can assist in making public health decisions and in setting priorities for prevention strategies.

DEFINITIONS, ASSUMPTIONS, AND ANALYTIC PRINCIPLES

The technique described is used to assess intervention costs and consequences and is referred to as the Basic Assessment Scheme for Intervention Costs and Consequences methodology (BASICC). It provides a minimum standard for analyzing the cost-effectiveness of any prevention program. If this analysis is not completed, resources cannot be allocated effectively for the intervention because information concerning the effects of spending resources are inadequate. Data for this methodology should be collected for new programs and for existing programs for which these data have not been available. A discussion of standards for a comprehensive cost-effectiveness analysis was published in A Practical Guide to Prevention Effectiveness: Decision and Economic Analysis (2). The following are the underlying principles of BASICC.

A societal perspective should be used when BASICC principles are applied. The societal perspective is usually preferred for public health decisions because all resources and benefits associated with the intervention, regardless of who pays for or who receives them, are included in the analysis. Sometimes, an additional perspective may be appropriate. Other possible perspectives include that of service providers (e.g., a clinic, an institution, an agency, or individual providers); business firms; health insurance companies and health-care reimbursers; government; and program participants.

Several fundamental terms used in BASICC methodology have been defined in previous CDC reports (1,2).

Efficacy refers to the scientific basis for "what works" in reducing adverse health outcomes. It is the improvement in health outcome that a prevention strategy can produce in expert hands under ideal circumstances. Effectiveness refers to the health impact of an intervention in typical community settings.

Cost, at a minimum, refers to a) the resources expended in developing, implementing, and maintaining the intervention in a community and b) the health-care resources saved in avoiding treatment for the health outcomes prevented in the community (i.e., direct costs).

A more complete analysis includes productivity costs and intangible costs. Productivity costs refer to the cost to an individual and society from the loss of time for productive pursuits and the value of that time (11). Intangible costs (e.g., pain and suffering) are those for which a monetary value cannot be assigned.

Although all costs associated with prevention resources and benefits should be included in a complete analysis, BASICC includes only intervention costs and savings from direct costs. This exclusion of other costs may understate the benefits as well as the side effects of prevention.

Net cost calculations enable the costs of a prevention program to be compared with the medical treatment costs saved from health outcomes averted by the program. Net cost, which is defined as

Cost(program) + Cost(Side Effects) - Cost(Health Outcome Averted)

is the recommended endpoint for BASICC.

The cost of resources needed for a prevention program can be classified as either fixed or variable. Over a reasonable range of program size, fixed costs do not vary with the quantity of output (e.g., costs associated with facilities, capital equipment, and administrative salaries are the same regardless of the number of program clients). Variable costs change with the level of output and respond proportionately to change in volume of activity (e.g., provider time and materials increase as the number of program clients increases). The sum of fixed and variable costs for a specified number of clients is the total cost of a prevention program.

In cost analyses, average and marginal costs can be calculated. The average cost of a program, defined as the total cost of the program divided by the total units of output produced, includes a proportion of rent, utilities, and equipment, even if expansion of facilities is not required to implement the intervention. Marginal cost is defined as the additional cost required to produce an additional unit of output. Marginal costs include only variable costs because of the assumption that, in the short term, an additional unit of output can be produced with no increase in fixed costs. In BASICC, marginal costs are considered for interventions that are implemented in preexisting facilities (i.e., fixed costs associated with providing services are not included). The average intervention costs are considered for programs that do not use existing facilities or that require expansion of existing facilities. Average costs, however, are not interchangeable with marginal costs.

In prevention programs, a time lag always occurs between the intervention and any effect on health outcomes. This time lag must be considered in conducting any analysis, because decision makers prefer that benefits occur as soon as possible after a program is implemented. This time preference is incorporated into BASICC by discounting costs and effects that will occur in future years applied to the present. The discount factor can be calculated directly or can be found in many finance or accounting books or on financial calculators. The formula for discounting a cost or a benefit that will occur in the future (at time T) is as follows:

event or cost occurring in year T / (1+ discount rate) exp T

The recommended discount rate for BASICC is 3% (2) but is usually varied over a reasonable range (e.g., 0 to 8%) in a sensitivity analysis.

METHODS

The six data elements necessary for completing BASICC are as follows.

1. A complete description of the program, the units in which the service(s) are provided, and the time frame of the program.

2. Health outcome(s) averted by the prevention program and the estimated time between its implementation and when the health outcome is averted.

3. The rates and societal burden of the health outcome.

4. The preventable fraction for the health outcome, with the program in place and used in a realistic manner (i.e., the proportion of the health outcome averted through the program).

5. Intervention costs per unit of intervention, including the cost of any intervention side effects.

6. Direct medical treatment cost of the health outcome prevented.

   Net costs can be calculated from these six elements.

Description of the Program

A comprehensive description of a prevention program includes the following items (12):

1. a statement of the objectives;

2. a description of the target population;

3. a measure of effectiveness;

4. an understanding of external environmental constraints on program effectiveness;

5. a list of resources required;

6. a method for day-to-day management of the program;

7. a mechanism for long-range planning and reaction to environmental changes;

8. the implementation strategy to be used for a new program or changes in a program; and

9. evidence that the program benefits the participants (i.e., scientific evidence of effectiveness).

   A time frame for assessing the intervention and the health outcome must

be selected. A full year of cost (resource) data is suggested to compensate for any seasonal variation.

Averted Health Outcomes

Prevention programs should a) improve the quality of life, b) reduce the incidence or severity of a disease or injury, and c) reduce premature death through early detection or interventions to reduce risks or exposures associated with incidence. A prevention program should have a specific objective: to avert or reduce the occurrence of a specific health outcome. Prevention effectiveness analyses are generally incidence based; therefore, the analysis should include all adverse health outcomes that are caused or prevented during the lifetime of the participant as a result of the prevention program.

Rates and Economic Burden of the Health Outcome

Information about the incidence and prevalence of the health outcome with and without exposure to the prevention program should be summarized. The rate and number of cases of the health outcome in the geographic area of interest by relevant demographic and risk group provide an assessment of the magnitude of the problem being prevented. The economic burden of the health outcome also should be summarized and should be described in the context of the same demographic and risk categories as the health outcome of interest.

Preventable Fraction

A prevention intervention is rarely 100% effective. Because the entire population at risk may not be reached and because compliance among persons reached may not be universal, programs generally do not work in practice in the same way that efficacy studies or theoretical models predict. Thus, the preventable fraction of health outcomes that will actually be averted by the intervention should be determined. Formulas for calculating the preventable fraction may be found in the Guide (2). Alternatively, the following approximation for the preventable fraction may be used, in which rate refers to the rate of the health outcome both with (designated as "I") and without ("O") the intervention, the proportion (p) of the population reached, and the proportion who comply.

{{Rate(I) - Rate(O)} / Rate (O)} x p (reached) x p (who comply)

Intervention Resource Costs

Resources expended by prevention programs during a specified time period must be defined and quantified by constructing an inventory of resources and their unit costs. A detailed discussion of this may be found in the Guide (2) and elsewhere (13). When a marginal analysis is considered, the program resources are determined from a list such as the following:

1. Variable costs:

• direct provider time by provider type for each type of service in intervention;

• supplies and materials for each type of service;

• laboratory or other tests for each type of service.

  Costs to participants in the programs may include:

• time required of client for the intervention;

• expenses of client for the intervention.

• laboratory controls;

• development of new educational or promotional materials;

• facilities, including rent and utilities specific to the intervention;

• equipment specific to the intervention;

• maintenance on facilities and equipment specific to the intervention;

• administrative and support staff specific to the intervention;

• other direct costs of providing services within intervention (e.g., travel, reused materials, and professional development).

  The above list suggests types of resources that are used in most

prevention programs and is not meant to be all inclusive.

Each item on the inventory has a unit cost (e.g., salary, fringe benefits per hour of provider time, or price paid for a laboratory test or reimbursement rate per mile of travel). In assessments such as the example that follows, those resources that are not new or expanded are not included in this resource inventory.

The total cost of a prevention program is determined by summing the total variable costs and the total fixed costs incurred by the program during the designated time period. The total variable costs are obtained by multiplying the unit cost of each variable input by the quantity used during the period and summing over all variable inputs. The quantity of variable inputs used is a function of the level of output of the program (e.g., the number of clients served). The total fixed costs are obtained by multiplying the unit cost of each fixed input by the quantity used and summing over all fixed inputs used. The quantity of fixed inputs (e.g., building rent) is independent of the level of output of the program over the range of outputs under consideration.

Once the unit costs and fixed costs are determined for all resources during the time period of interest, a summary table can be prepared. Included in the following example are the resources (column A); their unit costs (column B); the quantity of the resource used during the time period of interest (column C); and the total resource cost ({column D} = {column B} X {column C}). By summing the values in column D, the total cost of the intervention during the time period can be estimated.

For example, a 6-week group smoking-cessation program with 25 participants, resources might include provider time for the intervention (i.e., class time and preparation) and a brochure and progress calendar for each client. A room at a university student center is rented specifically for this program. Participants attend six 45-minute classes and travel an average of 30 minutes per class (6 X 1.25 hours = 7.5 hours). The median wage in the area is $10 per hour. The client's payment for the program is not included because that would count the resources used twice: once from the agency's and once from the participant's perspective. The societal perspective counts resource costs only once.

In the resource inventory for this program (Table_1), the total cost of the resources used is divided by the number of participants to calculate the cost per participant.

This calculation provides an estimate of the expected cost per participant in the intervention. If a program anticipates that 1,500 persons will participate in this intervention during the next year, the expected cost will be the cost per unit of intervention multiplied by 1,500, in this case, $11.13 X 1,500 = $16,695. Including participant costs, the expected cost for 1,500 participants is $86.13 X 1,500 = $129,195. Multiplying the expected cost per participant by the expected number of participants provides an accurate estimate only if the per-client fixed costs do not change as the program expands. The inclusion of participant costs is necessary if the societal perspective is taken.

Treatment Costs for the Health Outcome

For every health outcome averted, the direct costs associated with the health outcome are also prevented. This benefit of the prevention intervention is accounted for in the net cost calculation, which includes medical treatment costs and nonmedical costs (e.g., household assistance or hospice costs) (2).

For some health outcomes, medical treatment costs are available. For infectious diseases or acute health conditions (e.g., injuries), the cost per episode is needed. For chronic diseases (e.g., diabetes), the discounted lifetime cost or cost per person-year is more useful. Nonmedical costs are often more difficult to determine.

The difference in rates of the health outcome both with and without the intervention, as applied to the target population, is used to determine the number of cases of the health outcome that are prevented. The cost per health outcome is then multiplied by the number of cases averted to estimate the expected savings in treatment cost.

THE CALCULATION OF NET COSTS

The net cost of the program can be calculated by using the six elements described previously. Net cost can be summarized as the cost of the intervention and side effects for 10 exp n persons minus the direct costs associated with the expected number of cases averted in the same 10 exp n persons. The accuracy of this estimate depends on whether the calculated per-unit fixed costs remain constant for this number of participants.

If the net cost is negative, then the savings associated with the health outcomes prevented exceed the cost of the prevention program. Even if the net costs are positive, the public health reasons for the intervention may be compelling, particularly given that productivity losses and intangible costs (e.g., changes in the quality of life associated with the health outcome) have been excluded from the analysis. The decision to implement an intervention should be based on the value that participants and society place on preventing the health condition, not on whether the intervention is cost saving.

To continue the example on smoking cessation, an example of a net cost calculation follows (Table_2).

Therefore, the net cost = $86,130 - $310,470 = -$224,340

DISCUSSION

During 1992 and 1993, CDC published eight prevention effectiveness assessments in the MMWR (3-10). These reports demonstrated an approach to prevention effectiveness analyses that was based on information obtained from previously published studies. Although each analysis followed the same general framework and content guide, the information available for each analysis differed. As a result, each report differed in format and content. The BASICC methodology provides a minimum standard approach for analyzing prevention strategies.

Several factors influence the cost effectiveness of a public health program. How a program is implemented has a direct effect on cost. For example, a water fluoridation program (3) is available to all persons, regardless of age, in a community that has a common water supply. In 1989, approximately 135 million U.S. residents used water that had adjusted or natural fluoride concentrations considered adequate for improved dental health. However, persons who reside in rural areas and obtain their drinking water from individual wells cannot be reached with a community-based program. Thus, a program for municipal water supply fluoridation can only target populations that have unfluoridated municipal water supplies.

The size of an intervention and the population exposed to the intervention can also affect costs. For example, an infection-control program for a 1,000-bed hospital might not be four times more expensive than one for a 250-bed hospital (5).

Measurements of an intervention effect might not be empirically based on the population of interest. For example, tractor rollover protection data from agricultural tractors (6) should not be expected to apply to garden tractors. In addition, extrapolating from one population to another should be done with caution (e.g., data concerning use of bicycle helmets in Australia {9} might not reflect behaviors and expectations in the United States).

An unforeseen effect of environmental changes could alter the estimated preventable fraction. When the time period between the intervention and the benefit is short and the environment is similar, the intervention will be more likely to have the desired effect. For example, estimates of the effect of smoking cessation programs could be influenced during the next decade by new federal excise tax policies and new treatment technology.

A single adverse health outcome might be prevented by several different associated prevention programs. For example, risk for coronary heart disease is associated with smoking, hypertension, hypercholesterolemia, low levels of physical activity, and, in women, estrogen loss. The independent risk reduction attributed to a program to increase physical activity is difficult to assess.

Some single interventions (e.g., vaccinations) occur a limited number of times per lifetime; other interventions (e.g., water fluoridation and mammography screening) are continual or are repeated many times during a person's lifetime. The cost and effect of 1 year of fluoridation or of one mammography screening may be misleading. The analysis should compare the total lifetime costs for the prevention program -- rather than the costs for one application of the program -- with the adverse health outcomes averted as a result of the program.

Prevention programs can take place in various settings and sites, including treatment centers. For example, breast cancer screening costs (4) are variable. The total cost of the technician and the radiologist fees, the use of the mammography screening device and materials, and the visit-associated administrative duties ranges from less than $50 to several hundred dollars. An effective program can be simply a transfer of resources and costs from one site to another. Results of BASICC analyses can enable decision makers to deliver more health benefits from prevention services with the limited resources available.

Health outcomes are inherently not comparable. For example, tractor rollover protection systems not only prevent death but also reduce the severity of nonfatal injuries (6). In fact, most injuries are not fatal; fatal injuries, however, are more likely to be reported than are nonfatal injuries. A common standard of measure, such as years of healthy life gained (14,15), should be used to enhance comparability whenever possible to facilitate comparison across interventions for different health outcomes. All resource costs should be included when different intervention strategies are compared.

Currently, most resources are allocated for the prevention of health outcomes without the use of a standard methodology and without an answer to the two questions: "what works?" and "how much does it cost?" The BASICC methodology for prevention effectiveness provides a minimum standard approach for a more rational allocation of resources and for more informed decision making.

In Memoriam

Robin Gorsky, who was instrumental in bringing the principles of operations research and cost-effectiveness analysis to public health, died before this report went to press. She was a strong supporter and friend of CDC.

References

1. Teutsch SM. A framework for assessing the effectiveness of disease and injury prevention. MMWR 1992;41(No. RR-3).

2. Haddix A, Teutsch SM, Shaffer P, Dunet D, eds. Prevention effectiveness: a guide to decision analysis and economic evaluation. New York: Oxford University Press, 1996 (in press).

3. CDC. Public health focus: fluoridation of community water systems. MMWR 1992;41:372-5,381.

4. CDC. Public health focus: mammography. MMWR 1992;41:454-9.

5. CDC. Public health focus: surveillance, prevention, and control of nosocomial infections. MMWR 1992;41:783-7.

6. CDC. Public health focus: effectiveness of rollover protective structures for preventing injuries associated with agricultural tractors. MMWR 1993;42:57-9.

7. CDC. Public health focus: effectiveness of smoking-control strategies

• United States. MMWR 1992;41:645-7,653.

TABLE 1. Example of a resource inventory
=====================================================================================================
(A)               (B)                   (C)                        (D) = (B) X (C)

Resource          Unit cost             No. units used             Resource cost
----------------------------------------------------------------------------------
Provider time     $20 per hour          6 hours                         $120.00
Brochure          $28 per 100 items     25 items                          $7.00
Calendar          $0.05 per item        25 items                          $1.25
Room              $25 per session       6 sessions                      $150.00
Participant       $10 per hour          7.5 hours X 25 clients         $1875.00 *
                                          per group

Total cost, excluding participant costs +                               $278.25

Total cost, including participant costs                               $2,153.25
----------------------------------------------------------------------------------
* Cost per participant = $2,153.25 / 25 participants = $86.13
+ If participant costs are excluded, the cost per participant is $278.25 / 25 participants = $11.13
=====================================================================================================

TABLE 2. Method for net cost calculation
===============================================================================
2A. Information necessary for net cost calculation
-----------------------------------------------------------------------------
Data                                             Cost
-----------------------------------------------------------------------------
Cost of intervention per participant             $86.13
Hypothetical medical care costs per case of
  smoking-related disease                        $55,000
Rate of smoking-related disease among
  smokers who are not participants               10 cases per 1,000 smokers
Rate of disease among participants               2 cases per 1,000 nonsmokers
===============================================================================
2B. Example of a net cost calculation
-----------------------------------------------------------------------------
For 1,000 persons:                               Cost calculations
-----------------------------------------------------------------------------
Intervention costs                               $86,130
Cases of smoking-related disease averted         10 - 2 = 8
Medical costs of disease averted                 8 X $55,000 = $440,000
Medical costs equally spaced and discounted,
  3% per year, 24 years                          0.7056 X $440,000 = $310,470
===============================================================================

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

Page converted: 09/19/98

HOME  |  ABOUT MMWR  |  MMWR SEARCH  |  DOWNLOADS  |  RSS |  CONTACT POLICY  |  DISCLAIMER  |  ACCESSIBILITY

Morbidity and Mortality Weekly Report Centers for Disease Control and Prevention 1600 Clifton Rd, MailStop E-90, Atlanta, GA 30333, U.S.A Department of Health and Human Services

This page last reviewed 5/2/01