 |
 
|
 |
 |
| |
 |
|
|
| ||||||||||
|
|
|
|
|
|
|
||||
| ||||||||||
 |
|
|
|
Persons using assistive technology might not be able to fully access information in this file. For assistance, please send e-mail to: mmwrq@cdc.gov. Type 508 Accommodation and the title of the report in the subject line of e-mail. A Framework for Assessing the Effectiveness of Disease and Injury PreventionU.S. Department of Health and Human Services Public Health Service Centers for Disease Control Epidemiology Program Office Atlanta, Georgia 30333 The MMWR series of publications is published by the Epidemiology Program Office, Centers for Disease Control, Public Health Service, U.S. Department of Health and Human Services, Atlanta, Georgia 30333. SUGGESTED CITATION Centers for Disease Control. A framework for assessing the effectiveness of disease and injury prevention. MMWR 1992;41(RR-3) (inclusive page numbers). Centers for Disease Control William L. Roper, M.D., M.P.H., Director The production of this report as an MMWR serial publication was coordinated in: Epidemiology Program Office Stephen B. Thacker, M.D., M.Sc., Director Richard A. Goodman, M.D., M.P.H. Editor, MMWR Series R. Elliott Churchill, M.A. Project Coordinator, Scientific Communications Program Public Health Publications Branch Suzanne M. Hewitt, Chief Mark W. Crowe, M.A., Project Editor Morie E. Miller, Editorial Assistant Information Resources Management Activity Sandra L. Ford, Visual Information Specialist Contents Glossary Background Role of Public Health Agencies Approaches to Prevention Assessing Prevention Effectiveness Prevention: From Research into Practice Methodologies for Assessing the Effectiveness of Disease and Injury Prevention Strategies Efficacy, Effectiveness, and Safety Attributable Risk and Prevented Fraction Economic Studies Additional Approaches Comments References Glossary * Adverse event (outcome) -- Any disease or injury, e.g., premature death or unnec essary morbidity. Attributable risk -- The theoretical reduction in the rate or number of cases of an adverse outcome that can be achieved by elimination of a risk factor. Cost benefit -- An economic analysis in which all costs and benefits are converted into monetary (dollar) values and results are expressed as dollars of benefit per dollars expended. Cost effectiveness -- An economic analysis assessed as a health outcome per cost expended. Cost utility -- An economic analysis assessed as a quality-adjusted outcome per net cost expended. Decision analysis -- An analytic technique in which probability theory or probabilistic information processing is used to obtain a quantitative approach to decision making. Demonstration settings -- A population- or clinic-based environment in which prevention strategies are field tested. Direct costs -- Costs associated with prevention activities and the health-care system (compare with Indirect costs). Discounting -- A method for adjusting the value of future costs and benefits. Discounting -- expressed as a present dollar value -- is based on the time-value of money; i.e., a dollar today is worth more than it will be a year from now (even if inflation is not considered). Distributional effects -- The manner in which the costs and benefits of a preventive strategy affects different groups of people in terms of demographics, geographic location, and other descriptive factors. Effectiveness -- The improvement in health outcome that a prevention strategy can produce in typical community-based settings. Efficacy -- The improvement in health-outcome effect that a prevention strategy can produce in expert hands under ideal circumstances. Fixed costs -- The portion of total costs of a program incurred even when output is nil, e.g., costs associated with overhead, facilities, and overhead salaries (compare with Variable costs). Health promotion -- Disease and injury prevention strategies that depend on behavior change in individuals. Health protection -- Disease and injury prevention strategies that depend on changes in an individual's environment. Indirect costs -- Costs not directly associated with prevention and health-care activities that accrue to individuals (e.g., loss of time from work), society (e.g., disability payments), or employers (e.g., decreased productivity). Meta-analysis -- A systematic, quantitative method for combining information from multiple studies in order to derive the most meaningful answer to a specific question. Assessment of different methods or outcome measures can increase power and account for bias and other effects. Outcome measures -- Disease and injury morbidity and mortality that are the target of prevention programs. Payer -- An organization responsible for payment of health-care costs. Premature mortality -- a) Any preventable death. b) Deaths that occur before a specified age, most often age 65 or the average life expectancy of a certain population. Preventable fraction -- The proportion of an adverse health outcome that potentially can be eliminated as a result of a prevention strategy. Prevented fraction -- The proportion of an adverse health outcome that has been eliminated as a result of a prevention strategy. Prevention (primary, secondary, tertiary) -- A framework for categorizing prevention programs based on the stage of the natural history of a disease or injury: Primary prevention -- An intervention implemented before there is evidence of a disease or injury. This strategy can reduce or eliminate causative risk factors (risk reduction). Secondary prevention -- An intervention implemented after a disease has begun, but before it is symptomatic (screening and treatment). Tertiary prevention -- An intervention implemented after a disease or injury is established. This strategy can prevent sequelae. Preventive medical services -- Clinical services provided to patients to reduce or prevent disease, injury, or disability. These are preventive measures provided by a health-care professional to a patient. Preventive strategies (clinical, behavioral, environmental) -- A framework for cate gorizing prevention programs based on how the prevention technology is deliv ered, i.e., provider to patient (clinical), individual responsibility (behavioral), or alteration in an individual's surroundings (environmental). Process measures -- Steps in a (prevention) program logically required in order for the program to be successful, e.g., knowledge, attitudes, and behaviors may be targeted by a prevention program for the prevention of an adverse health outcome. Recipients of services (beneficiaries) -- Any individual who benefits from a preven tion strategy; used most often in the context of medical services. Safety -- An assessment of the level and acceptability of risk of adverse outcomes that occur as a result of a prevention technique in the context of a specific prevention strategy and disease or injury outcome. Sensitivity analysis -- A quantitative method for assessing the impact of individual factors in a model by varying values of those factors and observing the effect on the outcome. Technology -- Techniques, devices, drugs, or procedures used to reduce the risk of an adverse health outcome. Unnecessary morbidity -- Any preventable disease, injury, or disability. Variable costs -- The portion of the total cost that increases with greater output, e.g., the costs associated with increasing the number of persons seen in an education program. Willingness to pay -- An approach for determining the value of a health outcome based on society's valuation of (willingness to pay for) that outcome. A Framework for Assessing the Effectiveness of Disease and Injury Prevention Steven M. Teutsch, M.D., M.P.H. ** Summary This report presents background information for a series of MMWR articles on evaluating the effectiveness of disease and injury prevention strategies, describes issues surrounding the assessment of effectiveness, and provides an overview of methods used in such assessments. BACKGROUND Public health practice relies on scientifically sound strategies for improving the quality of life and reducing unnecessary morbidity and premature mortality. With the rapid increase in innovative technologies emerging from scientific research, the number of conditions for which potential public health action might be undertaken is proliferating rapidly. In addition, a variety of innovative approaches can be utilized to implement each new and existing technology. Because resources are usually limited, it is imperative that there are systematic mechanisms for assessing the effectiveness and safety of each of those technologies as future directions for prevention are considered. In 1988, the United States expended 18.4 billion health-care dollars on prevention, or 3.4% of total health expenditures. An additional 14.4 billion dollars were spent for nutrition, sewers, and clean water (1). The federal government alone paid for 48% of these costs, with state and local governments contributing another 18%. To balance the costs and benefits of prevention activities, the development of new techniques and their delivery to all segments of the population should be based on an understanding of the efficacy, effectiveness, and costs associated with these new techniques. In order to deliver effective early-detection and preventive services, decision makers require timely and valid information about the potential of proposed prevention measures to reduce occupational and environmental exposures and of strategies designed to alter health-related behavior. Information on the effectiveness of prevention is needed by many people. Public health professionals and policy makers at the state, national, and local levels make judgments about public health priorities, select prevention strategies, and allocate resources. Health-care practitioners and payers confront decisions about clinical strategies for delivering preventive services. Information on the efficacy, effectiveness, and costs provides a basis for optimal utilization of techniques. The U.S. Preventive Services Task Force and other groups have developed recommendations based on assessments of efficacy and effectiveness (2-4). These efforts provide a scientific basis for determining which procedures should be incorporated into the routine care of patients and suggest when and how the procedures should be performed. They also identify instances in which more research is needed to improve the data base on effectiveness. Role of Public Health Agencies As the lead federal agency for prevention, CDC has emphasized evaluation of prevention programs (5). As part of that continuing effort, CDC is strengthening efforts to assure that public health priorities and program strategies maximize the health of the population relative to the resources expended. Critical reviews of current scientific knowledge and alternative programmatic approaches require CDC and state and local health departments to make an ongoing commitment to examine the scientific underpinnings for disease- and injury-control activities and to modify programs to capitalize on new opportunities and new information. CDC assesses the effectiveness of prevention efforts systematically, thereby complementing efforts by the Agency for Health Care Policy and Research to assess the effectiveness of medical procedures (6). A forthcoming issue of the MMWR (weekly) will inaugurate a monthly series of articles on prevention effectiveness. Each article will highlight current knowledge regarding a specific prevention strategy, including efficacy, effectiveness, safety, and economic aspects. The topics to be covered have been selected on the basis of their current priority for public health, as assessed by their inclusion in the National Health Promotion and Disease Prevention Objectives (7); the existence of current CDC programs; and the availability of data. The articles are intended to focus on CDC's mission in prevention, to provide approaches to evaluating the effectiveness of prevention strategies, and to present specific examples of the effectiveness of a broad array of prevention strategies. The series will highlight issues particular to these prevention strategies. This MMWR Recommendations and Reports provides a conceptual overview of the issues of prevention effectiveness that will be addressed in the MMWR (weekly) during 1992 and 1993. Specific methodologies will be presented within the context of the individual articles and will provide a basis for recommendations regarding more standardized approaches to conducting assessments of prevention effectiveness to be presented in future issues of Recommendations and Reports. Approaches to Prevention Two complementary frameworks for conceptualizing prevention programs are presented below. These frameworks suggest a range of process and outcome measures that should be assessed. The first framework includes three components (clinical, behavioral, and environmental) and considers how the prevention technology is delivered. The second also includes three components (primary, secondary, and tertiary) and considers the stage of a disease or injury at which the intervention is targeted. Framework I: Delivery of Prevention Technologies Clinical prevention strategies. The traditional medical model for preventive services, early detection, and treatment relies on one-to-one, provider-to-patient interaction, such as underlies screening, vaccination, and diagnosis and early-treatment programs. These interventions generally occur within the traditional health-care delivery system. Behavioral prevention strategies. Behavioral change models (health promotion) use a broad array of strategies to encourage lifestyle changes, such as exercise, smoking cessation, and healthful diets. Accomplishment of these behavior changes may require changing a person's knowledge and attitudes, as well as the behaviors, of individuals or groups. This is a complex, sequential process. Environmental prevention strategies. Environmental strategies (health protection)--such as safe water, fluoridation, lead abatement, regulations on public smoking, seat-belt laws, and safer highways--generally require societal commitment for the implementation of the extensive interventions needed. Once these changes are made, they require little individual effort from the beneficiary and can have far-reaching impact. Obtaining clinical services or effecting behavioral changes require that individuals make personal efforts to take necessary actions. Preventive environmental services, on the other hand, are for the most part passive, requiring little or no action on the part of the beneficiary. Framework II: Targeting Intervention by Stage of Disease or Injury Primary prevention. Primary prevention is the reduction or control of causative factors for a health problem and includes reducing risk factors--such as smoking to prevent lung cancer or sex education to reduce sexually transmitted diseases--and environmental exposures--such as reducing ambient lead to prevent intellectual impairment. This category also includes health-service interventions, such as vaccinations or such preventive ``therapy'' tools as fluoridated water supplies or dental sealants. Secondary prevention. Secondary prevention involves early detection and treatment, such as mammography for detecting breast cancer or contact tracing for detecting and treating persons with sexually transmitted diseases. Tertiary prevention. Tertiary prevention involves providing appropriate supportive and rehabilitative services to minimize morbidity and maximize quality of life, such as rehabilitation from injuries. It includes preventing secondary complications among individuals with disabilities, such as shoulder overuse syndrome among wheelchair users or bedsores among those confined to bed. Rational choices can only be made based on valid and timely information on the efficacy, effectiveness, and cost of each prevention strategy. This information allows comparison of alternative approaches for an individual condition-- e.g., the relative effects of seat belts, passive restraints, safer highways, or more efficient and available emergency medical services on reducing morbidity and mortality from motor-vehicle crashes. Sound data facilitate difficult choices among disparate conditions, such as genetic counseling to reduce birth defects or screening and treatment programs for persons with diabetic retinopathy. Assessing Prevention Effectiveness The scientific approach to evaluating the effectiveness of prevention strategies may be termed ``assessing prevention effectiveness.'' Prevention effectiveness includes the following: Identification of efficacious and effective strategies (see Efficacy, Effectiveness, and Safety) to reduce morbidity and mortality and improve the quality of life. Determination of the potential and practical consequences of those strategies, including social, legal, ethical, and economic factors. Determination of the economic impact of a prevention strategy. Determination of optimal methods for implementing those strategies. Evaluation of the impact of prevention programs. Assessment of the effectiveness of prevention strategies allows for comparison of alternate approaches both within and among prevention programs. For instance, mammography can detect breast cancer early enough to allow timely treatment, which, in turn, can reduce mortality for women in the age group greater than 50 years. The cost effectiveness of screening women less than 50 years is less clear (8). In a different context, lead exposure, even at low levels, has been shown to decrease mental development among children, but data demonstrating the impact of lead abatement programs on mental development are more tenuous (9). When there are insufficient resources to do everything that is desirable, how are decision makers to make rational choices among such seemingly disparate endeavors? How can information be collected to help structure prevention programs? For example, in screening for breast cancer, what are the advantages and disadvantages of programs targeted to special high-risk populations compared with strategies appropriate for the general population? The roots of prevention effectiveness lie in the assessment of technology. Technology assessment is broad in scope and includes ``. . . any process of examining and reporting properties of a technology . . ., such as safety, efficacy, feasibility, and indications for use, cost, and cost-effectiveness, as well as social, economic, and ethical consequences, whether intended or unintended'' (10). Prevention: From Research into Practice An idealized model for developing and implementing prevention strategies is shown (Figure 1). The process begins with the development of prevention technologies by researchers in the basic sciences. The researchers identify those technologies that may reduce unnecessary morbidity and premature mortality. Techniques are demonstrated to be efficacious through the application of additional research under carefully controlled conditions. As these techniques are applied at the community level, the impact and cost of the interventions can be assessed first in demonstration settings and then in routine community settings. Improvements in methods to effect these changes can then be incorporated iteratively. This idealized model oversimplifies an iterative process. Moreover, there is pressure to move rapidly from basic and applied research to widespread implementation before the appropriate evaluation studies can be completed. Consequently, there are often gaps in what is known about the efficacy, effectiveness, safety, or economic impact of specific prevention strategies. The articles in the MMWR (weekly) series will identify many of these gaps. They will suggest how these gaps have been addressed, despite the lack of definitive answers. An effort to fill these gaps should form the basis for additional studies. While prevention activities can readily be defined as successful if they delay or avert morbidity and mortality, actual measurement of the nonoccurrence of events is difficult in the absence of controls. Controlled trials, however, are often limited by fiscal, ethical, or other constraints. This Recommendations and Reports focuses on the difficult problems of assessment that arise as a consequence of these constraints. Articles in the MMWR (weekly) series will describe examples of how prevention effectiveness can be assessed. METHODOLOGIES FOR ASSESSING THE EFFECTIVENESS OF DISEASE AND INJURY PREVENTION STRATEGIES The effectiveness of a prevention strategy should be assessed at each stage of its development and implementation (Figure 1), and there are methods that can be used at each step. In general, once studies demonstrate the efficacy of a technique, epidemiologic, mathematical, or economic models can be constructed to estimate the potential future impact of an intervention program. If programs already exist, data and results from actual experience can be used to modify the programs and to provide more accurate estimates of changes in outcomes and associated costs and benefits. The techniques described in this section should be used and interpreted in the context of their temporal relationship to the development and implementation of the prevention strategy. Efficacy, Effectiveness, and Safety Efficacy The first question that should be addressed about any prevention technique is ``does it work?'' What is the scientific basis for using the technique? How good is that information? ``Efficacy'' is defined as the effect obtained with a specific technique in expert hands under ideal circumstances. Determination of the efficacy of a prevention strategy requires doing a critical review of the studies that have been done, with the goal of assessing the quality of the science and the magnitude of the impact. Criteria for clinical preventive strategies have been developed by the Clinical Preventive Services Task Force (2) and the Canadian Task Force on the Periodic Health Examination (3) for this purpose. In the medical milieu, the sine qua non for efficacy studies has been a randomized clinical trial of health outcomes with and without use of the technology being assessed. Most medical technologies, however, are not subjected to this level of scrutiny because, despite their importance, these studies are prohibitively costly in terms of time, effort, and money. Consequently, smaller or methodologically less desirable studies often must be used to assess efficacy. Effectiveness Once a technique, such as a seat-belt or smoking-cessation program, is known to be efficacious, it is necessary to answer ``how well does it work in the real world?'' Thus, effectiveness is the impact of the prevention activity in practical application. It takes into consideration not only the efficacy of the intervention, but the practical aspects of delivering it to people as part of their routine activities. Problems of access, follow up, quality assurance, and individual behavior in the context of existing legal, health-care, and social systems are all elements of the evaluation of effectiveness. Although efficacy is usually determined under carefully controlled conditions, by their very nature, effectiveness studies must be done in the setting in which the intervention will be conducted, i.e., in communities. Therefore, determination of the impact of prevention strategies most often comes from community demonstration projects and prevention programs. Safety Prevention techniques are associated with potential hazards. These hazards may be direct results of the prevention technology as it might be used or misused in routine practice, e.g., surgical complications from biopsies of lesions originally detected with mammography. They may also be indirectly related to the procedures, e.g., anxiety created by the finding of a suspicious lesion with mammography or risks associated with an inappropriate intervention resulting from a screening test. Initial safety data emerge from efficacy studies, and additional information is generated when this kind of evaluation is applied on a broader scale. Attributable Risk and Prevented Fraction Attributable risk and prevented fraction provide quantitative measures of the impact of risk factors and interventions. Attributable risk The theoretical limit of what can be achieved can be assessed by the population attributable risk (PAR). The attributable risk for any risk factor is a measure of the proportion or number of cases of the disease or injury that could be eliminated if the risk factor had not occurred. For conditions with a one-to-one correspondence between a risk factor and an outcome (e.g., smoking and lung cancer), the attributable risk is calculated as follows: Pe (RRe-1)PAR = 1 + Pe (RRe-1)where Pe is the population prevalence of exposure to the risk factor and RRe is the relative risk associated with the risk factor. The PAR is dependent on the strength of the risk factor (the relative risk), as well as the prevalence of the risk factor in the population. When the relative risk and prevalence of multiple levels of a risk factor are known -- e.g., mild, moderate, or severe hypertension for ischemic heart disease-- the PAR can be expressed as follows: |gS (Pe (RRe-1))PAR = 1\f+\f|gS (Pe (RRe-1))where the summation is over each level of a risk factor. The PAR is used in cost-effectiveness and cost-benefit analyses (see later discussion) to examine the marginal costs for achieving each additional level of reduction in risk. Prevented fraction Although the PAR calculates the theoretical limit of what can be achieved with eliminating a risk factor, the prevented fraction addresses what can be achieved with a preventive service actually delivered. For example, the attributable risk for smoking assumes that the risk of lung cancer for all smokers is eliminated and their level of risk returns to that for nonsmokers. In addition, the preventable fraction takes into account the proportion of smokers that one could persuade to quit smoking and also the delayed effect from the smokers' prior risk. The relative risks for causative factors used to calculate attributable risks are greater than one, which reflects the additional risk associated with the presence of a risk factor. In contrast, the relative risks used to calculate the prevented fraction are less than one, which reflects the reduction in risk that occurs as a result of an intervention. In community settings, the intensity of introducing an intervention and the resources that can be brought to bear are usually less than in the efficacy studies conducted earlier. In addition, although participants can generally be selected for efficacy studies, public health programs target all the individuals in the risk group. Motivation and participation of a general population are lower than among persons in a research project. Both these factors reduce the impact of prevention strategies when they are translated from the research setting into the community, in much the same way that effectiveness is diluted from efficacy, as described previously. Prevented fraction is one measure used to determine what can actually be achieved in a community setting, such as the number of cases of a disease prevented by a vaccine program. It is calculated as follows: P1 (1-RR)PF = RR + P1 (1-RR)where PF refers to preventable fraction. In addition, Pe for an attributable risk is a measure of the proportion of the population that has the risk factor; for the calculation of preventable fraction, P1 measures the proportion of the population that is at risk and that accepts the intervention. Prevented fraction can most accurately be assessed after a prevention strategy has been widely implemented. The term ``preventable fraction'' should perhaps be reserved for situations in which estimates are used to assess potential impact prior to widespread implementation of a strategy. Economic Studies Prevention technologies may be aimed at disparate health outcomes, which range, for example, from the use of fluoride to reduce caries to the use of folate to prevent neural-tube defects. Economic analyses allow us to compare interventions in ways that consider both the costs and the benefits (11-13). To be useful, however, these analyses need to fulfill basic criteria. Analyses of prevention strategies can be interpreted only in comparison with alternatives, e.g., doing nothing, using other methods, or addressing different problems. Although many public health practitioners are unfamiliar with the methods of economic analyses, standard methods exist to ensure that the approaches are indeed comparable. Cost-effectiveness analysis measures dollars expended per health outcome attained, e.g., dollars per year of life saved. Cost-benefit analyses convert health outcomes into a dollar value based on how much society values the outcome or is willing to pay for the outcome. This conversion of morbidity and mortality into dollar terms requires methods for placing a value on various states of being. Cost-benefit analysis expresses the results as the dollars expended per dollars of benefit achieved. A third approach (cost-utility analysis) involves converting these outcomes to another indicator, the quality-adjusted life-year. Cost-utility analysis can be seen as lying methodologically between cost-effectiveness and cost-benefit analysis. All three, however, require a measure of the effectiveness of the prevention strategy and are based on models that allow the calculation of benefit in terms of efficacy and effectiveness. Early in the development of a prevention strategy, it is frequently necessary to make assumptions about the quantitative relationships using models. For example, in an exercise program (14), a model for determining the number of adverse events that could be prevented was constructed on the basis of a given cohort, epidemiologic data about the relationship of exercise to cardiovascular disease and injuries, and adherence to exercise as part of a medical regimen. Models are often useful in comparing alternate strategies, e.g., the use of one versus two doses of vaccine, or the different impacts associated with vaccinating people at various ages. When the accuracy of those estimates is uncertain, the impact of changes in these assumptions can be assessed by sensitivity analyses. These assessments examine the magnitude of the change in outcome as the variables in question are changed. Sensitivity analyses provide information on the probable range of estimates that can be derived from the model and can identify areas in which research is needed to acquire more precise estimates. Economic analyses can be performed from a variety of perspectives. The societal perspective takes the broadest view, yet begs the question of the impact on specific population subgroups. Employers may be interested primarily in their health insurance costs, loss of time from work, and improvements in productivity. An individual may assess the out-of-pocket costs and perceived benefits of the care. The government may focus on the costs associated with disability, Medicare, and Medicaid payments. Insurers may be concerned with expenditures for services rendered. Each perspective is legitimate in its own right. What is important is that the perspective of the study be clearly specified and that only studies using similar perspectives be compared. Cost A first step in economic analyses is to determine the total program cost. From a societal perspective, both direct and indirect costs should be considered. For these analyses, there are direct program costs -- including personnel, equipment, and space -- and indirect costs -- including costs of time to the recipient of the program, lost time from work, and travel. Similarly, direct benefits include costs saved that would have been directly associated with the outcome (e.g., health-care costs), as well as indirect benefits (e.g., earned wages and productivity). Alternatively, the value of benefits can be assessed in terms of society's willingness to pay for them. As the magnitude of the program changes, costs may also be described as fixed (required regardless of the scope of the program) and variable (increases with number of participants). Cost effectiveness Cost-effectiveness analyses are most commonly performed. Effectiveness data on outcomes are more generally available and more readily understood than cost-benefit analysis outcomes (in which value is translated into dollars). Therefore, only cost-effectiveness and its variant, cost-utility analysis, are presented here. Cost-effectiveness analysis combines the cost of implementing the intervention with the effectiveness of the intervention. Operationally, it is defined as the net cost divided by the net effectiveness, or as follows: Net Cost / Cost Effectiveness = Adverse Outcomes Averted where Net Cost = CostProgram + CostSideEffects - CostDisease - CostIndirect.Costs are broken down into the cost of the prevention program (CostProgram), the costs of the side effects of the program (CostSideEffects) less the costs of diagnosis and treatment associated with the disease (CostDisease) -- which are avoided as a result of the prevention program -- and the indirect costs (CostIndirect) saved or incurred as a result of changes in productivity, morbidity, and mortality. CostProgram includes the costs for management and operation (space, equipment, materials, personnel time, travel, overhead, follow up, and treatment) plus participants' time and expenses. CostSideEffects includes the expenses associated with incidental findings and hazards and psychological effects associated with the program itself. CostDisease includes the direct costs of care that would have been associated with management had the disease not been prevented. CostIndirect includes the indirect costs from absenteeism, loss of productivity, and transfer payments that are averted as a result of preventing the adverse outcome. For example, for an exercise-promotion activity (17): CostProgram includes the direct cost of exercise (equipment) and counseling. CostSideEffects includes the costs associated with exercise-induced injuries (medical costs, loss of time from work). CostDisease is the direct (medical care) and indirect costs (loss of earnings from disability and premature death) that would have been associated with coronary artery disease had it not been prevented. Cost Indirect reflects the cost (value) of the time spent exercising. Cost effectiveness is most meaningfully measured in terms of outcomes -- cost per life saved or cost per case averted. Other measures can be used, such as cost per milligram of serum cholesterol lowered or cost per person screened. Enumerated processes -- such as the number of mammograms performed -- are easier to measure and are useful for monitoring program activities; they are less useful in determining program effectiveness, unless the process measured has a proven relationship to the outcome. Cost-utility analysis In a cost-utility analysis, the net benefit is often expressed as the number of life-years saved. This can be modified to adjust for morbidity averted and side effects caused as follows: Adjusted Life-Years = Saved Unadjusted Life-Years + Saved Morbidity Reduced - Side Effects Caused Weighing factors are used to convert morbidity and side effects to the same units as life-years. Weighing factors are derived in two ways: a) a determination of the length of life a person would be willing to forego if s/he could be freed of the morbidity, or b) the risk of death one would be willing to accept if s/he could be freed of the morbidity. In the exercise-promotion example above (17), cardiovascular morbidity prevented and morbidity from injuries incurred were converted into quality-adjusted life-years and summed with the unadjusted years of life saved to determine the adjusted life-years saved. However, adjusted life-years are subjective determinations. Discounting Benefits from a prevention program accrue in the future, whereas costs are incurred in the present. Because a dollar in the present is worth more than a dollar in the future (how much would someone have to repay you a year from now to actually reimburse you for a dollar you lend today?), the value of future dollars is adjusted to their present value. This is done to reflect the cost of the money expended at present, but for which benefits do not accrue until the future. The rate of return foregone is called the discount rate. Although of modest importance over the short run, even relatively low discount rates have very large impact when the benefits are far in the future. At a compound rate of 4%, benefits that occur 18 years in the future are worth only half of today's dollar value (without considering additional losses in value due to inflation). Although the need for discounting is well recognized, the rate used for discounting (excluding inflation) varies substantially from researcher to researcher. Standardization of methodology Much as standard methods for age adjustment allow us to compare mortality rates from populations with markedly different age distributions, standard methods for cost-effectiveness analyses are needed to ensure that results of these studies can be compared. The perspective of studies must be the same, items included in the analysis and their values must be comparable, and other procedures such as discounting must be done in a standard fashion. Although many excellent cost-effectiveness studies have been done, this level of standardization for prevention studies does not currently exist. Additional Approaches Many studies yield contradictory results or have insufficient power to address critical questions. Expert panels, literature reviews, and consensus conferences may provide judgments based on available data, but they are highly subject to both the level of the information presented and the interests of the participants (5). Meta-analysis provides a method for estimating effect across a large number of studies (15-17). Unlike the expository style and methods of literature reviews, meta-analyses utilize specific criteria for examining each study. These criteria include the study design, the population studied, the size of the study, statistical methods, and results. In determining overall effect, the results of the studies can be weighted by study quality or sample size, as well as by factors relevant to particular questions of interest. Prospective and retrospective studies on the suitability of interventions for the target populations and on the legal, ethical, and distributional effects should also be conducted. COMMENTS The upcoming MMWR (weekly) series will provide public health decision makers with information about the impact that prevention programs can make on the health of their communities. This information will also provide an opportunity for examining how information can be organized and for determining the methods most suitable for public health interventions. Discussion and careful examination of the MMWR (weekly) series should lead to more systematic ways for assessing and enhancing the effectiveness of public health programs. References
** The author was assisted by Stephen Thacker, Willard Cates, Jr., William R. Taylor, David Holtgrave, Richard Rothenberg, Edwin Kilbourne, Lyle Conrad, Donna Stroup, and Barbara Holloway. Disclaimer All MMWR HTML documents published before January 1993 are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices. **Questions or messages regarding errors in formatting should be addressed to mmwrq@cdc.gov.Page converted: 08/05/98 |
| |||||||
This page last reviewed 5/2/01
|
