Lesson 5: Public Health Surveillance

Section 7: Evaluating and Improving Surveillance

Surveillance for a disease or other health-related problem should be evaluated periodically to ensure that it is serving a useful public health function and is meeting its objectives. Such an evaluation: (1) identifies elements of surveillance that should be enhanced to improve its attributes, (2) assesses how surveillance findings affect control efforts, and (3) improves the quality of data and interpretations provided by surveillance.

Although the aspects of surveillance that are emphasized in an evaluation can differ, depending on the purpose and objectives of surveillance, the evaluation's overall scope and approach should be similar for any health-related problem. The evaluation usually begins by identifying and interviewing key stakeholders and by collecting background documents, forms, and reports. The evaluation should address the purpose of surveillance, objectives, and mechanics of conducting surveillance; the resources needed to conduct surveillance; the usefulness of surveillance; and the presence or absence of the characteristics or qualities of optimal surveillance. The outcome of the evaluation should provide recommendations for improvement.(9, 27, 28) We discuss these main components in the following sections.

Stakeholders

Stakeholders are the persons and organizations who contribute to, use, and benefit from surveillance. They typically include public health officials and staff, health-care providers, data providers and users, community representatives, government officials, and others interested in the health condition under surveillance. Stakeholders should be identified not only because they contribute to or use surveillance results, but also because they might be interested in, and can contribute to the evaluation. Stakeholders should be engaged early in the evaluation process because some might have a hand in implementing recommendations that emerge from the evaluation. Evaluations conducted without early buy-in from those responsible for conducting surveillance are often viewed as unwanted criticism and interference from outsiders and are usually ignored.

Purpose, objectives, and operations

The evaluation should start with a clear statement of the purpose of surveillance, which usually facilitates prevention or control of a health-related problem. The purpose should be followed by clearly stated objectives describing how surveillance data and their interpretations are used. Considering the information needed for effective prevention and control of the health problem is also helpful. For example, an objective of surveillance for gonorrhea might be to detect individual cases and their contacts so that both can be treated. To meet this objective, sufficient information will be needed to identify cases and contacts for follow-up. To characterize the purpose, objectives, and operations of surveillance, addressing the questions at the beginning of this lesson will be helpful.

Sketching a flow chart of the method of conducting surveillance is recommended. First, identify gaps in the evaluator's knowledge of how surveillance is being conducted. Second, provide a clear visual display of the activities of and flow of data for surveillance for those not familiar with it (Figure 5.9).

Usefulness

Usefulness refers to whether surveillance contributes to prevention and control of a health-related problem. Note that usefulness can include improved understanding of the public health implications of the health problem. Usefulness is typically assessed by determining whether surveillance meets its objectives. For example, if the primary objective of surveillance is to identify individual cases of disease to facilitate timely and effective control measures, does surveillance permit timely and accurate identification, diagnosis, treatment, or other handling of contacts when appropriate?

Usefulness of surveillance is influenced greatly by its operation, including its feedback mechanism to those who need to know, and by the presence or absence of the characteristics of optimal surveillance. Qualities or characteristics described previously in this lesson and in Appendix A affect the operation and usefulness of surveillance. Evaluation of surveillance requires assessment, either qualitatively or quantitatively, of each characteristic.

Figure 5.9 Simplified Diagram of Surveillance for a Health Problem

Image Description

Source: Centers for Disease Control and Prevention. Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines working group. MMWR 2001;50(No. RR-13): p. 8.

Resource requirements (personnel and other costs)

These costs are usually assessed in light of the objectives of surveillance and its usefulness and against the expected costs of possible modifications or alternatives to the way in which surveillance is conducted.

Recommendations

The purpose of evaluating surveillance for a specific disease is to draw conclusions and make recommendations about its present state and future potential. The conclusions should state whether surveillance as it is being conducted is meeting its objectives and whether it is operating efficiently. If it is not, recommendations should address what modifications should be made to do so. Recommendations must recognize that the characteristics and costs of conducting surveillance are interrelated and potentially conflicting. For example, improving sensitivity can reduce predictive value positive and increase costs. For surveillance, recommendations should be prioritized on the basis of needs and objectives. For example, for syndromic surveillance, timeliness and sensitivity are critical, but high sensitivity increases false alarms, which can drain limited public health resources. Each characteristic must be considered and balanced to ensure that the objectives of surveillance are met. (See Appendix E for an assessment of and recommendations for notifiable disease surveillance.)

Recommendations should be realistic, feasible, and clearly explained. Feedback to health facilities and stakeholders is an important, but sometimes neglected, part of the evaluation. Certain recommendations might be unpopular and will need convincing justification. When possible, include an estimate of the time and resources needed to implement the changes. Prioritizing plans and developing a timetable for surveillance improvements might be helpful. A method for ensuring that improvements are initiated in a timely fashion is critical to the evaluation's ultimate success.(9, 29)

References (This Section)

9. Protocol for the evaluation of epidemiological surveillance systems [monograph on the Internet]. Geneva: World Health Organization [updated 1997; cited 2006 Jan 20]. Available from: http://whqlibdoc.who.int/hq/1997/WHO_EMC_DIS_97.2.pdf.

27. Centers for Disease Control and Prevention. Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines working group. MMWR 2001;50(No. RR-13):1–35.
28. Centers for Disease Control and Prevention. Framework for evaluating public health surveillance systems for early detection of outbreaks; recommendations from the CDC Working Group. MMWR 2004;53(No. RR-5):1-13.
29. World Health Organization Regional Office for Africa and the Centers for Disease Control and Prevention. Technical Guidelines for Integrated Disease Surveillance and Response in the African Region. Harare, Zimbabwe and Atlanta, Georgia, USA. July 2001: 1–229.