Modeling in Real Time During the Ebola Response

Background

During CDC’s response to the 2014–2016 Ebola virus disease (Ebola) epidemic in West Africa, U.S. and international public health decision-makers and stakeholders needed information early in the epidemic. This information included the number of Ebola cases that could be expected over time; the resources and personnel needed to respond adequately; and the impact of interventions, such as Ebola treatment units (ETUs), community care centers (CCCs), and safe burials. On August 4, 2014, CDC activated a Modeling* Task Force (CDC Modeling) and incorporated the task force into its incident management structure.

Models were used at the outset of the Ebola response in early September 2014 to estimate the impact of the epidemic with and without intervention. These models indicated not only that public health agencies had the means to stop the epidemic by using existing tools and strategies but also that the international community needed to act quickly with sufficient resources to stop the spread of the epidemic. The simple models (1–3) used by CDC Modeling enabled decisions to be made quickly during the response. This report summarizes 1) CDC Modeling’s role, accomplishments, and impact; 2) key issues, challenges, and lessons learned; and 3) suggestions for modeling in future responses.

Personnel conducting this assessment comprised both CDC Modeling staff and other CDC staff. All documents produced by CDC Modeling during August 4, 2014–July 13, 2015, were assessed: five publications, approximately 40 internal memoranda that included multiple versions documenting the modeling process, 1,000 technical consulting e-mails, and 30 presentations, as well as numerous meeting notes. In-depth after-action discussions with CDC Modeling staff aided understanding of how models were developed and used in the response, the difficulties encountered, and the impact of the models on decision making. Other assessments were the amount and type of data used (data requirements), how readily data were available, and the time available before response leadership needed preliminary results (turnaround time). Within the context of CDC’s Ebola response, “impact” referred to the use of models to provide information for the response and make decisions. Models associated with major projects that resulted in a publication, written report, or internal memorandum were categorized into 1 of 3 phases of the West African epidemic curve: 1) start and incidence acceleration, 2) peak and incidence deceleration, and 3) final phase and extinguishing (Figure). Eight reviewed projects resulted in either a publication or an internal, predecisional memorandum.

CDC Role, Accomplishments, and Impact

Start and Acceleration

Initial modeling questions concerned resource needs and thus predicted the number of Ebola cases that could be expected over time with and without isolation, treatment, and safe burials. These estimates enabled CDC Modeling to evaluate the impact that interventions, such as ETUs, CCCs, and safe burials, could have on the epidemic. On the basis of these questions and the knowledge available at the time about virus characteristics and transmission, CDC Modeling developed a simple Microsoft Excel spreadsheet–based model called EbolaResponse (4). This model included input values that could be easily changed (e.g., the number of Ebola patients placed in ETUs) and thus, could estimate potential outcomes if the disease remained unchecked and assess the relative impact of interventions. Because data from the field were limited, CDC Modeling calculated a correction factor for underreporting (4). In late August 2014, the correction for underreporting was estimated to be approximately 2.5; in other words, the true case count was 2.5 times greater than the reported case count.

By using data available through late August 2014, CDC Modeling estimated that 550,000 total reported Ebola cases could occur in Liberia and Sierra Leone (1.4 million when corrected for underreporting) by January 20, 2015, if no additional interventions or behavior changes occurred and if current parameters continued without change (4). Conversely, CDC Modeling predicted that transmission would decline substantially by mid-January 2015 if approximately 70% of Ebola patients were placed into ETUs or CCCs (or an equivalent) and if safe burials were conducted when needed. If this 70% goal could be reached, it would “bend the curve,” causing transmission to drop off substantially. If a large-scale response was delayed, the projected number of cases at the epidemic’s peak most likely would more than double and thus require more resources to control (4).

Initial estimates from EbolaResponse were published in September 2014 (4), with scenarios predicting that each month of delayed response would cause approximately 3.1 times more cases. Perhaps the most important message contained in the report was that public health agencies and the international community needed to act quickly with sufficient resources to stop the epidemic. On the basis of this information and other factors, including the United Nations Ebola virus disease outbreak overview of needs and requirements document (5), CDC leadership and U.S. government officials recommended a rapid increase in aid for the Ebola response. International donors provided approximately U.S. $154.6 million to support Ebola response activities in West Africa, including approximately $71 million from the United States (5). By March 2015, 10 countries had supplied approximately U.S. $2.2 billion in aid, with the United States providing approximately $1.05 billion of that amount (6). Later analyses demonstrated how the increase in resources helped to ensure that the actual number of cases was far less than if prompt action had not been taken (7).

In another modeling project, CDC Modeling analyzed the regional spread of Ebola in West Africa. CDC Modeling used geographic information system software and various regression models to identify factors that could be used to calculate the probability of individual areas becoming affected next (8) and helped to provide data to decision-makers about allocating resources for surveillance, especially in the countries surrounding Guinea, Liberia, and Sierra Leone.

CDC also needed data on the staff, equipment, and materials required to operate a typical ETU. CDC Modeling helped conduct a cost analysis to determine the budget needed to start up and run up to 1,000 ETU beds for 6 months, using the cost per bed from an interagency partner and publicly available data. CDC used this information internally to help guide resource allocation decisions.

Another study estimated the impact of ETUs and CCCs on Ebola transmission (7). Results suggested that during September 23–October 31, 2014, hospitalizing approximately 20% of all Ebola patients in ETUs prevented an estimated 2,244 cases. In addition, placing 35% of patients in CCCs or equivalent community settings that prevent transmission through reduced contact with patients, coupled with the use of safe burials, prevented an estimated 4,487 cases. Together, these interventions prevented an estimated 9,097 cases (7). The findings of this analysis provided evidence that interventions were working.

In September 2014, CDC Modeling began producing weekly predecisional memoranda for internal CDC use only, which provided estimates of the number of active cases (e.g., persons with Ebola and in need of a bed, either in an ETU or CCC) based on updated case counts from the field. These weekly updates provided senior leadership with situational awareness about the epidemic as it evolved. As data and case reporting improved through November 2014–March 2015, the need for projections to support decision making declined. However, projections indicated that the situation could change quickly and bolstered the need for public health agencies to avoid becoming complacent (4) (CDC, unpublished data, 2014).

Peak and Deceleration

As the epidemic progressed, public health officials developed plans to increase ETU capacities and developed methods to isolate patients in non-ETU settings to disrupt Ebola transmission (5,6). Senior U.S. leadership authorized personnel, funds, and supplies to help control the Ebola epidemic (5,6). Various philanthropic organizations and the U.S. Public Health Service agreed to operate ETUs and CCCs built by the U.S. military and others, and funded by the U.S. government. The governments of the United Kingdom, France, China, and other countries also helped to build and support treatment units that were run by international nongovernmental organizations and agencies (9). Once increased resources were allocated, questions arose about stocking and staffing the ETUs and preventing Ebola’s spread to patients within ETUs whose illnesses had been misclassified as Ebola. In one analysis, modelers considered ways to prevent Ebola’s spread to febrile persons with malaria whose illness had been misclassified as Ebola. Modelers analyzed the feasibility of treating all contacts of Ebola patients for malaria to prevent the onset of febrile malaria and subsequent admission to ETUs. If implemented, the intervention could avert admissions even for low levels of treatment compliance (10). Only a few clinics used this strategy. Although these strategies were implemented on a small scale, they might be useful on a wider scale for future responses.

CDC Modeling also contributed to the domestic U.S. response when the first imported case of Ebola prompted CDC and health departments to collaborate to improve hospital preparedness. State and local public health planners needed to know where in the United States travelers from West Africa were most likely to arrive, where they might seek treatment, and whether the United States had enough facilities designated to treat patients with Ebola. In a research letter, CDC Modeling and the U.S. Department of Health and Human Services’ Biomedical Advanced Research and Development Authority estimated the rate of new Ebola cases expected in the United States based on three categories of persons arriving from Liberia, Sierra Leone, and Guinea: 1) travelers who were not health care workers, 2) health care workers, and 3) medical evacuees (11). The rate of new infections in the United States was multiplied by treatment length to determine the number of Ebola patients expected to need treatment at any given time while the epidemic in West Africa continued. According to this analysis, the capacity of Ebola treatment centers in the United States (49 hospitals with 71 total beds) was sufficient to care for the model’s highest estimated number of patients with Ebola, with a large reserve capacity if epidemic conditions worsened (11).

Final Phase and Extinguishing

By May 2015, transmission of Ebola in West Africa had diminished substantially, and the response focused on eliminating transmission (12). Ebola virus was cultured from the semen of male Ebola survivors several months after clinical illness. Therefore, sex partners of these Ebola survivors could be infected but unaware of their infection while the illness is incubating. CDC Modeling was asked to project how often a person who acquired Ebola through sexual transmission and in whom the illness is incubating might arrive from West Africa into the United States (13). Using data from May and June 2015, modelers estimated that the projected frequency of a person traveling from West Africa who has acquired Ebola through sexual transmission and whose Ebola is incubating to be one traveler every 2.75 to 8.3 years (CDC, unpublished data, 2015). These estimates were specific for May and June 2015. As long as a resurgence of Ebola does not occur, the risk of importation will decline over time as the number of survivors capable of transmitting Ebola declines.

CDC Modeling produced a predecisional memorandum for internal CDC use that provided estimates of the potential impact of rapid Ebola diagnostic tests, specifically, the ability to rapidly test a patient with fever or other symptoms possibly indicative of Ebola. Although nucleic acid tests are more accurate, they require well-established laboratories and fully trained personnel. Rapid diagnostic tests produce quick results, are simple to perform, and do not require electricity, which is an important consideration in remote areas. Decision-makers needed to know the best possible strategies for using rapid diagnostic tests and how decreasing or low prevalence of Ebola might affect potential strategies for using these tests. The model results suggested that using rapid tests during low-prevalence periods most likely would require a second sequential confirmatory test at the treatment center to decrease the false-positive test results (CDC, unpublished data, 2015). This modeling provided evidence to support the use of rapid Ebola tests in low-prevalence settings as an effective screening tool to rule out Ebola infection (<1% false negatives), enabling patients with Ebola-like symptoms, but with negative rapid test results, to be treated outside of Ebola isolation units. In addition, this modeling predicted the number of false-positive (and true negative) rapid test results that could be expected at various disease prevalence levels in the community (CDC, unpublished data, 2015).

Key Challenges and Lessons Learned

Throughout the response, relevant data were not always available, and available data frequently contained inconsistencies that took time and effort to resolve. In addition, reporting delays made the incidence of Ebola difficult to accurately calculate, a crucial input in the models produced by CDC Modeling. Expert opinion was needed when data were not available (e.g., when data from the field were limited and a correction factor was used to estimate the actual number of cases). Even when adequate data existed, because no data sharing agreements had been developed and executed, questions arose about who owned the data and who could use them for analysis. As a result, some modeling projects were delayed.

The urgency of the Ebola response required a short turnaround time for projects. CDC Modeling typically was given ≤1 week to answer questions. To reduce errors and ensure the reproducibility and accuracy of the estimates in this short time frame, the CDC Modeling had two teams that cross-checked each other’s calculations, and other modelers at CDC reviewed the calculations during the clearance process. Each team had to clearly document its modeling methods so that other modelers could replicate the model. Therefore, for each model, CDC Modeling provided a technical appendix accessible to scientists within and outside of CDC.

Models and findings needed to be shared with the public, technical experts, responders, and other stakeholders. However, communicating about modeling is difficult when persons unfamiliar with modeling have difficulty interpreting what the estimates mean and understanding the nature of assumptions, uncertainty, and context. For example, if one assumption is disputed, nonmodelers might perceive this dispute as a reason to dismiss the entire model, rather than understand that defining assumptions and improving inputs is part of model development. Publishing manuscripts in scientific journals was, by itself, insufficient for communication. The immediate and primary purpose of the models was to provide information for decision-making, whereas publishing articles about the work provides information for future emergency responses.

Modeling in Future Responses

The benefits of incorporating modeling into major emergency responses were clear in the 2014–2016 Ebola epidemic response (14). Models provided critical decision-making tools in real time and helped demonstrate to public health authorities that the epidemic could be stopped by using existing tools and strategies (4). Although initial model estimates of Ebola represented a worst-case scenario, the international community responded to ensure that these dire predictions would not be realized (14). The following comments were made regarding the accuracy of CDC models that forecasted the trajectory of the epidemic: “the model predicted that when the tipping point was reached, transmission would decline rapidly. This prediction was shown to be accurate in the following months in Liberia and Sierra Leone. . . . The predictions also closely matched the actual case trajectory after effective intervention” (14).

In future emergency responses, modeling can be improved in several ways. First, flexibility is needed to enable data collection to focus on data needs relative to the size of the epidemic and to collect the types of data modelers can use to produce improved, more accurate models (e.g., number of cases that can be expected over time based on available knowledge about pathogen characteristics and transmission, as well as the impact that interventions could have on the epidemic). For example, in a large-scale epidemic, data collection might focus on small amounts of very specific data from sentinel surveillance (1). Second, data-sharing agreements should be in place before an event. Finally, promoting ongoing dialogue will ensure that scientists and audiences understand data limitations and what can and cannot be reliably concluded from models.

Modeling is an important but underused public health tool. Prioritizing modeling in future responses will take hard work, commitment, education, and an openness in public health to new disciplines and approaches.