The focus of National Syndromic Surveillance Program in coming years will be on:

• Improving data
  • access,
  • quality,
  • representativeness and
  • timeliness;
• Increasing the capabilities and technology supporting:
  • syndromic surveillance data collection,
  • processing, and
  • provisioning activities;
• And finally, working with the syndromic surveillance community to further the science and practice of syndromic surveillance.

If there are collaborating state or local health departments (e.g., metropolitan statistical areas [MSAs], groups of states, multiple counties, etc.) representing an area with a total combined population equal to or greater than one million people, as estimated by the U.S. Census 2013 estimate, a single entity and application representing the collaboration of entities seeking funding should be submitted with Memorandums of Understanding (MOUs) or Memorandums of Agreements (MOAs) from partner organizations included with the application.

The BioSense Investment Review and the BioSense Governance Group recommended increasing the representativeness of the data in order to represent or describe regional and national population health trends.  CDC has determined that requiring a population catchment area of at least 1 million people, encouraging more strategic onboarding of hospitals, and increasing funding for individual awards by reducing the total number of awards, will facilitate efforts to make the data more representative. Determination of awards will include consideration of geographic region of the applicant so there is better national representation. CDC encourages joint applications, especially in regions of the US that are not well represented, such as the West Coast.

The highest priority is to obtain data from the traditional data sources, such as emergency departments, urgent care centers, and hospital admissions.  It is also important to capture data that is representative of the target population. Other data might be helpful but should be considered strategically given limited resources.

All applications will be reviewed and eligibility will be determined based upon the criteria outlined in the Funding Opportunity Announcement.  It is critically important for responses to CDC-RFA-OE15-1502 to adhere to the eligibility information outlined in the Funding Opportunity Announcement.

Yes, CDC will continue to provide support for onboarding activities.

Both the awardee and CDC are responsible for data quality. CDC will join with awardees to assess data quality reports that could include measures stated in the Funding Opportunity Announcement, among others. CDC is developing operational definitions of the measures.

Both CDC and the jurisdiction are responsible for this activity. CDC will produce and disseminate data quality reports for selected measures. Jurisdictions provide guidance on additions and modifications that add actionable information to the reports. The reports will be tools that enable jurisdictions to identify data-quality problems and take action to resolve them. CDC will provide more guidance about this after the National Syndromic Surveillance Program launches.

No.  Continued active engagement by the jurisdiction is the foundational goal and expectation for this activity. It is critical for CDC and jurisdictions to work together to identify evolving data quality problems, collaboratively define them, and implement solutions.

Activity 2.b) Conduct data quality and timeliness assessments for data submitted to the National Syndromic Surveillance Program Platform.

Measure:
Quarterly data quality and timeliness reports will include at a minimum:

1. Proportion of records with missing fields in data feeds [quantitative indicator]
2. Proportion of records with invalid values in data feeds [quantitative indicator]
3. Proportion of emergency department visit data that meet quality criteria defined by the National Syndromic Surveillance Community of Practice core group [quantitative indicator]
4.  Proportion of records available in the front end application of the National Syndromic Surveillance Program Platform within less than 48 hours after patient admission to the emergency department [quantitative indicator]

Measure #1: Proportion of records with missing fields in data feeds
This measure will focus on critical data elements that data processing or key analytical processes are dependent upon in order to deliver successful and presumably valid standard results to the end user.   Initial reports for Measure #1 will focus on the following critical data elements:  Patient ID, Facility ID, Event Date

Measure #2: Proportion of records with invalid values in data feeds [quantitative indicator]
Data Quality Category: Validity
This measure will focus on critical data elements that contain unusual values or values that are not reconciled against an associated "master references file" (e.g., jurisdiction’s master facility table; ZIP code master file; element-specific standard value set).  Initial reports for Measure #2 will focus on the following critical data elements:  Facility ID, ZIP code

Measure #3: Proportion of ED visit data that meet quality criteria [quantitative indicator]
Data Quality Category: TBD; possibly covers the gamut of Completeness, Validity, Accuracy, Consistency, Uniqueness, Timeliness, and Representativeness  
CDC and the National Syndromic Surveillance Community of Practice will collaboratively develop criteria used to assess the quality of emergency department visit data. CDC will produce reports based on the National Syndromic Surveillance Community of Practice guidance.  As with other measures, both CDC and the jurisdiction have dual responsibilities for this activity.

Measure #4: Proportion of records available in the front end application of the National Syndromic Surveillance Program Platform within less than 48 hours after patient admission to the emergency department [quantitative indicator]
Data Quality Category: Timeliness

Data associated with a patient encounter “travels” through various stages and levels of processing.   Consider these “data stops” as if the data are traveling from one location to another, stopping at various points along the way until they arrive at their final destination.  Processing stages include the following, listed in order of Data Stops:

Jurisdiction Data Feeds to CDC

Data Stop #1: Patient encounter (visit date)
Data Stop #2: Message created
Data Stop #3: Message sent to CDC
Data Stop #4: Message arrives at CDC

CDC Processing of data

Data Stop #5: CDC processes message into raw level “stage 1” data
Data Stop #6: CDC processes “stage 1” data into “meaningful use” data for analytic use
Data Stop #7: “Meaningful Use” data collapsed and summarized by syndrome
Data Stop #8: Syndrome level data are pushed to end-user application/analytic tool

For the National Syndromic Surveillance Program Funding Opportunity Announcement, a DUA is a Data Use Agreement that is a formal agreement between an awardee and the Association for State and Territorial Health Officials (ASTHO), which hosts the National Syndromic Surveillance Program Platform cloud environment.  Awardees must complete individual DUAs with hospitals and other sources of emergency department data (health information exchanges, etc.) within their jurisdiction.  CDC is also developing a DUA that can be used to describe data sharing between CDC and the jurisdiction in more detail than does the ASTHO DUA.  Awardees will not be required to sign the CDC DUA but they may consider doing so.

Applicants that have existing DUAs with the cloud steward, ASTHO, are required to submit a copy of their DUA with their application for CDC-RFA-OE15-1502.  Applicants that do not have a current DUA with ASTHO are required to submit a letter signed by the health officer committing to signing the DUA within the first six months of funding.

If you, as an applicant, currently have a sub DUA with the state but are planning on responding directly to CDC-RFA-OE15-1502, you will need to submit a letter signed by your health officer committing to signing the DUA with ASTHO within the first six months of funding.

If the information within the DUA changes then a new DUA may need to be renegotiated.  You may want to share your questions with the national syndromic surveillance community to learn how other practitioners are handling this issue.

Awardees are not required to share their raw or patient-level data with CDC for surveillance purposes.  Keep in mind that in order to administer the system and provide basic data quality checks, CDC or its contractor will have to see data.  However, jurisdictions are encouraged to allow their data to be used for surveillance purposes in order to generate regional and national aggregated results using analytic tools on the platform.  In addition, CDC will collaborate with jurisdictions as needed for surveillance or quality assurance purposes such as reviewing records for message validation or accurate binning.

It is important to understand that CDC will have access to these data only under certain circumstances:  

• When it is necessary for CDC and its contractors to see data in order to onboard a facility and start their data flowing through the application.  This access occurs only for the purpose of getting the system working for the local jurisdiction and the access will occur in consultation and collaboration with the jurisdiction.  
• When CDC and its contractors must ensure a data quality baseline and this determination requires access to the data.  Evaluating data quality involves developing techniques for data quality assessment, problem resolution, and developing analytic data processes to satisfy syndromic surveillance needs (collapsing records into visits, syndrome binning, geospatial coding, etc.)
• CDC can have access to data for analytic or surveillance purposes only if the local jurisdiction’s administrator enables the application to share data with CDC.  The jurisdiction determines what level of data they want to share (state aggregate, county aggregate, or data details).  Also this is not necessarily a permanent decision.  A jurisdiction might determine it is important to share data details for a particular time period concerning a specific event response.  Afterwards, they could turn off sharing or adjust data access to a new level.  
  • When data are shared for surveillance and analytic purposes, CDC will use National Syndromic Surveillance Program data for detection of hazards that threaten public health and to characterize the event at the national, state, and local levels.   
  • CDC looks forward to the opportunity to work with the community to continue developing and evaluating statistical algorithms for anomaly detection.  As various data analytic processes are developed and tested, they will be implemented into standardized methodologies for using National Syndromic Surveillance Program data for outbreak event detection, characterization, and improved situation awareness at all public health levels.

When appropriate at the federal level, CDC data analysts will work with jurisdictions to:

• monitor and report syndromic surveillance results,
• report agreed upon pertinent information to the CDC Emergency Operations Center and to state and local health departments, and
• provide technical assistance to state and local health departments and to CDC programs in how best to use National Syndromic Surveillance Program data for agreed upon event- or disease-specific studies.

The National Syndromic Surveillance Community of Practice will encourage local use of syndromic surveillance, ensures appropriate data sharing, and supports a regional and national view of enhanced surveillance.  

The National Syndromic Surveillance Community of Practice will provide its members with many opportunities, such as: 

• sharing how noteworthy accomplishments were achieved and the lessons learned,
• engaging with syndromic surveillance subject matter experts (SMEs),
• receiving peer-to-peer mentoring and technical assistance,
• accessing a repository of tools, reports, discussions, and more, and
• sharing a professional identity.  

Furthermore, across organizational units the National Syndromic Surveillance Community of Practice promotes coordination, standardization, and synergies for efficient use of resources through collaborative efforts. Through the National Syndromic Surveillance Community of Practice and access to shared analytic tools and services in the National Syndromic Surveillance Program Platform, public health programs and practitioners can access aggregated analyses of disease patterns and share more detailed data when needed to identify, investigate, and monitor public health threats that cross municipal boundaries.

For the National Syndromic Surveillance Program cloud, the cloud steward is the Association of State and Territorial Health Officials (ASTHO).  They are responsible for managing the Data Use Agreement (DUA) each National Syndromic Surveillance Program applicant must sign in order to share syndromic surveillance data within the National Syndromic Surveillance Program Platform.

Cloud technology is one or more virtual servers accessible through the internet.  When put into practice, this technology shifts workload from local computers to a network of virtual computers. Local computers no longer have to do all the heavy lifting when it comes to running applications. The network of virtual computers that make up a cloud handles them instead. Hardware and software demands on the user's side decrease. The only thing the user's computer needs to be able to run is the cloud computing system's interface software, which can be as simple as a Web browser, and the cloud's network takes care of the rest.

Applicants may include in their budget a request for funding for a subcontractor.  The request for a subcontractor should include a brief statement explaining the need for the subcontractor and the qualifications of the subcontractor to meet the needs. CDC (Procurement and Grants Office [PGO] and program) reviews and approves/deny based on criteria from PGO.

Starting in 2014, CDC required all new Funding Opportunity Announcements to include basic performance and evaluation elements to enhance accountability and improve results:

• a program logic model that illustrates how required awardee activities support achievement of program goals,
• performance measures used to monitor awardee progress on funded activities, and
• development of an evaluation plan by awardees that will identify areas in need of process improvement (e.g., what factors hinder timely negotiation of a DUA in my jurisdiction?) and/or answer broader questions of achievement of results/impact (e.g., are the data useful for timely response and mitigation?)

Awardees must report quarterly on certain designated performance measures.   Other measures require reporting on a semi-annual or annual basis (e.g., participation in the annual grantee meeting). Through development of the evaluation plan, CDC will work with awardees to clearly define when to report results for relevant performance measures.

CDC will provide reporting templates for awardees to use for reporting on performance measures.  These templates will be used for quarterly discussions between CDC and awardees.

Yes. The plan should reflect key questions that will result in improvement and/or demonstrate results (outcome evaluation).  The performance measures required by the Funding Opportunity Announcement will be included in the plan.

No.  CDC will provide awardees with technical assistance to create and implement their evaluation plans.  A grantee may choose to use some of its award to secure the necessary skills to conduct process or outcome evaluations outlined in the approved plan.

Performance and evaluation data will be discussed on quarterly calls among the “team” (awardee, CDC project officer, and CDC evaluator) to help drive program improvement.  These discussions will help identify ways to address challenges and barriers to progress, including technical assistance needs.

The recruitment plan will identify which hospitals in an awardee’s jurisdiction will be asked to submit their emergency department data to the National Syndromic Surveillance Program Platform. The plan contains criteria that hospitals must meet to be considered as a desired partner. CDC will provide core criteria for selection of hospitals and provide technical assistance as needed to develop the recruitment plan.

A jurisdictional working group is composed of different stakeholders who can help improve syndromic surveillance practices at the jurisdictional level.  Ideally, if one doesn’t currently exist, the working group should be established within 6 months of the awardee receiving funding.

Yes, the work plan should be shared with the project officers during quarterly updates.

A success story is a narrative that describes one or more public health practice achievements resulting from using syndromic surveillance data.

• A “use case” is an instance where an awardee used the National Syndromic Surveillance Program Platform to analyze syndromic surveillance data, report a potential problem that could affect their population’s health, and possibly spur action by health department officials.
• A “success story” describes a use case that went one step further in that the detection of a key public health event through monitoring the National Syndromic Surveillance Program Platform led to a positive outcome, such as successful mitigation of an outbreak or interventions to prevent or reduce drug overdose, injuries, etc.

Examples of peer-to-peer mentoring include, but are not limited to:

• sharing information and analytic code by a state or local health department that has more experience with syndromic surveillance in order to better detect a disease outbreak or other event of public health concern,
• conducting data sharing workshops and collaborating with regional partners to address barriers to sharing, and
• sharing proven best practices that successfully address challenges in obtaining Data Use Agreements (DUAs) with key hospital facilities.

The applicant has the option to include the work plan in the project narrative and/or to prepare a more detailed work plan and upload it as a separate file.