Evaluation of the Colorectal Cancer Control Program (DP15-1502): Using Program Data to Measure Health Systems Change

Colorectal Cancer Control Program Evaluation: Challenges and Lessons Learned

Authors:

Krishna Sharma (Presenter)
Centers for Disease Control and Prevention

Kristy Kenney, Centers for Disease Control and Prevention
Amy DeGroff, Centers for Disease Control and Prevention
Janet Royalty, Centers for Disease Control and Prevention
Justin Uhd, Centers for Disease Control and Prevention

Public Health Statement: Colorectal cancer (CRC) is the second leading cause of cancer deaths among cancers that affect both men and women, yet screening rates remain low, particularly among underserved populations.

Purpose: The Colorectal Cancer Control Program includes 30 grantees funded by the Centers for Disease Control and Prevention (CDC) to increase CRC screening rates in partner health systems through implementation of evidence-based interventions (EBIs) and supporting activities (SAs).

Methods/Approach: The primary outcome for the program is clinic-level CRC screening rates. Grantees report data to CDC from all participating clinics and complete an annual survey. This presentation highlights challenges and lessons learned from the first year of the evaluation.

Results: Grantees reported extracting data from electronic health record systems (47%) and collecting reliable data on screening rates (50%) as a moderate or serious challenge. A majority of grantees reported a medium-to-high level need for training on collecting, managing, and analyzing data (60%) and evaluating EBIs and SAs (73%). Grantees identified measuring process implementation and performance (e.g., quality of partnerships) as a challenge. Evaluators also found it challenging to isolate the impact of the program on clinic screening rates.

Conclusions/Implications: Close collaboration between CDC and grantees is fundamental to successful evaluation efforts. CDC hosted grantee forums and facilitated candid communications and knowledge sharing amongst grantees. Feedback from grantees helped CDC develop appropriate data collection tools and understand contextual issues that affect the interpretation of data. Evaluation and baseline data collection tools help track implementation progress and outcomes early in the process.

Using Clinic-Level Data to Assess Impact of Health Systems Change in the Colorectal Cancer Control Program

Authors:

Kristy Kenney (Presenter)
Centers for Disease Control and Prevention

Anamika Satsangi, Centers for Disease Control and Prevention
Amy DeGroff, Centers for Disease Control and Prevention
Janet Royalty, Centers for Disease Control and Prevention

Public Health Statement: CDC’s Colorectal Cancer Control Program (CRCCP) funds 30 grantees to implement evidence-based interventions (EBIs) within primary care clinics to increase colorectal cancer (CRC) screening rates. CDC designed an evaluation to measure clinic-level changes in CRC screening rates, the outcome of interest.

Purpose: To develop a standardized evaluation approach to allow for comparisons across grantees.

Methods/Approach: To standardize data collection and reporting, CDC developed a data dictionary, a guidance document on how to measure screening rates, data collection forms with validation features, and a web-based data reporting system. Grantees report clinic-level data including a baseline record and an annual record, thereafter. Data are descriptive (e.g., number of patients, type of clinic) and longitudinal (e.g., screening rate, EBIs implemented, monitoring and quality improvement activities).

Results: In program year 1 (PY1), grantees recruited 420 clinics serving 718,850 patients ages 50–75. Over 70% of clinics are federally qualified health centers (FQHCs).The median CRC baseline screening rate is 31%. Nearly all clinics are receiving either weekly or monthly support from grantees to implement EBIs and improve data quality. Preliminary analysis indicate an increase in the median screening rate across clinics.

Conclusions/Implications: Conducting high quality, systematic evaluations of Federal programs is difficult when hundreds of implementation sites are involved. Standardized data collection and reporting systems facilitate quality assurance and improvement. Early results from the CRCCP suggest that public health programs can effectively integrate with primary care to improve outcomes. Program reach is significant, and, as reflected in the low baseline screening rates, grantees have partnered with high need clinics.

Methods for Conducting Cost Effectiveness Analyses of CDC’s Colorectal Cancer Control Program

Authors:

Florence Tangka (Presenter)
Centers for Disease Control and Prevention

Sujha Subramanian, RTI International
Sonja Hoover, RTI International

Public Health Statement: Health centers and health departments spend significant amounts of money screening people for colorectal cancer. However, there is little quantitative information on the cost-effectiveness of the activities to promote screenings.

Purpose: The objective of this on-going study is to assess the cost-effectiveness of the colorectal cancer promotion strategies used by CDC’s Colorectal Cancer Control Program (CRCCP) to increase screening uptake, and to improve diagnostic resolution and timeliness of treatment for adults aged 50–75 years.

Methods/Approach: Using a customized data collection tool for selected CRCCP grantees, we will collect information on program expenditures, quantities of labor and non-labor resources; and proportions screened, at baseline and during follow-up periods of 12–18 months. We will use the collected information to evaluate the cost-effectiveness of program screening promotion activities and determine the cost per additional person screened. We will also use a previously validated microsimulation model to estimate the cost per quality adjusted life years for the different screening promotion interventions evaluated.

Results: We anticipate that costs and effectiveness of screening promotion strategies will vary across sites. In order to better understand why this may occur, we will speak with grantees and health center representatives and staff in person and via telephone. We will learn about their screening promotion strategies and processes as well as additional details about the populations they serve. For many of the sites, we will assist local staff to collect expenditures, resource use and time data. When we are not able to assist in-person, we will provide training over the phone to collect the study information, and we will be available for consultations.

Conclusions/Implications: We anticipate that similar strategies will be employed across different grantees to promote and increase colorectal cancer screening uptake, but the cost of the strategies and changes in screening uptake will differ because of multiple factors, such as the electronic reporting system, staff training and education, and local barriers to implementations.

Evaluating Program Management and Implementation in CDC’s Colorectal Cancer Control Program

Authors:

Amy DeGroff (Presenter)
Centers for Disease Control and Prevention/p>

Justin Uhd, Centers for Disease Control and Prevention
Stephanie Melillo, Centers for Disease Control and Prevention
Lindsay Gressard, Centers for Disease Control and Prevention

Public Health Statement: Evaluation of implementation and management of the Colorectal Cancer Control Program (CRCCP) can help to improve grantee programs and increase accountability.

Purpose: The CRCCP requires grantees to implement evidence-based interventions (EBIs) and supporting activities to increase colorectal cancer screening rates in health systems and clinics. Evaluation of grantees’ program design, implementation, and management can identify important factors contributing to program successes or challenges. This presentation will describe these evaluation efforts and present findings.

Methods/Approach: We designed an annual Grantee Survey to assess program management, implementation activities, use of health IT, non-health system partnerships, data use, and training and technical assistance needs. We also conducted analysis of grantees’ approved Program Year 2 budgets to determine allocation of resources, staffing patterns, and contracted partners and their activities.

Results: Grantees predominately allocated funding to Contracts/Consultants and Personnel Costs budget categories (50% and 32%, respectively). Among Contracts/Consultants, 45% are with Health Care Systems and Clinics. Contracted partners are most often budgeted to conduct Evaluation and Performance Management and support EBI implementation. Grantees budget for an average of seven personnel comprising 2.7 full time equivalents, mostly in Program Management, Evaluation, and Leadership positions (73%). Five grantees obtained additional funding to support their programs. Grantees partner with various non-health system organizations, both funded and unfunded, including the American Cancer Society and state primary care associations.

Conclusions/Implications: While grantees are implementing and managing their programs in various ways, they are consistently utilizing partnerships, evaluation support, and program managers and leaders to implement their programs.