BRFSS Data Quality, Validity, and Reliability

BRFSS Data Quality and National Estimates

There have been numerous studies that have examined issues related to the reliability and validity of the BRFSS and the system's ability to provide both valid national estimates, within state estimates and comparisons across states (see bibliography). While many of these studies look at particular topic areas, the annotated list of publications below provides information across topics and illustrates the future of the BRFSS as a reliable and valid source of information on health related issues:

1. Pierannunzi C, Hu S, Balluz L. A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011 2013, 13:49
   
   In order to assess the reliability and validity of prevalence estimates taken from the BRFSS, scholarship published from 2004–2011 dealing with tests of reliability and validity of BRFSS measures was compiled and presented by topics of health risk behavior. Assessments of the quality of each publication were undertaken using a categorical rubric. Higher rankings were achieved by authors who conducted reliability tests using repeated test/retest measures, or who conducted tests using multiple samples. A similar rubric was used to rank validity assessments. Validity tests which compared the BRFSS to physical measures were ranked higher than those comparing the BRFSS to other self-reported data. Literature which undertook more sophisticated statistical comparisons was also ranked higher. Overall findings indicated that BRFSS prevalence rates were comparable to other national surveys which rely on self-reports, although specific differences are noted for some categories of response. BRFSS prevalence rates were less similar to surveys which utilize physical measures in addition to self-reported data. There is very little research on reliability and validity for some health topics, but a great deal of information supporting the validity of the BRFSS data for others.
2. Hu SS, Pierannunzi C, Balluz L. Integrating a multimode design into a national random-digit–dialed telephone survey. Prev Chronic Dis 2011; 8(6):A145.
   
   This study examines how the BRFSS is integrating multiple modes of data collection to enhance validity. The research includes information on the incorporation of cellular telephones in parallel with ongoing, monthly landline telephone BRFSS data collection, and mail follow-up surveys. In addition the article describes how a pilot study in which respondents’ physical measurements are taken was conducted to assess the feasibility of collecting these data for a subsample of adults in 2 states. This article provides an overview of these new modes of data collection.
3. Fahimi M, Link M, Schwartz DA, Levy P, Mokdad A. Tracking chronic disease and risk behavior prevalence as survey participation declines: statistics from the Behavioral Risk Factor Surveillance System and other national surveys. Prev Chronic Dis 2008; 5(3).
   
   To assess the impact of trends in response rates on the quality of BRFSS estimates these authors compared selected health and risk factor estimates from BRFSS with similar estimates from the National Health Interview Survey (NHIS) and the National Health and Nutrition Examination Survey (NHANES). The article compares weighted estimates for outcome measures, including overall measures and measures for 12 population subgroups. These surveys produced similar estimates for several outcome measures, although significant differences were observed as well. Many of the observed differences were found to have limited consequences for implementing related public health programs; other differences may require more detailed examination. In general, the range of BRFSS estimates examined here tends to parallel those from NHIS and NHANES, both of which have higher rates of participation.
4. Li C, Balluz LS, Ford ES, Okoro CA, Zhao G, Pierannunzi, C. A Comparison of Prevalence Estimates for Selected Health Indicators and Chronic Diseases or Conditions from the Behavioral Risk Factor Surveillance System, the National Health Interview Survey, and the National Health and Nutrition Examination Survey, 2007-2008. Preventive Medicine. 2012 Jun; 54(6):381-7.
   
   This article compares the prevalence estimates of selected health indicators and chronic diseases or conditions among three national health surveys (BRFSS, NHANES, and NHIS) in the United States. The authors find that the prevalence estimates of current smoking, obesity, hypertension, and no health insurance were similar across the three surveys, with absolute differences ranging from 0.7% to 3.9% (relative differences: 2.3% to 20.2%). The prevalence estimate of poor or fair health from BRFSS was similar to that from NHANES, but higher than that from NHIS. The prevalence estimates of diabetes, coronary heart disease, and stroke were similar across the three surveys, with absolute differences ranging from 0.0% to 0.8% (relative differences: 0.2% to 17.1%). While the BRFSS continues to provide invaluable health information at state and local level, it is reassuring to observe consistency in the prevalence estimates of key health indicators of similar caliber between BRFSS and other national surveys
5. Nelson DE, Powell-Griner E, Town M, Kovar MG. A comparison of national estimates from the National Health Interview Survey and the Behavioral Risk Factor Surveillance System. American Journal of Public Health, 2003; 93:1335–1341.
   
   The study compares national estimates from the National Health Interview Survey (NHIS) and the BRFSS. The authors compared data from the two surveys on smoking, height, weight, BMI, diabetes, hypertension, immunization, lack of insurance coverage, cost as a barrier to medical care, and health status. Overall the national estimates were similar for 13 of the 14 measures examined. Small differences according to demographic characteristics were found for height and BMI, with larger differences for health status. These conclude that although estimates differed within subgroups, the BRFSS provided national estimates comparable to those of the NHIS and that BRFSS national data could provide rapidly available information to guide national policy and program decisions.
6. Mokdad AH, Stroup DF, Giles WH. Public health surveillance for behavioral risk factors in a changing environment: recommendations from the Behavioral Risk Factor Surveillance team. MMWR, 2003; 52 (RR-9):1–12.
   
   The article provides an overview of the BRFSS and describes the challenges for BRFSS in effectively managing an increasingly complex surveillance system that serves the needs of numerous programs while facing changing telecommunication technology and the greater demand for more local-level data. To determine options and recommendations for how best to meet BRFSS future challenges, a 2-day strategy workshop was held and attended by survey research specialists. The workshop featured presentations on the current system; emerging technologic challenges; telephone surveying techniques; program perspectives of CDC, partner organizations, and states; and recommendations for change. The report summarizes the recommendations resulting from that workshop.

For Other References on BRFSS-Related Reliability and Validity Studies Click Here.