1012 — Recommendations from HSR&D's Panel on Statistics and Analytics on VHA Datasets

Kashner TM, Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC; Chen GJ, The University of Kansas School of Medicine, Kansas City, KS; Golden RM, University of Texas at Dallas, Richardson, TX; Henderson WG, Colorado School of Public Health, Aurora, CO; Henley SS, Martingale Research Corporation, Plano, TX; Maciejewski ML, Durham VA Medical Center, Durham, NC; Meterko M, Boston VA Medical Center, Boston, MA; Olsen MK, Durham VA Medical Center, Durham, NC; Vijan S, VA Ann Arbor Healthcare System, Ann Arbor, MI; Vogel WB, North Florida/South Georgia Veterans Health System, Gainesville, FL

Objectives:
We announce the findings, guidelines, and recommendations representing the work of the expert Panel on Statistics and Analytics on VHA Dataset, as sponsored by HSRandD (SDR#13-426). The purpose for the Panel was to assess ways to improve, if possible, the quality of analytic methods that investigators propose in HSRandD merit reviewed applications. Areas are biostatistics, including computational and mathematical statistics, econometrics, epidemiology, informatics, psychometrics, and qualitative research.

Methods:
To assess availability, expertise, and use of methodologists in preparing HSRandD applications, we administered in 2014 a web-based survey to all PIs who had submitted an HSRandD application between October 1, 2012 through December 31, 2013, and to all research administrators (ACOS-RD, Administrative Officers, COIN Directors) representing VA medical centers who support an Office of Research. To provide guidelines for investigators when preparing methods sections to HSRandD applications, and assist reviewers in assessing those applications, the Panel conducted literature reviews and identified specific analytic and statistical challenges (exclusive of qualitative research) that commonly face health services researchers in VA. The Panel also recommended acceptable and state-of-the-art strategies to address those challenges. Finally, a simple simulation study was conducted to serve as an example of how statistical performance can vary between acceptable and state-of-the-art methods.

Results:
96% of applications involved at least one methodologist, with 70% involving biostatisticians, 66% qualitative researchers, 40% econometricians, 40% informaticians, 30% epidemiologists, and 26% psychometricians. Both availability and expertise varied widely across COIN and nonCOIN medical centers, with 40% relying on VA affiliated universities for biostatistical support. Finally, 20% of administrators reported that limited availability or expertise of a methodologist was a primary and/or secondary cause of why a worthy application was not submitted to HSRandD for consideration. Guidelines were prepared describing challenges and recommending acceptable and state-of-the-art strategies covering 12 critical topics in methodology.

Implications:
To uphold recommended guidelines, additional resources for methodologists may be necessary to ensure VA maintains its leadership role in health services research.

Impacts:
We discuss the seven recommendations the Panel made to HSRandD to improve access to, and use of, qualified methodologists when investigators prepare applications for submission to HSRandD.