Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

Randomized Program Evaluations

The purpose of the Randomized Program Evaluation Service Directed Research (SDR) projects is to support innovative research utilizing randomized designs to study the impact of new policies and programs planned for wide-spread implementation within VA on utilization, patient outcomes, and Veteran experience. The U.S. Department of Veterans Affairs (VA) is undergoing a major reorganization to transform the way it provides services for its Veterans nationally. This transformation has already involved new policies such as the Veteran's Choice Act which involves a dramatic shift in the delivery of VA health care services and evolution of the Veterans Health Administration (VHA)'s role as a payer in addition to provider of health care for millions of Veterans nationwide. Moreover, the reorganization under MyVA includes new initiatives designed to emphasize Veteran engagement, shared VA services (e.g., combining health care, benefits, and customer service), and Veteran centered care. This follows a period of other major national initiatives within VHA, including the national implementation of the Patient Aligned Care Teams (PACT), implementation of new telehealth and connected health programs, promotion of patient-centered care, and adoption of several specialty-care transformational initiatives, among others. Assessment of the impact of these programs is still ongoing, and in some cases is complicated by the lack of a strong, contemporaneous comparison group. HSR&D's randomized program evaluation service-directed research projects involve the systematic, but purposefully random, allocation of sites that will undergo a new practice, policy, or program. A key advantage to randomization is that it provides the best opportunity to detect a true effect of a new program, since it reduces the potential influence of secular trends or systematic variation across sites, patients, and providers (for example, the possibility that sites that adopt a new program first are more committed or more experienced in delivering it - or conversely, motivated to do so by poor baseline performance). Identifying the true effect of a new program or policy can prevent wasted effort and expense on ineffective rollouts, and ultimately produce greater return on the resources invested. Randomization also allows an equitable opportunity for patients or their providers to receive a new program where it isn't possible to deploy the program or policy to all sites simultaneously. These advantages led the Office of Management and Budget to strongly support randomized program evaluations through its Improve Government Performance Policy and more recently, the passage of the Evidence-Based Policymaking Commission Act to support the U.S. Government in establishing standards for national administrative/survey data and analyses methods to conduct randomized program evaluations of new policies. This approach to data-driven decision-making has been referred to as "evidence-based policy" and reflects a recent Institute of Medicine (IOM) update on Learning Health Care Systems which advocates better integration of randomized designs to evaluate new or augmented clinical practices or policies in health care systems. Within the VHA, such initiatives are institutionally sanctioned by leadership and evaluated with the purpose of determining whether they were implemented as intended, with an eye towards assessing effectiveness, impact, and ultimately, sustainability. Such an evaluation asks not only "does the program work?" but also "where does it work? What makes it work? How can we make it work in the real world?" This portfolio includes program evaluation topics recently solicited from VHA operational and program leaders and reviewed by HSR&D and QUERI and further evaluated for suitability for a randomized evaluation by the Partnered Evidence-based Policy Resource Center (PEPReC). The goal of PEPReC is to support the development of high-priority policy, planning, and management initiatives and quantitative program evaluations to improve the quality and efficiency of VA health care. Selected randomized program evaluation topics were evaluated for the scientific support for the program, availability and validity of relevant data, feasibility of randomization, likelihood of demonstrating a measureable effect within a 12-18 month period, and clinical/policy importance and include: 1.Veteran-Directed Home & Community Based Services, with the Office of Geriatrics and Extended Care 2.Predictive Model-Based Targeted Risk Mitigation For Patients Receiving VA Opioid Prescriptions Who Are At High Risk Of Adverse Events, with the Office of Mental Health Operations 3.Risk Stratified Enhancements To Clinical Care: Targeting Care For Patients Identified Through Predictive Modeling As Being At High Risk For Suicide, with the Office of Mental Health Operations 4.Impact Of Mobile Teledermatology On Skin Care Delivery And Patient Outcomes, with the Office Of Connected Care

For general information about Randomized Program Evaluations, please contact:

Stephen E. Marcus, PhD, MPH
Scientific Portfolio Manager
Health Services Research and Development (10X2H)
Email: stephen.marcus@va.gov


Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.