Innovative methods are needed to bring guideline recommendations to health professionals to assist them in providing evidence-based clinical care for Veterans. We used hypertension as a model to study guideline implementation in primary care practice through an innovative clinical decision support (CDS) system: ATHENA-HTN (hypertension). This project was funded through the VISN Implementation Collaborative for HTN to support innovations in implementing evidence-based clinical practices through collaboration between VISN leadership and HSR&D investigators.
The project had 4 Specific Aims:
(1). To update the ATHENA-Hypertension (HTN) clinical decision support (CDS) system for evidence-based management of primary hypertension to new VA/DoD hypertension guidelines and to enhance the integration with the VA Computerized Patient Record System (CPRS).
(2). To implement ATHENA-HTN in primary care clinics in 5 medical centers (stations) in VISN 1 to present recommendations to clinicians for individual patient management at the time of the patients' primary care outpatient clinic visits. We aimed to install ATHENA-HTN at sites geographically remote from the development site to give a realistic assessment of the work involved in deploying the system.
(3). To evaluate the implementation in a clinical trial.
(4). To explore the organizational impact of the guideline implementation. From an organizational perspective, we conceptualized that the innovative program implementation required changes in roles, routines, and rules.
(1) ATHENA-HTN presents patient-specific evidence-based recommendation for HTN management to primary care providers (PCPs) at point-of-care. We updated the knowledge-base(KB) for ATHENA-HTN in consultation with hypertension and primary care experts, and tested it offline.
(2) We worked with VISN1 New England Healthcare Network staff to install ATHENA-HTN remotely on VISN1 VA servers and to provide linkages to the server from individual clinic computers. We included hospital-based clinics and community-based-outpatient-clinics (CBOCs), and physician (MD) PCPs and nurse-practitioner/physician-assistant (non-MD) PCPs. Since some sites had thin-client computing environments, we developed a thin-client version. We established nightly extracts of relevant VistA data for patients with scheduled primary care clinic visits and validated data extracts. We conducted daily pre-computes of recommendations, applying exclusion criteria for patients for whom standard recommendations do not apply, and established methods for on-the-fly updates including updated blood pressures (BPs) at the time of the visit. We evaluated PCPs' responses to ATHENA-HTN in a post-intervention survey that addressed and surveyed issues in following HTN guidelines.
(3) We evaluated the implementation in a clinical trial with randomization by substation (to obviate contamination with the same workgroups), stratified on station (medical center), PCPs' average years at VA, and presence/absence of a pharmacist in the primary-care-clinics. We analyzed patient data extracted from the electronic health record (VistA) to ascertain impact of the intervention on PCP prescribing. For patients who did not have a BP above target during the intervention period, future planned analyses will examine guideline-adherence of choice of antihypertensive therapy. For patients who did have a BP above target during a visit with their PCPs (an index visit), primary analysis focused on intensification of drug therapy for hypertension. We identified which of these events occurred first during the year after the index visit: intensification of therapy (Event1), return of patient's BP to below target (Event2), patient seen in clinic and BP either remained above target or was not measured (Event3, undesired event), or patient had neither a return primary care clinic visit nor a repeat BP (Event4, right-censoring). We compared mean proportions of events per PCP between study arms using mixed model regression adjusting for medical center, clinic type (CBOC or not), PCP discipline (MD vs non-MD), presence of pharmacist, and mean age of PCP's panel of patients as fixed effects, and substation as a random effect to account for clustering of PCPs. We fit an initial model including all 4 events and did not detect a difference between study arms for mean proportion of event 4; final findings are based a denominator of event types 1-3.
(4) We maintained an organizational database in which we recorded steps in system implementation and maintenance.
(1)The updated ATHENA-HTN performed well in offline testing and tester-physician use.
(2) We successfully deployed ATHENA-HTN for a training period at 5 sites in VISN1. One site withdrew after the training period; results are reported here for the 4 sites participating in the intervention phase. Participation at the 4 sites included 20 substations (10 in each study arm), with 76 PCPs (30 control arm, 46 intervention), of whom there were 38 each MDs and non-MD. All 46 PCPs allocated to active intervention received displays of ATHENA-CDS. 41 of 46 (89%) of PCPs responded to the post-intervention survey: Many PCPs reported that they found ATHENA-HTN to be Useful or Very Useful in reminding them: of management of hypertension 34/41(83%), of intensifying treatment when the patient's BP was above target 34/41(83%), and of considering thiazides as part of a multi-drug regimen 27/41(66%). They reported that Often or Very Often the information presented was useful 29/40(73%), and had an impact on their choice of treatment 18/39(46%). The system was rated as having excellent or good readability 30/41(73%) and ease of navigation 27/41(66%).
Regarding reasons for not following guideline recommendations in the survey, PCPs designated the following Often or Very Often: time constraints 23/41(56%), patient had poor medication adherence 22/41(54%), changing medication could adversely affect patient adherence 24/41(59%), a previous trial of the recommended medication was not effective or was not tolerated by the patient 18/39(46%), and patient comorbidities 21/40(53%).
(3). 17,436 patients met study eligibility criteria. Of these, 5334 (30.6%) had an index visit with BP above target; of whom 2534/5334 (47.5%) were within 5mmHg of target. Mean proportions of each event per PCP by study arm were Event1 25.4% control vs 30.6% intervention (p>0.1); Event2 59.6% vs 55.2.0% (p>0.1); Event3 16.2% vs 13.7% ( (p>0.1). A difference in mean proportions of Event1 between study arms was indicated for the MD subgroup only (25.6% control vs 35.5% intervention, F=4.14, p=0.072). The mean proportion of Event1 declined in those panels that averaged over 70 years in age (p=0.015). No significant association was detected between the remaining covariates and any of the outcomes (p>0.6).
(4). VA experienced 2 major organizational changes during early implementation with major impact on deployment of CDS: re-organization of VA Office of Information Technology (OI&T) and the introduction of new data-security policies. Organizational changes entailed rules (e.g., previously approved IT required many new layers of approval), and confusion about new roles (e.g., who is allowed to give permission for certain steps in deployment?). Organizational changes required extensive time for establishing new routines, lengthening the deployment phase from a few days to up to a year or more.
Results showed that it was feasible to encode complex clinical guidelines into computable formats in the ATHENA-HTN system and to implement the system remotely in VA medical centers geographically distant from system developers. ATHENA-CDS provides a generalizable (beyond HTN) method to display evidence-based recommendations, tailored to specific patients, to health professionals at the point-of-care. PCPs found the CDS system to be both usable and useful.
Proportions of BP control across both study arms were much higher than anticipated during drafting of the RFA, with almost 70% of patients having BP control at all clinic visits during the study period. Furthermore, among patients who did have a BP above target, almost 60% returned to a controlled BP without any intensification of drug therapy on average within PCP panels, suggesting that PCPs and their patients were using good judgment in deferring intensification. PCPs reported concerns about changing or intensifying drug therapy for many patients. We also found that intensification occurred less often when the mean age of the panel of patients was older; possibly because older patients are more likely to have comorbid conditions that make intensification of drug therapy riskier. These findings support the need for quality measures for HTN that take account of individual patient factors.
- Aquino Shluzas L, Cronkite RC, Chambers DA, Hoffman BB, Breeling J, Musen MA, Owens DK, Goldstein MK. Organizational factors affecting implementation of the ATHENA-Hypertension clinical decision support system during the VA’s nation-wide information technology restructuring: a case study. Health Systems. 2014 Nov 1; 3(3):214-234.
- Cucciare MA, Ketroser N, Wilbourne P, Midboe AM, Cronkite R, Berg-Smith SM, Chardos J. Teaching motivational interviewing to primary care staff in the Veterans Health Administration. Journal of general internal medicine. 2012 Aug 1; 27(8):953-61.
- Steinman MA, Goldstein MK. When tight blood pressure control is not for everyone: a new model for performance measurement in hypertension. Joint Commission Journal on Quality and Patient Safety. 2010 Apr 1; 36(4):164-72.
- Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, Scates N, Tu S, Goldstein MK. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain medicine (Malden, Mass.). 2010 Apr 1; 11(4):575-85.
- Martins SB, Lai S, Tu S, Shankar R, Hastings SN, Hoffman BB, Dipilla N, Goldstein MK. Offline testing of the ATHENA Hypertension decision support system knowledge base to improve the accuracy of recommendations. AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium. 2009 Nov 18; 539-43.
- Chan AS, Shankar RD, Coleman RW, Matins SB, Hoffman BB, Goldstein MK. Leveraging point-of-care clinician feedback to study barriers to guideline adherence. AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium. 2009 Mar 9; 915.
- Bosworth HB, Olsen MK, Dudley T, Orr M, Goldstein MK, Datta SK, McCant F, Gentry P, Simel DL, Oddone EZ. Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. American heart journal. 2009 Mar 1; 157(3):450-6.
- Goldstein MK. Using health information technology to improve hypertension management. Current Hypertension Reports. 2008 Jun 1; 10(3):201-7.
- Michel M, Trafton JA, Martins SB, Wang D, Tu S, Johnson NA, Goldstein MK. Improving Patient Safety using ATHENA-Decision Support System Technology: Opioid Therapy for Chronic Pain. AHRQ Advances in Patient Safety: New Directions and Alternative Approaches. 2009 Jul 1; Vol 4.
- Goldstein MK. EHR Data for Complex Clinical Decision Support (CDS) and Quality Improvement: opportunities and challenges. 2012 Oct 10.
- Goldstein MK. Clinical Decision Support for Hypertension Management Integrated with Electronic Health Record: ATHENA-HTN. 2011 Aug 17.
- Goldstein MK. ATHENA Clinical Decision Support System: Aspects of Design and Implementation. [Cyberseminar]. 2010 Feb 18.
- Hwang TS, Martins SB, Tu SW, Wang DY, Heidenreich PA, Goldstein MK. Automating Quality Review for Heart Failure Through a Performance Measurement System. Poster session presented at: Society for Medical Decision Making Annual Meeting; 2014 Oct 22; Baltimore, MD.
- Goldstein MK. Implementing Real-Time Clinical Decision Support for Health Professionals Within Workflow: ATHENA-CDS. For Symposium, "Overcoming Challenges in Developing and Implementing Technology-Based Tools to Improve Health-Related Decisions and Behaviors: Lessons Learned". Presented at: Society of Behavioral Medicine Annual Meeting and Scientific Sessions; 2013 Mar 21; San Francisco, CA.
- Goldstein MK. Putting CDS to Good Use: Thinking Outside the EHR box. Paper presented at: Office of the National Coordinator for Health Information Technology Clinical Decision Support Federal Collaboratory; 2012 Dec 5; Washington, DC.
- Tu S, Goldstein MK, Peleg M, Martins SB. Knowledge-Based Decision-Support Systems for Implementing Clinical Practice Guidelines. Presented at: American Medical Informatics Association Annual Symposium; 2012 Nov 2; Chicago, IL.
- Goldstein MK, Martins SB, Schlosser J, Kim N, Farwell W, Holmes T, Musen M, Tu S, Chambers DA, Keng T, Hoffman B. Reaction of health professionals to chronic disease Clinical Decision Support (CDS). Presented at: American Public Health Association Annual Meeting and Exposition; 2012 Oct 29; San Francisco, CA.
- Goldstein MK, Corley AM, Martins SB, Tu SW, Furman AE, Oshiro CM. Stakeholder Input to Clinical Decision Support (CDS) for Complex Chronic Disease. Poster session presented at: Society for Medical Decision Making Annual Meeting; 2012 Oct 19; Phoenix, AZ.
- Aquino Shluzas L, Cronkite RC, Chambers DA, Owens DK, Goldstein MK. Implementation of the ATHENA-HTN clinical decision support system. Poster session presented at: American Medical Informatics Association Translational Bioinformatics / Clinical Research Informatics Annual Joint Summits on Translational Science; 2012 Mar 19; San Francisco, CA.
- Goldstein MK, Martins SB, Schlosser J, Kim N, Farwell W, Holmes TH, Musen M, Tu SW, Chambers DA, Keng T, Hoffman BB. Attitudes Toward Hypertension Clinical Decision Support (CDS) System. Poster session presented at: Bay Area Clinical Research Annual Symposium; 2011 Nov 4; San Francisco, CA.
- Martins SB, Tu SW, Hoffman BB, Farwell W, Schlosser J, Kim N, Chambers DA, Holmes TH, Goldstein MK. Hypertension and Diabetes: Blood Pressure Goals Endorsed by VA Primary Care Providers. Presented at: VA HSR&D National Meeting; 2011 Feb 18; National Harbor, MD.
- Goldstein MK. HSR&D Future Directions II: Medical Informatics. Paper presented at: VA HSR&D Career Development Annual Meeting; 2010 Feb 26; San Francisco, CA.
- Goldstein MK. From Evidence to Patient Care: Decision Support with Actionable Guidelines. Paper presented at: American Medical Informatics Association Annual Symposium; 2009 Nov 15; San Francisco, CA.
- Goldstein MK. Health Informatics Tutorial: Knowledge-Based Decision Support Systems for Implementing Clinical Practice Guidelines. Paper presented at: American Medical Informatics Association Annual Symposium; 2009 Nov 14; San Francisco, CA.
- Goldstein MK. From Evidence to Patient Care: Decision Support with Actionable Guidelines. Paper presented at: Agency for Healthcare Research and Quality Annual Conference; 2009 Nov 12; Rockville, MD.
- Cronkite RC, Yeh GS, Chambers DA, Breeling J, Goldstein MK. The Four R's: Effect of Roles, Routines, Rules and Reorganization on Implementation of a clinical Decision Support System. Poster session presented at: VA QUERI National Meeting; 2008 Dec 10; Phoenix, AZ.
- Goldstein MK, Martins SB, Chan A, Coleman RW, Oddone EZ, Bosworth HB, Shoenberger D, Harris AH, Tu SW, Musen M, Hoffman BB. ATHENA Hypertension Decision Support System: Impact on Hypertension Management in Primary Care Setting. Presented at: Agency for Healthcare Research and Quality Translating Research Into Practice and Policy Annual Meeting; 2006 Jul 1; Washington, DC.
- Goldstein MK, Martins SB, Chan AS, Coleman RW, Bosworth HB, Oddone EZ, Shlipak MG, Harris AH, Lavori P, Hoffman BB. Impact of decision support for hypertension on clinicians' prescribing and patients' blood pressures. Presented at: VA HSR&D National Meeting; 2006 Feb 1; Arlington, VA.
- Goldstein MK. Clinicians interaction with a guideline-based decision support system for hypertension in primary care clinics: ATHENA DSS. Paper presented at: Scientific Basis of Health Services International Annual Conference; 2003 Sep 1; Washington, DC.
Health Systems, Cardiovascular Disease
Technology Development and Assessment
Decision support, Diabetes, Hypertension, QUERI Implementation