Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

TRX 02-216 – HSR Study

TRX 02-216
Human Factors and the Effectiveness of Computerized Clinical Reminders
Emily S. Patterson, PhD MS
Cincinnati VA Medical Center, Cincinnati, OH
Cincinnati, OH
Funding Period: July 2003 - June 2006
Computerized clinical reminders (CRs) are in widespread development to improve compliance with established clinical guidelines. Despite promising results from randomized controlled trials of CRs, effectiveness in practice is mixed. Human factors, the study of the interaction of humans and decision support systems in complex environments, may provide new explanations for why CRs are less effective an intervention than anticipated. In prior work, eleven human factors barriers to the use of 9 HIV clinical reminders were identified through ethnographic observation and semi-structured interviews. These barriers included design, organizational, team, provider, and contextual barriers as well as interactions with other supporting artifacts.

The aims are to identify human factors barriers and facilitators to the use of computerized clinical reminders, assess their prevalence across the VHA, and recommend local and national changes to design, implementation, and use.

Three studies combine qualitative and quantitative approaches. Study 1: Ethnographic observations to identify barriers and facilitators to the use of clinical reminders at four sites selected by convenience sampling. Study 2: National provider survey of randomly selected VHA physicians to determine prevalence rates. Study 3: Simulated use in a laboratory setting with intake nurses to evaluate whether a redesign of clinical reminders improves performance and efficiency of use as compared to the current design.

Study 1 findings are that barriers to use included (1) workload, (2) poor usability, (3) lack of flexibility, (4) coordination issues between nurses and providers, and (5) a tendency to perform documentation while not with the patient. Facilitators to use included (1) the ability to document problems and receive prompt feedback from those responsible for maintaining the clinical reminders system, (2) location of the computer workstations, (3) improving integration of reminders into "normal" workflow, and (4) strategies for reducing the overall number of clinical reminders at a site. Study 2 findings are that 1) more than two-thirds of VHA primary care respondents always use clinical reminders to some extent, typically during the patient visit, 2) physicians who provide more direct patient care are more likely to use more clinical reminders per patient, 3) physicians who have been in practice at the VHA for five to nine years are more likely to use more clinical reminders than physicians who have practiced fewer than five years, and 4) physicians who perceive a greater workload use fewer clinical reminders per patient. Study 3 findings are that a redesigned interface significantly increased efficiency and user satisfaction, and decreased mental workload and frustration, for six simulated patient-provider outpatient visits in a laboratory setting.

Improving the ability to effectively and easily use clinical reminders would likely increase and standardize the quality of care provided and increase the efficiency of care for all outpatients in the VHA.

External Links for this Project

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


Journal Articles

  1. Fung CH, Tsai JS, Lulejian A, Glassman P, Patterson E, Doebbeling BN, Asch SM. An evaluation of the Veterans Health Administration's clinical reminders system: a national survey of generalists. Journal of general internal medicine. 2008 Apr 1; 23(4):392-8. [view]
  2. Spina JR, Glassman PA, Belperio P, Cader R, Asch S. Clinical relevance of automated drug alerts from the perspective of medical providers. American journal of medical quality : the official journal of the American College of Medical Quality. 2005 Jan 1; 20(1):7-14. [view]
  3. Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. Journal of the American Medical Informatics Association : JAMIA. 2005 Jul 1; 12(4):438-47. [view]
  4. Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. Journal of Biomedical Informatics. 2005 Jun 1; 38(3):189-99. [view]
  5. Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JA, Roth EM, Asch SM. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. Journal of the American Medical Informatics Association : JAMIA. 2007 Sep 1; 14(5):632-40. [view]
Conference Presentations

  1. Fung CH, Tsai J, Lulejian A, Patterson E, Asch SM. A national survey of clinical reminder use and barriers to clinical reminder use in an integrated healthcare system. Paper presented at: Society of General Internal Medicine Annual Meeting; 2006 Apr 1; Los Angeles, CA. [view]
  2. Militello L, Patterson E, Tripp-Reimer T, Asch SM, Fung CH, Glassman P, Anders S, Doebbeling BN. Clinical Reminders: Why don't they use them? Presented at: Human Factors and Ergonomics Society Annual Meeting; 2004 Sep 1; Santa Monica, CA. [view]
  3. Asch SM. Lessons from implementation of an electronic health record in an integrated health delivery system: The Veterans Health Administration (VHA). Paper presented at: AcademyHealth Annual Research Meeting; 2006 Jun 1; Seattle, WA. [view]
  4. Asch SM. Results of a National Comparative HIV Quality Improvement Initiative Within the Veteran's Administrative Healthcare System. Paper presented at: Human Factors and Ergonomics Society Annual Symposium on Human Factors and Ergonomics in Health Care; 2004 Sep 1; New Orleans, LA. [view]
  5. Fung CH, Woods JN, Asch SM, Glassman P, Doebbeling BN. Variation in Implementation of Computerized Clinical Reminders in the VHA. Paper presented at: VA HSR&D National Meeting; 2004 Mar 10; Washington, DC. [view]

DRA: Health Systems
DRE: none
Keywords: Organizational issues
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.