Return to 2001 Abstacts List

Timothy R Dresselhaus, VA San Diego Heath Care System; UCSD School of Medicine; Center for the Study of Provider Behavior; Jeff Luck, VA Center for the Study of Health Care Provider Behavior; John W Peabody, VA San Francisco Health Care System; Department of Edpidemiology and Biostatistics, Institute for Global Health, University of San Francisco; RAND

Objectives: Payers and institutions use chart abstraction to measure physician performance, despite its underestimation of the quality of care due to recording bias. We wondered if the medical record might also overestimate the quality of care through false, and potentially unethical, documentation by providers. To determine this, we compare the quality of care as documented in the medical record with the reports of actor patients.

Methods: Twenty physicians in the primary care clinics of two Veterans Affairs Medical Centers were randomly selected among consenting residents and faculty (97% agreed to participate). Data were collected from standardized (actor) patients, who served as the gold standard and presented undetected as patients to physician subjects, and from the medical record generated from these visits. Quality criteria were developed from national guidelines and a modified Delphi technique for four common medical conditions. These were then recorded by a standardized patient or abstracted from the medical record. Physician subjects completed 160 evaluations of standardized patients (8 cases x 20 physicians). We determined the proportion of criteria reported in the medical record but not by the standardized patient (false positives). We also determined the distribution of false positives according to domain (history, physical exam, diagnosis, treatment), study site, physician subjects, and actor patients. False positive rates at the two study sites were compared by t-test.

Results: Compared to the gold standard of standardized patients, false positives were identified in the medical record for 6.4% of measured items overall. False positives were higher for physical examination (13.5%) and diagnosis (14.6%) than for history (3.8%) and treatment (3.4%). The difference in false positive rates between site 1 (7.1%) and site 2 (5.7%) was not statistically different (p = 0.34). The proportion of false positives for individual physician subjects ranged from 2.2% to 13.0% and for actor patients from 1.4% to 11.6%.

Conclusions: These results suggest that chart abstraction may overestimate the quality of care due to false positives. The clustering of false positives in the domains of physical examination and diagnosis suggests that these are not incidental occurrences or under-reporting by actor patients. Though false positives in the physical examination could result from careless documentation by physician subjects, they may indicate intentional misrepresentation of the process of care, perhaps to up-code a visit or save time by adding an exam element not performed. Such fabrication would violate ethical standards essential to the integrity of clinical practice, potentially putting patients at risk by including misinformation in the medical record. By contrast, documentation of diagnosis in the medical record but not by the actor patient would represent an important lapse in communication of essential information.

Impact: These data indicate that some physicians may actually perform less care than they report in the medical record. Improved evaluation methods are needed to detect such irregularities as well as guidelines to determine appropriate actions when potentially unethical conduct is identified in studies of quality.