3099 — Function-Based Risk Adjustment in Statistical Models: Systems vs. Components
Vogel WB (Rehabilitation Outcomes Research Center) , Berlowitz DR
(Center for Health Outcomes, Quality, and Economics Research), Tsilimingras D
(Center for Health Outcomes, Quality, and Economics Research), Hoenig HM
(Duke University), Young LJ
(Rehabilitation Outcomes Research Center), Cowper DC
(Rehabilitation Outcomes Research Center), Duncan PW
(Rehabilitation Outcomes Research Center), Wing KL
(Rehabilitation Outcomes Research Center)
To compare the predictive performance of the Function Related Groups–Discharge Motor Function (FRG-DMF) risk adjustment system with that system’s main components, the Functional Independence Measure (FIM) and patient age. We examined the ability of these two approaches to explain variations in three- and six-month post-discharge mortality in logistic regression models estimated from observational data on VA stroke patients.
Data from the VA’s Integrated Stroke Outcomes Database (ISOD) for FY01 and the Beneficiary Identification Records Locator Subsystem (BIRLS) database were used to identify VA stroke discharges and post-stroke mortality. Each of the resulting 2,545 discharges was placed in one of the 20 FRG-DMF categories according to the FRG-DMF logic. Logistic regressions were then run comparing the within-sample predictive performance of the FRG-DMF system and the predictive performance of that system’s primary inputs, admission total FIM score and age.
The use of FIM score and age yielded very similar predictive power to the FRG-DMF risk adjustment system for both three-and six-month mortality. The FRG-DMF models yielded areas under the Receiver Operating Characteristics (ROC) curve of .818 and .797 for three- and six-month mortality, respectively, while the FIM-age models yielded areas of .806 and .784. Moreover, concordance percentages were similar at 81.0 and 79.4 for three- and six-month mortality for the FRG-DMF models, and 80.3 and 78.2 for the FIM-age models.
These findings have significant implications for researchers looking to risk adjust outcomes in statistical models. Clinically meaningful classification systems such as the FRG-DMF system are very useful for descriptive comparisons of outcomes. However, researchers who risk adjust outcomes in statistical models may wish to use more direct patient-specific measures when little predictive power is sacrificed. These more direct measures can yield considerable advantages in small samples where the many categories of risk adjustment systems result in small cell sizes and overfitting.
Our results show that simpler statistical specifications can yield predictive results on a par with more complex risk adjustment systems, thereby enabling risk adjustment in smaller samples.