Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

2006 HSR&D National Meeting Abstract

1065 — Development of the QUASII (QUality Assessment of Studies in Informatics Implementations) Instrument

Author List:
Phansalkar S- (George E. Wahlen VAMC, Salt Lake City, UT)
Staggers N (College of Nursing, University of Utah, Salt Lake City, UT)
Weir CR (George E. Wahlen VAMC, Salt Lake City, UT)

We are conducting a meta-analysis of the efficacy of clinical informatics solutions (CIS) implementations. Quality assessment of studies in informatics poses problems peculiar to this domain, including: failure to specify the independent variable (e.g. the intervention and confounders of the implementation process), failure to model the relationships between variables, reporting only significant variables, inappropriate unit of analysis, and a lack of generalizability to other interventions. In this study, we describe the development of an instrument, QUASII (QUality Assessment of Studies in Informatics Implementations), designed to detect biases specific to CIS implementations.

Current quality assessment instruments were reviewed. A modified Delphi approach was used to identify pertinent elements. Each item is scored on a 7-point Likert scale ranging from ‘Completely Inadequate’ to ‘Completely Adequate’. Inter-rater reliability for the weighting of the constructs to be included in QUASII, was evaluated.

Nine quality assessment instruments were used to develop an initial list of 52 items. Following expert consensus, the final instrument included 9 constructs specific to informatics: Confounders of CIS Implementation, Clinical Unit of Analysis, Clinical Applicability, Unintended Organizational Co-Interventions, Equality of User Groups, Construct Validity of CIS Intervention, Causal Model of CIS Effects (e.g. underlying mediating and moderating variables), Specification of Dependent Variables (e.g. publishing only significant results), and Appropriate Generalizability of Findings. Intraclass correlation coefficient (ICC) between three experts was 0.78; (95% CI: 0.56, 0.90).

QUASII is the first instrument for the assessment of study quality specific to informatics. Further, QUASII describes biases that are peculiar to CIS implementations and uses a weighted scale for the measurement of its constructs. This enables the measurement of change that is attributable to the CIS alone and prevents inferences that may lead to false estimates of the impact of a CIS intervention.

VA has a strong culture of deploying CIS for improving patient care. The development of an instrument to critically appraise the quality of informatics studies can help derive proper conclusions about their applicability to VA hospitals and the dissemination of research findings in CIS implementations into clinical practice.

Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.