BACKGROUND/RATIONALE:
The Centers for Medicare and Medicaid Services (CMS) publicly reports Agency for Healthcare Quality and Research's Patient Safety Indicator (PSI) rates on Hospital Compare. Currently, the PSI rates are only reported internally within the VA, although this may change in the future once present-on-admission (POA) data are incorporated into the rates. As the PSIs become more visible to VA hospitals, key VA stakeholders (e.g., hospital directors, quality managers) should be aware of the strengths and limitations of the PSIs. Education may help VA stakeholders gain knowledge about the PSIs prior to public reporting, in addition to informing them on how to interpret their PSI rates and use the PSIs as for screening, case finding, and quality improvement (QI).

OBJECTIVE(S):
The overall study goal was to educate VA stakeholders about the PSIs in order to enhance knowledge and use. Specific objectives included: 1) assessing stakeholders' needs related to the PSIs and identifying potential barriers/facilitators to using the PSIs for QI; 2) developing an educational program that best meets stakeholders' needs; 3) implementing the program that was developed and periodically assessing stakeholders' satisfaction with the program; and 4) assessing the effectiveness of the program.

METHODS:
This was a one year mixed methods implementation study. Objective 1: We conducted telephone interviews with patient safety managers and quality/performance improvement managers at eight VA hospitals to assess educational preferences and organizational factors related to how improvement activities are prioritized at the local hospital. Based on the information collected, we then developed a web-based survey to assess pre-program educational needs; this was administered nationwide to VA stakeholders (ranging from hospital directors to patient safety managers) at 132 VA hospitals and patient safety officers and quality management officers from the 21 Veterans Integrated Service Networks (VISN). Objective 2: We developed an educational program that incorporated stakeholders' feedback from the survey/interviews. Objective 3: We implemented the program to and obtained ongoing feedback from participating stakeholders through web-based surveys. Objective 4: Upon program completion, we administered a survey to participating stakeholders to assess effectiveness of the program.

FINDINGS/RESULTS:
1) Of 16 potential participants, we interviewed nine participants by telephone. Findings included: Most respondents did not recall receiving any training or education related to the PSIs although many had heard of the PSIs from various meetings. Respondents suggested ways in which seminars through Live Meeting could be made more interactive, such as through videos. In addition, respondents noted that priorities at many VA hospitals were set in response to VA central office and VISN mandates, although many were also generated in-house. We also administered a pre-program survey to 782 potential participants with a total of 181 respondents. Findings from the survey included: 88% strongly agreed/agreed that they were interested in learning more about the PSIs; respondents were most interested in learning how to use the PSIs for monitoring trends and how to interpret the PSIs; 83% of respondents preferred to learn about the PSIs through web conferencing; and 62% of respondents expressed interest in learning about the PSIs through Q&A sessions and pre-recorded/archived materials.

2) We developed an educational program consisting of six cyber seminars that were offered through LiveMeeting. Each session's presentation was 30 minutes with 15 minutes for Q&A. The format of the presentations occurred using an interview-type style (moderator asked questions; presenters responded to questions). Internal VA reports, containing PSI rates, were used as a learning tool to walk through various topics related to the PSIs while providing concrete, relevant examples.

3) Of 181 potential participants, 82 participants signed up to take part in the educational program; on average, nine participants attended the cyber seminars that were offered via LiveMeeting. The program was implemented over the course of 13 weeks. Some of the covered topics included: an overview of PSI rates contained in internal reports, how to interpret PSI rates, how to use the PSIs, and organizational factors to consider. Feedback from the surveys, administered immediately after conclusion of each presentation, assured us that our presentations were clear and concise; we were making complicated topics understandable; and participants enjoyed our use of the interview style format (which broke up the monotony that can occur during a one speaker presentation).

4) We administered the survey to 82 participant; a total of 16 individuals responded. Findings from the survey administration included: most respondents were satisfied with what was learned through the program; thought the session topics chosen for the program were useful; would recommend this program to their staff/colleagues; felt that the knowledge gained from the program would be put to direct use; and felt they had the knowledge to use the PSIs for QI. Some of the reasons that contributed most to their learning about the PSIs were: the structure of the program (e.g., LiveMeeting, archived/recorded sessions on study SharePoint site to access on their own time, availability of archived handouts) and the content (e.g., clear, concise, and applied clinical examples). Some of the reasons that contributed least to their learning about the PSIs were: lack of face-to-face training, lack of available time to join the sessions, and technical issues. Recommendations for improving the program included: having multiple LiveMeeting sessions on each topic and having participants work through activities based on what was learned in each session (such as conducting a drill down on data and discussing QI opportunities).

IMPACT:
Through our program, VA participants learned how to interpret and use the PSIs. In turn, this education could be useful to participants as they will be better informed about how to interpret their PSI rates and will also know more about the strengths and limitations of the PSIs. The education received could also lead to development of QI projects using the PSIs, improving patient safety and the quality of care delivered to Veterans. This study also should help to inform VA efforts about public reporting when VA decides to publicly release PSI rates on the VA and CMS Hospital Compare websites. Future studies using other types of interventions are needed to complement this approach.

---

External Links for this Project

---

PUBLICATIONS:

Journal Articles

1. Rivard PE, Parker VA, Rosen AK. Quality improvement for patient safety: project-level versus program-level learning. Health care management review. 2013 Jan 1; 38(1):40-50. [view]

DRA: Health Systems
DRE: Prevention
Keywords: none
MeSH Terms: none