Disseminating a Dashboard for VA Purchased Community Nursing Homes

Objective: To evaluate the impact of the CNH Dashboard implementation on the quality of contracted CNHs relative to non-contracted CNH in a VAMC Market.

Stepped wedge Dashboard Implementation The investigators propose a stepped wedge implementation of the CNH Dashboard, randomized at the VISN level (n=3-4 per period). The stepped wedge allows for maximization of GEC resources with a systematic roll out to provide sufficient education to every randomized VISN (see education program below). Every 3 months, we will bring a new randomized group of VISNs 'online' with the CNH Dashboard (Table 2). The investigators selected the VISN level as a unit of randomization, because there is integration between VAMCs within a VISN. With approximately 4 VAMCs in each VISN, this release schedule is robust, but manageable with support from this award. The investigators will pilot test the CNH Dashboard in 3 VISNs to finalize the information upload, dashboard, and educational program.

Table 2: Stepped wedge plan for CNH Dashboard roll out Randomized VISN Group Months

-6 to 0 Months 1-3 Months 4-6 Months 7-9 Months 10-12 Pilot VISNs CNH Dashboard CNH Dashboard CNH Dashboard CNH Dashboard CNH Dashboard

1. Usual CNH Quality CNH Dashboard CNH Dashboard CNH Dashboard CNH Dashboard
2. Usual CNH Quality Usual CNH Quality CNH Dashboard CNH Dashboard CNH Dashboard
3. Usual CNH Quality Usual CNH Quality Usual CNH Quality CNH Dashboard CNH Dashboard
4. Usual CNH Quality Usual CNH Quality Usual CNH Quality Usual CNH Quality CNH Dashboard

Dashboard Hosting The CNH Dashboard will be hosted on a VA GEC intranet website. The advantage of a intranet site is that it allows controlled access to the site for the stepped wedge implementation. As the project continues, the investigators will enlist the assistance of the Indianapolis VAMC Center for Applied Systems Engineering to guide the transition toward an electronic dashboard that is accessible throughout the VA intranet.

Educational Program The investigators will design an educational program with the goals of a) understanding about the quality of CNH as presented in the five star rating system, b) orientation to the dashboard, c) identifying low performing CNH facilities, and d) available options for contracting with higher quality facilities. This educational program will include elements of didactic training, as well as coaching throughout the implementation. The effectiveness of this educational program will be determined by anonymous participant survey. The education program will be rolled out to VISN and facility geriatric program leaders.

Primary Analysis While the investigators are randomizing by VISN, the investigators will analyze by VAMC, because that is where purchased CNH selection occurs. For each VAMC, we will measure the ratio of the average quality of purchased CNH facilities (observed) to non-contracted CNH (expected) in the VAMC market. The VAMC average observed to expected (O/E) ratio will serve as the primary measure for this analysis. The primary reference baseline period will consist of the average O/E in the 6 months prior to CNH Dashboard implementation. The intervention period, during which the CNH Dashboard is released to the randomized group, will last for 3 months during which the educational program will be delivered and assessed. The primary outcome will be the average O/E in the 12 months after the roll out period ends. Each VAMC will act as their own control. Our primary analyses will consist of the VAMC average O/E ratio of Contracted CNH facilities in the 6 months prior to the intervention and the 12 months post-intervention (Figure 6). From Figure 3, the investigators expect that this will be a normal distribution and, therefore, will use a Student's t-test to compare the average O/E ratios.

Currently, CNH contracts are generally written with annual renewals of a 5-year master contract. Thus, if quality is low, a contract can be discontinued annually. However, there is substantial overhead in establishing a 5-year contract. While the investigators do not expect the CNH quality to be the only measure of CNH selection, we anticipate that it will become a factor in non-renewal and new contract initiation. Presently, each VAMC contracts with an average of 12 CNHs and thus, there are 2-3 facilities per year which are up for contract. Changing the contracting process will take time and with GEC resources; the investigators are thus committed to follow up for 12 months for each of the four groups after CNH Dashboard roll out.

Secondary Analysis To protect the internal validity of the stepped wedge implementation, the investigators must perform secondary analyses with additional reference periods.12 This helps to account for secular trends and intervening events which can influence the study outcome. For example, if CMS were to revise the criteria for the 5-star ranking system downward during the CNH Dashboard implementation, the investigators would introduce a type 2 error (incorrect acceptance of the null). In addition to the changes in contracted CNH quality as the dashboard is released, the investigators expect that there will be movement toward higher quality facilities among VAMCs prior to their randomized roll out. This change is a result of VAMCs taking action because they know they are being examined with respect to their contracted CNH quality prior to seeing the CNH Dashboard. VAMCs will either pressure contracted CNHs to improve or change contracting practices. As a result, we expect that the O/E ratio will rise naturally during the course of the study. Additionally, secular trends in CNH selection, VA approval for fee authority, OIG investigations, news stories, and VA organizational decisions could influence the selection and focus on CNH quality.

To address this natural change, the investigators propose secondary analyses with additional reference periods. Our first additional reference period for the randomized groups will be the O/E in the 6 months prior to CNH Dashboard implementation (months -6 to 0), which will be compared to the O/E of the primary analysis period. Second, we will examine the change O/E for the primary reference period. We will compare the O/E of randomized group 1 to the other randomized groups using Student's t-tests to determine if there has been significant baseline change over the course of the evaluation. Finally, the investigators will examine the O/E in the reference period as a function of time. This will particularly pertain to Groups 3 and 4 which have a longer window (12 and 15 months respectively) by which we can examine the temporal change in O/E without the intervention.