Develop, Implement and Assess Effectiveness of Early Warning Score (EWS) for Moneragala District General Hospital

Introduction: Early detection and timely interventions are important determinants of clinical outcome in people with acute illness. Adverse outcomes including unplanned transfer to intensive care (ICU), cardiac arrest and death are usually preceded by acute physiological changes manifesting as alterations in vital signs. Usage of early warning scores (EWS) based on bedside vital sign observations may help early detection, improve outcome of patients and reduce healthcare cost.

Effectiveness of EWS in predicting deterioration of seriously ill patients has been demonstrated in high income countries (HICs). However, these scores developed in HICs have been shown to lose sensitivity and specificity when applied to a low income setting. It is imperative to explore the usefulness of EWSs in Sri Lanka. If the results are positive, widespread adaptation of these scores can significantly contribute to improved patient outcome, better utilization of ICU services and cost effective healthcare provision.

The study will take place in district general hospital (DGH) Moneragala, in Sri Lanka, a lower middle income country (LMIC). The hospital has nearly 450 beds and over 800 staff members serving over 50000 patients per year and approximately 500 cardiac arrests per year. It has four medical wards, two surgical wards and 5 other wards. A wedge shaped interventional study was designed to investigate whether a "setting tested" early warning system protocol can be implemented in a rural district general hospital of a LMIC using a local TTT model to reduce cardiac arrests and admissions to ICU.

Objectives

• To describe the characteristics, including EWS, of patients resuscitated at DGH, Moneragala.
• To validate a suitable EWSs at DGH, Moneragala.
• To examine the effectiveness of the validated EWS as part of a training and implementation bundle to reduce the incidence of cardiac arrests, ICU admissions and mortality.

Methodology Study component 1- Retrospective study All clinical variables and patient characteristics of past two years (01.04.2013-30.06.2015) collected retrospectively from BHTs of all cardiac arrests (approximately 200) of DGH Moneragala. Vital signs and laboratory measurements 24 and 48 hours before cardio respiratory emergency and at admission to hospital will be extracted. The availability of variables required for the calculation of various EWSs will be noted.

Data collection tool: Pre-defined data sheets. Study process: All clinical variables and patient characteristics will be collected retrospectively from BHTs. Vital signs and laboratory measurements 24 and 48 hours before cardio respiratory emergency and at admission to hospital will be extracted. The availability of variables required for the calculation of various EWSs will be noted.

Statistical analysis: Descriptive statistics will be used to describe the characteristics of the resuscitated patients. Mean and standard deviation will be used for normally distributed continuous variables while median and inter quartile ranges will be used for skewed distributions of continuous variables. Count and percentages will be used for discrete variables. The availability of physiological variables will also be illustrated using counts and percentages.

Study component 2- The component two of the study is aimed at selecting and validating an EWS that is capable of predicting cardiac arrest with high sensitivity and specificity A prospective cohort design, conducted at all units of DGH, Moneragala (except ICU) on all consecutive in-patient admissions for a period of 3 months.

Sample size: This phase will be used to determine the ability of EWSs to predict patient outcome with regards to cardiac arrest, ICU admission and death. Assuming that the best performing EWS will achieve an area under the ROC (AUROC) curve of .80 and an alpha of 0.05 with 80% power, comparison of an EWS that performs "worse" (AUROC curve = 0.80) will require 28 cardiac arrest patients. With a cardiac arrest rate of approximately 1% (cardiac arrests/total admissions), and 56,000 admissions per year. this means that each EWS should be performed on 2800 patients. Under the same assumptions, comparison of an EWS with an AUROC curve = 0.70 will require n=108 patients with cardiac arrest; thus, 10,800 patients should be tested with each EWS. With 56000 admissions per year 3 months of phase 2 will be adequate for this.

Data collection tools: Data will be collected from bed head tickets using pre-defined data sheets. Demographic details and physiological data will be recorded on admission to ward. Physiological data will be collected twice daily by these medical/nursing officers.

National Early Warning Score (NEWS), Modified Early Warning Score (MEWS), Standardized Early Warning Score (SEWS), Patient At Risk Score (PARS), Leeds Early Warning Score (LEWS), The Assessment Score for Sick patient Identification and Step-up in Treatment (ASSIST), Cardiac Arrest Risk Triage (CART) and VitalpacTMEarly Warning Score (ViEWS) will be tested in this phase to select the best performing one, as described below.

Statistical analysis: Data will be analyzed using STATA 13. Selection of the EWS will be based on the discrimination (using area under the receiver operating characteristic curve), calibration (using the Hosmer-Lemeshow Ĉ-statistic) and accuracy (using Brier score).

Study component 3- The purpose of this component is to examine the effectiveness of the selected EWS at improving pre-defined patient outcomes.

This is an experimental step wedge design. Study setting and population: Same as study two Study period: Twelve months Sample size: Currently, early detection of deterioration by means of EWS prior to cardiac arrest is 0%. The current cardiac arrest team only attends to patients when a cardiac arrest is detected. It was anticipate that, with the EWS identified in Part 2 of the study, the early detection of cardiac arrests will increase to 50-75%. To detect a difference of 50% in one ward, with 90% power and an alpha of 0.05, would require a sample size of 15 cardiac arrest patients. The estimated cardiac arrest rate at this study site is <1%. Assuming a cardiac arrest rate of approximately 1%, the selected EWS as part of EWS should be performed on 1,500 patients.

This means the study will be powered to detect this in at least 7 of the 12 wards when investigators implement the EWS over the 12 months it takes to recruit all wards. However addition data collection for three months will enable (power) to detect this in almost all the wards.

Data collection tools and study variables Patient data collected from bed head tickets using pre-defined data sheets. Interviewer administered questionnaires will be used to assess the success of the training for nurses and doctors. Successfulness of course delivery in each section will be measured separately. Data sheets will be used to monitor the implementation of the intervention and outcome measures.

Study variables include the same variables as in study 2. The indicators to monitor the implementation of the intervention will also be gathered. Success of implementation will be evaluated using process and outcome measures. These will include indicators to monitor the implementation of the intervention (completeness of observations, use of EWS, appropriate escalation by the nursing team) and outcome indicators (patients suffering cardiac arrests, detected and missed by EWS, ICU admissions and in-hospital mortality). Data will also be collected to monitor the success of the training programme (retention of knowledge).

Intervention and study process Introduce EWS: An EWS that is appropriate for use in the study setting will be adapted and all participants will be educated on this.

Training of staff: The participating Doctors and Nurses from each ward will be trained on early detection and management of clinically deteriorating patients based on the EWS selected. The training will be implemented in a stepped wedge method with a new ward absorbed in to the program monthly.

As each ward has the EWS introduced, an acute care training (ACT) course will be delivered to its staff. The ACT course will comprise of preparation using dedicated e-learning platform (http://nics-training.com/?page_id=403) followed by a 2-day structured, multi-modal training package focused on acute care skills for ward nurses and doctors, comprising short lectures, problem based learning and practical skills stations The ACT course will be delivered to small groups, fortnightly by a faculty of local trainers (Doctors, nurse tutors and nursing officers) who have received a five day preparatory train the trainer(TTT) course led by experienced doctor and nurse trainers (part of investigators' study team) modeled on previous efforts. As in phase 2, process and outcome data, will be collected by trained data collectors.

Follow up: Formative and summative training assessments will be conducted on training participants to measure the effectiveness of the programs, before and after training. Coaching, support and feedback will be provided to the faculty every two months to ensure maintenance of quality. Any issues faced by the staff during implementation will be identified and appropriate remedial action will be taken.

The end points will be proportion of patients detected (and missed) by the system and the number of unexpected cardiac arrests and ICU admissions.

Comparison of outcomes: The "at risk" population will be calculated as a proportion of those who have actual cardiac arrests, and tested whether this will be at least 50% or 75% whereas the value is now 0% (there is no systematic detection of at-risk patients currently, with cardiac arrest team attending only after a declared cardiac arrest. This will be done for each ward separately as well as to the group as whole. Matched group outcomes for ICU admission rates, mortality and cardiac arrests will be compared. These will be assessed for each ward and for the whole group before and after the EWS/RRS implementation. Wilcoxon signed-rank test and McNemar test will be used to compare non parametric continuous and discrete variables of matched group outcomes measured, respectively

Pre and post training: The impact of the training on the knowledge, skills and confidence of the staff in the management of the deteriorating patient before and after the training will be assessed, as above (appendix C, D and E).

Knowledge retention: Knowledge retention will be measured 3 months and 6 months after the training.

Statistical analysis: Prevalence of unanticipated ICU admissions, cardiac arrests and hospital mortality will be calculated for the hospital overall, as well as by ward (using the total number of admissions as the denominator), before and after implementation of the EWS selected from component 2 of the study. The effectiveness of the EWS system will be assessed by comparison of the proportion of cardiac arrest cases that are identified early before and after use of the EWS. Risk ratios with 95% confidence intervals will be calculated with shared frailty specified for a ward to account for any within ward clustering. The formative and summative assessments will be used to compare, in a paired manner, the effect of the training program on ward staff. The outcome analysis and statistical tests used to compare outcomes before and after implementing the EWS is described in section 3.5 comparison of outcomes

Ethical clearance obtained from the Ethics review Committee of the Faculty of Medicine, University of Colombo (EC-15-034).