Does an Indian Version of the International Classification of Disease Injury Severity Score Predict Mortality in Four Public Hospitals in Urban India?

Introduction

In 2013 trauma was estimated to cause 4,8 million deaths, which is more than HIV/AIDS, tuberculosis, malaria and maternal conditions combined (1). Ninety per cent of these deaths occur in lower-middle income countries (LMIC) and an estimated two million lives could be saved annually by improved quality of care (2,3). India is considered a lower-middle income country with over 1 million annual trauma deaths (1). In 2020 trauma is estimated to be the third leading cause of death in the country (4). Hence, efforts to strengthen trauma care in India are urgently needed.

Trauma patients constitute a heterogeneous population, making trauma research and outcome comparison over time and between contexts difficult but important (5). Accounting for factors such as selection bias, difference in care and case mix is crucial for correct conclusions (6,7).To enable this several tools or scores have been developed including the Injury Severity Score (ISS) and the Trauma and Injury Severity Score (TRISS) (8,9). The use of these scores as part of quality improvement programmes has been associated with improved trauma care (10).

In ISS and TRISS the severity assigned to each injury is based on expert consensus. In contrast, the international classification of disease (ICD) injury severity score (ICISS) was developed using a more data driven approach (9,11). This score is based on survival risk ratios assigned to ICD injury codes to estimate an individual patient's probability of survival. According to a recent systematic review ICISS outperforms ISS derived methods (12), but so far almost all research on ICISS is from high income countries. Therefore, our research question is, does an Indian version of ICISS predict mortality in four public hospitals in urban India?

Study design

We will conduct a retrospective registry based study to derive and temporally validate a new version of ICISS.

Setting

We will use data from an ongoing prospective cohort study called Towards Improved Trauma Care Outcomes (TITCO) in India that started in four public university hospitals across urban India. The four centres are Lokmanya Tilak Municipal General Hospital in Mumbai, King Edward Memorial Hospital in Mumbai, Jai Prakash Narayan Apex Trauma Center in Delhi, and the Institute of Post-Graduate Medical Education and Research and Seth Sukhlal Karnani Memorial Hospital in Kolkata. The data used in this study was collected between October 2013 and January 2015.

Trained project officers conducted all data collection. The project officers had a health master's degree or higher education. They worked eight hours a day and rotated between day, evening and night shifts. There was one project officer for each hospital. The project officers where continuously supervised and trained by project management. Patients were followed up until discharge, death or 30 days, whichever came first.

Source and method of participant selection

Project officers included all consecutive patients that fitted the eligibility criteria. Data for patients admitted during the project officers' shifts were collected using a combination of direct observation and extraction from patient records. Data for patients admitted outside of their shifts was collected retrospectively from patient records within days of patient arrival. All patients discharged before 30 days where considered alive at 30 days.

Data sources/measurements

Data on covariates were extracted from patient records or from the patients or their accompanying relatives. Injuries were also extracted from patient records, including imaging reports and operation notes and were then coded using ICD-10. We will calculate the SRR for to each unique ICD-10 code using SRR=A/(A+B), where A denotes the number of surviving patients with a specific ICD-code and B is the number of non-surviving patients with the same specific ICD-code. The calculated SRR gets a value between zero and one. One represents 100% survival and zero represents 0% survival. We will calculate the final ICISS score for each patient as the product of all individual SRRs. Hence, ICISS also ranges from 0 to 1 and should be interpreted as the patient's probability of survival. This method is commonly referred to as the conventional ICISS.

Bias

The personal collecting the data were observers and did not take part in the actual care. During the conversion from injuries in free text to ICD-10 codes the coders will be blinded to patient demographics and outcomes. ICD-10 coding will be done after completing World Health Organization (WHO) ICD-10 online training module and after achieving over 80% agreement in several samples of 50 injuries compared to an external coder.

Study size

We will use all available data from TITCO and create a temporal split sample, using the earlier data for derivation and the most recent data for validation. These two samples are henceforth referred to as the derivation sample and validation sample respectively. We will first estimate the required sample size of the validation sample to include the most recent 200 consecutive events, i.e. patients who died within 24 hours, and all non-events enrolled during the same time period.

We use mortality within 24 hours for our sample size calculation as we want the study to be powered for secondary outcomes also. This effective sample size will allow us to detect a significant difference in discrimination and calibration of ICISS between derivation and calibration samples at 80% power and a 5% significance level. We will include all remaining patients in the derivation sample.

Quantitative variables

We will analyse all quantitative variables as continuous.

Statistical methods and analyses

The derivation and validation of ICISS will be conducted as two separate steps, described below. We will use R for all statistical analyses. We will assess predictive performance in terms of discrimination and calibration. Discrimination will be assessed by calculating the area under the receiver operating characteristics curve (AUROCC) and calibration will be assessed by comparing observed and predicted outcomes visually in a calibration plot and statistically by calculating the calibration slope. Confidence intervals for predictive performance measures will be estimated using a bootstrap approach.

We interpret overlapping confidence intervals as evidence of lack of a significant difference. Parametric and non-parametric exact tests will be used as appropriate, with 95% confidence intervals and a 5% significance level. Our main analysis will be a complete case analysis, in which we exclude observations with missing values in any of the following variables: age, sex, mechanism of injury, transfer status, and outcome. Observations with no injuries reported will be assigned an ICISS of 1 and for each observation the final ICISS will be calculated based only on SRR for ICD-codes that occurred in at least 10 observations in the derivation sample.

Derivation

We will derive SRR in the derivation sample for each of the outcomes and used them to calculate ICISS for each patient. In other words, we will calculate one set of SRR for mortality within 30 days, henceforth referred to as SRR-30D, and one set of SRR for mortality within 24 hours, henceforth referred to as SRR-24H. We will then calculate two ICISS for each patient. We will use similar denotation to refer to these ICISS, i.e. ICISS-30D and ICISS-24H. Finally, we will assess the performance of ICISS-30D in predicting mortality within 30 days and within 24 hours, and repeated this analysis for ICISS-24H.

Validation

We will use the SRR-30D and SRR-24H that we derive in the derivation sample to calculate ICISS-30D and ICISS-24H in the validation sample. We will then assess the performance of ICISS-30D in predicting mortality within 30 days and within 24 hours, and the performance of ICISS-24H in predicting mortality within 30 days and within 24 hours. Finally, the performance of each model in the validation sample will be compared with the same model's performance in the derivation sample.

Sensitivity analyses

We will conduct four sensitivity analyses. In the first sensitivity analysis we will include observations with missing values in covariates but with complete outcome data. In the second sensitivity analysis we exclude observations without any reported injury. In the third sensitivity analysis we will calculate ICISS based on all available SRR, regardless of how frequently the corresponding ICD-10 codes occurr in the dataset. Finally, we will calculate ICISS for each patient based only on unique ICD-10 codes, in other words, each ICD-10 code will only be allowed to contribute one SRR to ICISS even if it occurrs more than once in the same patient.

1. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet [Internet]. 2014 Dec 17 [cited 2014 Dec 19];385(9963):117-71. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4340604\&tool=pmcentrez\&rendertype=abstract
2. Chandran A, Hyder AA, Peek-Asa C. The global burden of unintentional injuries and an agenda for progress. Epidemiol Rev [Internet]. 2010 Jan [cited 2015 Dec 6];32:110-20. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2912603\&tool=pmcentrez\&rendertype=abstract
3. Mock C, Joshipura M, Arreola-Risa C, Quansah R. An estimate of the Number of Lives that Could be Saved through Improvements in Trauma Care Globally. World J Surg. 2012;36(5):959-63.
4. Joshipura MK. Trauma care in India: current scenario. World J Surg [Internet]. 2008 Aug [cited 2016 Jan 19];32(8):1613-7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18553048
5. Rutledge R. The goals, development, and use of trauma registries and trauma data sources in decision making in injury. Surg Clin North Am [Internet]. 1995 Apr [cited 2016 Feb 22];75(2):305-26. Available from: http://www.ncbi.nlm.nih.gov/pubmed/7900000
6. Newgard CD, Fildes JJ, Wu L, Hemmila MR, Burd RS, Neal M, et al. Methodology and analytic rationale for the American College of Surgeons Trauma Quality Improvement Program. J Am Coll Surg [Internet]. 2013 Jan [cited 2016 Feb 22];216(1):147-57. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23062519
7. Krumholz HM. Mathematical models and the assessment of performance in cardiology. Circulation [Internet]. 1999 Apr 27 [cited 2016 Feb 22];99(16):2067-9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/10217642
8. Baker SP, O'Neill B, Haddon W, Long WB. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma [Internet]. 1974 Mar [cited 2015 Apr 16];14(3):187-96. Available from: http://www.ncbi.nlm.nih.gov/pubmed/4814394
9. Rutledge R, Osler T, Emery S, Kromhout-Schiro S. The end of the Injury Severity Score (ISS) and the Trauma and Injury Severity Score (TRISS): ICISS, an International Classification of Diseases, ninth revision-based prediction tool, outperforms both ISS and TRISS as predictors of trauma patient survival,. J Trauma [Internet]. 1998 Jan [cited 2016 Feb 15];44(1):41-9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/9464748
10. Glance LG, Osler T. Beyond the major trauma outcome study: benchmarking performance using a national contemporary, population-based trauma registry. J Trauma [Internet]. 2001 Oct [cited 2016 Feb 22];51(4):725-7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/11586166
11. Meredith JW, Evans G, Kilgo PD, MacKenzie E, Osler T, McGwin G, et al. A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma [Internet]. 2002 Oct [cited 2016 Feb 22];53(4):621-8; discussion 628-9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12394857
12. Gagné M, Moore L, Beaudoin C, Batomen Kuimi BL, Sirois M-J. Performance of ICD-based injury severity measures used to predict in-hospital mortality: a systematic review and meta-analysis. J Trauma Acute Care Surg [Internet]. 2015 Dec 26 [cited 2016 Feb 4]; Available from: http://www.ncbi.nlm.nih.gov/pubmed/26713976