Inter-rater Reliability of Preoperative Mortality Risk Calculators

Identifying high-risk patients for perioperative treatment and decision-making remains a challenge due to difficulty in reliably estimating morbidity and mortality risks. Assessing surgical risk helps allocating resources, obtaining informed consent, and making shared decisions with a multidisciplinary team (MDT). Multimorbidity is increasing globally in an ageing population with a growing burden of chronic diseases. It has been shown that high-risk non-cardiac surgical patients suffer disproportionally from perioperative complications. Preoperative mortality risk calculators are available and may help to identify high-risk non-cardiac surgical patients already before surgery, triggering efforts to lower the burden of possible complications, e.g., by extended monitoring or specified treatments. A systematic review suggested a significant risk of bias in developing current preoperative risk calculators due to lack of external validation, highlighting the need for enhanced performance and reliability to ensure their effectiveness in clinical practice. Low reliability and performance may, despite their general availability, be why that preoperative risk calculators are not yet consistently used in clinical practice. For daily clinical use, good predictive performance, low inter rater variability and user friendliness are essential. Discrepancies in predictor measurements can cause miscalibration, changes in discriminatory ability, and overall accuracy, leading to clinically relevant variability in risk calculator results. Previous studies have shown that physicians must trust a mortality risk calculator before utilization. High-risk patients suffer especially from complications, and it has been shown that the complications often result in death perioperatively. Therefore, adequate preoperative calculation of mortality risks and early recognition of high-risk non-cardiac surgical patients could benefit from reliable preoperative risk calculation. These high-risk patients can then be discussed in a preoperative multidisciplinary discussion to lower complications and perioperative deaths.

The current study evaluated the inter-rater reliability of calculating preoperative mortality risk scores for high-risk non-cardiac surgical patients in clinical practice among five anesthesiologists. The hypothesis was that the available risk calculators would show moderate to good reliability. For this purpose, five anesthesiologists used the following three risk calculators: the preoperative score to predict postoperative morbidity [POSPOM, the American College of Surgeons surgical risk calculator, and the surgical outcome risk tool [SORT].