Assessment of Artificial Intelligence Algorithms for ROTEM

Rotational thromboelastometry (ROTEM) is a point-of-care viscoelastic testing method used to assess the coagulation status of patients undergoing high-risk surgical procedures, such as cardiac surgery and liver transplantation. While ROTEM-guided transfusion algorithms have improved clinical outcomes, the accurate interpretation of ROTEM results remains complex and heavily dependent on clinical experience.

This prospective observational validation study aims to assess the accuracy and clinical decision-making performance of artificial intelligence (AI)-based language models in interpreting ROTEM findings. The study will compare the AI-based evaluations to expert consensus in terms of both diagnostic accuracy and treatment recommendations.

De-identified ROTEM case data will be converted into structured clinical vignettes, which will be independently interpreted by at least three experienced clinicians (serving as the gold standard) and AI models. Each AI system will be prompted with ROTEM parameters in a standardized format and asked to assess coagulopathy type and suggest appropriate treatment options. ROTEM interpretation algorithms, such as Görlinger's protocol, will be provided as background context to ensure consistent guidance.

The study will include adult patients (≥18 years) undergoing elective cardiac surgery or liver transplantation, provided complete and technically valid ROTEM results are available. The main outcome of the study is the agreement between AI-based and expert decisions regarding the need for treatment. Secondary outcomes include diagnostic classification of coagulopathy, concordance in treatment recommendations, and standard accuracy metrics (sensitivity, specificity, PPV, NPV, overall accuracy, and Cohen's Kappa).

This study does not involve any direct patient interventions or changes in treatment based on AI output. All data will be anonymized before analysis, and informed consent will be obtained from all participants.

As part of the AI evaluation and expert comparison, each ROTEM clinical scenario will be assessed based on a standardized set of 14 structured clinical questions. These questions are designed to determine both the presence and type of coagulopathy, as well as the appropriate treatment recommendations. The specific questions are:

Is there evidence of coagulopathy based on the ROTEM findings?

Do the ROTEM results indicate hyperfibrinolysis?

Do the findings suggest the presence of residual heparin effect?

Is there evidence of fibrinogen deficiency?

Is there evidence of thrombocytopenia or platelet dysfunction?

Do the results indicate coagulation factor deficiency?

Do the findings suggest protamine overdose?

Is treatment not required at this time?

Is antifibrinolytic therapy indicated?

Should protamine be administered?

Should fibrinogen or fibrinogen-containing products be administered?

Should platelet transfusion be performed?

Should prothrombin complex concentrate (PCC) or fresh frozen plasma (FFP) be administered?

If bleeding continues, should reassessment be done after 10-15 minutes?

Each question will be answered as "Yes" or "No" by both the AI system and the expert panel, and the responses will be used to calculate diagnostic and treatment concordance metrics.