AzTrials
AzTrials

Intracranial HemorrhagesAI-REACTNCT06018545

AI Assisted Reader Evaluation in Acute Computed Tomography (CT) Head Interpretation

An observational study of Intracranial Hemorrhages and Acute Ischemic Stroke, run by Oxford University Hospitals NHS Trust at 4 sites in United Kingdom.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838). The purpose of the study is to assess the impact of an Artificial Intelligence (AI) tool called qER 2.0 EU on the performance of readers, including general radiologists… Read more about this study

Who is it for?
Any ageHealthy volunteers can take part
What is being tested?
Ground truthing and Reading
How many people
33 participants
Where
4 sitesUnited Kingdom
Study dates
1 Jun 2023to 1 Jun 2025
Phase
Observational
Are there results?
None posted in the registry

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages Any age
  • All sexes
  • Healthy volunteers accepted
  • Child, Adult, Older adult

Study population: Setting: Readers will be recruited from the following four hospital Trusts (secondary and tertiary level: * Guy's \& St Thomas NHS Foundation Trust * Northumbria Healthcare NHS Foundation Trust * NHS Greater Glasgow and Clyde * Oxford University Hospitals NHS Foundation Trust Participants: 30 volunteer participant readers will be selected from the following groups: * Emergency Medicine Consultants and Registrars (5 Consultant, 5 Registrar (ST3-6), 5 junior (F1-ST2) * General Radiologist Consultants and Registrars (5 Consultant, 5 Registrar) * 5 CT Radiographers

Who may be able to join

  • Radiologists/Radiographers/ED clinicians who review CT head scans as part of their clinical practice

Who cannot join

  • Neuroradiologists.
  • Non-radiologist groups: Clinicians with previous formal postgraduate CT reporting training
  • Emergency Medicine group: Clinicians with previous career in radiology/neurosurgery to registrar level

Trial locations and contacts

4 sites in 1 country.

United Kingdom4 sites
  • NHS Greater Glasgow and Clyde

    Glasgow, G12 0XH

  • Guy's & St Thomas NHS Foundation Trust

    London, London, SE1 7EH

  • Northumbria Healthcare NHS Foundation Trust

    Newcastle upon Tyne, NE27 0QJ

  • Oxford University Hospitals NHS Foundation Trust

    Oxford, Oxfordshire, OX3 9DU

Other trials in United Kingdom

About this study

Official title: AI Assisted Reader Evaluation in Acute CT Head Interpretation

This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838).

The purpose of the study is to assess the impact of an Artificial Intelligence (AI) tool called qER 2.0 EU on the performance of readers, including general radiologists, emergency medicine clinicians, and radiographers, in interpreting non-contrast CT head scans. The study aims to evaluate the changes in accuracy, review time, and diagnostic confidence when using the AI tool. It also seeks to provide evidence on the diagnostic performance of the AI tool and its potential to improve efficiency and patient care in the context of the National Health Service (NHS). The study will use a dataset of 150 CT head scans, including both control cases and abnormal cases with specific abnormalities. The results of this study will inform larger follow-up studies in real-life Emergency Department (ED) settings.

Being tested
Ground truthing, Reading
Keywords
Radiology, Head tomography, Emergency medicine, Radiographer, Artificial intelligence
Medical subject headings
Intracranial Hemorrhages, Ischemic Stroke, Hydrocephalus, Cerebral Infarction, Brain Edema, Brain Injuries

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • Ground truthingOther

    Two Consultant neuroradiologists will independently review the images to establish the 'ground truth' findings on the CT scans which will be used as the reference standard. In the case of disagreement, a third senior neuroradiologist's opinion will be sought for arbitration.

  • ReadingOther

    All 30 readers will review all 150 cases, in each of two study phases. The readers will provide their opinion on the presence or absence of some acute abnormalities, including intracranial haemorrhage, infarct, midline shift and fracture. They will provide a confidence in their diagnosis (10-point visual analogue scale), and a single click point to mark the location of each abnormality that they consider as being present. The time taken for each scan will be automatically recorded.

Study groups in NCT06018545
Study groupRoleWhat they receive
Readers30 readers will be recruited across four NHS trusts including ten general radiologists, fifteen emergency medicine clinicians, and five CT radiographers of varying seniority. Readers will interpret each scan first without, then with, the assistance of the AI tool, with an intervening 4-week washout period. Using a panel of neuroradiologists as ground truth, the stand-alone performance of qER will be assessed, and its impact on the readers' performance will be analysed as change in accuracy, mean review time per scan, and self-reported diagnostic confidence. Subgroup analyses will be performed by reader professional group, reader seniority, pathological finding, and neuroradiologist-rated difficulty.
Receives: Other: Reading
Ground truthersTwo Consultant neuroradiologists will independently review the images to establish the 'ground truth' findings on the CT scans which will be used as the reference standard. In the case of disagreement, a third senior neuroradiologist's opinion will be sought for arbitration. A difficulty score will be assigned to each scan by the ground truthers using a 5-point Likert scale.
Receives: Other: Ground truthing

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Reader performance: Sensitivity, specificity, comparative between with and without AI assistance.

    Reader performance will be evaluated as sensitivity, specificity, with and without AI assistance.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  2. Reader performance: Positive and negative predictive value, comparative between with and without AI assistance.

    Reader performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV), with and without AI assistance.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  3. Reader performance: Area Under Receiver Operating Characteristic Curve (AUROC), comparative between with and without AI assistance.

    Reader performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC), with and without AI assistance.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  4. Reader speed: Mean time taken to review a scan, with versus without AI assistance.

    Reader speed will be evaluated as the man time taken to review a scan, using time unite of seconds.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  5. Reader confidence: Self-reported diagnostic confidence on a 10 point visual analogue scale, with vs without AI assistance.

    On the reading platform (RAIQC), one of the questions asks the level of confidence that the participant has in their diagnostic opinion. The question offers a scale of 1 to 10, where 1 is not confident, and 10 is highly confident.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  6. qER (AI algorithm) performance: Sensitivity and specificity

    qER performance will be evaluated as sensitivity, specificity.

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  7. qER (AI algorithm) performance: Positive and negative predictive value.

    qER performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV).

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

  8. qER (AI algorithm) performance: Area Under Receiver Operating Characteristic Curve (AUROC).

    qER performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC)

    Measured: During 6 weeks, which is the period for reading or reviewing the cases/scans.

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted30 Aug 2023
  2. Study start1 Jun 2023
  3. Primary completion1 Sep 2023
  4. Study completion1 Jun 2025

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Observational
Observational model
Cohort
Time perspective
Retrospective
Enrolment
33 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT06018545
Sponsor's protocol ID
310995 - A
Acronym
AI-REACT
Status last verified
Nov 2025
First submitted
25 Jul 2023
Last updated
24 Nov 2025
Results posted
No
Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No

Publications

  • Juszczyk K, Ireland K, Thomas B, Kroon HM, Hollington P. Reduction in hospital admissions with an early computed tomography scan: results of an outpatient management protocol for uncomplicated acute diverticulitis. ANZ J Surg. 2019 Sep;89(9):1085-1090. doi: 10.1111/ans.15285. Epub 2019 Jun 17.

    PubMed 31206250
  • Chan J, Fan KS, Mak TLA, Loh SY, Ng SWY, Adapala R. Pre-Operative Imaging can Reduce Negative Appendectomy Rate in Acute Appendicitis. Ulster Med J. 2020 Jan;89(1):25-28. Epub 2020 Feb 18.

    PubMed 32218624
  • Greenhalgh R, Howlett DC, Drinkwater KJ. Royal College of Radiologists national audit evaluating the provision of imaging in the severely injured patient and compliance with national guidelines. Clin Radiol. 2020 Mar;75(3):224-231. doi: 10.1016/j.crad.2019.10.025. Epub 2019 Dec 19.

    PubMed 31864722
  • Lin E, Yuh EL. Computational Approaches for Acute Traumatic Brain Injury Image Recognition. Front Neurol. 2022 Mar 9;13:791816. doi: 10.3389/fneur.2022.791816. eCollection 2022.

    PubMed 35370919
  • Sheth SA, Giancardo L, Colasurdo M, Srinivasan VM, Niktabe A, Kan P. Machine learning and acute stroke imaging. J Neurointerv Surg. 2023 Feb;15(2):195-199. doi: 10.1136/neurintsurg-2021-018142. Epub 2022 May 25.

    PubMed 35613840
  • Yeo M, Tahayori B, Kok HK, Maingard J, Kutaiba N, Russell J, Thijs V, Jhamb A, Chandra RV, Brooks M, Barras CD, Asadi H. Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J Neurointerv Surg. 2021 Apr;13(4):369-378. doi: 10.1136/neurintsurg-2020-017099. Epub 2021 Jan 21.

    PubMed 33479036
  • Chilamkurthy S, Ghosh R, Tanamala S, Biviji M, Campeau NG, Venugopal VK, Mahajan V, Rao P, Warier P. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 2018 Dec 1;392(10162):2388-2396. doi: 10.1016/S0140-6736(18)31645-3. Epub 2018 Oct 11.

    PubMed 30318264
  • Guo Y, He Y, Lyu J, Zhou Z, Yang D, Ma L, Tan HT, Chen C, Zhang W, Hu J, Han D, Ding G, Liu S, Qiao H, Xu F, Lou X, Dai Q. Deep learning with weak annotation from diagnosis reports for detection of multiple head disorders: a prospective, multicentre study. Lancet Digit Health. 2022 Aug;4(8):e584-e593. doi: 10.1016/S2589-7500(22)00090-5. Epub 2022 Jun 17.

    PubMed 35725824
  • Lee JY, Kim JS, Kim TY, Kim YS. Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm. Sci Rep. 2020 Nov 25;10(1):20546. doi: 10.1038/s41598-020-77441-z.

    PubMed 33239711
  • Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, Suever JD, Geise BD, Patel AA, Moore GJ. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. 2018 Apr 4;1:9. doi: 10.1038/s41746-017-0015-z. eCollection 2018.

    PubMed 31304294
  • Davis MA, Rao B, Cedeno PA, Saha A, Zohrabian VM. Machine Learning and Improved Quality Metrics in Acute Intracranial Hemorrhage by Noncontrast Computed Tomography. Curr Probl Diagn Radiol. 2022 Jul-Aug;51(4):556-561. doi: 10.1067/j.cpradiol.2020.10.007. Epub 2020 Nov 15.

    PubMed 33243455
  • Wardlaw JM, Mair G, von Kummer R, Williams MC, Li W, Storkey AJ, Trucco E, Liebeskind DS, Farrall A, Bath PM, White P. Accuracy of Automated Computer-Aided Diagnosis for Stroke Imaging: A Critical Evaluation of Current Evidence. Stroke. 2022 Jul;53(7):2393-2403. doi: 10.1161/STROKEAHA.121.036204. Epub 2022 Apr 20.

    PubMed 35440170
  • Finck T, Moosbauer J, Probst M, Schlaeger S, Schuberth M, Schinz D, Yigitsoy M, Byas S, Zimmer C, Pfister F, Wiestler B. Faster and Better: How Anomaly Detection Can Accelerate and Improve Reporting of Head Computed Tomography. Diagnostics (Basel). 2022 Feb 10;12(2):452. doi: 10.3390/diagnostics12020452.

    PubMed 35204543
  • Warman R, Warman A, Warman P, Degnan A, Blickman J, Chowdhary V, Dash D, Sangal R, Vadhan J, Bueso T, Windisch T, Neves G. Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage. Cureus. 2022 Oct 13;14(10):e30264. doi: 10.7759/cureus.30264. eCollection 2022 Oct.

    PubMed 36381767
  • Dyer T, Chawda S, Alkilani R, Morgan TN, Hughes M, Rasalingham S. Validation of an artificial intelligence solution for acute triage and rule-out normal of non-contrast CT head scans. Neuroradiology. 2022 Apr;64(4):735-743. doi: 10.1007/s00234-021-02826-4. Epub 2021 Oct 8.

    PubMed 34623478

Questions about this trial

What is this clinical trial studying?

An observational study of Intracranial Hemorrhages and Acute Ischemic Stroke, run by Oxford University Hospitals NHS Trust at 4 sites in United Kingdom.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 1 inclusion and 3 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 24 Nov 2025. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 4 sites in United Kingdom. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Reader performance: Sensitivity, specificity, comparative between with and without AI assistance. (time frame: During 6 weeks, which is the period for reading or reviewing the cases/scans.). The study lists 8 primary outcomes in all. It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT06018545, which also lists 23 linked publications.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 24 Nov 2025.

Free account

Keep searching with a free account

You’ve used your 3 free searches for this session. Log in or create a free AzTrials account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in