Pulmonary DiseaseNCT04221100
xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation by Radiologists
A study testing Radiograph interpretation for pulmonary abnormalities in people with Pulmonary Disease, run by 1QB Information Technologies Inc..
Trial summary
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Pulmonary Disease
- What is being tested?
- Radiograph interpretation for pulmonary abnormalities
- Type of treatment
- Medical device
- Could I receive a placebo?
- Yes, some participants may receive a placebo
- How many people
- 0 participants
- Where
- No sites listed yet
- Study dates
- Mar 2021to May 2021
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Radiologist currently practicing at a Pureform Radiology clinic in Calgary, Canada.
Who cannot join
- Radiologists not currently practicing at a Pureform Radiology clinic in Calgary, Canada.
- Physicians currently practicing at a Pureform Radiology clinic in Calgary, Canada, but that are not radiologists by training.
About this study
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
Read the detailed description
To investigate the effect of xrAI for radiologists that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 10 to 14 radiologists to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group.
To analyse the effect of xrAI, the investigators will estimate the average treatment effect (ATE) for each group by comparing the performance of the treatment and control groups using randomization-based inference (Green and Gerber, 2012).
- Conditions
- Pulmonary Disease
- Being tested
- Radiograph interpretation for pulmonary abnormalities
- Medical subject headings
- Lung Diseases
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Radiograph interpretation for pulmonary abnormalitiesDevice
The pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax.
Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure.
Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis.
| Study group | Role | What they receive |
|---|---|---|
| Control | Placebo comparator | Participants will review the 500 chest radiographs without the assistance of xrAI Receives: Device: Radiograph interpretation for pulmonary abnormalities |
| Treatment | Experimental | Participants will review the 500 chest radiographs with the assistance of xrAI Receives: Device: Radiograph interpretation for pulmonary abnormalities |
What is being measured
The primary outcome is the main question the study is designed to answer.
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset.
Accuracy= (TP+FP) / (TP+FP+TN+FN)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal.
Precision= TP / (TP+FP)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset).
Recall= TP / (TP+FN)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Secondary outcomes (3)
Mean of radiologist accuracy (as defined in outcome 1)
Investigators will calculate the mean of the accuracy of all participants in each group.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of radiologist precision (as defined in outcome 2)
Investigators will calculate the mean of the precision of all participants in each group.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of radiologist recall (as defined in outcome 3)
Investigators will calculate the mean of the recall of all participants in each group.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted9 Jan 2020
- Study startMar 2021 (estimated)
- Primary completionMay 2021 (estimated)
- Study completionMay 2021 (estimated)
Why it stopped: Not enough participants enrolled in the study
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Pulmonary DiseaseStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind
- Who is masked
- Participant
- Enrolment
- 0 participants (actual)
Sponsor and investigators
- Lead sponsor
- 1QB Information Technologies Inc.
- Sponsor type
- Industry (pharma, biotech, devices)
- Responsible party
- Sponsor
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04221100
- Sponsor's protocol ID
- xrAI ClinicalEvaluation Rads
- Status last verified
- Feb 2021
- First submitted
- 6 Jan 2020
- Last updated
- 23 Feb 2021
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Undecided: The results of this study will be shared though we have not yet decided the format (publication in a medical journal, conference, white paper) not the type of information that will be shared (individual anonymized participant data or outcomes of the study). We will update this section once the IPD sharing plan is confirmed.
Study at a glance
From the registry record, last updated by the sponsor on 23 Feb 2021. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation by Radiologists
- ClinicalTrials.gov ID
- NCT04221100
- Acronym / sponsor ID
- xrAI ClinicalEvaluation Rads
- Conditions
- Pulmonary Disease
- Study type
- Interventional
- Recruitment status
- WithdrawnReason stopped: Not enough participants enrolled in the study
- Enrolment
- 0 participants (actual)
- Lead sponsor
- 1QB Information Technologies Inc.Industry (pharma, biotech, devices)
- Study start
- Mar 2021 (estimated)
- Primary completion
- May 2021 (estimated)
- Study completion
- May 2021 (estimated)
- First posted
- 9 Jan 2020 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
Detailed description
To investigate the effect of xrAI for radiologists that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 10 to 14 radiologists to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group.
To analyse the effect of xrAI, the investigators will estimate the average treatment effect (ATE) for each group by comparing the performance of the treatment and control groups using randomization-based inference (Green and Gerber, 2012).
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (participant)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Control | Placebo comparator | Device: Radiograph interpretation for pulmonary abnormalitiesParticipants will review the 500 chest radiographs without the assistance of xrAI |
| Treatment | Experimental | Device: Radiograph interpretation for pulmonary abnormalitiesParticipants will review the 500 chest radiographs with the assistance of xrAI |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (1 listed)
- Radiologist currently practicing at a Pureform Radiology clinic in Calgary, Canada.
Key exclusion criteria (2 listed)
- Radiologists not currently practicing at a Pureform Radiology clinic in Calgary, Canada.
- Physicians currently practicing at a Pureform Radiology clinic in Calgary, Canada, but that are not radiologists by training.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Radiograph interpretation for pulmonary abnormalitiesThe pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax. Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure. Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis. | Device | Control, Treatment |
Primary outcome
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset. Accuracy= (TP+FP) / (TP+FP+TN+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal. Precision= TP / (TP+FP) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset). Recall= TP / (TP+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
3 secondary outcomes
Mean of radiologist accuracy (as defined in outcome 1)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of radiologist precision (as defined in outcome 2)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of radiologist recall (as defined in outcome 3)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Results and safety
The study was withdrawn before enrolling participants, so there are no results.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 23 Feb 2021.
- Individual participant dataSharing planned: Undecided. The results of this study will be shared though we have not yet decided the format (publication in a medical journal, conference, white paper) not the type of information that will be shared (individual anonymized participant data or outcomes of the study). We will update this section once the IPD sharing plan is confirmed.
Related trials for Pulmonary Disease
RecruitingPhase 2
A Study of Brenipatide (LY3537031) in Adult Participants With Moderate-to-Severe Chronic Obstructive Pulmonary Disease (COPD)
Pulmonary Disease, Chronic Obstructive
RecruitingEarly Phase 1
Testing the Safety of Dapagliflozin Prior to Surgery for the Treatment of Patients With Stage IA Lung Adenocarcinoma
Lung Adenocarcinoma, Stage IA1 Lung Cancer American Joint Committee on Cancer (AJCC) v8
Enrolling by invitationObservational study
Screening for Hematology Branch Protocols
Hematologic Disease and Disorders, Donors
RecruitingObservational study
Evaluation for NCI Surgery Branch Clinical Research Protocols
Synovial Cell Cancer, Melanoma
RecruitingPhase 3
A Study to Learn About the Study Medicine Called PF-08046054/SGN-PDL1V Versus Docetaxel in Adult Participants With Previously-Treated Programmed Cell Death Ligand 1 (PD-L1) Positive Non-Small-Cell Lung Cancer (NSCLC)
Non-small Cell Carcinoma, Non-Small Cell Lung Cancer Metastatic
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
Questions about this trial
What is this clinical trial studying?
A study testing Radiograph interpretation for pulmonary abnormalities in people with Pulmonary Disease, run by 1QB Information Technologies Inc..
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: withdrawn, as of the sponsor's last update on 23 Feb 2021. Status can change between updates, so confirm with the study contact before making plans.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. Some participants in this study may receive a placebo. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study was withdrawn before enrolling participants, so there are no results.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Number of abnormalities identified divided by number of total of images analyzed (accuracy) (time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 23 Feb 2021.