Pulmonary DiseaseNCT04153045
xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation
A study testing Radiograph interpretation for pulmonary abnormalities in people with Pulmonary Disease, run by 1QB Information Technologies Inc. at 1 site in Canada.
Trial summary
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Pulmonary Disease
- What is being tested?
- Radiograph interpretation for pulmonary abnormalities
- Type of treatment
- Medical device
- Could I receive a placebo?
- Yes, some participants may receive a placebo
- How many people
- 28 participants
- Where
- 1 siteCanada
- Study dates
- 12 Feb 2020to 21 Apr 2020
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Family doctors currently practicing at a site within Saskatchewan Health Authority
- Nurse practitioners currently practicing at a site within Saskatchewan Health Authority
- Emergency medicine physicians currently practicing at a site within Saskatchewan Health Authority
- Internists currently practicing at a site within Saskatchewan Health Authority
- Pulmonologists currently practicing at a site within Saskatchewan Health Authority
- Radiologists currently practicing at a site within Saskatchewan Health Authority
Who cannot join
- Physicians not currently practicing at a site within Saskatchewan Health Authority.
- Physicians currently practicing at a site within Saskatchewan Health Authority as neither family doctors, nurse practitioners, emergency medicine physicians, internists, pulmonologists, nor radiologists.
Trial locations and contacts
1 site in 1 country.
Canada1 site
1QB Information Technologies Inc
Vancouver, British Columbia, V6E 4B1
About this study
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by family doctors, nurse practitioners, emergency medicine physicians, internists, pulmonologists, and radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
Read the detailed description
To investigate the effect of xrAI for clinicians that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 36 clinicians to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group within each group of clinicians (family doctor, nurse practitioner, emergency medicine physician, internist, pulmonologist, radiologist). Within each group of clinicians, investigators will randomly assign half to treatment or control group. This randomized complete block design ensures that an equal number of each group of clinical professionals are represented in the treatment and control groups.
To analyse the effect of xrAI, the investigators will estimate the average treatment effect (ATE) for each subgroup by comparing the performance of the treatment and control groups using randomization-based inference (Green and Gerber, 2012).
- Conditions
- Pulmonary Disease
- Being tested
- Radiograph interpretation for pulmonary abnormalities
- Medical subject headings
- Lung Diseases
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Radiograph interpretation for pulmonary abnormalitiesDevice
The pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax.
Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure.
Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis.
| Study group | Role | What they receive |
|---|---|---|
| Control | Placebo comparator | Participants will review the 500 chest radiographs without the assistance of xrAI Receives: Device: Radiograph interpretation for pulmonary abnormalities |
| Treatment | Experimental | Participants will review the 500 chest radiographs with the assistance of xrAI Receives: Device: Radiograph interpretation for pulmonary abnormalities |
What is being measured
The primary outcome is the main question the study is designed to answer.
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset.
Accuracy= (TP+FP) / (TP+FP+TN+FN)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal.
Precision= TP / (TP+FP)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset).
Recall= TP / (TP+FN)
TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Secondary outcomes (3)
Mean of physician accuracy (as defined in outcome 1)
Investigators will calculate the mean of the accuracy of all participants in a group and the mean of the accuracy of all participants across all the groups.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of physician precision (as defined in outcome 2)
Investigators will calculate the mean of the precision of all participants in a group and the mean of the precision of all participants across all the groups.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of physician recall (as defined in outcome 3)
Investigators will calculate the mean of the recall of all participants in a group and the mean of the recall of all participants across all the groups.
Measured: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted6 Nov 2019
- Study start12 Feb 2020
- Primary completion21 Apr 2020
- Study completion21 Apr 2020
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Pulmonary DiseaseStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind
- Who is masked
- Participant
- Enrolment
- 28 participants (actual)
Sponsor and investigators
- Lead sponsor
- 1QB Information Technologies Inc.
- Sponsor type
- Industry (pharma, biotech, devices)
- Collaborators
- Saskatchewan Health Authority - Regina Area
- Responsible party
- Sponsor
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04153045
- Sponsor's protocol ID
- xrAI ClinicalEvaluation Stage1
- Status last verified
- Sep 2020
- First submitted
- 4 Nov 2019
- Last updated
- 22 Sep 2020
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Undecided: The results of this study will be shared though we have not yet decided the format (publication in a medical journal, conference, white paper) not the type of information that will be shared (individual anonymized participant data or outcomes of the study). We will update this section once the IPD sharing plan is confirmed.
Study at a glance
From the registry record, last updated by the sponsor on 22 Sep 2020. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- xrAI - Improving Quality and Efficiency in Chest Radiograph Interpretation
- ClinicalTrials.gov ID
- NCT04153045
- Acronym / sponsor ID
- xrAI ClinicalEvaluation Stage1
- Conditions
- Pulmonary Disease
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 28 participants (actual)
- Lead sponsor
- 1QB Information Technologies Inc.Industry (pharma, biotech, devices)
- Collaborators
- Saskatchewan Health Authority - Regina Area
- Study start
- 12 Feb 2020 (actual)
- Primary completion
- 21 Apr 2020 (actual)
- Study completion
- 21 Apr 2020 (actual)
- First posted
- 6 Nov 2019 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
xrAI (pronounced "X-ray") serves as a clinical assistance tool for trained clinical professionals who are interpreting chest radiographs. The tool is designed as a quality control and adjunct, limited, clinical decision support tool, and does not replace the role of clinical professionals. It highlights areas on chest radiographs for review by an interpreting clinician.
The objective of this study is to utilize machine learning and artificial intelligence algorithms (xrAI) to improve the quality and efficiency in the interpretation of chest radiographs by family doctors, nurse practitioners, emergency medicine physicians, internists, pulmonologists, and radiologists.
The hypothesis is that the addition of xrAI's analysis will reduce inter-observer variability in the interpretation of chest radiographs and increase participants' sensitivity, recall, and accuracy in pulmonary abnormality screening.
Detailed description
To investigate the effect of xrAI for clinicians that interpret chest radiographs as part of their daily responsibilities, the investigators have designed a randomized control trial.
The pulmonary abnormalities detected by xrAI and included in the definition of abnormal are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), and pneumothorax.
To assess the causal effect of xrAI the investigators randomly assign 36 clinicians to either treatment (x-ray images processed by xrAI) or control (no xrAI processing) groups. Participants will only review images once. Each participant will perform 500 radiograph interpretations in total.
Participants in the control group will be asked to interpret the same 500 images without xrAI's analysis.
To increase the precision of the estimate and better investigate potential differences between clinical professionals, investigators block randomize the assignment of treatment or control group within each group of clinicians (family doctor, nurse practitioner, emergency medicine physician, internist, pulmonologist, radiologist). Within each group of clinicians, investigators will randomly assign half to treatment or control group. This randomized complete block design ensures that an equal number of each group of clinical professionals are represented in the treatment and control groups.
To analyse the effect of xrAI, the investigators will estimate the average treatment effect (ATE) for each subgroup by comparing the performance of the treatment and control groups using randomization-based inference (Green and Gerber, 2012).
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (participant)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Control | Placebo comparator | Device: Radiograph interpretation for pulmonary abnormalitiesParticipants will review the 500 chest radiographs without the assistance of xrAI |
| Treatment | Experimental | Device: Radiograph interpretation for pulmonary abnormalitiesParticipants will review the 500 chest radiographs with the assistance of xrAI |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (6 listed)
- Family doctors currently practicing at a site within Saskatchewan Health Authority
- Nurse practitioners currently practicing at a site within Saskatchewan Health Authority
- Emergency medicine physicians currently practicing at a site within Saskatchewan Health Authority
- Internists currently practicing at a site within Saskatchewan Health Authority
- Pulmonologists currently practicing at a site within Saskatchewan Health Authority
Key exclusion criteria (2 listed)
- Physicians not currently practicing at a site within Saskatchewan Health Authority.
- Physicians currently practicing at a site within Saskatchewan Health Authority as neither family doctors, nurse practitioners, emergency medicine physicians, internists, pulmonologists, nor radiologists.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Radiograph interpretation for pulmonary abnormalitiesThe pulmonary abnormalities detected by xrAI and included in the definition of abnormality are as follows: any linear scar or fibrosis, atelectasis, consolidation, abscess or cavity, nodule, pleural effusion, severe cases of emphysema and COPD (mild cases with hyperinflation but not significant emphysema are not flagged), pneumothorax. Participants in the treatment group will interpret 500 images presented alongside the results of xrAI's processing in a dark room and asked to categorize each image into one of the following categories: lungs are clear, at least one pulmonary abnormality is present, not sure. Participants in the control group will be asked to interpret the same 500 images as the treatment group but without xrAI's analysis. | Device | Control, Treatment |
Primary outcome
Number of abnormalities identified divided by number of total of images analyzed (accuracy)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Accuracy is defined as the ratio of the images where the physician's prediction matched the labels of the dataset. Accuracy= (TP+FP) / (TP+FP+TN+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the total of abnormalities identified (precision)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Precision is defined as the probability of a radiograph being abnormal if a physician makes the determination that it is abnormal. Precision= TP / (TP+FP) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
Number of true abnormalities identified divided by the sum of true abnormalities identified and abnormalities missed (recall)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Recall is defined as the probability of a physician catching an abnormality in an image if one exists (based on the labels of the dataset). Recall= TP / (TP+FN) TP (true positives) = cases interpreted as abnormal that are abnormal; FP (false positives) = cases wrongly interpreted to be abnormal; TN (true negatives) = cases correctly interpreted to be normal; FN (false negatives) = abnormal cases wrongly interpreted as normal.
3 secondary outcomes
Mean of physician accuracy (as defined in outcome 1)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of physician precision (as defined in outcome 2)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Mean of physician recall (as defined in outcome 3)
Time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 22 Sep 2020.
- Individual participant dataSharing planned: Undecided. The results of this study will be shared though we have not yet decided the format (publication in a medical journal, conference, white paper) not the type of information that will be shared (individual anonymized participant data or outcomes of the study). We will update this section once the IPD sharing plan is confirmed.
Related trials for Pulmonary Disease
Active, not recruitingPhase 1/2
A Beta-only IL-2 ImmunoTherapY Study
Advanced Solid Tumor, Unresectable Solid Tumor
RecruitingPhase 2/3
Phase 2 Trial of Adagrasib Monotherapy and in Combination With Pembrolizumab and a Phase 3 Trial of Adagrasib in Combination in Patients With a KRAS G12C Mutation KRYSTAL-7
Advanced Non-Small Cell Lung Cancer, Metastatic Non-Small Cell Lung Cancer
RecruitingPhase 3
A Clinical Trial of Adjuvant Intismeran (V940) With or Without Pembrolizumab Coformulated With Berahyaluronidase Alfa (MK-3475A) in High-Risk Stage I Non-Small Cell Lung Cancer (V940-014)
Non-small Cell Lung Cancer
RecruitingObservational study
Asthma and Primary Arterial Hypertension: an Inflammatory Link?
Asthma Bronchial
Not yet recruitingObservational study
Early Switch in Moderate ARDS
Acute Respiratory Distress Syndrome (ARDS)
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Radiograph interpretation for pulmonary abnormalities in people with Pulmonary Disease, run by 1QB Information Technologies Inc. at 1 site in Canada.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 6 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 22 Sep 2020. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Canada. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. Some participants in this study may receive a placebo. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Number of abnormalities identified divided by number of total of images analyzed (accuracy) (time frame: Time needed to analyze 500 images. Participants will be asked to completed the exercise within 2 weeks.). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 22 Sep 2020.