Validation of an Artificial Intelligence Enabled Diagnostic Support Software (ArtiQ.Spiro) in Primary Care Spirometry Datasets - a Retrospective Analysis

This is a retrospective analysis of existing clinical datasets with consecutive spirometry collected in a primary care setting in the UK. Individual patient data will be included if the individual meets the study protocol eligibility criteria.

Clinical datasets will be de-identified (name, date of birth, address, postcode, occupation GP, ethnicity, medications data removed). Individuals will be identified by a study ID number. The de-identified datasets will contain the minimum information needed for spirometry and ArtiQ.Spiro - namely age, smoking history, height, weight, primary respiratory symptom - and the deidentified data exported from the primary care spirometry software.

ArtiQ.Spiro Evaluation (Index Tests for Diagnosis and Quality):

A deidentified dataset will be provided to a machine learning analyst who will apply the machine learning algorithm of ArtiQ.Spiro. For each individual, the algorithm will produce a preferred diagnosis (highest probability diagnostic category) (Index Test for Diagnosis) and an assessment of spirometry quality (Acceptable, Usable, Not Acceptable/Usable) (Index Test for Quality). No clinical information outside of the spirometry dataset nor reference standard data will be made available to the analyst.

Reference Standard for Diagnosis:

The clinical dataset, together with available primary care records and secondary care records, will be used by the senior members of the direct clinical care team (Consultants in Respiratory Medicine with an interest in integrated respiratory care) to provide a reference standard for diagnosis. For each individual, two consultants will provide a diagnosis independently and blinded to the index test (ArtiQ.Spiro) output. If there is agreement, this diagnosis will be taken as the reference standard for diagnosis for the individual. If there is no agreement, a third consultant outside the direct clinical care team will be provided with the same information (but deidentified) to act as final arbitrator.

Reference Standard for Quality:

A deidentified dataset will be provided to a specialist respiratory physiologist. He/she will grade the quality of each spirometric session according to the official American Thoracic Society / European Respiratory Society 2019 Technical Statement for Standardization of Spirometry. For each patient, the quality of the spirometry session will be graded according to one of three categories: Acceptable, Usable, Not Acceptable/Usable. This will act as the reference standard for quality. The respiratory physiologists will be blinded to the output from the Index Test (ArtiQ.Spiro). The respiratory physiologists will also record time taken to evaluate the dataset.

Data Analysis:

Data analysis will be performed by the research team who will be independent to the direct clinical care team and the respiratory physiologists who will be providing the reference standards for diagnosis and quality respectively.