Automated Detection of Malarial Retinopathy in Patients Diagnosed With Cerebral Malaria

A prospective, pivotal, multi-center clinical study was conducted at 12 healthcare facilities (hospitals and clinics) across two countries in Africa, Malawi and Nigeria; with the goal to validate the performance of ASPIRE in detecting malarial retinopathy (MR) in the retinal images of diagnosed cerebral malaria (CM) patients. The clinical evaluation was conducted in compliance with 21 CFR parts 50, 56, and 812, under the institutional review board (IRB) and/or independent ethics committee (IEC) approval provided by an internal IRB or IEC of the respective institution or clinical facility, and in accordance with good clinical practice (GCP). The internal IRB or IEC's that reviewed the study application are in compliance with the definition in 21 CFR 812.3(t) and GCP. Informed consent was obtained from all enrolled subjects.

The study recruited and imaged N = 834 pediatric patients below 21 years of age with clinical diagnosis of cerebral malaria (intended target population). Of these, 141 patient-cases were rejected either by human graders (reference standard) or by the image quality analysis (IQA) algorithm due to inadequate image quality in the patient-case, or had a missing ASPIRE output for MR detection. Remaining N=693 patient-cases with adequate image quality were used for the validation of MR detection algorithm.

The clinical study relied on the out-of-United States (OUS) clinical data due to the minuscule prevalence of malaria in the US (on an average, 2000 malaria cases are detected per year in the US, or 0.0006% of the US population is diagnosed with malaria). Malaria is not endemic and does not regularly occur or spread in the US. The greatest prevalence and risk of CM and malaria is in the sub-Saharan region of Africa. Over 90% of malaria cases in Africa are due to the Plasmodium falciparum parasite, which causes CM and MR; for which the clinical sites and facilities in Sub-Saharan African countries were engaged to enroll CM patients and collect clinical validation data. The ASPIRE software application used in this clinical study conducted outside the United States is identical to the ASPIRE software application developed in the US.

The study recruited N = 834 pediatric patients clinically diagnosed with cerebral malaria. The study population represented the intended population for the use of this device. All recruited patients were imaged with VistaView retinal camera operated by the users who were minimally skilled healthcare providers such as nurses or medical technicians. Before any patient-participant was recruited and imaged using VistaView camera, ASPIRE users underwent a one-time standardized training program on how to acquire images, how to reattempt imaging and improve image quality if ASPIRE gave an inadequate image quality result, and how to submit images for analysis to ASPIRE. All users underwent the same standardized training program using VistaView camera, irrespective of their educational qualifications, clinical experience, clinical environment, and the type of clinical setting. No additional training was provided to users for the duration of the study.

Study population: The target/intended population for ASPIRE is pediatric population under 21 years of age, clinically diagnosed with cerebral malaria. The enrollment was conducted sequentially at all clinical sites. The study investigators, local principal investigators, and study coordinators were trained to comply with GCP and study protocol and were committed to conduct the study accordingly, although no written commitments were obtained.

Demographics: The study aims to collect data that spans the gender, race, and ethnicity distribution from subjects that meet the inclusion criteria.

Enrollment and informed consent: Subjects were enrolled at the time of diagnosis of CM. If the patients met the inclusion criteria, the patient's parents or guardian were approached with information regarding the study. If parents/guardians agreed to participate, clinic personnel followed and gained informed consent. No subject sampling was done to enroll subjects in the clinical study. Thus, enrollment was done in consecutive order as subjects clinically diagnosed with CM were identified and consent was obtained. No incentives were provided to subjects for study participation, except transportation as needed.

Description of protocol/study design: This clinical study is designed to collect data to establish the safety and effectiveness of ASPIRE when compared to the clinical reference standard. A total of 834 subjects clinically diagnosed with CM were recruited to participate in this study. These subjects were enrolled through 12 clinical sites. The study procedure was as follows:

1. Pupillary light response is checked and then pupils are dilated with 0.5% tropicamide or 10% phenylephrine.
2. A trained user (imager) captures retinal images using VistaView camera (at least 8 photos per eye: recommended as 4 optic disc centered, 4 macula centered, and 2 peripheral retinal photos).
3. The retinal images captured by VistaView camera are transmitted to a smartphone storage folder automatically.
4. The ASPIRE application accesses the stored retinal images and analyzes them for image quality using ASPIRE analysis software. The software either accepts or rejects each image based on adequate or inadequate image quality, respectively. A graphical display on smartphone is shown to the user that displays the captured image and whether it was accepted (passed image quality analysis) or rejected (failed image quality analysis).
5. The user takes additional retinal images if ASPIRE identifies that a sufficient number of adequate quality images is not available for processing, and then submits them again for ASPIRE's processing.
6. Once a sufficient number of adequate quality retinal images is available, images are analyzed for detection of malarial retinopathy (MR) by ASPIRE analysis software.
7. The analysis of retinal images for image quality and MR detection is fully automatic and does not require manual inputs or intervention from the user.

Reference Standard: Using an internationally recognized grading system developed by the Malarial Retinopathy grading consensus group at the University of Liverpool (UoL), United Kingdom; the retinal images were independently reviewed and graded to determine the presence of malarial retinopathy lesions such as retinal whitening and hemorrhages, and the quality of retinal images. For each participating subject, the retinal images were graded independently by three experienced and validated graders. First, two ophthalmologists graded the images, and in case of significant differences in the two independent gradings, the third ophthalmologist independently adjudicated the grades. The ophthalmologists demonstrate over 15 years of experience in working with cerebral malaria patients and/or in grading and screening for malarial retinopathy lesions, and all of them received a training course in the grading of malarial retinopathy as per the MR grading protocol developed by the Malarial Retinopathy grading consensus group at the University of Liverpool (UoL), United Kingdom. Throughout the study, the graders and study staff were masked to the patient history, ASPIRE algorithm training, and/or ASPIRE algorithm outputs/results. The grading of retinal images by validated ophthalmic graders was used as the clinical reference standard for detecting the presence of malarial retinopathy, based on the definition of reference standard by the US FDA as the "best available" method for establishing the presence or absence of the target condition, that is acceptable within the medical, laboratory, and regulatory community. The reference standard for each subject was categorized as:

• Positive, if retinal whitening and/or hemorrhages were detected (MR detected)
• Negative, if retinal whitening and hemorrhages were not detected (MR not detected)
• Ungradable, if image quality was inadequate/questionable to grade MR (Inadequate photo quality)

Study Hypothesis: The study investigators aim to demonstrate MR detection performance of ASPIRE on a validation dataset collected with sequential enrollment of clinically diagnosed CM patients, with or without malarial retinopathy. About 61% of CM-diagnosed patients show manifestation of CM in retina, in form of retinal lesions, called malarial retinopathy (MR). The remaining 39% of CM-diagnosed patients may not exhibit MR. So, the prevalence of MR in CM-diagnosed patients is 61%. The study hypothesis is that the sensitivity and specificity of detecting MR using ASPIRE's MR detection software is non-inferior to the clinical reference standard, which is defined as the reading of retinal images for MR detection by ophthalmic specialists such as ophthalmologists.

Statistical power and sample size calculation: To establish that ASPIRE's software-based test is non-inferior to the standard test (ophthalmologist's reading of images) that exhibits sensitivity of 89% and specificity of 87%; the investigators require a minimum number of disease-positive (MR-positive) "cases" of 366, and minimum number of disease-negative (MR-negative) "controls" of 234. In total, the investigators require a minimum sample size of 600 CM-diagnosed patients to validate the proposed ASPIRE test. ASPIRE's minimum required performance goals/thresholds are defined at 82% for sensitivity and 80% for specificity, reflecting the requirement to prove that ASPIRE's software-based test is non-inferior to the standard test (with a non-inferiority margin of 7%), and based on anticipated enrollment numbers and prespecified regulatory requirements.

The investigators propose to validate ASPIRE's software-based test performance on the MR-positive and MR-negative patients sample and verify if the resulting sensitivity/specificity is within the margin of difference. Upon verification, it can be concluded that the software-based test (ASPIRE) is non-inferior to the standard test within a non-inferiority margin of 7%.

ASPIRE processing: All images for each patient-case are processed by ASPIRE software. As per ASPIRE's image quality analysis (IQA) protocol, each image is processed by the IQA algorithm to determine a probability score for the image to have adequate quality. If the image is found to be inadequate quality, it is rejected from further processing for malarial retinopathy detection. The algorithm for MR detection requires at least four images with clinically acceptable/adequate quality to complete the processing of a patient-case. If the IQA algorithm does not find at least four images of adequate quality in a patient-case, then the case is rejected from the processing for malarial retinopathy (MR) detection, until additional images are captured and submitted for processing.

The MR detection algorithm uses the adequate quality case to determine the probability of detecting MR by combining individual MR probability scores of all adequate quality images obtained from the patient. The result for each clinically diagnosed CM patient is either "Malarial retinopathy detected", "Malarial retinopathy not detected", or "Inadequate photo quality" when adequate-quality images are not present. The ASPIRE results for each patient are compared to the clinical reference standard.

Study Data: The study recruited and imaged N = 834 patients with clinical diagnosis of cerebral malaria, and it was mutually exclusive from the patient data previously used for training the ASPIRE algorithm. Out of 834 patient-cases, 141 cases were rejected either due to inadequate image quality in patient-cases as determined by the human graders (33), or IQA algorithm (63), or the case had a missing ASPIRE output for MR detection (45). The adequate quality retinal image data of N=693 patients was used to validate the MR detection algorithm. This validation data included 394 MR-positive patients and 299 MR-negative patients, per the clinical reference standard. The prevalence of MR in inadequate quality data (determined by IQA algorithm) was 61%, against the clinical reference standard determined by human graders.