SepsisPhase 2HindSight P2NCT04005001
Machine Learning Sepsis Alert Notification Using Clinical Data
A Phase 2 study testing HindSight and InSight in people with Sepsis and Severe Sepsis, run by Dascena at 3 sites in United States.
Trial summary
Machine learning is a powerful method to create clinical decision support (CDS) tools, when training labels reflect the desired alert behavior. In our Phase I work for this project, we developed HindSight, an encoding software that was designed to examine discharged patients' electronic health records (EHRs), identify… Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Sepsis, Severe Sepsis, Septic Shock
- What is being tested?
- HindSight and InSight
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 37,986 participants
- Where
- 3 sitesUnited States
- Study dates
- 25 Sep 2021to 31 Aug 2022
- Phase
- Phase 2
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- During the study period, all patients over the age of 18 presenting to the emergency department or admitted to an inpatient unit at the participating facilities will automatically be enrolled in the study, until the enrollment target for the study is met
Who cannot join
- Patients under the age of 18
- Prisoners
Trial locations and contacts
3 sites in 1 country.
United States3 sites, 3 recruiting
- Recruiting
Cooper University Health Care
Camden, New Jersey, 08103
Sharad Patel
Snehal Gandhi
- Recruiting
Cape Regional Medical Center
Cape May, New Jersey, 08210
Andrea McCoy
- Recruiting
Baystate Health
Springfield, Massachusetts, 01199
Gregory Braden
About this study
Official title: Using Clinical Treatment Data in a Machine Learning Approach for Sepsis Detection
Machine learning is a powerful method to create clinical decision support (CDS) tools, when training labels reflect the desired alert behavior. In our Phase I work for this project, we developed HindSight, an encoding software that was designed to examine discharged patients' electronic health records (EHRs), identify clinicians' sepsis treatment decisions and patient outcomes, and pass those labeled outcomes and treatment decisions to an online algorithm for retraining of our machine-learning-based CDS tool for real-time sepsis alert notification, InSight. HindSight improved the performance of InSight sepsis alerts in retrospective work. In this study, we propose to assess the clinical utility of HindSight by conducting a multicenter prospective randomized controlled trial (RCT) for more accurate sepsis alerts.
Read the detailed description
We will evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis alert notification. HindSight identifies clinicians' sepsis-related decisions in the electronic health records of former patients and passes those events to InSight, thus supplying InSight with labeled examples of true positive sepsis cases for retraining. In our retrospective work, we have shown that HindSight enables InSight to adapt to site-specific deviations of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. In our Phase I work, HindSight achieved an area under the receiver-operating characteristic (AUROC) of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). In Aim 1, we will prospectively validate HindSight's performance on real-time patient data streams in three diverse hospitals non-interventionally. In Aim 2, we will evaluate the effect of the tool in a prospective, interventional RCT. HindSight will first be evaluated by live deployment at four academic and community hospitals, during which time it will not provide alerts of future sepsis onset. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm).
- Conditions
- Sepsis, Severe Sepsis, Septic Shock
- Being tested
- HindSight, InSight
- Keywords
- Sepsis, Machine learning algorithm, Clinical decision support
- Medical subject headings
- Sepsis, Shock, Septic
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
HindSightOther
Also called HindSight-Clinical decision support (CDS) system for sepsis alert notification
HindSight will examine the dynamic trends of clinical measurements taken from a patient's EHR and analyzes correlations between vital signs to alert for the onset of sepsis.This machine learning based tool is optimized by encoder and utilizes periodic retraining to improve its performance over time.
InSightOther
Compared to the ability of the InSight software's recognition of sepsis onset to HindSight's performance. The study determines if the HindSight software has equivalent or better performance than the InSight software.
| Study group | Role | What they receive |
|---|---|---|
| Experimental | Experimental | The experimental arm will involve patients monitored by HindSight. Receives: Other: HindSight |
| Control | Active comparator | The control arm will involve patients monitored by InSight. Receives: Other: InSight |
What is being measured
The primary outcome is the main question the study is designed to answer.
Rate of reduction in false alerts
The primary outcome measure of interest will be false alert reduction. Successful completion of Aim 1 will be demonstrated by a positive predictive value (PPV) in a live clinical setting for which the lower bound of the 95% confidence interval meets or exceeds the benchmark from prior retrospective studies. Meeting the retrospective PPV benchmark indicates that prospective CDS quality reflects retrospective CDS quality, and is sufficiently high to reduce alarm fatigue and improve clinical utility. Success of Aim 2 is contingent upon achieving a 15% relative reduction in false alerts when comparing between the two treatment arms (p < 0.05; Fisher's Exact Test).
Measured: Through study completion, human subjects involvement will occur for an average of eight months
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted2 Jul 2019
- Study start25 Sep 2021
- Primary completion31 Aug 2022 (estimated)
- Study completion31 Aug 2022 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (5)Papers the sponsor has linked to this study.
- Other trials for SepsisStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase 2
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Triple blind
- Who is masked
- Participant, Care provider, Investigator
- Enrolment
- 37,986 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Dascena
- Sponsor type
- Industry (pharma, biotech, devices)
- Collaborators
- Cape Regional Medical Center, Cooper University Medical Center, Baystate Health, National Institute on Alcohol Abuse and Alcoholism (NIAAA)
- Responsible party
- Sponsor
- Investigators
- Jana Hoffman, PhD, Principal investigator, Dascena
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04005001
- Sponsor's protocol ID
- 19-569185
- Other IDs
- 2R44AA030000-02
- Acronym
- HindSight P2
- Status last verified
- Apr 2022
- First submitted
- 28 Jun 2019
- Last updated
- 3 May 2022
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Desautels T, Calvert J, Hoffman J, Mao Q, Jay M, Fletcher G, Barton C, Chettipally U, Kerem Y, Das R. Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting. Biomed Inform Insights. 2017 Jun 12;9:1178222617712994. doi: 10.1177/1178222617712994. eCollection 2017.
PubMed 28638239Calvert J, Mao Q, Rogers AJ, Barton C, Jay M, Desautels T, Mohamadlou H, Jan J, Das R. A computational approach to mortality prediction of alcohol use disorder inpatients. Comput Biol Med. 2016 Aug 1;75:74-9. doi: 10.1016/j.compbiomed.2016.05.015. Epub 2016 May 24.
PubMed 27253619Calvert JS, Price DA, Barton CW, Chettipally UK, Das R. Discharge recommendation based on a novel technique of homeostatic analysis. J Am Med Inform Assoc. 2017 Jan;24(1):24-29. doi: 10.1093/jamia/ocw014. Epub 2016 Mar 28.
PubMed 27026611Calvert J, Mao Q, Hoffman JL, Jay M, Desautels T, Mohamadlou H, Chettipally U, Das R. Using electronic health record collected clinical variables to predict medical intensive care unit mortality. Ann Med Surg (Lond). 2016 Sep 6;11:52-57. doi: 10.1016/j.amsu.2016.09.002. eCollection 2016 Nov.
PubMed 27699003Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017 Nov 9;4(1):e000234. doi: 10.1136/bmjresp-2017-000234. eCollection 2017.
PubMed 29435343
Study at a glance
From the registry record, last updated by the sponsor on 3 May 2022. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Using Clinical Treatment Data in a Machine Learning Approach for Sepsis Detection
- ClinicalTrials.gov ID
- NCT04005001
- Acronym / sponsor ID
- HindSight P2 / 19-569185
- Conditions
- Sepsis, Severe Sepsis, Septic Shock
- Keywords
- Sepsis, Machine learning algorithm, Clinical decision support
- Phase
- Phase 2
- Study type
- Interventional
- Recruitment status
- Unknown status
- Enrolment
- 37,986 participants (estimated)
- Lead sponsor
- DascenaIndustry (pharma, biotech, devices)
- Collaborators
- Cape Regional Medical Center, Cooper University Medical Center, Baystate Health, National Institute on Alcohol Abuse and Alcoholism (NIAAA)
- Study start
- 25 Sep 2021 (actual)
- Primary completion
- 31 Aug 2022 (estimated)
- Study completion
- 31 Aug 2022 (estimated)
- First posted
- 2 Jul 2019 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Machine learning is a powerful method to create clinical decision support (CDS) tools, when training labels reflect the desired alert behavior. In our Phase I work for this project, we developed HindSight, an encoding software that was designed to examine discharged patients' electronic health records (EHRs), identify clinicians' sepsis treatment decisions and patient outcomes, and pass those labeled outcomes and treatment decisions to an online algorithm for retraining of our machine-learning-based CDS tool for real-time sepsis alert notification, InSight. HindSight improved the performance of InSight sepsis alerts in retrospective work. In this study, we propose to assess the clinical utility of HindSight by conducting a multicenter prospective randomized controlled trial (RCT) for more accurate sepsis alerts.
Detailed description
We will evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis alert notification. HindSight identifies clinicians' sepsis-related decisions in the electronic health records of former patients and passes those events to InSight, thus supplying InSight with labeled examples of true positive sepsis cases for retraining. In our retrospective work, we have shown that HindSight enables InSight to adapt to site-specific deviations of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. In our Phase I work, HindSight achieved an area under the receiver-operating characteristic (AUROC) of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). In Aim 1, we will prospectively validate HindSight's performance on real-time patient data streams in three diverse hospitals non-interventionally. In Aim 2, we will evaluate the effect of the tool in a prospective, interventional RCT. HindSight will first be evaluated by live deployment at four academic and community hospitals, during which time it will not provide alerts of future sepsis onset. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm).
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Triple blind (participant, care provider, investigator)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Experimental | Experimental | Other: HindSightThe experimental arm will involve patients monitored by HindSight. |
| Control | Active comparator | Other: InSightThe control arm will involve patients monitored by InSight. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (1 listed)
- During the study period, all patients over the age of 18 presenting to the emergency department or admitted to an inpatient unit at the participating facilities will automatically be enrolled in the study, until the enrollment target for the study is met
Key exclusion criteria (2 listed)
- Patients under the age of 18
- Prisoners
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| HindSightAlso: HindSight-Clinical decision support (CDS) system for sepsis alert notificationHindSight will examine the dynamic trends of clinical measurements taken from a patient's EHR and analyzes correlations between vital signs to alert for the onset of sepsis.This machine learning based tool is optimized by encoder and utilizes periodic retraining to improve its performance over time. | Other | Experimental |
| InSightCompared to the ability of the InSight software's recognition of sepsis onset to HindSight's performance. The study determines if the HindSight software has equivalent or better performance than the InSight software. | Other | Control |
Primary outcome
Rate of reduction in false alerts
Time frame: Through study completion, human subjects involvement will occur for an average of eight months
The primary outcome measure of interest will be false alert reduction. Successful completion of Aim 1 will be demonstrated by a positive predictive value (PPV) in a live clinical setting for which the lower bound of the 95% confidence interval meets or exceeds the benchmark from prior retrospective studies. Meeting the retrospective PPV benchmark indicates that prospective CDS quality reflects retrospective CDS quality, and is sufficiently high to reduce alarm fatigue and improve clinical utility. Success of Aim 2 is contingent upon achieving a 15% relative reduction in false alerts when comparing between the two treatment arms (p < 0.05; Fisher's Exact Test).
Results and safety
Results are not available in the source record.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 3 May 2022.
- Background: PubMed 28638239Desautels T, Calvert J, Hoffman J, Mao Q, Jay M, Fletcher G, Barton C, Chettipally U, Kerem Y, Das R. Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting. Biomed Inform Insights. 2017 Jun 12;9:1178222617712994. doi: 10.1177/1178222617712994. eCollection 2017.
- Background: PubMed 27253619Calvert J, Mao Q, Rogers AJ, Barton C, Jay M, Desautels T, Mohamadlou H, Jan J, Das R. A computational approach to mortality prediction of alcohol use disorder inpatients. Comput Biol Med. 2016 Aug 1;75:74-9. doi: 10.1016/j.compbiomed.2016.05.015. Epub 2016 May 24.
- Background: PubMed 27026611Calvert JS, Price DA, Barton CW, Chettipally UK, Das R. Discharge recommendation based on a novel technique of homeostatic analysis. J Am Med Inform Assoc. 2017 Jan;24(1):24-29. doi: 10.1093/jamia/ocw014. Epub 2016 Mar 28.
- Background: PubMed 27699003Calvert J, Mao Q, Hoffman JL, Jay M, Desautels T, Mohamadlou H, Chettipally U, Das R. Using electronic health record collected clinical variables to predict medical intensive care unit mortality. Ann Med Surg (Lond). 2016 Sep 6;11:52-57. doi: 10.1016/j.amsu.2016.09.002. eCollection 2016 Nov.
- Background: PubMed 29435343Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017 Nov 9;4(1):e000234. doi: 10.1136/bmjresp-2017-000234. eCollection 2017.
- Individual participant dataSharing planned: No
Related trials for Sepsis
RecruitingPhase 3
Selective Antibiotics When Symptoms Develop Versus Universal Antibiotics for Preterm Neonates
Sepsis, PROM, Preterm (Pregnancy)
RecruitingObservational study
Metabolic and Functional Study of γδ T Cells in Critically Ill Patients
Sepsis, Critical Illness
Not yet recruiting
Blood Purification in Septic Children
Sepsis, Blood Purification
Active, not recruitingObservational study
Biomarker-enhanced Artificial Intelligence Based Pediatric Sepsis Screening Tool
Sepsis
RecruitingPhase 3
A Trial of Adjunctive Fludrocortisone (50mcg, 100mcg or Placebo) in Critically Ill Patients With Septic Shock Receiving Hydrocortisone, Evaluating Survival and Functional Recovery and the Influence of Transcriptomic Response
Septic Shock
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Jana Hoffman, PhD principal investigator
Dascena
3 sites in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A Phase 2 study testing HindSight and InSight in people with Sepsis and Severe Sepsis, run by Dascena at 3 sites in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: status not confirmed, as of the sponsor's last update on 3 May 2022. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 3 sites in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
What does this phase mean?
Phase 2 studies give the treatment to a larger group to see whether it has the intended effect and to keep checking safety.
Not every study follows the same path, and some studies (observational studies, device studies) have no phase.
Are results available for this study?
Results are not available in the source record.
How can a doctor refer a patient or contact the research team?
Contact Jana Hoffman, PhD (2158806619, jana@dascena.com), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Rate of reduction in false alerts (time frame: Through study completion, human subjects involvement will occur for an average of eight months). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT04005001, which also lists 5 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 3 May 2022.