Critical IllnessNCT06996626
Real-time NOMA Evaluation
A study testing Revealed Alerts and Unrevealed Alerts in people with Critical Illness, run by David T Huang at 2 sites in United States.
Trial summary
Alerts related to outlier clinician behavior are generated in real-time by an intelligent system continuously scraping EHR (electronic health record) data. These alerts are passed to the bedside and their potential impact on bedside clinical behavior is evaluated. Read more about this study
- Who is it for?
- Ages 18 to 100 years
- Conditions the study is for
- Critical Illness
- What is being tested?
- Revealed Alerts and Unrevealed Alerts
- Type of treatment
- Medical device
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 7,445 participants
- Where
- 2 sitesUnited States
- Study dates
- 23 Jun 2025to 30 Dec 2026
- Phase
- Interventional
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 to 100 years
- All sexes
- No healthy volunteers
- Adult, Older adult
Who may be able to join
- All patients in the Presbyterian and Montefiore ICUs
Who cannot join
- None
Trial locations and contacts
2 sites in 1 country.
United States2 sites
UPMC Montefiore
Pittsburgh, Pennsylvania, 15213
UPMC Presbyterian
Pittsburgh, Pennsylvania, 15213
About this study
Official title: Real-time Evaluation of an Outlier-based Alerting System
Alerts related to outlier clinician behavior are generated in real-time by an intelligent system continuously scraping EHR (electronic health record) data. These alerts are passed to the bedside and their potential impact on bedside clinical behavior is evaluated.
Read the detailed description
A clinician-informed AI model will generate outlier alerts from real-time review of the EHR (electronic health record) of UPMC Presbyterian/Montefiore ICU patients. These alerts will first be reviewed by an ICU clinician, along with the patients' EHR, for clinical relevance. For those alerts deemed potentially relevant, the ICU clinician will contact the treating ICU clinician (eg, an ICU pharmacist, physician, advanced practice provider) and discuss the alert. The treating ICU clinician will take whatever action, including no action, they deem best.
- Conditions
- Critical Illness
- Being tested
- Revealed Alerts, Unrevealed Alerts
- Medical subject headings
- Critical Illness
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Revealed AlertsDevice
Bedside reveal of alerts generated by the alerting system
Unrevealed AlertsDevice
Alerts will be generated but not revealed.
| Study group | Role | What they receive |
|---|---|---|
| Unrevealed Alerts | Active comparator | Not described Receives: Device: Unrevealed Alerts |
| Revealed Alerts | Experimental | Not described Receives: Device: Revealed Alerts |
What is being measured
The primary outcome is the main question the study is designed to answer.
Rate of Clinical Actions Following Revealed vs. Non-Revealed Alerts
The primary outcome compares the proportion of alerts that lead to documented clinical actions when revealed to treating ICU clinicians versus when not revealed. Alerts are generated by a decision support system and reviewed daily by ICU study clinicians. On average, 30 alerts are reviewed per ICU per day, with approximately 5 alerts revealed to the treating clinicians. The analysis uses a stepped wedge design with ICU beds as the unit of analysis, where each ICU acts as its own control. The outcome will assess whether revealing alerts increases the rate of appropriate clinical actions taken, as compared to when alerts are withheld.
Measured: From time of ICU admission until ICU discharge
Secondary outcomes (10)
Rate of Any Clinically Responsive Action Following Alerts
For each alert, a range of clinical actions may be deemed "responsive," including but not limited to the specific recommended action. This outcome assesses the differential rate of any clinically appropriate response (whether or not it matches the recommended action) between alerts that are revealed versus not revealed to treating ICU clinicians.
Measured: Up to 90 days after ICU admission
True Positive Alert Rate (TPAR) by Study Group and Alert Type
This outcome evaluates the overall and alert-specific True Positive Alert Rate (TPAR), defined as the proportion of alerts that are associated with a clinically appropriate action. TPARs will be calculated separately for the intervention group (alerts revealed), the control group (alerts not revealed), and the combined population. Comparisons will assess whether revealing alerts is associated with a higher TPAR across alert categories.
Measured: Up to 90 days after ICU admission
Rate of Non-Responsive Actions Following Alerts
This outcome assesses the difference in the rate of clinical actions that are not considered responsive to the alert (i.e., actions taken that do not address the alert's content or recommended intervention) between the intervention group (alerts revealed) and the control group (alerts not revealed). This helps evaluate potential unintended or off-target responses to alerting.
Measured: Up to 90 days after ICU admission
Overall Alert Rate
Total number of alerts generated per ICU per day.
Measured: Through study completion, an average of 2 years
Alert Rate per Alerting Model
Number of alerts generated per model type per ICU per day.
Measured: Daily, up to 90 days
Delay Between Alert Generation and First Responsive Action
Median time from alert generation to the first documented responsive clinical action.
Measured: Measured continuously through study completion, an average of 2 years
ICU Length of Stay
Total number of days each participant spends in the ICU during the index hospitalization.
Measured: Up to 90 days after ICU admission
Hospital Length of Stay
Total number of days each participant spends in the hospital during the index hospitalization.
Measured: up to 90 days after hospital admission
In-Hospital Mortality
Proportion of participants who die during the index hospital stay.
Measured: Up to 90 days after hospital admission
Time Trend of True Positive Alert Rate (TPAR)
Evaluate changes in the True Positive Alert Rate (TPAR) over time during the study period to assess model performance stability and potential temporal variation in responsiveness.
Measured: Through study completion, an average of 2 years
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted30 May 2025
- Study start23 Jun 2025
- Primary completion30 Jun 2026
- Study completion30 Dec 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Critical IllnessStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Sequential
- Model description
- Step wedge RCT
- Masking
- None (open label)
- Enrolment
- 7,445 participants (estimated)
Sponsor and investigators
- Lead sponsor
- David T Huang
- Sponsor type
- University, hospital or other
- Collaborators
- National Institute for Biomedical Imaging and Bioengineering (NIBIB)
- Responsible party
- Sponsor-investigator, David T Huang, University of Pittsburgh
- Investigators
- David Huang, MD, Principal investigator, University of Pittsburgh
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06996626
- Sponsor's protocol ID
- STUDY25020040
- Other IDs
- 5R01EB032752-10
- Status last verified
- Jun 2026
- First submitted
- 14 May 2025
- Last updated
- 23 Oct 2026
- Results posted
- No
- Data monitoring committee
- Yes
- FDA-regulated drug
- No
- FDA-regulated device
- Yes
- Individual data sharing
- No: The investigators do not currently have a data sharing plan for sharing with future investigators. If the investigators decide to share data in the future with investigators conducting similar research both inside and outside of the University of Pittsburgh, the investigators will contact the Office of Sponsored Programs before sharing de-identified research data to determine whether an agreement needs to be executed.
Study at a glance
From the registry record, last updated by the sponsor on 23 Oct 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Real-time Evaluation of an Outlier-based Alerting System
- ClinicalTrials.gov ID
- NCT06996626
- Acronym / sponsor ID
- STUDY25020040
- Conditions
- Critical Illness
- Study type
- Interventional
- Recruitment status
- Active, not recruiting
- Enrolment
- 7,445 participants (estimated)
- Lead sponsor
- David T HuangUniversity, hospital or other
- Collaborators
- National Institute for Biomedical Imaging and Bioengineering (NIBIB)
- Study start
- 23 Jun 2025 (actual)
- Primary completion
- 30 Jun 2026 (actual)
- Study completion
- 30 Dec 2026 (estimated)
- First posted
- 30 May 2025 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Alerts related to outlier clinician behavior are generated in real-time by an intelligent system continuously scraping EHR (electronic health record) data. These alerts are passed to the bedside and their potential impact on bedside clinical behavior is evaluated.
Detailed description
A clinician-informed AI model will generate outlier alerts from real-time review of the EHR (electronic health record) of UPMC Presbyterian/Montefiore ICU patients. These alerts will first be reviewed by an ICU clinician, along with the patients' EHR, for clinical relevance. For those alerts deemed potentially relevant, the ICU clinician will contact the treating ICU clinician (eg, an ICU pharmacist, physician, advanced practice provider) and discuss the alert. The treating ICU clinician will take whatever action, including no action, they deem best.
Study design
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Sequential
- Masking
- None (open label)
- Number of arms
- 2
- Model description
- Step wedge RCT
| Arm | Type | Receives |
|---|---|---|
| Unrevealed Alerts | Active comparator | Device: Unrevealed Alerts |
| Revealed Alerts | Experimental | Device: Revealed Alerts |
Who may qualify
- Ages
- 18 to 100 years
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (1 listed)
- All patients in the Presbyterian and Montefiore ICUs
Key exclusion criteria (1 listed)
- None
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Revealed AlertsBedside reveal of alerts generated by the alerting system | Device | Revealed Alerts |
| Unrevealed AlertsAlerts will be generated but not revealed. | Device | Unrevealed Alerts |
Primary outcome
Rate of Clinical Actions Following Revealed vs. Non-Revealed Alerts
Time frame: From time of ICU admission until ICU discharge
The primary outcome compares the proportion of alerts that lead to documented clinical actions when revealed to treating ICU clinicians versus when not revealed. Alerts are generated by a decision support system and reviewed daily by ICU study clinicians. On average, 30 alerts are reviewed per ICU per day, with approximately 5 alerts revealed to the treating clinicians. The analysis uses a stepped wedge design with ICU beds as the unit of analysis, where each ICU acts as its own control. The outcome will assess whether revealing alerts increases the rate of appropriate clinical actions taken, as compared to when alerts are withheld.
10 secondary outcomes
Rate of Any Clinically Responsive Action Following Alerts
Time frame: Up to 90 days after ICU admission
True Positive Alert Rate (TPAR) by Study Group and Alert Type
Time frame: Up to 90 days after ICU admission
Rate of Non-Responsive Actions Following Alerts
Time frame: Up to 90 days after ICU admission
Overall Alert Rate
Time frame: Through study completion, an average of 2 years
Alert Rate per Alerting Model
Time frame: Daily, up to 90 days
Delay Between Alert Generation and First Responsive Action
Time frame: Measured continuously through study completion, an average of 2 years
ICU Length of Stay
Time frame: Up to 90 days after ICU admission
Hospital Length of Stay
Time frame: up to 90 days after hospital admission
In-Hospital Mortality
Time frame: Up to 90 days after hospital admission
Time Trend of True Positive Alert Rate (TPAR)
Time frame: Through study completion, an average of 2 years
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 23 Oct 2026.
- Individual participant dataSharing planned: No. The investigators do not currently have a data sharing plan for sharing with future investigators. If the investigators decide to share data in the future with investigators conducting similar research both inside and outside of the University of Pittsburgh, the investigators will contact the Office of Sponsored Programs before sharing de-identified research data to determine whether an agreement needs to be executed.
Related trials for Critical Illness
Recruiting
Cycling Within an Interactive and Immersive Virtual Environment During Critical Illness: a Patient-centred Approach for Lower Limb Rehabilitation
Critical Illness
Not yet recruitingObservational study
Dynamic Inflammatory Changes and Nutritional Risk Scores in ICU Patients Aged 65 Years and Older
Critical Illness, Geriatric Patient
Recruiting
Individualised Blood Pressure Targets Versus Standard Care Among Critically Ill Patients With Shock
Critical Illness, Shock
Recruiting
Characterization and Longitudinal Monitoring of Muscular Phenotype in Patients Admitted to Intensive Care for Sepsis
Sepsis, Myopathy Critical Illness
Not yet recruitingObservational study
Stress Hyperglycemia Ratio and Glucose Variability in Critically Ill Patients
Critical Illness, Hyperglycemia
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
David Huang, MD principal investigator
University of Pittsburgh
2 sites in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Revealed Alerts and Unrevealed Alerts in people with Critical Illness, run by David T Huang at 2 sites in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: under way, not looking for participants, as of the sponsor's last update on 23 Oct 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 2 sites in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Rate of Clinical Actions Following Revealed vs. Non-Revealed Alerts (time frame: From time of ICU admission until ICU discharge). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 23 Oct 2026.