Patient SafetyNCT07823062
Specialist Review of Harm Ratings for Clinical Responses Acronym: NOHARM
An observational study of Patient Safety and Medical Errors, run by Stanford University.
Trial summary
The purpose of this study is to determine how closely potential-harm ratings assigned using the NOHARM rubric agree with independent ratings from specialty-matched physicians. The study uses reconstructed clinical consultation cases and previously generated responses written by artificial intelligence systems or… Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Patient Safety, Medical Errors
- How many people
- 80 participants
- Where
- No sites listed yet
- Study dates
- 21 Sep 2026to 30 Oct 2026
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Study population: Licensed or registered attending- or consultant-level physicians practicing in one of the medical specialties represented in the NOHARM benchmark in the United States, Canada, United Kingdom, or Australia. Participants will be recruited through professional networks, institutional communications, and collaborating clinical communities and will complete specialty-matched rating activities remotely.
Who may be able to join
- Licensed or registered physician in good standing
- Attending- or consultant-level physician actively practicing in allergy and immunology, cardiology, dermatology, endocrinology, gastroenterology, hematology, infectious disease, nephrology, neurology, or pulmonology
- Practicing in the United States, Canada, United Kingdom, or Australia
- Assigned clinical materials match the physician's specialty
- Able to read and respond in English
- Access to a computer with a stable internet connection
- Willing and able to complete at least one remote specialist-rating engagement of approximately three hours
Who cannot join
- Does not meet the professional, career-level, specialty, geographic, or credential-verification requirements above
- Prior involvement with assigned cases, responses, or rubric information that could compromise independent rating
- Unable to complete the remote rating activities in English
- Unable or unwilling to follow the independent-rating procedures
- For validation-stage assignments, participation in the rubric-revision process
Trial locations and contacts
No sites are listed yet. Contact the study team or a site near you to ask about taking part.
About this study
Official title: Specialist Physician Validation of the NOHARM Harm-Rating Rubric for Clinical Responses
The purpose of this study is to determine how closely potential-harm ratings assigned using the NOHARM rubric agree with independent ratings from specialty-matched physicians. The study uses reconstructed clinical consultation cases and previously generated responses written by artificial intelligence systems or physicians. The materials contain no patient identifiers or protected health information. Board-certified attending physicians from 10 medical specialties will review assigned case-response pairs through a secure website. For each response, they will indicate whether it contains an error, describe the error when applicable, and rate the greatest potential patient harm as none, mild, moderate, or severe. Reviewers will not be shown the response source, existing rubric rating, or other reviewers' ratings. Two specialists will initially review each response; a third may review it if the first two ratings differ. The study includes a development stage, possible rubric revision, and a separate validation stage.
- Conditions
- Patient Safety, Medical Errors
- Keywords
- Artificial Intelligence, Clinical Response Evaluation, Harm Assessment, Clinical Benchmark, Rubric Validation
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
| Study group | Role | What they receive |
|---|---|---|
| Specialist Physician Reviewers | Licensed or registered attending- or consultant-level physicians practicing in one of the represented medical specialties in the United States, Canada, United Kingdom, or Australia. Participants will independently review and rate reconstructed clinical case-response pairs during the development stage, validation stage, or both. Physicians involved in rubric revision will not provide validation-stage ratings. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Inverse-Probability-Weighted Confirmation of Rubric-Severe Classifications
The inverse-probability-weighted proportion of responses classified as severe by the NOHARM rubric that are also classified as severe by specialist consensus: P(specialist severe | rubric severe). This measure will be calculated separately for the development and validation stages. Values range from 0 to 1; higher values indicate greater confirmation. The prespecified threshold is 0.70.
Measured: At completion of the development and validation stages, up to 2 months
Inverse-Probability-Weighted Capture of Specialist-Severe Classifications
The inverse-probability-weighted proportion of responses classified as severe by specialist consensus that are also classified as severe by the NOHARM rubric: P(rubric severe | specialist severe). This measure will be calculated separately for the development and validation stages. Values range from 0 to 1; higher values indicate greater capture. The prespecified threshold is 0.70. Both co-primary measures must meet this threshold.
Measured: At completion of the development and validation stages, up to 2 months
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted16 Sep 2026
- Study start21 Sep 2026 (estimated)
- Primary completion25 Oct 2026 (estimated)
- Study completion30 Oct 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Patient SafetyStudies of the same condition, open and under way.
- More trials run by Stanford UniversityThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Cohort
- Time perspective
- Prospective
- Enrolment
- 80 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Stanford University
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Jonathan Chen, Stanford University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07823062
- Sponsor's protocol ID
- 85610
- Status last verified
- Sep 2026
- First submitted
- 10 Sep 2026
- Last updated
- 16 Sep 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Yes
Study at a glance
From the registry record, last updated by the sponsor on 16 Sep 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Specialist Physician Validation of the NOHARM Harm-Rating Rubric for Clinical Responses
- ClinicalTrials.gov ID
- NCT07823062
- Acronym / sponsor ID
- 85610
- Conditions
- Patient Safety, Medical Errors
- Keywords
- Artificial Intelligence, Clinical Response Evaluation, Harm Assessment, Clinical Benchmark, Rubric Validation
- Study type
- Observational
- Recruitment status
- Not yet recruiting
- Enrolment
- 80 participants (estimated)
- Lead sponsor
- Stanford UniversityUniversity, hospital or other
- Study start
- 21 Sep 2026 (estimated)
- Primary completion
- 25 Oct 2026 (estimated)
- Study completion
- 30 Oct 2026 (estimated)
- First posted
- 16 Sep 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The purpose of this study is to determine how closely potential-harm ratings assigned using the NOHARM rubric agree with independent ratings from specialty-matched physicians. The study uses reconstructed clinical consultation cases and previously generated responses written by artificial intelligence systems or physicians. The materials contain no patient identifiers or protected health information. Board-certified attending physicians from 10 medical specialties will review assigned case-response pairs through a secure website. For each response, they will indicate whether it contains an error, describe the error when applicable, and rate the greatest potential patient harm as none, mild, moderate, or severe. Reviewers will not be shown the response source, existing rubric rating, or other reviewers' ratings. Two specialists will initially review each response; a third may review it if the first two ratings differ. The study includes a development stage, possible rubric revision, and a separate validation stage.
Study design
- Observational model
- Cohort
- Time perspective
- Prospective
- Number of arms
- 1
| Arm | Type | Receives |
|---|---|---|
| Specialist Physician Reviewers | – | –Licensed or registered attending- or consultant-level physicians practicing in one of the represented medical specialties in the United States, Canada, United Kingdom, or Australia. Participants will independently review and rate reconstructed clinical case-response pairs during the development stage, validation stage, or both. Physicians involved in rubric revision will not provide validation-stage ratings. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
- Study population
- Licensed or registered attending- or consultant-level physicians practicing in one of the medical specialties represented in the NOHARM benchmark in the United States, Canada, United Kingdom, or Australia. Participants will be recruited through professional networks, institutional communications, and collaborating clinical communities and will complete specialty-matched rating activities remotely.
Key inclusion criteria (7 listed)
- Licensed or registered physician in good standing
- Attending- or consultant-level physician actively practicing in allergy and immunology, cardiology, dermatology, endocrinology, gastroenterology, hematology, infectious disease, nephrology, neurology, or pulmonology
- Practicing in the United States, Canada, United Kingdom, or Australia
- Assigned clinical materials match the physician's specialty
- Able to read and respond in English
Key exclusion criteria (5 listed)
- Does not meet the professional, career-level, specialty, geographic, or credential-verification requirements above
- Prior involvement with assigned cases, responses, or rubric information that could compromise independent rating
- Unable to complete the remote rating activities in English
- Unable or unwilling to follow the independent-rating procedures
- For validation-stage assignments, participation in the rubric-revision process
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Primary outcome
Inverse-Probability-Weighted Confirmation of Rubric-Severe Classifications
Time frame: At completion of the development and validation stages, up to 2 months
The inverse-probability-weighted proportion of responses classified as severe by the NOHARM rubric that are also classified as severe by specialist consensus: P(specialist severe | rubric severe). This measure will be calculated separately for the development and validation stages. Values range from 0 to 1; higher values indicate greater confirmation. The prespecified threshold is 0.70.
Inverse-Probability-Weighted Capture of Specialist-Severe Classifications
Time frame: At completion of the development and validation stages, up to 2 months
The inverse-probability-weighted proportion of responses classified as severe by specialist consensus that are also classified as severe by the NOHARM rubric: P(rubric severe | specialist severe). This measure will be calculated separately for the development and validation stages. Values range from 0 to 1; higher values indicate greater capture. The prespecified threshold is 0.70. Both co-primary measures must meet this threshold.
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 16 Sep 2026.
- Individual participant dataSharing planned: Yes
Related trials for Patient Safety
RecruitingObservational study
A Study in Children and, Adults With Congenital Thrombotic Thrombocytopenic Purpura (cTTP) Treated With Adzynma
Thrombotic Thrombocytopenic Purpura (TTP)
RecruitingPhase 1/2
The Safety and Efficacy of Daratumumab in Patients With Refractory Aplastic Anemia
Aplastic Anemia, Refractory
Active, not recruitingPhase 2
Safety and Efficacy of Empagliflozin in Hemodialysis
End Stage Renal Disease
RecruitingPhase 3
Study to Assess the Efficacy, Pharmacokinetics, Safety and Tolerability of Atrasentan in Pediatric Patients With Primary IgAN
Berger Disease, Bergers Disease
Active, not recruitingPhase 4
A Study Evaluating the Effectiveness and Safety of Risdiplam Administered as an Early Intervention in Pediatric Participants With Spinal Muscular Atrophy After Gene Therapy
Muscular Atrophy, Spinal
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
Questions about this trial
What is this clinical trial studying?
An observational study of Patient Safety and Medical Errors, run by Stanford University.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 7 inclusion and 5 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: opening soon, as of the sponsor's last update on 16 Sep 2026. Status can change between updates, so confirm with the study contact before making plans.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
Contact Jonathan H Chen, MD, PHD (650-725-3655, jonc101@stanford.edu), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Inverse-Probability-Weighted Confirmation of Rubric-Severe Classifications (time frame: At completion of the development and validation stages, up to 2 months). The study lists 2 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 16 Sep 2026.