Brain CancerCOSMONCT07307157
Head-to-Head Evaluation of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI System Versus Pathologist-Only Review
An observational study of Brain Cancer and Lung Cancer (Diagnosis), run by Harvard Medical School (HMS and HSDM) at 1 site in United States.
Trial summary
This study evaluates the diagnostic performance of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI system for cancer subtype classification and compares it head-to-head with pathologist-only review. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Brain Cancer, Lung Cancer (Diagnosis), Renal Cancer
- What is being tested?
- Digital Pathology Evaluation
- How many people
- 30 participants
- Where
- 1 siteUnited States
- Study dates
- 12 Jun 2025to 31 Jan 2026
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Study population: We will recruit pathologists from international academic medical centers, hospital systems, and diagnostic pathology practices across North America (United States), Europe (Austria, Hungary), and the Asia-Pacific (Taiwan, Hong Kong, South Korea, China, India) region. Participating sites will include major academic institutions with established pathology departments, with recruitment targeting expertise in neuropathology, pulmonary pathology, and urologic pathology.
Who may be able to join
- Board-certified pathologist with expertise in neuropathology, pulmonary pathology, urologic pathology, or general anatomical pathology
- Minimum of 3 years of clinical diagnostic experience
- Active clinical practice involving diagnostic pathology slide review
- Willingness to independently review and diagnose up to 300 de-identified whole-slide images
- Ability to access the study platform and complete case reviews within the specified study timeline
- Provision of informed consent for study participation
Who cannot join
- Prior involvement in the design or validation of the COSMO AI system
- Inability to commit sufficient time to complete assigned case reviews
- Presence of significant financial conflicts of interest related to the study outcomes
Trial locations and contacts
1 site in 1 country. Contact the study team or a site near you to ask about taking part.
About this study
This study evaluates the diagnostic performance of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI system for cancer subtype classification and compares it head-to-head with pathologist-only review. Pathologists will independently review de-identified whole-slide images derived from up to 300 patients across three anatomical sites (brain, lung, kidney) and provide diagnostic assessments. In parallel, COSMO will process the same cases offline to generate independent predictions, enabling direct comparison of diagnostic accuracy between human experts and the AI system.
The study will characterize the diagnostic accuracy of COSMO and pathologists, inter-observer agreement, and variations in performance across anatomical sites and cancer types with different incidence rates. Results will establish how COSMO compares to pathologists on identical cases and will inform the development of AI-assisted diagnostic systems in clinical practice.
Read the detailed description
Study Rationale and Background Diagnostic accuracy in cancer subtype classification varies significantly among pathologists due to differences in expertise, experience, and access to diagnostic resources. The emergence of AI systems in pathology offers the potential to enhance diagnostic performance and consistency in cancer classification. However, direct empirical comparisons of AI-based predictions and pathologists' diagnostic performance on identical cases remain limited in the literature.
Study Aims This head-to-head comparative study aims to: (1) evaluate the diagnostic performance of the COSMO AI system in cancer subtype classification across multiple anatomical sites; (2) characterize the diagnostic accuracy of experienced pathologists on the same cases; (3) directly compare diagnostic performance metrics between COSMO and pathologists; and (4) examine concordance patterns and performance variation by anatomical site, cancer incidence category, pathologist experience, and case complexity.
Study Setting and Participants The study will involve up to 25 board-certified pathologists with 3 to 10+ years of diagnostic experience, recruited from institutions across North America, Europe, and the Asia-Pacific region. Participating pathologists will have domain expertise in neuropathology, pulmonary pathology, urologic pathology, or general anatomical pathology.
Cases and Stratification The study will employ de-identified archival whole-slide images representing up to 300 patients with confirmed reference diagnoses, including 100 brain cancers, 100 lung cancers, and 100 kidney cancers. Cases will be stratified by cancer type and incidence category (common vs. rare or uncommon), consistent with World Health Organization (WHO) guidelines.
Data Collection Pathologists will independently review each case and provide diagnostic classifications along with confidence assessments using a 5-point scale. The digital pathology interface will automatically record time-to-diagnosis metrics. COSMO will process the same cases offline to generate independent diagnostic predictions and confidence scores. Both pathologist and AI predictions will be evaluated against established reference standard diagnoses.
Analysis Framework The primary analysis will characterize diagnostic performance metrics (including accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC)) for both pathologists (at the individual and aggregated levels) and the COSMO system. Secondary analyses will assess performance stratified by anatomical site, cancer incidence category, and pathologist experience level.
- Conditions
- Brain Cancer, Lung Cancer (Diagnosis), Renal Cancer
- Being tested
- Digital Pathology Evaluation
- Keywords
- Artificial Intelligence, Whole-Slide Images, Ontology, Multimodal
- Medical subject headings
- Brain Neoplasms, Lung Neoplasms, Disease, Kidney Neoplasms
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Digital Pathology EvaluationDiagnostic test
Digital Pathology Evaluation
| Study group | Role | What they receive |
|---|---|---|
| AI-Based Evaluation using COSMO | Not described | |
| Pathologist-Based Evaluation | Not described Receives: Diagnostic Test: Digital Pathology Evaluation |
What is being measured
The primary outcome is the main question the study is designed to answer.
Diagnostic performance
Diagnostic performance of the COSMO AI system and pathologists in identifying cancer subtypes across brain, lung, and kidney tumors, as assessed by accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC). We will include both overall comparisons and stratified evaluations by anatomical site and cancer incidence category (common vs. rare or uncommon).
Measured: Periprocedural (at the time of slide review)
Secondary outcomes (5)
Inter-Observer Agreement Among Pathologists
Diagnostic concordance among participating pathologists, measured by Fleiss' kappa, intraclass correlation coefficient (ICC), and pairwise concordance rates.
Measured: Periprocedural (at the time of slide review)
Pathologist-COSMO AI Concordance
Agreement patterns between pathologist diagnoses and COSMO AI predictions, including proportion of concordant cases overall and stratified by anatomical site, cancer incidence category, and pathologist experience level.
Measured: Periprocedural (at the time of slide review)
Diagnostic Confidence
Mean confidence scores (5-point scale) reported by pathologists during diagnostic assessment, stratified by anatomical site, cancer incidence category, and diagnostic correctness (correct vs. incorrect).
Measured: Periprocedural (at the time of slide review)
Time-to-Diagnosis
Mean diagnostic time (in seconds) required by pathologists to provide cancer subtype classification, stratified by anatomical site, cancer incidence category, and pathologist experience level.
Measured: Periprocedural (at the time of slide review)
Diagnostic Performance Stratified by Pathologist Experience
Diagnostic accuracy of pathologists stratified by years of clinical experience (3-5 years, 6-10 years, >10 years) to assess the relationship between experience level and diagnostic performance in cancer subtype classification.
Measured: Periprocedural (at the time of slide review)
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted29 Dec 2025
- Study start12 Jun 2025
- Primary completion31 Jan 2026 (estimated)
- Study completion31 Jan 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Brain CancerStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Cohort
- Time perspective
- Prospective
- Enrolment
- 30 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Harvard Medical School (HMS and HSDM)
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Kun-Hsing Yu, Harvard Medical School (HMS and HSDM)
- Investigators
- Kun-Hsing Yu, MD, PhD, Principal investigator, Harvard Medical School (HMS and HSDM)
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07307157
- Sponsor's protocol ID
- Yu Lab COSMO Study
- Acronym
- COSMO
- Status last verified
- Dec 2025
- First submitted
- 13 Dec 2025
- Last updated
- 29 Dec 2025
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No: Individual pathologist diagnostic assessments will not be shared to protect evaluator anonymity and privacy. De-identified case data and aggregated performance metrics will be made available through published results and supplementary materials. The protocol document will be uploaded to enable full methodological transparency.
Study at a glance
From the registry record, last updated by the sponsor on 29 Dec 2025. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Head-to-Head Evaluation of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI System Versus Pathologist-Only Review
- ClinicalTrials.gov ID
- NCT07307157
- Acronym / sponsor ID
- COSMO / Yu Lab COSMO Study
- Conditions
- Brain Cancer, Lung Cancer (Diagnosis), Renal Cancer
- Keywords
- Artificial Intelligence, Whole-Slide Images, Ontology, Multimodal
- Study type
- Observational
- Recruitment status
- Enrolling by invitation
- Enrolment
- 30 participants (estimated)
- Lead sponsor
- Harvard Medical School (HMS and HSDM)University, hospital or other
- Study start
- 12 Jun 2025 (actual)
- Primary completion
- 31 Jan 2026 (estimated)
- Study completion
- 31 Jan 2026 (estimated)
- First posted
- 29 Dec 2025 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study evaluates the diagnostic performance of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI system for cancer subtype classification and compares it head-to-head with pathologist-only review. Pathologists will independently review de-identified whole-slide images derived from up to 300 patients across three anatomical sites (brain, lung, kidney) and provide diagnostic assessments. In parallel, COSMO will process the same cases offline to generate independent predictions, enabling direct comparison of diagnostic accuracy between human experts and the AI system.
The study will characterize the diagnostic accuracy of COSMO and pathologists, inter-observer agreement, and variations in performance across anatomical sites and cancer types with different incidence rates. Results will establish how COSMO compares to pathologists on identical cases and will inform the development of AI-assisted diagnostic systems in clinical practice.
Detailed description
Study Rationale and Background Diagnostic accuracy in cancer subtype classification varies significantly among pathologists due to differences in expertise, experience, and access to diagnostic resources. The emergence of AI systems in pathology offers the potential to enhance diagnostic performance and consistency in cancer classification. However, direct empirical comparisons of AI-based predictions and pathologists' diagnostic performance on identical cases remain limited in the literature.
Study Aims This head-to-head comparative study aims to: (1) evaluate the diagnostic performance of the COSMO AI system in cancer subtype classification across multiple anatomical sites; (2) characterize the diagnostic accuracy of experienced pathologists on the same cases; (3) directly compare diagnostic performance metrics between COSMO and pathologists; and (4) examine concordance patterns and performance variation by anatomical site, cancer incidence category, pathologist experience, and case complexity.
Study Setting and Participants The study will involve up to 25 board-certified pathologists with 3 to 10+ years of diagnostic experience, recruited from institutions across North America, Europe, and the Asia-Pacific region. Participating pathologists will have domain expertise in neuropathology, pulmonary pathology, urologic pathology, or general anatomical pathology.
Cases and Stratification The study will employ de-identified archival whole-slide images representing up to 300 patients with confirmed reference diagnoses, including 100 brain cancers, 100 lung cancers, and 100 kidney cancers. Cases will be stratified by cancer type and incidence category (common vs. rare or uncommon), consistent with World Health Organization (WHO) guidelines.
Data Collection Pathologists will independently review each case and provide diagnostic classifications along with confidence assessments using a 5-point scale. The digital pathology interface will automatically record time-to-diagnosis metrics. COSMO will process the same cases offline to generate independent diagnostic predictions and confidence scores. Both pathologist and AI predictions will be evaluated against established reference standard diagnoses.
Analysis Framework The primary analysis will characterize diagnostic performance metrics (including accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC)) for both pathologists (at the individual and aggregated levels) and the COSMO system. Secondary analyses will assess performance stratified by anatomical site, cancer incidence category, and pathologist experience level.
Study design
- Observational model
- Cohort
- Time perspective
- Prospective
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| AI-Based Evaluation using COSMO | – | – |
| Pathologist-Based Evaluation | – | Diagnostic Test: Digital Pathology Evaluation |
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
- Study population
- We will recruit pathologists from international academic medical centers, hospital systems, and diagnostic pathology practices across North America (United States), Europe (Austria, Hungary), and the Asia-Pacific (Taiwan, Hong Kong, South Korea, China, India) region. Participating sites will include major academic institutions with established pathology departments, with recruitment targeting expertise in neuropathology, pulmonary pathology, and urologic pathology.
Key inclusion criteria (6 listed)
- Board-certified pathologist with expertise in neuropathology, pulmonary pathology, urologic pathology, or general anatomical pathology
- Minimum of 3 years of clinical diagnostic experience
- Active clinical practice involving diagnostic pathology slide review
- Willingness to independently review and diagnose up to 300 de-identified whole-slide images
- Ability to access the study platform and complete case reviews within the specified study timeline
Key exclusion criteria (3 listed)
- Prior involvement in the design or validation of the COSMO AI system
- Inability to commit sufficient time to complete assigned case reviews
- Presence of significant financial conflicts of interest related to the study outcomes
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Digital Pathology EvaluationDigital Pathology Evaluation | Diagnostic test | Pathologist-Based Evaluation |
Primary outcome
Diagnostic performance
Time frame: Periprocedural (at the time of slide review)
Diagnostic performance of the COSMO AI system and pathologists in identifying cancer subtypes across brain, lung, and kidney tumors, as assessed by accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC). We will include both overall comparisons and stratified evaluations by anatomical site and cancer incidence category (common vs. rare or uncommon).
5 secondary outcomes
Inter-Observer Agreement Among Pathologists
Time frame: Periprocedural (at the time of slide review)
Pathologist-COSMO AI Concordance
Time frame: Periprocedural (at the time of slide review)
Diagnostic Confidence
Time frame: Periprocedural (at the time of slide review)
Time-to-Diagnosis
Time frame: Periprocedural (at the time of slide review)
Diagnostic Performance Stratified by Pathologist Experience
Time frame: Periprocedural (at the time of slide review)
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 29 Dec 2025.
- Individual participant dataSharing planned: No. Individual pathologist diagnostic assessments will not be shared to protect evaluator anonymity and privacy. De-identified case data and aggregated performance metrics will be made available through published results and supplementary materials. The protocol document will be uploaded to enable full methodological transparency.
Related trials for Brain Cancer
RecruitingPhase 3
Beginning Radiation Immediately With GammaTile at GBM Excision Versus Standard of Care
Glioblastoma
RecruitingPhase 2
BrAin Metastasis in tripLe Negateive Breast Cancer: IvoneScimab and Trop2 ADC
Triple-Negative Breast Cancer (TNBC), Brain Metastases
RecruitingPhase 1/2
Testing the Addition of an Anti-cancer Drug, Selinexor, to the Usual Chemotherapy Treatment (Temozolomide) for Brain Tumors That Have Returned After Previous Treatment
Recurrent Glioblastoma, IDH-Wildtype, Recurrent MGMT-Methylated Glioblastoma
Active, not recruiting
Effects of Whole-body Electrical Muscle Stimulation Exercise on Adults With Neuromuscular Disease
Neuromuscular Diseases (NMD), Amyotrophic Lateral Sclerosis
Active, not recruitingPhase 1
Prospective Exploratory Study of FAPi PET/CT With Histopathology Validation in Patients With Various Cancers
Bladder Carcinoma, Cervical Carcinoma
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
Kun-Hsing Yu, MD, PhD principal investigator
Harvard Medical School (HMS and HSDM)
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Brain Cancer and Lung Cancer (Diagnosis), run by Harvard Medical School (HMS and HSDM) at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 6 inclusion and 3 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: enrolling by invitation, as of the sponsor's last update on 29 Dec 2025. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Diagnostic performance (time frame: Periprocedural (at the time of slide review)). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 29 Dec 2025.