LLM in Urodynamic Education

Pressure-flow urodynamic interpretation requires simultaneous assessment of numeric parameters (PdetQmax, Qmax, cystometric capacity, voided volume), derived indices (BOOI, BCI), technical quality verification, and integration of ICS-defined diagnostic criteria across seven diagnostic categories (bladder outlet obstruction, detrusor overactivity, detrusor underactivity, hypocompliant bladder, hypercompliant bladder, atonic bladder, and normal urodynamic study). Mastery of this skill during residency training remains highly dependent on faculty availability, case exposure, and the quality of feedback provided, limiting standardization across training environments.

This prospective, single-center, assessor-blinded educational intervention study evaluates whether a standardized, modular LLM-assisted tutoring program can improve residents' diagnostic agreement with an expert-derived reference standard over the course of training. Eligible participants are urology residents currently enrolled in an accredited urology training program without prior formal urodynamic course certification.

A total of 83 archived pressure-flow urodynamic studies, performed using a single urodynamic system, were screened; after exclusion of studies with missing summary data, inadequate technical quality, or pediatric age, a final pool of 65 fully anonymized studies was retained. From this pool, two mutually exclusive case sets were constructed: an assessment case pool of 30 cases (10 cases each for pre-test, mid-test, and post-test, balanced for diagnostic category distribution and sex) and an educational case pool used exclusively during training modules. No case was used in more than one assessment time point or in both pools, minimizing the influence of case familiarity on assessed performance.

The reference standard for all assessment cases was established independently of the educational intervention: two experienced urologists, blinded to each other's assessments, independently classified each case across the seven diagnostic categories; disagreements were adjudicated by a third independent experienced urologist, whose decision served as the final reference standard. This reference standard was finalized prior to the start of resident assessments and remained unchanged throughout the study.

The educational intervention consists of five sequential modules delivered via standardized prompts using the Gemini LLM: (1A) theoretical instruction on ICS terminology and pressure-flow principles without case analysis; (1B) structured conceptual reinforcement addressing operational thresholds, sex-specific interpretation, and ambiguous findings; (2) observational learning, in which the LLM models expert-level step-by-step interpretation of training cases; (2B) visual pattern recognition, focused specifically on graphical identification of detrusor overactivity from filling-phase pressure curves; and (3) interactive learning, in which residents independently interpret cases before receiving structured comparative feedback against the LLM's structured assessment. All residents completed modules in an identical sequence using identical materials.

Resident performance is assessed at three time points - pre-test (before the intervention), mid-test (after modules 1A, 1B, 2, and 2B), and post-test (after module 3) - each comprising 10 independent pressure-flow cases evaluated under proctored, closed-book conditions without access to LLMs, reference materials, or prior assessment feedback. For each case, residents record numeric parameters, derived indices (BOOI, BCI), classification across the seven diagnostic categories (Present/Absent for detrusor overactivity, detrusor underactivity, hypocompliant bladder, hypercompliant bladder, atonic bladder, and normal urodynamic study; Present/Absent/Indeterminate for bladder outlet obstruction, with an additional Not Applicable option reserved for female patients), and an overall technical quality assessment.

The primary outcome is the resident-level pooled, unweighted Cohen's kappa between resident classifications and the expert reference standard, calculated by combining all diagnostic category decisions per resident per time point (10 cases × 7 categories, with female bladder outlet obstruction decisions excluded) into a single multicategory agreement table. Secondary outcomes include numeric extraction accuracy, BOOI/BCI calculation accuracy, technical quality assessment agreement, and diagnosis-specific agreement, each evaluated across the three time points. Change in pooled kappa across pre-test, mid-test, and post-test is analyzed using a within-subject repeated-measures approach.

All urodynamic data were fully anonymized prior to use, and no patient-identifiable information was retained or shared. Participation was voluntary, and written informed consent was obtained from all residents. The study was conducted in accordance with the Declaration of Helsinki and approved by the institutional non-interventional clinical research ethics committee.