Augmented Reality in Surgery

Background and rationale. Anterior cruciate ligament (ACL) reconstruction is one of the most frequently performed orthopaedic procedures worldwide. A well-documented determinant of long-term clinical outcome is the anatomical accuracy of femoral bone tunnel placement, whose centre must reproduce the native ACL footprint on the lateral wall of the intercondylar notch. Femoral tunnel malposition is recognised in the literature as a contributing factor to recurrent instability, graft failure and revision surgery. Anatomic tunnel placement is guided arthroscopically by the visual identification of bony and cartilaginous landmarks - most notably the resident's ridge (lateral intercondylar ridge) and the posterior cartilaginous margin of the lateral femoral condyle. These landmarks must be discriminated from the surrounding tissue on the 2D arthroscopic monitor image; their identification depends on operator experience and on the quality of the visual field, which can be degraded by soft-tissue debris, bleeding or suboptimal camera orientation. The reported association between surgeon case volume and surgical outcome reflects, in part, the learning curve associated with this visual task. Computer vision and artificial intelligence methods applied to surgical video offer a route to automated, reproducible recognition of these landmarks on the existing arthroscopic image stream, without modifying the hardware already present in the operating room.

Investigational system. The ARS (Augmented Reality in Surgery) software pipeline analyses arthroscopic video acquired from the standard arthroscopic camera column. Two functional modules are evaluated: a binary classifier that determines whether the field of view is sufficiently free of debris and adequately oriented to allow landmark identification, and a semantic segmentation module that delineates the resident's ridge, the posterior cartilaginous margin of the lateral femoral condyle and the candidate anatomical footprint zone for femoral tunnel placement. The algorithmic core is built by fine-tuning surgical vision foundation models on a dataset specifically curated for this task. The intended future deployment paradigm - outside the scope of the present study - is an unobtrusive visual overlay on the existing arthroscopic monitor, supporting the surgeon's visual interpretation. By design, the system is a decision-support tool: it does not pilot instruments, does not take autonomous decisions, and does not substitute the surgeon's judgement at any step of the procedure.

Study design. The study is a prospective, single-centre observational study conducted at IRCCS Galeazzi-Sant'Ambrogio (Milan, Italy). A retrospective component, based on previously acquired arthroscopic video material from ACL reconstructions performed within the same Unit and managed under applicable data-protection and consent provisions, is used for additional independent validation of the algorithms. The prospective component is fully integrated into the standard clinical pathway: no additional surgical step, device or instrument is introduced, no operative time is added, and the system is not used to guide any intraoperative decision during enrolment. The algorithm operates offline on the recorded material.

Video acquisition protocol. For each enrolled patient, six 5-second video segments are extracted from a single continuous intra-operative recording captured from the existing arthroscopic camera column at native 1920×1080 resolution and 60 frames per second. Five segments document the state of the intercondylar notch at progressive cleaning steps - corresponding to approximately 0%, 25%, 50%, 75% and 100% completion of soft-tissue debridement of the lateral wall - and the sixth segment is acquired with the surgical probe positioned on the posterior cartilaginous margin, without occluding the candidate footprint zone, in order to capture the instrument-anatomy spatial relationship in the same anatomical frame. During the procedure the surgeon verbally announces each stability moment to facilitate post-operative segment extraction. Cleaning of the lateral wall in the resident's ridge region is performed exclusively with radiofrequency ablation; motorised instrumentation is avoided in this region during enrolment to preserve the integrity of the bony ridge as a visual landmark.

Data management. Footage is pseudonymised at the point of acquisition; only intra-articular content is recorded, and no patient-identifying frames are produced. A structured naming convention is applied at ingestion. All study data are stored on institutional infrastructure under the governance of IRCCS Galeazzi-Sant'Ambrogio, processed in accordance with Regulation (EU) 2016/679 (GDPR) and applicable Italian implementing legislation. Each video segment is traceable to a single enrolment record on the institutional study management system, supporting source-data verification.

Annotation workflow. Annotation is performed on a self-hosted instance of the CVAT Community Edition platform, deployed on institutional infrastructure. Three annotation workstreams are configured: a field-classification workstream, in which each segment is assigned a class corresponding to the cleaning step; an anatomical-segmentation workstream, in which every extracted frame of the fully-cleaned segment is densely annotated with polyline labels for the resident's ridge and the posterior cartilaginous margin and with a polygon label for the candidate perforation zone; and an instrument-plus-anatomy workstream addressing the segment recorded with the surgical probe in position. An AI-collaborative labelling pipeline is employed, in which a vision foundation model generates mask proposals that the expert surgeon annotator subsequently refines. The reference standard is established by a senior surgeon with sub-specialty expertise in knee arthroscopy; a subset of cases is independently re-annotated to estimate inter-rater agreement, and discrepancies are adjudicated by consensus.

Algorithm training and performance assessment. Models are trained on the annotated prospective dataset, supplemented by the retrospective material where applicable, and evaluated on a held-out subset. Algorithm performance is summarised by classification accuracy, sensitivity, specificity and F1 score for the field-quality classifier, and by intersection-over-union (IoU) and Dice similarity coefficient at the per-frame level for the anatomical-segmentation outputs, reported with bootstrap confidence intervals. Pre-specified sensitivity analyses include stratification by cleaning step, by camera orientation and by surgical phase.

Quality assurance. Standard Operating Procedures cover video acquisition, file management and annotation. Each procedure is version-controlled and distributed to all study personnel. Automated data validation rules check naming-convention compliance, segment duration, resolution and frame rate at ingestion. Video segments failing technical quality checks - for example, due to excessive motion blur or to instrument occlusion of the candidate footprint zone in segments where the instrument should not be present - are excluded from the analysis dataset, and exclusions are documented at segment level. Adverse events are not anticipated as a direct consequence of the observational procedure; any clinical event occurring during the standard ACL reconstruction is documented in the institutional clinical record according to routine practice and is not part of the study endpoints.

Sample size and missing data. The required sample size is driven by the technical objective of training and validating image-analysis algorithms rather than by a clinical-effect estimate, and is specified in the study protocol. Handling of missing data and the segment-level exclusion criteria for technical inadequacy are pre-defined in the analysis plan.

Statistical analysis. The statistical analysis plan is developed in collaboration with an institutional biostatistician and is documented in the study protocol. Algorithm performance metrics are reported as point estimates with bootstrap confidence intervals; comparisons across strata, where applicable, use non-parametric tests appropriate to the per-frame or per-patient unit of analysis.

Regulatory positioning. The investigational software is used in this study under a research protocol and is not employed to guide intraoperative decisions during enrolment. Any future clinical deployment is anticipated under the European Medical Device Regulation framework as a decision-support software intended to assist the surgeon's visual interpretation, without autonomous guidance.

Ethics. The study is conducted in accordance with the Declaration of Helsinki, with applicable European and Italian regulations on observational research with health data, and with Regulation (EU) 2016/679 (GDPR). Approval by the competent Ethics Committee is obtained prior to study initiation.