Can the SurgInfoBot Improve the Consent Process for Endoscopy? A Randomised Controlled Trial

Introduction

Human-computer interactions or 'Chatbots' have been utilized in a variety of healthcare settings, including the promotion of positive healthcare behaviors, the deliverance of psychological therapy, and the performance of diagnostic tasks. The growth of the internet has enabled greater access to information regarding healthcare interventions, the truth and value of which cannot always be verified. The use of Chatbots - online programs which attempt to replicate person to person interactions - in providing healthcare information has been successfully employed by prior studies. Recent studies have investigated the role of chatbots in providing information in the post-operative period, across orthopaedic surgery, vascular surgery and urology. The value of such human-computer interactions in providing pre-procedural information is less well explored.

Standard methods of consenting patients for procedures may not always result in patients being fully informed; a 2004 study of patients undergoing screening sigmoidoscopy demonstrated that 39% of surveyed patients could describe no other indication than doctor recommendation for the procedure after undergoing standard consent. Only 19% of patients mentioned bleeding and perforation as possible complications. This study seeks to investigate the usability of a novel chatbot designed to provide peri-procedural information in two endoscopic procedures - diagnostic oesophagogastroduodenoscopy (OGD) and diagnostic colonoscopy.

A novel Chatbot - 'SurgInfoBot' has been developed in order to provide real-time, patient-driven peri-procedural information. This study primarily seeks to assess the effect of SurgInfoBot use on patient satisfaction with the consent process in endoscopy. It will also assess the usability of the chatbot according to the system usability scale and test performance according to the as-yet unvalidated Chatbot Usability Questionnaire (CUX). User engagement will be analyzed objectively using stored metrics. Comparison will be made between perceptions of the SurgInfoBot as an information source and other established patient information sources. The potential impact of the SurgInfoBot on peri-procedural anxiety will also be explored.

Methods

Trial design

This randomised controlled trial will be reported according to the CONsolidated Standards of Reporting Trials (CONSORT) 2010 guidelines. A single blinded parallel study with 1:1 randomisation will be performed. This study will be conducted at Tallaght University Hospital (Dublin, Ireland).

Participants

All patients undergoing diagnostic or screening colonoscopy or Oesophagogastroduodenoscopy (OGD) will be eligible for inclusion, provided they are aged 18 or over and have capacity to consent to participate in this trial. Patients undergoing sigmoidoscopy procedures will not be included, as the SurgInfoBot does not contain specific information regarding sigmoidoscopy. Patients undergoing planned procedures beyond those required for diagnostics (such as biopsy, campylobacter-like organism testing or polyp retrieval) will not be eligible for inclusion; this includes patients undergoing endoscopic stenting, endoscopic mucosal resection (EMR), planned endoscopic laser ablation, or other interventional procedures. Patients undergoing emergency or inpatient endoscopy will not be eligible for inclusion. This trial did not exclude patients who did not speak English as their first language, though patients requiring an interpreter will not be eligible for inclusion.

Patients were recruited from an academic teaching hospital in West Dublin. Eligible patients will be attending outpatient appointments with a single colorectal and general surgeon (DO K). All endoscopy procedures will be performed in the same recruiting institution.

Interventions

Participants will be randomised to receive either standard consent, or standard consent with SurgInfoBot access. The standard consent procedure is as follows: The rationale, risks, benefits, and alternatives for the intended procedure (diagnostic or screening OGD or colonoscopy) are discussed in the outpatient clinic by a consultant general surgeon (DOK) or members of the surgical team. Patients are given the opportunity to ask questions. All patients receive a written information leaflet, which includes contact details for the endoscopy department should patients wish to ask further questions. Those undergoing colonoscopy receive further instructions along with prescribed colonoscopy 'prep' medication by post. On the day of the procedure, patients are required to sign a standardised consent form.

Patients in the SurgInfoBot arm will be consented as above, but will also granted access to the SurgInfoBot, which they will be invited to use between their outpatient appointment and their endoscopy procedure date. Participants will be provided with an access link and Unique Study Identifier in order to pseudonymise participant data. Participants can access the SurgInfoBot as many times as they wish, and could ask any questions they see fit.

The 'SurgInfoBot' is a fully automated conversational agent, built as a web page that can be accessed via computer or smartphone. The SurgInfoBot was developed within the department of Surgical Affairs, Royal College of Surgeons in Ireland, using the Microsoft Chatbot Framework (Microsoft Corporation, 2019) and hosted by the Azure cloud system (Microsoft corporation, 2019). It is a command-based chatbot which relies on a database of questions and replies to answer user queries. A question bank with associated answers was developed initially by two general surgery residents (BM, CT) using national freely-available online resources where available, and high quality peer-reviewed articles where no national resources addressed the topic of concern. Content was verified for accuracy by a consultant colorectal/ general surgeon (DO K). An iterative process of content generation, usability testing, modification and re-testing was undertaken with healthy volunteers, in order to populate the SurgInfoBot's knowledge database and ensure adequate usability prior to patient use.

Randomisation

Randomisation will be performed using an online randomisation system (https://www.sealedenvelope.com/simple-randomiser/v1/lists) . After undergoing standard endoscopy consenting procedures as above, patients will be asked to consent to take part in this research study. Recruited patients will then be allocated the next available unique study identifier, randomly allocating patients to receive standard further consent information (printed information leaflet) or access to the SurgInfoBot (printed information leaflet and access instructions for the SurgInfoBot online interface). Both groups of patients will also receive a link (via e-mail) to the survey of baseline demographics which will completed before access to the SurgInfoBot is granted. Patients in the SurgInfoBot access arm will also receive an e-mail with their access details.

Baseline data

Baseline demographic information will be recorded. This includes participant gender, age, first language (English or other), level of educational attainment and electronic health literacy as measured using the eHEALS (eHealth Literacy Scale) score.

Primary Outcome Measure and Determination of Sample Size:

Patient satisfaction with the pre-procedural information received will be recorded using a 6-point Likert scale from '- - -' to '+ + +' using the following statement: "Overall, I am satisfied with the information I received before this procedure. This study was powered to detect a 0.5 point increase in satisfaction, based on a mean and standard deviation derived from a previous study utilising this scale by Huber et al. 201213, with a power of 80% and alpha of 0.05. Thirty-one participants were therefore required for recruitment in each group. Based on an anticipated drop-out rate of 20%, we aimed to recruit 70 patients in total (35 in each group).

Secondary Outcome Measures:

Knowledge:

Knowledge was assessed using a 5 item questionnaire using both open and multiple choice questions; participants were asked to identify the procedure they were having performed name the indication for having this procedure performed, identify the nature of sedation given, identify the risks of the procedure, and identify the alternatives of having the procedure performed. Items and responses were reviewed by a general surgery resident (CT) and colorectal/ general surgeon (DOK) for content validity. Knowledge questionnaires were administered immediately after standard consent was provided, and on the day of the procedure. Perceived knowledge was also assessed using a six-point Likert scale, based on participant responses to the statement: "I feel well informed about the planned procedure" from '- - -' to '+ + +'

Anxiety Peri-procedural anxiety was recorded on the day of the procedure using the State-Trait Anxiety Inventory for adults14

SurgInfoBot usability

SurgInfoBot usability was recorded using the validated ChatBot Usability Questionnaire developed by Holmes et al. (2019)15, a usability scale specifically designed for evaluating chatbots in healthcare.

Statistical Analysis

Categorical data will be presented as both absolute and relative frequencies, continuous data by mean and standard deviation. Comparisons between groups will be made using the Chi-Square test and t-tests. An effect size will be calculated for the primary outcome measure (Overall Satisfaction scores) - 95% confidence intervals will also be reported. Multivariate logistic regression will be used to identify predictors of complete satisfaction (defined as an overall satisfaction score of +++). All tests will be two-tailed, with p<0.05 considered statistically significant. All analyses will be performed with SPSS software (IBM, USA).