Linguistically Tailored Health Messages to Encourage Plant-Based Food Choices in Adolescents

This study is a five-arm randomized controlled trial targeting adolescents aged 11-16 years in the UK and Ireland. In the intervention arms, participants will receive daily messages promoting a plant-based diet from Monday to Friday for two weeks. The first arm serves as a passive control group, with participants receiving no messages throughout the study. In arm 2, participants will receive daily non-tailored messages for the entire two weeks. In arm 3, participants will receive linguistically tailored messages. Participants in arm 4 will receive both non-tailored and tailored messages for one week each: tailored messages in week 1 and non-tailored in week 2 (arm 4a), and non-tailored in week 1 and tailored in week 2 (arm 4b). This design supports both within-subject comparisons (tailored vs. non-tailored) and between-subject comparisons (e.g., control vs. intervention).

At the start of the program, participants will play a chat game to collect chat data for analyzing their linguistic style. These data will be used to perform a text style transfer on the intervention messages using a Large Language Model (LLM). Following this, the two-week intervention will commence with a baseline survey conducted during the first weekend. During weekdays (Monday to Friday), participants in the intervention arms (arms 2, 3, and 4) will receive daily messages promoting a plant-based diet through the research app Avicenna. Additionally, all participants, including those in the passive control group, will receive daily surveys about their lunchtime food choices. Participants in the intervention arms will also respond to questions about message perception. On the weekends following each intervention week, participants will complete comprehensive surveys (midpoint and final surveys). These surveys, along with the baseline survey, will assess all outcome measures, including dietary behaviors, determinants of behavior, perceptions of the message source, message processing, and text style characteristics.

Sample size

The expected effect size was based on two meta-analyses of online health promotion studies that included general tailoring (Krebs et al., 2010; Lustria et al., 2013). These studies reported an average effect size of 0.08 (Cohen's f) overall and 0.112 for diet-related interventions specifically. Based on an assumed effect size of 0.1 and a power of 80% for between-subjects comparisons, a sample size of 592 participants is required. Since within-subject comparisons typically yield higher statistical power, we aim for a target sample size of 600 participants (150 per arm) with complete responses.

Recruitment of participants

Participants will be recruited through a research panel (Norstat) in the UK and Ireland. Panel members with eligible children will be invited to participate in the study. As part of the recruitment survey, parents will first answer pre-screener questions to confirm their child meets the inclusion criteria. If eligible, parents will continue in the recruitment survey, which further includes information about the study and the opportunity to sign up their child and provide consent for their child's participation. Once parental consent is obtained, children will be contacted to take part in the study. Participants will receive research panel credits (i.e., financial compensation) for completing each stage of the study: playing the chat game, completing the baseline survey, daily surveys, the midpoint survey, and the final survey.

Randomization

Block randomization will be used to allocate participants across the intervention arms. Arms 4a and 4b will be combined into a single block since they feature identical conditions, but only counterbalanced. This results in four total blocks. With a target sample size of 600, we aim for 150 participants per block. Participants will be assigned to one of the four blocks via the automated block randomization function in Qualtrics, integrated into the recruitment survey.

Chat Game

We developed a web-based chat game, called BetweenUs, to collect chat data while meeting ethical, safety, and privacy standards. The purpose of this game is to generate a sample of authentic text messages, from which we can learn about each participant's individual linguistic style. The game was custom-built (i.e., no third-party software) and hosted on a Microsoft Azure server of Erasmus University. It can be accessed here: (https://movez.ecda.ai/chat/test1).

BetweenUs was co-designed by children in a co-creation session. Children provided input on how the game should look, what types of avatars should be used, the length of the game, and which topics they would like to talk about. The game was inspired by popular imposter games, such as Among Us. The game was further piloted among children for final feedback.

In the game, players will be randomly paired in groups of 4 participants within a chatroom. At the start of the game, the player will be guided by five instruction screens explaining the rules of the game. Afterwards, the player can start the game, during which they need to answer two questions: 1) Does the player like or dislike the topic presented (e.g., watching football)? 2) Does the player feel comfortable playing the imposter? Then, a role (investigator or imposter) will be randomly assigned to each player. Investigators will need to chat about their actual like/dislike of the presented topic, while imposters are instructed to take on the opposite role of their actual preference (e.g., "I dislike watching football, although I actually like it") without being caught by the other players.

The game is divided into three rounds of one-to-one conversations of 8 minutes, followed by a group conversation of 5 minutes. This way, we can capture the dynamics of one-to-one and group conversations, reflecting real-world chats. The game concludes with voting, asking the investigators to guess the imposter within the group. The investigator wins if they guess the imposter correctly, while the imposter wins if they do not receive the majority vote.

The chat game is entirely anonymous. Players are represented by animal-like avatars. No personal or identifiable information is collected during the game, and players are instructed to refrain from sharing personal details. If any personal or sensitive details (such as names, age, etc.) are shared in the chat, they will be removed/hashed. Additionally, the game topics are carefully chosen by researchers (considering suggestions from the co-creation session) to ensure sensitive subjects are avoided. Players also have the option to indicate their discomfort by taking on the role of an imposter during the game. Finally, the data is accessible only by researchers and saved on university servers.

Intervention messages

Ten intervention messages were developed to promote the benefits of plant-based eating in an autonomy-supportive manner. Each message targets one of the three most mentioned motives for plant-based eating (Miki et al., 2020): health motives, environmental motives, and animal welfare motives. The order of message motives is kept constant throughout the weekdays, with each day addressing one of the three motives across both weeks.

Message creation was guided by adolescents' preferences for intervention content. All messages were designed to be factual (including science-based statistics), concise (4-5 sentences), relevant to teens (e.g., addressing health concerns like acne), and autonomy-supportive (offering suggestions and tips rather than directives) (Hingle et al., 2013). The messages were intentionally written in a neutral linguistic style, avoiding the use of highly emotive language, non-standard syntax, or emojis.

These messages constitute the non-tailored messages and serve as the foundation for developing tailored messages through text style transfer using a Large Language Model (LLM). The non-tailored messages were written in an active, conversational voice to enable a fair comparison with the tailored versions, which are also expected to maintain an active voice following the style transfer.

Message neutrality was validated through both expert human evaluation and the Linguistic Inquiry Word Count 2022 (LIWC22) tool, which assessed each message across 22 linguistic dimensions, scoring from 0 (neutral) to 100 (extremely high) (Boyd et al., 2022). To ensure the messages were accurate, trustworthy, clear, engaging for adolescents, and primarily factual in style, multiple rounds of proofreading were conducted by senior researchers with expertise in youth communication.

Large Language Model

An LLM will be used to linguistically tailor the intervention messages. The primary task of the LLM is to learn each participant's conversational style using data collected from the chat game, and then adapt the intervention messages accordingly. This process will result in 10 linguistically tailored messages for each participant, while the content and factual information of the messages remain constant across both non-tailored and tailored versions.

The LLM Mistral-large was selected based on its current general performance scores and open-source availability. In a separate study, we evaluated the performance of various LLMs (Mistral and GPT) in tailoring intervention messages, focusing on text style transfer accuracy, content preservation (i.e., meaning similarity), and fluency (i.e., comprehensibility). All evaluated models demonstrated good capabilities in capturing users' conversational styles and scored very high on content preservation and fluency. Mistral-large was ultimately chosen as the most suitable model for this study.

First, Mistral-large will generate style-neutral equivalents of the chat data to create a parallel dataset for each participant. To maintain relevance, only messages exceeding the participant's median word count will be included, thereby excluding brief or non-meaningful messages (e.g., "hi"). This parallel dataset will serve as the training input for the model.

Next, the LLM will be explicitly prompted to act as a linguistic expert, analyzing the conversational style reflected in the parallel dataset. Specifically, the LLM will be guided to consider: a) the participant's use of function words (e.g., pronouns like "I," "we," "you"), b) their preferred tone (e.g., formal or informal, analytical or narrative), c) the stylistic words they commonly use (e.g., phrases, filler words), and d) the usage of emojis or emoticons in example sentences. The LLM will also provide an explanation of the text style transfer.

Research App

Daily messages and surveys will be delivered via a smartphone research app called Avicenna. Participants are required to download and register for this app using a unique ID prior to the start of the program. This app ensures anonymous participation in the research program.

Each school day, participants will receive a morning push notification prompting them to view a new message by opening the app. Additionally, the app will display a red badge whenever a new message is available. Messages can be accessed until midnight on the same school day, after which they will disappear from the app, and any unread messages will be marked as missing data for that day. Upon reading the message, participants will complete daily measures directly within the app. On weekends, participants will receive a notification reminding them to complete the weekend surveys (i.e., baseline, midpoint, and final surveys).