Cognitive Augmentation Via Multimodal Sensing and Auricular Neurostimulation

The objective of this study is to determine if active tAN stimulation can form a closed-loop system with the Galea biosensing headset to enhance cognitive performance. The study is designed as a randomized, blinded, sham-controlled trial to test the effects of active tAN stimulation in cognitive stimulation tasks (Flanker task, GradCPT, MATB and cybersickness tasks) towards improvement of cognitive performance.

The study occurs in three phases.

• Phase I is designed to establish a quantifiable relationship between biometric data and cognitive states (cognitive load, stress, attention and cybersickness) under cognitive stimulation tasks (Flanker task, GradCPT, MATB and cybersickness tasks).
• Phase II involves evaluation of manual tAN to affect cognitive state in an open-loop paradigm.
• The third phase involves using the cognitive state quantification from Phase I, and the quantified effects of tAN discovered in Phase II to examine a closed-loop cognitive augmentation system.

Preliminary Cognitive State Assessment:

Phase I of the experiment will establish a cognitive task performance baseline, and also allow for quantification of cognitive state metrics based on biomarkers. Participants will perform the following tasks: Flanker Task, GradCPT, MATB, and a cybersickness stimulation task. Each of these tasks have different quantification and scoring systems to evaluate performance, and each aim to induce different cognitive stimulation. Before each task, an idle baseline of cognitive state is recorded.

Open-loop Intervention:

In Phase II, all participants (Groups 1-2) will undergo the same set of tasks, this time while wearing the Sparrow Link tAN device, and Group 1 will experience active tAN, controlled by the investigator to determine optimal settings for affecting cognitive state.

Closed-loop Intervention:

In Phase III, the information on quantifying cognitive state determined in phase I and information on affecting cognitive state determined in phase II will be used to create a closed-loop intervention system using active tAN (Group 1 only), triggered based on conditions determined by the investigating team. Both groups will undergo the same set of tasks as in Phase I and Phase II, but Group 1 will receive active tAN and Group 2 will not.

Tasks Experimental paradigms and mechanisms that have been shown to induce stress, loss of attention, and cybersickness will be used to elicit physiological responses from study participants. These responses will be used to generate data for automatic characterization of the target cognitive states, determine how different levels of manual tAN affects these states, and test the efficacy of a closed-loop neurostimulation system. These paradigms and mechanisms are described below.

Multi-Attribute Task Battery (MATB) The Multi-Attribute Task Battery is a computer-based task designed to evaluate operator performance and workload. MATB provides a benchmark set of tasks analogous to activities that aircraft crew members perform in flight, although it can also be used by non-pilot individuals. The MATB requires simultaneous performance of monitoring dynamic resource management, and task tracking. The simultaneous performance of multiple tasks is a central feature of the MATB, and is the feature that is consistent with most operational systems, making it useful as a research platform.

Flanker Task The flanker task can be used to measure processing and selective attention (Eriksen, 1974). The flanker task requires participants to respond to a central target stimulus while ignoring flanking stimuli that may or may not be congruent with the target stimulus (Eriksen, 1974). These tasks become more challenging over time either by decreasing the time allowed for reactions or by introducing more conflicting information. A subject's responses and error rate can be used to gauge their attention and stress levels.

GradCPT A continuous performance task requires the participants to respond to frequent stimuli and inhibit responding to infrequent stimuli (Robertson, et al., 1997). GradCPT eliminates the offsets and onsets of visual stimuli between trials by using gradual transitions. As a result, GradCPT is more dependent on internal attention control and useful for studying sustained attention processes using physiological methods. The GradCPT task is used to test a subject's sustained and selective attention as well as response inhibition. Insufficient attention to tasks can result in unintended or incorrect inputs. Images show the first image transitioning to the second at 100%, 75%, 50%, 25% and 0% image coherence. Participants are required to press a button each time a city scene is presented (90% of trials) and withhold responses when infrequent mountain scenes are presented on the remaining 10% of trials (Fortenbaugh, et al., 2017).

Cybersickness Stimulation Mechanisms Cybersickness relates to the tendency of some users to exhibit symptoms similar to classical motion sickness both during and after a virtual reality (VR) experience. It is distinct from motion sickness in that the user is often stationary but has a compelling sense of self motion as a result of the immersive visual aspects of the simulation (LaViola Jr, 2000). The symptoms of cybersickness include dizziness, nausea, and lightheadedness (Davis \& Nalivaiko, 2014), and there are several factors that are known to induce cybersickness including latency (Stauffert et al. 2020) and loss of control (Davis \& Nalivaiko, 2014). Prevalence of sensitivity to cybersickness ranges from 20-95% with immersive VR experiences (Yildirim, 2020).

Sensing Technologies This section describes the technologies that will be used during the experiment to record physiological responses to stimuli and characterize cognitive state metrics.

Electroencephalography (EEG) Electroencephalography (EEG) records brain electrical activity via placement of electrodes on the scalp. One major advantage of using EEG is that it is noninvasive, meaning it is a safe and low-risk method for studying brain activity.

The Galea biosensing suite supports ten active EEG electrodes and two passive EEG electrodes. The ten active electrodes are located along the midline and in the frontal, parietal and occipital lobes. Specifically, their locations are F1, F2, C3, CZ, C4, P3, P4, PZ, O1, O2 as denoted by the 10-10 system for EEG recording. The two passive electrodes are located in the headset face mask and measure the Fp1 and Fp2 locations as denoted by the 10-10 system. The device has two attached clips that attach to the ears of the user. These ear clips provide common-mode signal rejection and serve as a reference voltage for the EEG electrodes.

Face Electromyography (EMG) Electromyography (EMG) relates to the measurement of muscles and the nerve cells (motor neurons) that control them. To measure EMG, electrodes are placed on the skin or inserted into the muscle to detect electrical activity. The resulting signals are then analyzed to provide information about the functioning of muscles and nerves.

The Galea biosensing suite includes dry EMG electrodes around a face mask that attaches to the XR component of the headset. These electrodes measure facial muscle activity and allow for estimation of face poses when the face is covered by the XR headset. Three pairs of electrodes are placed above the eyebrows in different orientations in order to target the Frontalis muscles that raise the eyebrows, as well as the Depressor Glabellae, Depressor Supercilli, and Currugator muscles that allow the eyebrows to lower. Additional electrode pairs are placed on each of the cheeks to target the Zygomatic Major and Zygomatic Minor muscles that allow the lips and cheeks to move. The electrodes are standard dry, flat electrodes coated in Ag/AgCl.

Photoplethysmography (PPG) Photoplethysmography (PPG) is a non-invasive method of measuring blood flow in blood vessels close to the skin's surface. PPG provides information about the volume of blood in a particular area and can be used to infer the pulse rate and oxygen saturation levels. PPG can be used to measure pulse rate variability, which closely approximates heart rate variability (Chuang et al., 2015).

A PPG sensor is integrated into one of the ear clips attached to the Galea biosensing suite. The clip is placed on the earlobe, which results in very high signal-to-noise (SNR) signals, relative to other locations and is a sufficient replacement for heart rate monitoring when compared to electrocardiogram (Vescio et al., 2018; Weiler et al., 2017).

Electrodermal Activity (EDA) Electrodermal activity (EDA) refers to changes in the electrical conductance of the skin which are caused by changes in sweat gland activity. EDA can be measured by attaching electrodes to the skin and recording changes in skin conductance or impedance. Both methods provide adequate information on the changes in skin conductance. Sweat gland activity is often associated with increased emotional arousal, particularly with stress and anxiety, and EDA is a well established physiological measure that is used to assess changes in arousal, attention and emotional states (Leiner et al., 2012). However, the limitation of EDA is that it only provides information about changes in skin conductance relative to a baseline, and does not indicate what specific cognitive state the change in conductance is linked to.

Galea measures EDA from the forehead, which provides a similar level of information to more peripheral measurement locations (Hossain et al., 2022).

Electrooculography (EOG) The Galea biosensing suite supports integrated EOG, which measures eye electrical activity. The system detects eye movements along the vertical and horizontal axes. When these movements occur, the eye movement acts as a dipole and causes large measurable fluctuations that can be recorded. The Galea's EOG sensors use the same flat Ag/AgCl electrodes that the EMG face sensors. They are placed above and below the right eye for vertical EOG and on the outside periphery of the left and right eyes for horizontal EOG.

Eye Tracking and Pupillometry Eye gaze tracking is achieved using infrared imaging sensors that map eye movements in real time. Pupillometry is the measure of pupil size changes in response to different scenarios or stimuli.

The Galea biosensing suite includes an integrated Varjo XR-3 head-mounted display (HMD). These HMDs use infrared eye cameras to track gaze, eye movement, and pupillometry. The integrated eye gaze tracking and pupillometry system samples at 200 Hz and is able to pick up subtle shifts in attention and concentration.

Neurostimulation Techniques Transcutaneous auricular neurostimulation (tAN) is a paradigm which targets both the trigeminal and vagus nerves on the auricle. tAN delivers mild electrical stimulation to modulate several cranial nerve branches in key dermatome regions through a small non-invasive device placed over the left ear (auricle). These regions cover or are adjacent to the afferent sensory innervation of several cranial nerves (V, VII, IX, X) and occipital nerves.