Developing Intelligent Wearable Algorithms

Globally, the life expectancy is gradually increasing, which leads to a demographic shift towards an increasing proportion of older people. For example, in Sweden, the number of individuals over the age of 75 will double during the next 50 years and the proportion will increase by more than 60% (Statistics Sweden, publicly available data). This means that more people will live longer and less people will be available to care for them. One of our major societal challenges to meet this demographic transition is to develop strategies for maintaining good health and quality of life in that age group and then when they inevitably become ill, how to treat them in the best possible way? One way forward is to capitalize on the opportunities the technological evolution have provided. In fact, Sweden have adopted a very ambitious goal as outlined in Vision2025 (www.ehalsa2025.se): In 2025, Sweden will be best in the world at using the opportunities offered by digitalization and eHealth to make it easier for people to achieve good and equal health and welfare, and to develop and strengthen their own resources for increased independence and participation in the life of society. However, digital technology alone is not sufficient to reach such a goal, other strategies that uses the technology to increase the health of individuals must also be present. One such strategy is to maintain a healthy level of physical activity through life. Physical activity is one of few behaviors that a human can change, on its own and to a low cost, that simultaneously produces significant physical and mental health benefits. To that end the ability to assess a person's habitual physical activity level, defined as "the hypothetical average around which that individual's physical activity varies" has for a long time been considered as the holy grail among physical activity researchers. For 30 years researchers have used accelerometers to objectively quantify free-living physical activity but in the last decade wearables have become hugely popular among the general population (Apple and Fitbit being the largest brands). Wearables, here defined as small portable devices with embedded sensors that measure health-related variables, claim to be able to collect long-term high-resolution data regarding health-related behaviour e.g., body posture, PA intensity, sleep, and much more. Wearables can also capture important aspects related to the cardiometabolic system such as heart rate, blood pressure, as well as support behaviour change. Thus, they can be viewed as a "Swiss army knife", since they provide a wide range of tools in one device. In theory, wearables hold a huge promise to be used to create evidence-based personalized PA in health promotion, disease prevention, and disease treatment. Compared to most research grade monitors the commercial wearables have several advantages that make them attractive for use in the healthcare system. They contain multiple sensors of relevance for the assessment of PA and related health outcomes, notably an accelerometer to detect biomechanical aspects of movement and an optical sensor to detect the physiological response to movement e.g., heart rate. Additional strengths of the commercial monitors compared to research grade monitors are that they are designed to be worn for a long time which gives an even better opportunity to assess the individual level of physical activity, especially among those the least active. However, during the last decade, there have been an explosion of different wearables and data processing methods without an established framework to evaluate their validity or reliability. This have caused great confusion driven by nonrobust device development and evaluation methods that do not reflect how they will be used in practice. Moreover, the divergence in summary estimates of physical activity within and between different brands prohibits an opportunity to pool data or to make direct comparisons between different studies. Other limitations that wearables have that makes them unsuitable for use, even among healthy individuals is that they are primarily designed for the fitness market and not for use within the health care system, they have not been evaluated in a transparent manner, they do not accurately capture physical activity among individuals with altered motorical pattern, e.g., elderly. Even though the hardware in most wearables are similar, the algorithms that translates the signals from the sensors to different interpretable outcomes (e.g., steps) are often proprietary and update with irregular intervals. Other important limitations include questionable patient confidentiality and data ownership. Neither the commercial nor research grade monitors can today assess physical activity at individual level since they make limited use of the wealth of data collected. If wearables can overcome these limitations, they hold much promise towards expanding the clinical repertoire of patient-specific measures, and they are considered an important tool for the future of precision health and personalized medicine. Considering the wealth of new wearable PA trackers entering the market without prior proof of validity it is fundamental to provide consumers, patients, health professionals and researchers with an open-source, evidence-based wearable with excellent validity and reliability.

Purpose and aims This project will start to merge the strengths of the commercial wearables with those of research grade monitors and modern data processing methods to overcome these limitations. The combination of information from accelerometry (biomechanics) and heart rate from the optical sensor (physiological response) improves the predictions regarding PA intensity and physical activity related energy expenditure on individual level. In addition, the optical sensor that are fitted in a wearable can estimate the blood pressure of an individual. The signals from the sensors will be analysed using artificial intelligence and machine learning to take better advantages of the rich data collected. Normally PA and health outcomes are being measured at different times, but wearables can collect data from multiple variables simultaneously and in real-time. This ability may provide novel details about the association between the physical activity behavior and the individuals clinical status. This have not yet been studied, most likely due to the complexity of acquiring and analyzing these data. However, the field have now moved to a point where the technology along with processing power and analytical tools exists. Thus, the time has come to explore the full possibility of combining real-time data on physical activity behaviour, health related outcomes and artificial intelligence. So that in the future citizens may be able to improve and maintain their health through device-based services and to make informed decisions that are based upon their personal health data.

The overarching purpose of DIWAH is to develop and validate artificial intelligence/machine learning based algorithms to assess physical activity and health related variables on individual level in real time for use within the health care system using open-source wearables.

This project will take the first steps by conducting a rigorous development and validation of the algorithms using a transparend phase-based framework. More specifically the project aim to develop algorithms for assessing:

• Physical activity
• Heart Rate
• Health related variables e.g., blood pressure and energy expenditure But also identify what underlying factors that improves the predictions such as age sex body composition and health related fitness levels.

Project description Study sample Around 50 apparently healthy adults (18-65 years, 50% females) will be recruited to this project. Recruitment will be done using a convenience sampling method from sport clubs, work sites and students at the University along with word of mouth.

There are no particular requirements to be included in the study except that the subjects must be apparently healthy and being able to jog for at least 30 minutes. Exclusion criteria includes known cardiovascular disease or being on a medication that influence the heart rate.

Study design Given that the hardware in all wearables are similar the focus is on developing as transparent algorithms as possible by following the framework outlined by Keadle et al. and simultaneously adhere to the expert recommendations of the INTERLIVE Network regarding HR monitoring. The Keadle-framework is inspired by drug development and suggests five phases: Mechanical signal testing, Laboratory testing, Semi-structured evaluation, Naturalistic validation, and Adoption. All of these phases will not be covered in the present project. The semi-structured evaluation phase is designed to identifies specific actives based on the signals from the sensors in the wearable, such as running or vacuuming. However, the physiological response and, thus, the health effects of physical activity would be similar for different kind of activities on the same intensity. For instance, the heart does not really care if one is running or shovelling snow if it is working with the same intensity. In addition, its ethically motivated to test the device in a patient population (phase five - Adoption) before the device provides valid data. Therefore, the semi-structured evaluation phase as well as the adoption phase will be omitted in the current project.

Phase 1 - Product Development Given that the hardware is similar in every wearable on the market, it is not of any importance exactly which specific brand of wearable will be used to develop and test our algorithms. The main idea with the first phase is to make sure that the wearable produces valid data and that the raw unprocessed data can be easily accessed. To this end two open-source wearables with hardware that meets the requirements for the assessment of physical activity (accelerometer), HR (optical sensor), health related variables and that also can communicate wireless with external devices using Bluetooth have been identified namely the Bangle.js 2 (https://www.espruino.com/Bangle.js2) and the Pinetime (https://www.pine64.org/pinetime/). Apps that will store the data locally at the devices and that allows for easy transfer to a external device for further processing and analysis of the data collected in phase 2 have been developed. Based on the outcome from phase 2 it is necessary before conducting phase 3 - naturalistic validation, to develop pre-processing methods, since the wearables memory capacity will not allow for storing the vast amount of raw data that will be collected during that phase. Thus, this phase will be an ongoing part of the entirety of the project.

Phase 2 - Laboratory development The goal of the phase 2 study is to develop and test algorithms to estimate the intensity of physical activity, energy cost of physical activity and blood pressure from the wearables signals under controlled laboratory conditions. In addition potential predictors that may increase the precision of the predictions will be investigated. The idea is that wearable data are collected at the same time as criterion measures, during a protocol from resting via walking and running at increasing speed until exhaustion. During the protocol they will be monitored using indirect calorimetry (Vyntus CPX, Vyaire Medical, Inc., Il, USA) and a heart rate monitor (Polar Electro Oy, Kempele, Finland) fitted around the chest, which fully comparable with a proper electrocardiogram (r\>0.99), simultaneously fitted with the wearable, a research grade accelerometer (Actigraph GT3+, Actigraphcorp, Pensacola, FL, USA) and a commercial wearable (Fit-Bit Sense, Fitbit international limited, Dublin, Ireland). This procedure will capture a continuous signal from the wearables at various physical activity intensities, from resting to maximal effort on individual level. This will allow to model that signal on group as well as on individual level, and to compare to the golden standard methods of indirect calorimetry and HR monitoring. Blood pressure data is goig to be collected using standard clinical monitors to compare an optical signal with that of a blood pressure meter.

Within the scope of the project, several several well-established health-related fitness tests will be conducted. For muscular strength, the hand-grip test will be used, and for flexibility, the sit and reach test. In addition, the timed up and go test (TUG) will be conducted [18]. Background data such as age, sex and body composition of the participants will also be collected. These tests are then used to explore the possibility to predict an individual's level of physical activity. The outcome of this phase is a set of models to count steps, and to detect the intensity of physical activity using the accelerometer, the optical sensor, the combination of the two, augmented with background data. It is also expected that a model to assess blood pressure using the wearables during rest is developed.

Phase 3 - Naturalistic validation The goal of the third phase of the project is to conduct a rigorous, independent validation of the algorithms in real-world conditions compared with gold standard measures. Thus after the development of algorithms in a controlled environment, the project will move on to test the wearable during free living. The same 50 participants (50% women, 18-65 years) from the second phase will ingest a dose of doubly labelled water (DLW), which is the golden standard to measure energy expenditure during free living with errors within ±3% from the true value as measured by respiratory chambers. Even if the DLW method is the golden standard, it is not often used as it is an expensive method involving enriching the body water of a subject with heavy hydrogen (2H) and heavy oxygen (18O), and then determining the difference in washout kinetics between both isotopes, which is being a function of carbon dioxide production, i.e., energy expenditure, using specialized equipment.

Based on the DLW kinetics, then it is possible to model the signals from the prototype wearable to find the best fit, i.e., how well the prototype assesses physical activity related energy expenditure. The outcome of this phase is expected to be an model that, based on the information provided by the wearable, assesses energy expenditure during free living conditions. The project also aim to determine the physical activity levels of the participants based on the information provided during phase two.

Data Analysis and Statistics In all phases this project will utilize the power of artificial intelligence, more specifically, various machine learning techniques to predict different aspects of physical activity, energy expenditure, and blood pressure on individual level. Machine learning automatically creates models based on data sets that have the potential of producing accurate predictions that control decisions in real-time without human interaction. The wearable(s) provide high-frequency raw data sampling from both accelerometry and optical sensor. Using this data for machine learning, provides good opportunities to indirectly measure physical activity and to individually advice healthy levels of physical activity. Several studies have tried to derive meaningful physical activity outcomes from accelerometer data collected via wrist-worn accelerometer. The studies based on machine learning aimed on the classification of physical activity types, e.g., sitting, standing, walking, running, etc., or PA intensity, e.g., sedentary, light, moderate, vigorous. They report a \>80% classification accuracy for activity type and a \>90% classification accuracy for intensity. Most of these studies use supervised machine learning approaches with labelled data, i.e., a ground truth is available, such as random forest, support vector machine, artificial neural network, and deep learning models. Only few studies use unsupervised machine learning models, such as hidden Semi-Markov models, and k-means clustering. Here, it is possible to create datasets with labelled data. However, since the overarching goal of this project is to develop a wearable that can adapt to individual users of the wearable, the learning approach must also be able to handle unlabelled data. To approach this issue reinforcement learning is suitable. Finally, since several background variables, including functional tests, have been collected they will be used to augment the models. Thus, the project will identify the best performing models to predict the physical activity intensity, frequency, and level from accelerometer and heart rate, and use that to predict energy expenditure and blood pressure. Best performing refers to both accuracy and explainability, and it is expected that different approaches excels in these two aspects. Moreover, based on more accurate measurements of physical activity and their accurate prediction using time-series data from accelerometers, an re-evaluation of the potential of functional tests as a point-in-time predictor for physical activity is going to be explored.