The FAB Score Inter-rater Reliability Study

Study purpose:

Burn survivors face numerous physical challenges (e.g. reduced range of motion, upper/lower limb impairments, pain, swelling and itching) that affect activities of daily living (e.g. feeding, hygienic self-care, dressing, getting in/out of bed, walking and stair climbing). Assessment of physical function and independence, through valid and reliable instruments, is essential during inpatient rehabilitation of adult burn survivors in order to optimise their physical recovery and determine likely discharge outcomes. The FAB assessment is routinely carried out on patients at St Andrew's Burns Centre and the predictive validity of the FAB score has previously been tested in adult burn patients, however it has yet to be tested for inter-rater reliability. Therefore, the main purpose of this study is to investigate the ability of the FAB score to collect accurate data, free from measurement error, when used by multiple raters observing the same clinical episode.

Study design:

A quantitative reliability study will be carried out. All patients admitted to adult inpatients at St Andrew's Burn Centre will be screened and recruited according to inclusion/exclusion criteria by the Chief Investigator. There will be no change to patients' routine care as it is standard protocol for patients admitted to the burns ward to receive a FAB assessment within 48h of admission (FAB 1), and 48h of discharge (FAB 2), with interim assessments performed as necessary to monitor patients' progress and aid in maintaining motivation. In order to avoid any risk of potentially causing patients distress from too many assessing therapists, a maximum of three clinicians will be selected from a convenience sample of the burns inpatient therapy team (n=12). For each episode, the most clinically senior of the three raters will conduct the FAB assessment. Each rater will score the patient individually and will be blinded to the other observers' results. If a patient requires the assistance of more than one physiotherapist/occupational therapist to complete a task, the clinician is permitted to report, "I am assisting/supervising/not assisting", but not state how this would be represented as a FAB score.

Data analysis:

Scores from each assessment episode will be imported into MedCalc statistical package. Calculation of the Intraclass correlation coefficient (ICC) will assess the agreement (and therefore the inter-rater reliability) of the results provided by the three therapists, both as individual FAB score components, and as an overall functional outcome measure. The ICC values range between 0 and 1, with values ≥0.70 considered 'good' agreement. Therefore, the desired ICC for the study will be set at 0.70 and ICC values will be interpreted as follows: ≤0.50 poor agreement, 0.50-0.70 moderate agreement, 0.70-0.90 good agreement, ≥0.90 excellent agreement and 1.00 perfect agreement. The statistical analysis of the ICC is obtained by analysis of variance (ANOVA). As there will be three different raters for each scoring episode, a One-Way Random Effects Model will be used in a fully crossed study design. A One-Way model will allow flexibility of rater selection depending on therapists' availability, as consistent participation on a busy hospital ward will not be achievable. As this model will not be able to separate rater and total variance (i.e. rater and error variances combined), both error and rater variances must be small to achieve a desirable agreement level.

Sample size:

In order to determine a sample size for the ICC it is necessary to have the desired ICC value, the expected confidence level, and the confidence level associated with the confidence interval. As ICC values ≥0.70 are considered 'good' agreement, the ICC will be set at 0.70. The expected confidence interval width is determined by a pre-specified power of at least 0.8 with 95% confidence interval. The confidence level associated with the confidence interval is examined in graphical form, with expected 95% confidence interval width plotted against the desired ICC value. This reveals that for a sample size yielding a 95% confidence interval, and confidence levels not exceeding 0.56 (0.70 x 0.80), the study will require a minimum of 60 measurements (FAB scores). As three raters will be present for each assessment, this will therefore require a minimum of 20 patient scoring episodes.

The study design, data analysis and sample size for this quantitative reliability study is based on a previous study investigating a different tool that uses a similar scoring method, the Functional Assessment for Burns in Critical Care (FAB-CC), at St Andrew's Burns Intensive Care Unit.