Tackling Two of the Most Important Unresolved Tasks in Reading Intervention

Overview

Poor readers face many risks. Low-literate children are at higher risk of attention difficulties, socio-emotional and mental health issues including low self-esteem, learned helplessness, anxiety, depression, suicidal ideation, self-harm, drug abuse and other health risks. Multiple socio-economic risks emerge from low literacy as well. We thus plan to extend programmatic work with a sub-set of the same sample of children we have worked with at Tier 1 (effective whole class) into Tier 2 (supplemental small group) reading intervention and then Tier 3 (further extended supplemental intervention) to examine how to use two promising interventions to solve one of the lingering challenges in reading intervention - improving reading of treatment non-responders in real school contexts run by regular teachers. We plan to take on the second unresolved task in reading intervention (impact at scale) by running this as a scaled up randomised control trial (RCT) in around 100 schools in multiple school boards in Quebec Alberta and Ontario, Canada.

Interventions and intervention responsiveness

The literature on Response-to-Intervention (RtI) documenting effective Tier 1 and Tier 2 early reading interventions studies suggests that an explicit pedagogical focus on both the grapho-phonemic and morphological aspects of the written English facilitate reading attainment. In previous work (Georgiou, 2016-21; Savage, 2012-17), we contributed to this literature by working with a sample of children in Alberta and Quebec (Georgiou, 2022; Georgiou et al., 2021; Savage et al., 2018; 2020). In these projects, we helped over 95% of students to become typical readers. However, 4.5% of children in our sample continued to struggle despite interventions. Such children have been labeled 'treatment resistors' (Blachman, 1994). More work is needed on these promising theory-driven interventions at tier 2. A very modest research base exists on interventions for these children at tier 3, where intensive, sustained small group or individual literacy tutoring is deployed in RtI models. Al Otaiba et al. (2014) identified only nine reasonably well-executed studies. Griffiths and Stuart (2013) described only promising case studies. Arias-Gundín and Llamazares (2021) systematically reviewed tiered interventions and reported finding three mixed Tier 2/Tier 3 interventions and six Tier 3 interventions. About half of the reported studies produced no significant treatment effects. Arguably, if these children did not benefit from existing interventions, the need for new theorised approaches is clear.

Theoretical contrasts in the 2 study arms

Both arms of our RCT receive phonics instruction because this is a well-evidenced approach (Fletcher et al., 2022). The two arms differ in the other part of their content. These elements and the rationale are described below.

Theoretical contrast in Arm 1): Set-for-Variability (SfV)

SfV involves mental flexibility in applying phonics to word reading. While many children learn SfV without explicit instruction, this may not be the case for treatment non-responders who struggle with phonic blending, vocabulary and matching blended pronunciations to known word pronunciations (Edwards et al., 2021; Tunmer and Chapman 2012). In SfV, phonic blending is broken into two teachable components: (1) assembly of a 'phoneme string' such as 'c'-'a'-'t' from the printed word 'cat' and (2) matching of this string to a known oral word pronunciation 'cat', followed by smooth orchestration of the two processes. The second process is rarely taught in standard phonics approaches. In our model, SfV is an explicitly taught strategy (Savage et al., 2022). Word-specific vocabulary is taught because vocabulary and synthetic phonic blending are co-requisites to effective SfV use. SfV is a generative strategy for text reading that directly aids self-teaching (Share, 1995) because SfV works both for words typically deemed 'regular' (e.g., 'cat') that do follow standard phonics rules and for 'exceptions' (e.g., 'go', 'he', 'wasp') that do not follow the standard phonics rules. SfV also aids reading comprehension (Savage et al., 2018).

Research has shown that sustained SfV intervention improves word reading and reading comprehension outcomes in Tier 2 interventions in multi-site trials run by graduate students for at-risk students in Grade 1 (Savage et al., 2018) and the benefits may transfer to French in dual-language French Immersion instruction contexts (Côté et al., 2021). We have also reported data on large, scaled Tier 1 and 2 trials run by teachers and educational assistants in authentic school contexts in Grades 1, 2, and 3 in Alberta (Georgiou, 2022; Georgiou et al., 2021) and Reception (kindergarten) and Grade 1 in England (Savage et al., 2022), showing clear effects of intervention on reading accuracy and comprehension. However, while we have promising data from Tier 2 and 3 trials run by trained graduate students, no data exists on Tier 2 and 3 SfV interventions after foundational vocabulary and phonological processes have first been supported, and on external validity trials where the intervention is run by regular teachers. There is also a need to establish effects of a comprehensive SfV intervention alone (Colenbrander et al., 2022). Most existing SfV interventions involve other distinct simultaneously delivered components (e.g., Lovett et al., 2000). Our delivery of SfV thus pinpoints unique effects of this variable on reading outcomes (reading accuracy). We specifically expect effects on reading regular and exception words and on measures of SfV itself between pre and post-tests.

Theoretical contrast in Arm 2: Morphology

Attention to teaching morphology may be crucial because of the allegedly morpho-phonemic nature of written English (Venezky, 1977) and the role morphology may play in word reading and reading comprehension development (Kirby and Bowers, 2017; Levesque et al., 2021). Prior intervention trials and systematic reviews suggest replicable effects in both typical and atypical readers (Goodwin and Ahn, 2010; 2013). Galuschka et al.'s (2020) meta-analysis of all well-designed interventions for older struggling readers suggests that effects of morphology-focused interventions may be stronger than phonics-focused interventions. However, Galuschka et al. caution against drawing strong conclusions due the modest number of well-executed studies currently available. Few morphology studies have been placed within a research RtI model at either tier 2 or tier 3. Indeed, there exist relatively few teacher-delivered morphology interventions and some morphological interventions have been reported to be a challenge for teachers to implement (Colenbrander et al., 2018), hence trials run by teachers in authentic school contexts are needed. Finally, we further explore individual differences in responsiveness to morphological interventions (Georgiou et al., 2021; Savage et al., 2023). We specifically expect effects on reading polysyllabic words and on measures of morphological awareness itself between pre and post-tests.

Study design

We thus seek to evaluate an Ontario- Quebec- and Alberta-based 2-arm Randomized Control Implementation Trial of an RtI model. The interventions we will share in Quebec Ontario and Alberta are based on proven Alberta site implementation (Dunn et al., 2024) that has now impacted policy. We extend this logic here to include the teachers of at-risk and older persistently poor readers (tier 2 and tier 3 interventions) across provincial sites.

In our tier 2, we will contrast professional development (PD) in phonics + morphology with PD in phonics + strategy (set-for-variability) based interventions in a 2-arm trial. In our tier 3 phase of the study, children in the two trial arms will receive more sustained intervention, including professional development (PD) content derived from the multi-componential interventions run earlier in Alberta contrasting phonics plus morphology with phonics plus set-for-variability.

We will use our proven models of effective sustained PD delivery with loops of multi-component supports for practice (Bell et al., 2022; McNally et al., 2019; Savage et al., 2013, 2022) in both intervention arms in both tiers of study.

The research questions for Studies 1 and 2 are:

Question 1: What are the effects of bringing PD on morphology and SfV to bear respectively to help teachers improve reading in Grades 2 and 3 for students who are poor readers and non-responsive to sustained evaluated evidenced school-wide Tier 1 interventions at tier 2? and where needed in Grade 3 at tier 3?

Question 2: Can we scale up these interventions using extensive teacher PD and support with effects across trial sites in Quebec, Alberta and Toronto school boards?

We will also explore who responds to intervention (e.g., moderating effects of prior attainment) and why? (e.g., mediating effects of morphology and SfV).

Theory of change

Our theory of change is an explicit causal one that tests the claim that our professional development to teachers on instruction in phonics plus set-for-variability then delivered by teachers in their regular classrooms impacts word reading outcomes through children showing improved use of phonic decoding in word reading and set for variability abilities. Cascading effects may be evident in reading comprehension subsequently.

Our theory of change is an explicit causal one that also tests the claim that our professional development to teachers on instruction in phonics plus morphology then delivered by teachers in their regular classrooms impacts word reading outcomes through children showing improved use of phonic decoding and morphological abilities. Cascading effects may be evident in reading comprehension subsequently.

Power calculation:

We used Power up (Dong and Maynard, 2013) to assess power for school samples of 80,90, and 100. The latter is described below.

There are eight key factors associated with statistical power of particular study samples and sample size requirements to achieve a specified statistical power in nested designs.

1. T he minimum relevant size impact (MDES, the smallest size impact it is important to detect, if it exists)
2. The adopted level of statistical significance (α) or probability of making a Type I error (i.e., concluding there is an impact when there really is not). Commonly, evaluators set alpha equal to .05.
3. Desired level of statistical power (1-β), where β is the probability of making a Type II error (failing to detect a true impact if it occurs). Commonly, evaluators adopt a power level of .80.
4. use of one-tailed or two-tailed testing, with two-tailed testing being most common.
5. Use covariates to reduce measurement error (Bloom, 2006; Bloom et al, 2007),
6. Assumption of fixed or random effects across sample blocks or clusters, which relates to the intended application of the study findings.
7. Intraclass Correlation (ICC)
8. Average cluster size

Table 1. Specific assumptions of our calculations

1. .1
2. Alpha = .05
3. Beta = .8
4. Two-tail
5. .53 (post-test) .44 (delayed post-test)
6. Random
7. .54
8. 8. 8 _________________________________________________________________ A sample of 100 schools gives us an acceptable MDES of 0.258. Selected References Georgiou, G., Savage, R., Dunn, K., Bowers, P., and Parrila, R. (2021). Examining the effects of Structured Word Inquiry on the reading and spelling skills of persistently poor grade 3 readers. Journal of Research in Reading, 44, 1, 131-153. doi:10.1111/1467-9817.12325 Savage, R.S., Georgiou, G., Parrila, R., and Maiorino, K. (2018). Preventative reading interventions teaching direct mapping of graphemes in texts and set-for-variability aid at-risk learners. Scientific Studies of Reading, 22, 225-247. doi: 10.1080/10888438.2018.1427753