The Swedish Versions of ADOS and ADI-R: Psychometric Validation

Objectives:

The present study seeks to determine for the first time the reliability and validity of the Swedish versions of the Autism Diagnostic Interview-Revised (ADI-R)and Autism Diagnostic Observation Schedule (ADOS). The combination of these two assessments is deemed the "gold standard" of diagnosing ASD in international practice and research. ADI-R and ADOS are widely used (translated into 15 languages) and have been rigorously and convincingly evaluated in other countries. The Swedish translations of the ADI-R (interview schedule, algorithms) and ADOS (observation schedules, algorithms) have not yet been tested for scientific properties and feasibility, hampering best clinical practice, evidence-based diagnostics and research premises for ASD in Sweden. Therefore, this study aims at standardizing the Swedish forms of the ADI-R and ADOS to and ensure their cross-cultural validity and clinical utility. A particular objective of the study is to establish the agreement of ADI-R and ADOS with register-based diagnoses. The latter is of paramount importance for judging the significance and generalizability of findings from large scale register research, which is a specific strength of Swedish science.

Instruments:

The ADI-R is a clinical diagnostic instrument for assessing ASDs in children, adolescents and adults. The ADI-R provides a diagnostic algorithm for autism as described in both the ICD-10 and DSM-IV-TR and necessary information for making a diagnosis of other ASD in accord with the classification manuals. The ADI-R is appropriate for children and adults with mental ages from about 18 months and above. It is a standardized, semi-structured clinical review for caregivers of children, adolescents and adults. The interview contains 93 items on three content areas: quality of social interaction, communication and language, and repetitive, restricted and stereotyped interests and behavior. The measure also includes other items relevant for treatment planning, such as self-injury and over-activity. Responses are scored by the clinician based on the caregiver's description of the child's behavior. All questions ask about current behavior, with the exception of a few behaviors that only occur during specific age periods. In addition to asking about current behavior, each question focuses on the time period when the behaviors were likely to be most pronounced - generally, between the ages of 4 and 5 years. The interview starts with an introductory question followed by questions about the subject's early development. The next 41 questions cover verbal and nonverbal communication. Questions 50 through 66 ask about social development and play. The next 13 questions deal with interests and behaviors. The final 14 questions ask about "general behavior," including questions about memory skills, motor skills, over-activity and fainting.

The ADI-R interview generates scores in each of the three content areas. Elevated scores indicate problematic behavior in a particular area. Scores are based on the clinician's judgment following the caregiver's report of the child's behavior and development. For each item, the clinician gives a score ranging from 0 (not autistic) to 3 (severely autistic). A classification of autism is given when scores in all three content areas of communication, social interaction, and patterns of behavior meet or exceed the specified cutoffs, and onset of the disorder is evident by 36 months of age. For other ASD information from the ADI-R is used informally: e.g. a diagnosis of Asperger syndrome is indicated, if the criteria for autism are fulfilled in face of normal language development, or a diagnosis of PDD-NOS, if social interaction deficits and another behavior domain exceed the cut-offs. This interviewer-based instrument requires substantial training in administration and scoring. A trained clinician can administer the ADI-R to the parent of a child suspected of autism in approximately 2 ½ hours.

The ADOS is a semi-structured assessment of communication, social interaction and play or imaginative use of materials for individuals suspected of having autism or other ASD. It consists of four modules (plus a new toddler module), each of which is appropriate for children, adolescents and adults of differing developmental and language levels, ranging from no expressive or receptive language to verbally fluent adults. The ADOS builds on standard activities that allow the examiner to observe the occurrence or non-occurrence of behaviors that have been identified as important to the diagnosis ASDs across developmental levels and chronological ages. The examiner selects the module that is most appropriate for a particular child, adolescent and adult on the basis of his/her expressive language level and chronological age. Structured activities and materials, and less structured interactions, provide standard contexts in which social, communicative and other behaviors relevant to ASDs are observed. In effect, the ADOS provides a 30 to 60 minute observation period during which the examiner presents the individual being assessed with numerous opportunities to exhibit behaviors of interest in the diagnosis of ASD through standard 'presses' for communication and social interaction. Module 1 is intended for individuals who do not consistently use phrase speech Module 2 is intended for individuals with some phrase speech who are not verbally fluent. Module 3 is intended for children for whom playing with toys is age- appropriate who are verbally fluent. Module 4 is intended for verbally fluent adolescents and adults. Though the superficial appearance of the different modules is quite varied, the general principles involving the deliberate variation of the examiner's behavior using a hierarchy of structured and unstructured social behaviors are the same. The standardization of the ADOS lies in the hierarchy of behavior employed by the examiner and the kinds of behaviors taken into account in each activity during the overall ratings. The activities serve to structure the interaction; they are not ends in themselves. The object is not to test specific cognitive abilities or other skills in the activities, but to have tasks that are sufficiently intriguing that the child or adult being assessed will want to participate. What the examiners not do is often as important as what they do. The ADOS is not trivial to administer, so that priori training is advised even for experienced clinicians.

The modules consist of between 10 to 15 activities with up to 31 accompanying items to code. Comparable to the ADI-R, items are scored 0 to 3 depending on the presence and severity of observed symptoms. Subsets of items in each module are used to generate diagnostic algorithms for each module. Items and the thresholds for classification of autism and other ASDs differ for each module. In the new diagnostic algorithms, classification is made on the basis of exceeding thresholds for social affect (social and communication domain combined), restricted and repetitive behaviors and a total score of both.

Studies on the reliability (e.g. internal consistency, inter rater, retest) and validity (e.g. diagnostic) of the original and many international adaptations of the ADI-R and ADOS have been satisfactory to excellent throughout. For instance, the initial work on the psychometric properties of the English ADI-R showed multi rater weighted Kappas almost exclusively exceeding .65 for all algorithms items, internal consistency ranging from .69 (stereotypies) to .95 (social interaction domain) and item retest reliabilities of mean weighted Kappa = .72. Diagnostic validity studies showed a classification accuracy of 94% compared to clinical expert consensus diagnosis. Comparable data is available for example for the German, Bulgarian, Greek and Dutch versions. The recently presented revised diagnostic algorithms of the ADOS showed sensitivities of >.91 and specificities of >.84 for the discrimination of autism and non-spectrum conditions in N = 1630 individuals with at least some words speech. The diagnostic validity of the new algorithms has been cross-validated and also demonstrated excellent psychometric values in independent cross-cultural studies.

Procedure and complementary assessments:

Located at the clinical sites or Karolinska Institute Center of Neurodevelopmental Disorders (KIND), we will examine subjects with suspicion of ASD, other mental disorders and typically developing using the ADI-R and the ADOS. Testing for general intellectual abilities will also be carried-out using the Wechsler Intelligence Scales for Children-IV, a multidimensional IQ test-battery to determine general IQ, performance and verbal IQ as well as a profile of intellectual capacities. They comprises core and supplemental subtests forming composite neuropsychological scores. In addition, the Raven matrices are collected, especially to cover ASD cases who show problems with IQ testing using the Wechsler's, e.g. due to low IQ or low verbal skills. Before using the instruments, involved researchers will receive extended training for these scales by the applicant, who is a certificated clinical neuropsychologist and recognized ADI-R and ADOS trainer. Other Swedish trainers for the ADI-R and ADOS (Vivianne Nordin, Eric Zander, Karin Olafsdottir) are included in these assessments. For subsequent inter rater reliability analyses all administrations are videotaped. Up to 10 independent raters skilled on the ADI-R and ADOS, but blind to the children's diagnoses or history, rate these videos in the course of the study. In order to determine retest reliability half of the participants re-examined after three months by an independent examiner.

Independent DSM-IV-TR/ICD-10 consensus diagnoses on all participants are established by experienced clinicians (child psychiatrists, clinical child psychologists).

Work plan:

Study 1: Reliability ADI-R

1. Internal consistency will be calculated for the three ADI-R algorithm behavior domain scales: social interaction, communication, repetitive-stereotyped behavior using Cronbach's alpha.
2. Multirater reliability is determined for the ADI-R between four independent raters for all algorithm items, the three behavior domains and overall diagnosis using % agreement, (weighted) Kappa and Pearson product moment correlation.
3. Retest reliability for the ADI-R is computed for a three-month interval on all algorithm items, the three behavior domains and overall diagnosis using % agreement, (weighted) Kappa and Pearson product moment correlation.

Study 2: Reliability ADOS

1. Internal consistency will be calculated for the three ADOS algorithm behavior domain scales: social interaction, communication, repetitive-stereotyped behavior using Cronbach's alpha.
2. ADOS multi-rater reliability is determined between four independent raters for all algorithm items, the three behavior domains and overall diagnosis using % agreement, (weighted) Kappa and Pearson product moment correlation.
3. Retest reliability for the ADOS is computed for a three-month interval on all algorithm items, the three behavior domains and overall diagnosis using % agreement, (weighted) Kappa and Pearson product moment correlation.

Significance/patient gain/impact:

The value and impact of the study for clinical practice and research on ASD in Sweden have been pointed-out several times before in this application. ASD are no rare disorders, affecting 1% of the general population, causing substantial burden to affected individuals, their families and society. The combined usage of the ADI-R and the ADOS is viewed the gold standard of diagnosing ASD in international child and adolescent psychiatry. They are also valuable tools for intervention and educational planning, and unlike many other scales, continuously updated and improved. Their introduction has been decisive for progress in many fields of basic and applied ASD research. Most guidelines for diagnosing and treating ASD of professional Societies list them as first choice instruments. Both of them have been adapted and successfully evaluated for many languages and cultures. Thus they are considered the best clinical practice scales for ASD. Unfortunately, they have not yet been standardized in Sweden, impeding better clinical practice for patients, the availability of evidence-based instruments for clinicians, and research prerequisites on ASD in Sweden. The present study aims at filling this gap, by broadly establishing reliability and validity of the Swedish translations of the ADI-R and ADOS.

In summary, this is a study aiming at a long-term improvement of clinical and scientific structures in Sweden. Its impact will mean enhancing evidence-based clinical practice for patients and experts and a stimulation of internationally competitive ASD research in Sweden.

Department/research team:

The study is conducted at the Center for Neurodevelopmental Disorders at Karolinska Institutet (KIND), Stockholm, Sweden. KIND is a competence center for research, education and development in child and adolescent psychopathology. KIND's mission is to mutually connect research at KI and clinical services in Stockholm County in order to generate research results of high clinical significance and patient gain. Regarding the development of clinical services and research structures, the construction, adaptation and evaluation of clinical scales is a core objective of KIND.