The Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R): A Scale to Assist the Diagnosis of Autism Spectrum Disorder in Adults: An International Validation Study

Abstract

The Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R) is a valid and reliable instrument to assist the diagnosis of adults with Autism Spectrum Disorders (ASD). The 80-question scale was administered to 779 subjects (201 ASD and 578 comparisons). All ASD subjects met inclusion criteria: DSM-IV-TR, ADI/ADOS diagnoses and standardized IQ testing. Mean scores for each of the questions and total mean ASD vs. the comparison groups’ scores were significantly different (p < .0001). Concurrent validity with Constantino Social Responsiveness Scale-Adult = 95.59%. Sensitivity = 97%, specificity = 100%, test–retest reliability r = .987. Cronbach alpha coefficients for the subscales and 4 derived factors were good. We conclude that the RAADS-R is a useful adjunct diagnostic tool for adults with ASD.

Introduction

The RAADS was designed to address a major gap in screening services for adults with autism spectrum disorders. With the increased prevalence of the condition and the fact that adults are being referred or self-referred for services or diagnosis with increasing frequency, this instrument is a useful clinical tool to assist clinicians with the diagnosis of this growing population of higher functioning individuals in adulthood. This study was designed to standardize the Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R) in a large number of subjects at nine centers on three continents.

The RAADS-R is a modified version of the Ritvo Autism Asperger’s Diagnostic Scale (Ritvo et al. 2008). The rationale for its development was the need for a clinical adjunct diagnostic tool. While several diagnostic scales for children and adolescents with ASD are available (Attwood 1998; Ehlers et al. 1999), only one peer- reviewed scale, The Autism-Spectrum Quotient (AQ) was designed specifically for adults (Baron-Cohen et al. 2001). Its authors emphasize that it is useful in identifying autistic traits and as a screening instrument but not as a diagnostic tool. The AQ is mailed in by the participant, unlike the RAADS-R, which is designed to be administered by a clinician in a clinical setting. Baron-Cohen et al. (2005) propose a diagnostic system as well: The Adult Asperger’s Assessment (AAA): A Diagnostic Method. While the AAA is a good system, it is a complete diagnostic system, unlike the RAADS-R. It is lengthy and complicated to administer and is still relatively new. The initial RAADS was developed around the same time period. The original RAADS contains 78 empirically derived questions that assess symptoms based upon DSM-IV-TR (American Psychiatric Association 2000) and ICD-10 World Health Organization (1992) diagnostic criteria that were extensively tested prior to inclusion on the scale. It proved accurate in discriminating autistic and Asperger’s subjects (N = 37) from comparison subjects (N = 57) when tested (e.g., sensitivity = 1, specificity = 1). Seventy-six of the 78 questions significantly distinguished ASD subjects from comparison subjects (Ritvo et al. 2008).

Questions on the initial RAADS assess developmental pathology in three symptom areas: language, social relatedness, and sensory-motor (re: DSM-IV-TR). After critical review and the results of a factor analysis, a revised 80-item version was developed (the RAADS-R) with the addition of a fourth symptom area (circumscribed interests), two questions, and several word clarifications. Like the RAADS, the RAADS-R is a self-report instrument, designed for adults (18 +) with average or above average intelligence.

In order to validate the RAADS-R, a uniform research protocol was developed to be implemented at nine research centers on three continents. The protocol assured uniformity of diagnosis and data acquisition, and its implementation in multiple centers afforded an opportunity to assess the applicability of the scale in a variety of international clinical settings: (a) UCLA; (b) The Child Study Center, Yale University; (c) University of Utah; (d) Mt. Sinai Medical School, New York; (e) The Geneva Center, Toronto, Canada; (f) King’s College, London, England; (g) Monash University, Melbourne, Australia; (h) Griffith University, Brisbane, Australia; and (i) ASPECT, Sidney, Australia.

At the time this research was concluding, the American Psychiatric Association’s DSM-V committee proposed new diagnostic criteria for Autism Spectrum Disorders (APA DSM-5 2010). These new guidelines propose combining individuals who meet the DSM-IV-TR criteria for Autistic Disorder and Asperger’s Disorder into one category called Autistic Spectrum Disorder (ASD). The results of this study will present subjects grouped according to both the DSM-IV-TR and proposed DSM-V criteria. There has been ongoing controversy among clinicians about the relationship of autism and Asperger’s disorder. Diagnostic approaches and definitions are described in the literature (Fitzgerald and Corvin 2001; Klin et al. 2005; Volkmar and Lord 1998).

Methods

Participants

Research Centers and Informed Consents

Nine centers on three continents were selected to provide a worldwide spectrum of English speaking subjects. Human Subject Protection Protocol and Consent to Participate Forms were approved by the California Institutional Review Board, Pasadena, California for all UCLA subjects, and served as models for the other centers to assure uniformity. Each center has ongoing clinical and research programs focusing on autism, and each center complied with their rules and regulations concerning the protection of human subjects. Each of the nine centers provided autistic and Asperger’s disorder subjects and some also contributed comparison subjects.

Participant Grouping

The study population included 4 groups as follows: (a) subjects with a diagnosis of Autistic Disorder (n = 66), (b) subjects with a diagnosis of Asperger’s Disorder (n = 135), (c) subjects with no previous DSM-IV-TR diagnosis (n = 276), and (d) subjects with an axis I DSM-IV-TR diagnosis other than Autistic Disorder, Asperger’s Disorder, or PDD NOS (n = 302). Subjects in this group had the following diagnoses: Social Phobia, Schizophrenia, Major Depressive Disorder, Bipolar Disorder Type I and II, Obsessive Compulsive Disorder, Dysthymic Disorder, Generalized Anxiety Disorder, Post Traumatic Stress Disorder, Psychotic Disorder NOS, Anorexia Nervosa and Polysubstance Dependence. For some data analyses, the above groups were collapsed into two groups as follows: (a) Autism Spectrum Disorder (ASD) subjects (N = 201), and (b) all comparison subjects (N = 578). No participants from either group declined to participate.

Inclusion Criteria

A standardized research protocol was implemented at each of the nine centers to assure uniformity of diagnosis. All ASD subjects had to meet the following criteria: (a) a clinical DSM-IV-TR diagnosis of autistic or Asperger’s disorder, (b) an ADI/ADOS or at least an ADOS module 4 scale to assure they met the cut off for ASD, and (c) a Full Scale IQ score of 80 or above. If prior formal IQ test results were not available, a WAIS or a WASI was administered. Constantino’s Social Responsiveness Scale-Adult Research Version (SRS-A) was obtained if a suitable informant accompanied the subject, as it is a second party rating scale (a second party rating scale is administered to a person other than the subject who answers questions about the subject, such as a family member or a friend). The SRS was selected because it is easy to administer and is reported to have good psychometric properties: internal consistency 0.91–0.97, test–retest reliability 0.84–0.97 and interrater reliability 0.74–0.95 (Bolte et al. 2008). The respondents completing the measure in this study were spouses, parents and siblings. A subset of subjects at UCLA was retested after an average of 1 year to determine test–retest reliability.

Two groups of comparison subjects were selected. The first consisted of 276 volunteers who did not have, and never had a DSM-IV-TR diagnosis. They were recruited from clinic staff members and students, and were individually administered the scale by a research staff member. The second group consisted of 302 volunteers recruited at an adult psychiatric out-patient clinic. They were formally diagnosed as having a DSM-IV-TR diagnosis other than ASD by an experienced board certified psychiatrist or a licensed psychologist who administered the scale individually to each. The majority of the comparison cases were recruited by the principal investigator and UCLA.

Materials

Design of the Revised Scale: The RAADS-R contains 64 symptom-based questions and 16 non-symptom based responses (see Appendix 1 for scale and Appendix 2 for scoring instructions). The questions were divided into four sub scales per DSM-IV-TR criteria as follows:

• 1. Social Relatednesss—total of 39 questions: 1, 6, 8, 11, 14, 17, 18, 25, 37, 38, 3, 5, 12, 28, 39, 44, 45, 76, 79, 80, 20, 21, 22, 23, 26, 31, 43, 47, 48, 53, 54, 55, 60, 61, 64, 68, 69, 72, 77.

• 2. Circumscribed Interests—total of 14 questions: 9, 13, 24, 30, 32, 40, 41, 50, 52, 56, 63, 70, 75, 78.

• 3. Language—total of 7 questions: 2, 7, 27, 35, 58, 66, 15.

• 4. Sensory Motor—total of 20 questions: 10, 19, 4, 33, 34, 36, 46, 71, 16, 29, 42, 49, 51, 57, 59, 62, 65, 67, 73, 74.

Procedures

Administration of the RAADS-R to ASD Subjects

Following initial screening, a clinician interviewed each subject to confirm the diagnostic information and IQ data, completed the ADI/ADOS or ADOS module 4, and then introduced the scale and explained how to answer the questions. The clinician remained with each subject as the questions were being answered to offer clarifications when asked, and to make sure answers were entered in the proper columns on the Likert scale. An SRS-A scale was obtained from cooperating accompanying adults when available.

Administration of the RAADS-R to Comparison Subjects

After initial diagnostic and IQ screening, all were given the RAADS-R to complete in the presence of a clinician. Each of the nine participating centers was asked to score, review, and mail in full packets of ASD subjects. Some of the participating centers contributed data to the comparison group subject pool.

Data Analysis

Data were coded by each center to preserve anonymity prior to being sent to UCLA. Data were then recoded and entered into an Excel spreadsheet and then transferred to SAS and MPlus for statistical analyses. Exploratory factor analysis was carried out using both orthogonal and oblique varimax rotation under maximum likelihood estimation. A scree plot of the eigenvalues was examined to determine the number of factors. A confirmatory factor analysis was carried out on the final factors using robust weighted least squares (WLSMV) estimation methods as the original items are ordinal. The factor loadings were compared across gender and ASD subgroups to assess constancy (differential item function analysis). Cronbach alpha was also computed to assess the internal consistency of the derived factors. Unadjusted means were compared using one-way analysis of variance and demographics adjusted means were compared using analysis of covariance methods. Associations between continuous variables were assessed using Pearson correlations. Sensitivity, specificity and accuracy of a given factor or score for distinguishing between ASD versus non ASD was computed using non-parametric ROC (receiver operator characteristic) methods. Unweighted accuracy is defined as the average of sensitivity and specificity. The optimal threshold (cut point) was defined as the value of the given factor or score that maximized accuracy.

Results

Subjects’ Demographics

Table 1 contains the sex distribution and demographic characteristics of the four subject groups.

Table 1 Demographic characteristics of the study sample

Diagnostic Accuracy (Sensitivity and Specificity)

The total RAADS scores ranged from 44 to 227 in the ASD subjects and from 0 to 65 in the comparison groups. By ROC curve analysis, we determined that the best threshold for distinguishing between the two groups was a score of 65. Using this value, all 578 comparison subjects were correctly predicted with no “false positives” (specificity = 100%).

Six ASD subjects from three of the nine centers scored below the threshold (non-ASD predicted range). These “false negatives” yielded a sensitivity of 97% (see Table 2).

Table 2 Characteristics of the only 6 ASD subjects (3%) whose mean RAADS-R scores were below 65 (false negatives) (mean age = 19.9)

Table 3 shows the sensitivity, specificity and accuracy for the original four subscale domains and the four factors as well as the total RAADS-R score. As can be seen, the total RAADS-R score has the highest accuracy.

Table 3 Accuracy in distinguishing ASD (N = 201) from non-ASD (N = 578) (ROC analysis)

Group Comparisons

One-way ANOVA comparisons across all groups, mean RAADS-R scores, and statistical characteristics of subject groups are contained in Table 4. The means and statistical characteristics obtained after collapsing the four groups into two (all ASD subjects vs. all comparison subjects) are shown in Table 5. The mean RAADS-R scores of the ASD, the autistic, and the Asperger’s disorder groups (Tables 3 and 4) were significantly different from the means of the comparison groups (p < .0001). This was also true at each of the nine research centers. The original subscale domains, the factors and the total score are all significantly different in ASD versus non ASD.

Table 4 One way ANOVA comparisons and statistical characteristics of participant groups

Table 5 Means and statistical characteristics for total score, ASD vs. comparison groups

An age and sex adjusted ANOVA of total RAADS-R scores and subscale scores, across diagnostic groups showed group differences. Sex was not significant as a main effect (p = .06) but sex-diagnosis was significant (p = .0043). The effect of diagnosis slightly varied by sex, however, diagnosis was the strongest effect. Age was significant as a main effect (p = .0035) and age-diagnosis effect was also significant (p < .05). In ASD subjects, an increase in age was positively correlated with an increase in mean RAADS-R scores. (Spearman correlation 0.37, p = .0003). However, age–sex–diagnosis effect was not significant (p = .1299). Repeated analyses after adjustment for sex and age between and among the subject groups showed significant differences between ASD and comparison groups (p < .0001).

Pair-Wise Comparison of Each Question

Post-hoc unequal variance t tests were performed for pair-wise comparison of each of the 80 questions. The ASD subjects had significantly greater scores on each of the individual RAADS-R items (N = 80) than either of the comparison groups (p < .0001 for all comparisons). Total RAADS-R mean scores between autistic and Asperger’s disorder subject groups were not significantly different.

Validity Measures

All ASD subjects met the research diagnostic criteria of each center and had an ADOS module 4 score in the autism range (these were inclusion criteria). SRS-A scales were administered to 69 first-degree relatives of ASD subjects. The concordance rate was 95.59% (all but three scored in the “autism” range).

Analysis of the Subscales

Mean scores and Cronbach alpha correlation coefficients were computed for each of the four subscale domains described previously: circumscribed interests = .903, language = .789, sensory motor = .905, social relatedness = .923 (for the four participant groups’ mean scores of the subscales, see Table 6).

Table 6 Statistical analysis of the subscales

Factor Analysis

Factor Analysis was carried out using both varimax (orthogonal) and oblique (non-orthogonal) varimax rotation. The oblique rotational method was preferred because it assumes the underlying constructs to be interrelated. In addition, the oblique model had simpler factor structure that was easier to interpret.

Four factors were identified using the oblique rotation method. These factors had a simpler factor structure with much less cross loading compared to the orthogonal results, making for easier interpretation. These final oblique factors also had a higher correlation with the original subscales compared to the orthogonal factors. Factors I and IV correlated most strongly with the social construct (r = .80). Factor II correlated most strongly with circumscribed interests (r = .78), factor III with sensory motor (r = .88). The language construct was not picked up with the four-factor model. The variables determined below are based on factor scores.

• 1. Factor I: Social Relatedness, containing questions related to empathy, intimacy and social language accounted for 23.9% of the variance. Questions: 1, 6, 8, 11, 13, 14, 15, 16, 18, 31, 37, 38, 43, 48, 52, 53, 58, 62, 66, 68, 72, 74, 77; α = .93.

• 2. Factor II: Circumscribed Interests, also with questions relating to social blindness, accounted for 25.0% of the variance. Questions: 3, 4, 5, 7, 9, 12, 16, 17, 20, 22, 25, 27, 28, 32, 33, 39, 41, 44, 45, 49, 50, 56, 60, 64, 76, 78, 79, 80; α = .95.

• 3. Factor III: Sensory Motor accounted for 22.4% of the variance. Questions: 2, 10, 19, 24, 29, 34, 35, 36, 40, 46, 51, 54, 57, 59, 65, 70, 71, 73; α = .87.

• 4. Factor IV: Social Anxiety accounted for 18.9% of the variance. Questions: 21, 23, 26, 30, 42, 47, 55, 61, 63, 67, 69, 75; α = .89.

A confirmatory factor analysis using these four factors showed that the coefficient of variation of the factor loadings was 20, 18, 18, and 19%, respectively, implying similar loadings for each item within a given factor. A comparison of factor loadings (differential item function) by gender and by ASD vs. non-ASD showed no significant differences with the mean difference of 18% or less.

Test–Retest Reliability

Fifteen ASD subjects and fifteen comparison subjects without another DSM-IV-TR diagnosis were retested at UCLA after a mean interval of 1 year (maximum interval 15 months, minimum interval 10 months) The initial RAADS-R mean score for the ASA subjects was 155, and when repeated was 149. The initial RAADS-R mean score for the comparison subjects was 20, and 21 when repeated (Table 7 contains the statistical analyses for test–retest data).

Table 7 Test-retest data statistical analysis

University/Center Effect

The total mean RAADS-R scores for the ASD subjects from each of the 9 centers were significantly different from the comparison groups (p < .0001) and there were significant differences between the universities (see Table 8).

Table 8 RAADS-R scores and ranges by research center: ASD subjects

Clinical Implications of RAADS-R Scores

A RAADS-R score of 65 or greater is consistent with a clinical diagnosis of ASD. A RAADS-R score of 64 or lower is not consistent with a diagnosis of ASD (sensitivity = 97%, specificity = 100%). It must be emphasized, however, that if a subject has a score of 64 or lower but clinical judgment indicates that ASD is present, the clinical judgment should take precedence. This is due to the many limitations of self-rating scales that will be elaborated upon in the discussion section.

Discussion

The RAADS-R is a highly specific (100%) and sensitive (97%) instrument that is useful as an adjunct clinical diagnostic tool. Based on the results of this study and the RAADS pilot study, a RAADS-R score of 65 or higher is consistent with a diagnosis of ASD. The ASD subjects’ scores for each question were significantly different from the comparison subjects’. Additionally, the mean RAADS-R scores of the ASD subjects and the comparison subjects were significantly different in each of the nine centers, as well as all of the centers combined. Both the clinically assigned question-domains and the statistically derived factors demonstrated high internal consistency. The RAADS-R had high test–retest reliability in a limited number of subjects. The RAADS-R was designed to assist clinicians in diagnosing adults (18+) with suspected ASD. It is designed to be administered by clinicians in a clinical setting. It is not intended to be a mail in or an online screening instrument. The questions are designed for individuals with average IQ and above. This is a population with mild or subclinical ASD. These individuals often escape diagnosis. The RAADS-R was specifically designed to capture that population.

Limitations

Scales based on self-reports have inherent limitations: First, a subject may not understand a question, and while answering to the best of their ability, give a misleading answer. This was mitigated in the present study by having a clinician remain with the subject to clarify possible misunderstandings during the administration of the RAADS-R.

Second is the issue of unawareness i.e., “denial” of symptoms or over reporting of symptoms. To control for this we added the SRS-A, which is a second party instrument. The concordance rate was 95.95%, which indicates good concurrent validity. However, since only 69 subjects were available to complete the SRS-A, further studies are warranted.

Six subjects whose clinical diagnoses and ADOS module 4 scores placed them in the ASD group, scored below the RAADS-R ASD cut off point of 65. Their mean age was 19.9 years. Each of these six “false negative” subjects was reported by their family members to be invested in appearing “as normal as they could be.” The ANOVA results show a positive correlation with age in the ASD subjects with older subjects scoring higher (see Results section). Possible explanations are: (a) the younger subjects were denying symptoms, (b) the younger subjects received treatment since childhood and now believed they were “normal,” and (c) three of the cases from Yale’s sample have low functioning siblings with autism. These subjects may see themselves relatively unaffected for this reason. This stresses the importance for clinicians to search very carefully for symptoms when diagnosing young adults, given their propensity to want to “look as normal as possible.”

Another confounding factor is that the same symptoms of ASD reported to be unobtrusive by some subjects may be serious to incapacitating in others. For instance, a young man reported that hypersensitivity to lights and sounds caused him to be “completely socially isolated,” while another with the same symptoms told us, “I go to carnivals, circuses, and state fairs, every chance I get—I enjoy the flashing lights and sounds of the barkers and noises the merry go round makes.” “And I stay as long as I can,” he added with a smile.

Factor Analysis

As noted in the Results section, four factors emerged from Factor Analysis. The Cronbach alpha coefficients were satisfactory (social relatedness = .93, circumscribed interests = .95, sensory motor = .87, social anxiety = .89), indicating good internal consistency. For the clinical domain, the Cronbach alpha for language was the weakest. Factor IV contains questions relating to social anxiety and anxiety in general. This factor is important to examine. Targeting the social anxiety questions within the instrument may help the clinician differentiate social anxiety from ASD. The two diagnoses are often difficult to differentiate.

University Effect

As shown in the results section, there were significant differences of the mean RAADS-R scores among the nine research centers. This is most likely due to a large variation in the number of subjects per center (maximum 53 to a minimum of 7). Since the mean RAADS-R score for each center was significantly different from the mean RAADS-R scores of the comparison subjects (p < .0001), it is likely that the differences among centers would diminish if those with fewer subjects were to expand their database. Further validation studies are indicated.

Relevance to DSM-V

Results show significant statistical differences between the autism and Asperger’s subjects and comparison subjects with and without another DSM-IV-TR diagnosis. There were no significant differences between the autism and Asperger’s groups, which supports the newly proposed DSM-V diagnostic criteria. Recently proposed DSM-V criteria combines the two groups, creating an Autism Spectrum Disorder diagnosis.

Conclusions

The RAADS-R proved to be highly accurate in discriminating between subjects with ASD and those without a DSM-IV-TR diagnosis and with another DSM-IV-TR diagnosis (sensitivity = 97%, specificity = 100%, test–retest reliability = .987, accuracy = 98.5%,). No difficulties were reported in administration even though subjects were evaluated at nine medical centers on three continents. Mean RAADS-R scores were highly significantly different (p < .0001) between ASD and comparison subjects at each of the nine centers, and all eighty questions were significantly different between ASD and comparison subjects. The fact that this was true even when the comparison cases had a variety of DSM-IV-TR diagnoses other than ASD makes the RAADS-R particularly useful in adult clinical settings.

Appendices

Appendix 1: The Ritvo Autism Asperger’s Diagnostic Scale-Revised (RAADS-R) and Scoring Instructions

Appendix 2: Instructions for Scoring the Scale

The scale contains two types of questions:

• 1. 64 questions describing specific symptoms of ASD. These are scored in order of severity from: “true now and when I was young” = 3, to “never true” = 0.

• 2. 16 questions describing non symptomatic (normative) behaviors. These are score: “true now and when I was young” = 0 to “never true” = 3. These questions have asterisks by their number (Table 9).

  Table 9 Scores for the four possible answers