The psychometric properties of the Quantitative-Checklist for Autism in Toddlers (Q-CHAT) as a measure of autistic traits in a community sample of Singaporean infants and toddlers

Participants were from a sample of mothers and children who were recruited in an ongoing prospective longitudinal birth cohort study in Singapore (GUSTO—Growing Up in Singapore Towards Healthy Outcomes¹; see [34] for more details). Participating infants were born between November 2009 and May 2011. Mother-child pairs were followed up at regular intervals from 12 weeks of gestation to 41 months of age, with planned follow-ups taking place currently until at least age 9.

From the “main GUSTO cohort” (n = 1162), a subsample of participants were invited for more detailed follow-up neurocognitive phenotyping and neuropsychological assessments at regular intervals, the “GUSTO Neurodevelopmental Cohort”². At 18 months, 431 toddlers and their caregivers participated in the scheduled neurocognitive follow-up; at 24 months, 514 caregiver-toddler pairs participated. The caregivers were invited to complete the Q-CHAT among other measures. Response rates for the Q-CHAT were 85.4 % at 18 months (n = 368; 54.2 % males) and 77 % at 24 months (n = 396; 51.4 % males), while 294 participants (52.5 % males) had Q-CHAT data at both time points. At 24 months, 359 (90.7 %) respondents were mothers, 16 (4.0 %) were fathers, 8 (2.2 %) were grandmothers, aunts, domestic helpers, or more than one respondent, and 13 (3.1 %) did not indicate their relationship to the child³.

Q-CHAT completers at 18 and 24 months were compared to the active GUSTO cohort on various demographic variables to examine the representativeness of the present sample (see Table 1). Q-CHAT participants at 18 and at 24 months did not differ significantly from the active GUSTO cohort on maternal age, maternal education, or household monthly income (all p > .05; Table 1). However, the 18-month Q-CHAT sample had somewhat more Malay, fewer Chinese, and fewer Indian children than the active GUSTO cohort. The 24-month Q-CHAT sample included more Malay, more Chinese, and fewer Indian children than the main GUSTO cohort (Table 1). Nevertheless, the sizes of these differences were small, indicating that overall, the Q-CHAT sample was reasonably representative of and similar to the active GUSTO cohort in most demographic variables.

Ethical approval was granted by the National Health Care Group Domain Specific Review Board and the Sing Health Centralized IRB and approved by the National University of Singapore IRB. All participating caregivers gave informed written consent before their participation. There was no obligation to take part in the study, and participants could withdraw at any time point without their standard medical care being affected in any way. Each family was reimbursed for every follow-up clinic or home visit and for the completion of caregiver-completed questionnaires.

All biological, genetic, neuropsychological, and behavioral data collected for the GUSTO study at all time points, including the measures used in the present study, were collected by a team of trained researchers and research assistants, including undergraduate and post-graduate psychology students for the neuropsychological and behavioral measures, under supervision by post-doctoral research fellows and senior research staff.

As per Allison et al. [17], incomplete or ambiguously answered Q-CHAT items were conservatively scored ‘0’. If seven or more Q-CHAT items were missing, then the checklist was excluded from analyses (n = 1, 0.3 % at 18; and n = 3, 0.9 % at 24 months). For the CBCL at 24 months, questionnaires with more than eight missing items were excluded (n = 10, 2.5 %; [41]. For those with eight or fewer missing items, a conservative score of ‘0’ was given to any missing items [42].

All statistical analyses were conducted using the Statistical Package for Social Sciences (SPSS), Version 21.0. Descriptive statistics were first calculated, and data were explored for normality and outliers. An Exploratory Factor Analysis (EFA) using Principal Axis Factoring was then conducted to investigate the factor structure of the Q-CHAT at 18 months. EFA was selected, as it assumes that items on the Q-CHAT cluster according to underlying constructs. Oblique direct oblimin rotation was performed, as psychological constructs/factors are often correlated [43].

Cronbach’s alphas were then calculated to examine the Q-CHAT’s total and derived factor items’ internal consistency at both time points. One sample t tests compared total Q-CHAT scores in this Asian sample against Q-CHAT scores obtained in published literature with samples from the UK. Item-total correlations were also examined using Spearman’s rho non-parametric analyses. The relationship between 18- and 24-month Q-CHAT scores was examined with Pearson’s r correlation analyses for total and factor scores and Spearman’s rho for the Q-CHAT items, while paired t tests or non-parametric Wilcoxon’s tests examined differences between Q-CHAT total, factor, and item scores between 18 and 24 months. Finally, convergent validity between the Q-CHAT and the CBCL 1.5-5’s Total Problems, Internalizing, Withdrawn, and DSM-oriented PDD problem scales was examined using either Pearson’s r or Spearman’s rho correlations, depending on the scores’ distribution. Because of the number of statistical tests carried out, the magnitude of effect sizes was carefully considered in the interpretation of the findings.

Q-CHAT scores at 18 and 24 months were normally distributed (see Figs. 1 and 2). Table 2 presents Q-CHAT descriptive statistics at 18 and 24 months.

Mean Q-CHAT total scores at 18 (M = 35.6, SD = 7.2) and 24 months (M = 33.2, SD = 7.7) in this sample were significantly higher (i.e., more autistic traits) than those in Allison and colleagues’ [17] unselected sample of 754 UK toddlers with a mean age of 21.2 months (M = 26.7, SD = 7.8; one sample t(367) = 23.6, p < .001 at 18 months; t(395) = 16.8, p < .001 at 24 months) with large calculated effect size differences (d = 1.18 and .84, respectively). Similarly, our scores were significantly higher than those of Auyeung et al. [31] from 129 19-month-old toddlers (M = 26.55, SD = 7.08; t(367) = 24.0, p < .001 at 18 months; t(395) = 17.18, p < .001 at 24 months), with large calculated effect size differences (d = 1.26 and .90, respectively). Differences were small when our sample’s total scores at 18 and 24 months were compared to those reported by Wong et al. [30] for 2-year-old premature children in the UK (M = 33.7, SD = 8.3; t(367) = 4.98 p < .001 at 18 months; t(395) = −1.3, p = .19 at 24 months; calculated effect sizes were small, d = .24 and − .06 for the two time points, respectively).

The distribution of the 25 Q-CHAT item ratings at both time points can be found in Table 3, while mean item scores are presented in Table 4. Use of caregiver’s hand as a tool (item 12) and Echolalia (item 18) were the two most highly endorsed behaviors at 18 and 24 months, respectively. Unusual gestures (item 19) was the least frequently reported behavior at both time points.

At 18 months, all but two of the Q-CHAT items were significantly and positively associated with the total Q-CHAT score, with small to medium effect sizes (.12 ≤ rho ≤ .47, all p < .001; items 5, Protoimperative Pointing, and 9, Pretend Play, p < .01; item 12, Using hand as tool, p < .05). Two items, Echolalia (item 18; rho = .01, p = .84) and Looks at caregiver’s face to check reaction in unfamiliar situation (item 21; rho = .10, p = .06) were not statistically significantly associated with the total score at 18 months.

At 24 months, all but two of the items significantly correlated with the total Q-CHAT score with small to large effect sizes (.15 ≤ rho ≤ .56; all p < .001; items 19, Gestures, and 21, Looks at caregiver’s face to check reaction when faced with unfamiliar situation, p < .01). Only Echolalia (item 18) and Pretend play (item 9) did not correlate significantly with the total Q-CHAT score (rho = −.04 and .10, respectively; both p > .05).

No previous factor analysis of the Q-CHAT has yet been published in the literature, with the exception of the unpublished factor analysis by Allison and colleagues [33]. For this reason, an EFA was carried out. The Kaiser-Meyer-Olkin (KMO) statistic was .79 (above the threshold of .50) and the Bartlett’s Test of Sphericity was significant (χ ² (300) = 1802.86, p < .0001), indicating that the Q-CHAT data at 18 months were suitable for EFA [44].

Use of the Kaiser criteria initially suggested a solution of seven factors with Eigenvalues > 1, which explained 53.4 % of the total variance. However, examination of the scree plot (Fig. 3) indicated one point of inflexion after the third factor, thus the more parsimonious three-factor structure was selected. The EFA was re-run specifying three factors. A factor loading of 0.3 was chosen as the threshold above which items would be retained in the factor structure. Items 3 (Lines objects up), 13 (Walks on tiptoes), and 14 (Ease of adapting to changes in routine) were thus removed, as their factor loadings failed to reach the accepted threshold value. The resultant three factors explained a total of 38.1 % of the variance (factor 1, 19.5 %; factor 2, 10.95 %; and factor 3, 7.66 %). The pattern matrix, which employed direct oblimin rotation, is shown in Table 5.

Factor 1 (social-communicative autistic traits) comprised 10 items, of which only Use of hand as tool had a negative factor loading (item 12; the only reverse scored item in factor 1). Factor 2 (non-social/behavioral autistic traits) consisted of eight positively loaded items, all of which were reverse-scored. Finally, factor 3 (speech/language development) consisted of four items, of which only Echolalia (item 18) had a positive factor loading (see Fig. 4).

Following the factor analysis, raw factor scores were calculated for both time points through the summation of scores corresponding to items loading onto each factor (see Table 2 and Fig. 4 ). Correlations between the three factor scores were of small effect sizes: r = −.18 and − .14 (factors 1 and 2 at 18 and 24 months, respectively; p = .001); .16 and .17 (factors 1 and 3 at 18 and 24 months; p = .003); and .07 and .19 (factors 2 and 3; p = .18, ns and < .01 at 18 and 24 months, respectively).

Internal consistencies of the Q-CHAT total and factor 3 (speech/language) items were suboptimal (α = .53 and .63 at 18 months; .60 and .63 at 24 months, respectively). Items in factors 1 (social-communicative autistic traits) and 2 (non-social/behavioral autistic traits) had acceptable internal consistencies (α = .76 and .69, respectively, at 18 months; .75 and .71 at 24 months). No item removal resulted in improved internal consistency values for any of the factor scores, except for the removal of item 18 (Echolalia), which somewhat improved the internal consistency of the total items from .53 to .57 at 18 months. However, the improved Cronbach’s alpha remained suboptimal.

The Q-CHAT total and factor scores at 18 months were highly positively correlated with the respective total and factor scores at 24 months with medium to large effect sizes (see Table 2). Most Q-CHAT individual items at 18 months also correlated significantly and positively with medium to large effect sizes with the same items at 24 months (.23 ≤ r ≤ .59; all p < .001; see Table 4). In addition to examining the relationship between 18- and 24-month Q-CHAT scores, the extent to which the scores obtained at the two time points were comparable (i.e., consistent) was also examined using Bland-Altman plots (see Fig. 5). Bland-Altman analysis is a graphical method which allows the comparison of two different measures (or two measurements obtained from the same measure at different time points). In this plot, the difference between each participant’s Q-CHAT 18- and 24-month score is plotted against the mean of the two measurements. The extent to which the two scores are comparable (in agreement) is defined by whether the difference scores largely fall within the two lines indicating the limits of agreement (+/−1.96 SD) [45]. Figure 5 shows that the Q-CHAT difference scores between 18 and 24 months are within two standard deviations of the average difference scores of the two measurements (lines of agreement), indicating overall stability and agreement (consistency) in autistic trait scores at the two time points.

Significantly fewer total autistic traits were reported by caregivers at 24 compared to 18 months with a medium effect size (see Table 2). Mean social-communicative autistic trait scores (factor 1) significantly increased from 18 to 24 months with a small effect size increase, while non-social/behavioral autistic trait scores (factor 2) decreased over time with a medium effect size (see Table 2). Finally, speech/language scores (factor 3) were significantly higher at 18 months as compared to 24 months with a large effect size, indicating, as developmentally expected, improvements in speech/language use (see Table 2).

At the item level, mean scores in items 4 (Ease of understanding child’s speech), 8 (Number of spoken words), 11 (Sniffs/licks unusual objects), 16 (Does the same thing over and over again), and 23 (Twiddles objects repetitively) decreased significantly from 18 to 24 months, with medium to large effect sizes, indicating lower frequency of these behaviors at 2 years of age. A significant mean score increase was observed from 18 to 24 months only for items 3 (Lines objects up) and 9 (Engages in pretend play), with medium effect sizes (see Table 4).

Q-CHAT total scores at 18 and 24 months were significantly positively correlated with raw scores from the 24-month CBCL’s Internalizing Problems (r = .33 and .39, respectively), Withdrawn (.32 and .36), and PDD subscales (.30 and .35) with medium effect sizes (all p < .001). These three CBCL subscale scores were also significantly positively correlated with all three Q-CHAT Factor scores at both 18 and 24 months with small to medium effect sizes (.12 < r < .29, all p < .05), except factor 2 (non-social/behavioral items) at 18 months, which did not correlate significantly with the CBCL PDD scale at 24 months (r = .11, p = .07).

The normal distribution of Q-CHAT scores at both 18 and 24 months is consistent with our present dimensional understanding of autistic traits. The higher mean autistic trait scores identified in this sample of Asian toddlers as compared to the original Q-CHAT study’s UK sample [17] are also in line with a small number of other studies also reporting higher autistic traits in Asian adults without ASD [11, 46]. In fact, our Q-CHAT data from unselected Singaporean toddlers with an average gestational age of 38.3 weeks at birth (SD = 1.5 weeks) were comparable to those reported by Wong and colleagues [30] in prematurely born children from the UK. In a recently published study of Chinese school-aged children without ASD, CAST mean scores (M = 7.8, SD = 3.7) [47] were also significantly higher than those reported in the original CAST prevalence study in the UK [26] with a large effect size difference (M = 4.73, SD = 3.57; calculated d = .84). However, not all studies have consistently reported cross-cultural differences in mean scores obtained from caregiver-reported measures of autistic traits (i.e., [19, 21]). Thus, it is possible that differences in cultural experiences and interpretation of childhood behaviors may only partly explain the observed differences in caregiver- or self-reported autistic traits between the two samples (see [48] for a review of autism across cultures). Our sample was also demographically different to that of Allison and colleagues [17], as mean age at which mothers left formal education was higher in our study, with a large calculated effect size difference (d = .64). Other studies have failed to find significant ethnic/country differences (i.e., [49, 50]) and, interestingly, the mean CBCL PDD subscale raw score in our sample (M = 3.71; SD = 3.21) was comparable to the “omnicultural” mean PDD subscale raw score reported in [37] from 24 societies, including Singapore. Thus, although it is possible that ethnic/cultural differences and family/child differences may affect caregiver report of children’s autistic traits, this has yet to be investigated systematically (see, however, [51] for recent work on this) and further research is required to better understand possible socio-cultural influences in caregiver- or self-reported autistic traits.

Our study proposed a three-factor structure of Q-CHAT autistic traits in an unselected sample: a social-communicative factor, a non-social/behavioral factor and a speech/language factor. These were the same three factors that were identified in the unpublished factor analysis of the Q-CHAT [33]. Although derived in an unselected sample, the first two factors—social/communicative and non-social/behavioral autistic traits—reflect the DSM-5’s dyad of diagnostic criteria for ASD [1], providing some evidence towards a similar factor structure of autistic traits and related symptoms across both unselected and clinical populations. Other recent factor analytic studies in unselected and clinical samples also support the clustering of social communication and interaction items in a single factor (e.g., [52, 53]) and the independent contribution of stereotyped and repetitive behaviors as a separate factor [54, 55]. Finally, the third Q-CHAT factor, comprising only four items, is not specific to autistic traits, but is more developmental in its focus on speech/language.

Other studies examining other measures of autistic traits or symptoms of ASD have proposed similar factor structures to the one found in the present study for the Q-CHAT. Sun and colleagues [47] recently examined the psychometric properties of the Mandarin Chinese version of the CAST [26] in 737 4–11-year-old children from Mainland China. They found a similar symptom presentation to that reported in Western populations and a two-factor solution for its 28 items (social/communication factor and stereotyped/inflexible language and behavior factor). Matson et al. [56] also showed a three-factor structure of socialization/non-verbal communication, repetitive behavior/restricted interests, and communication on the Baby and Infant Screen for Children with aUtIsm Traits—Part 1 (BISCUIT—Part 1).

Internal consistencies were suboptimal for the Q-CHAT total and Q-CHAT speech/language factor scores (ranging from .52 to .63), but acceptable for the two main Q-CHAT factors of social/communication and non-social/behavioral autistic traits (.69 to .76). Rather than being a unitary measure, as suggested by researchers employing other measures for which a single factor structure has been proposed (e.g., [52, 57]), our findings suggest that autistic traits may be better conceptualized as social and non-social traits. The small correlation between these two factor scores (r = −.18) highlights their largely independent relationship (see also [54, 58, 59] for a research review). Test-retest reliability was .60–.64 for the total and the two main factor scores after 6 months, and the Bland-Altman analyses showed overall consistency in the scores from 18 to 24 months. Higher test-retest reliability of .82 for the Q-CHAT was reported by Allison et al. [17], but re-test in their study was only 1 month later; likewise, higher test-retest reliabilities have been reported for the SRS [6]; however, most other studies either investigated autistic traits in older children, where such traits are likely to be more stable, or carried out the second assessment a few weeks only after the first one, rather than after a longer interval of 6 months, as in the present study.

Finally, examination of convergent validity of the Q-CHAT total and factor scores showed that all scores, except factor 3 speech/language scores, were positively associated with the CBCL’s PDD subscale with a medium effect size. Autistic-trait-related behaviors at 18 and 24 months were also positively associated with internalizing and withdrawn behaviors at 24 months and autistic symptoms as measured by the CBCL, providing evidence that the relationship between internalizing traits and autistic traits found in school-aged children [60] is also present in unselected samples of toddlers as young as 18 months. Constantino et al. [52] and Duku et al. [61] reported similarly positive, but larger, correlations between the SRS and CBCL internalizing scores (.47 and .68) in their studies of 4- and 9-year-old children with diagnoses of ASD, respectively.

Overall, Q-CHAT total and factor scores were generally stable between 18 and 24 months. Individual Q-CHAT items also showed high mean stability after 6 months. Nevertheless, there was also evidence of change over time: mean non-social/behavioral autistic factor scores decreased significantly (indicating fewer autistic traits) during this period (d = .45), as did mean speech/language development factor scores (d = .56). These changes in factor and item scores are likely expected as part of normative development in young children. For example, there was a significant reduced frequency of repetitive action (item 16, d = .51) and significantly increased number of spoken words (item 8; d = .72) by 24 months. These changes are indicative, respectively, of the decrease in obsessionality and levels of high repetition (i.e., [62]) and the improvements in language comprehension and verbal expression of typically developing toddlers. This trajectory of decreasing repetitive speech and behavioral patterns in unselected toddlers likely contrasts a trajectory of persisting or increasing such behaviors in children with ASD (i.e., [63, 64]).

This study is one of the first to quantify the distribution of autistic traits in an unselected population of young Asian toddlers. The organization of autistic traits into social and non-social/behavioral traits proposed by this study based on EFA is consistent with the DSM-5’s current structure for ASD. This was a prospective study, thus avoiding recall bias from caregiver reports [65].

As this study was embedded in the larger multi-time point GUSTO study, some data were lost due to participant attrition or non-participation at certain time points, resulting in a lower sample size available for analyses when making comparisons between measures or examining the Q-CHAT over the two time points. Return rates for the Q-CHAT were thus modest. However, Q-CHAT respondents’ characteristics did not differ significantly from the main GUSTO cohort, making our results likely generalizable to the larger cohort. In addition, internal consistencies of the total Q-CHAT score and the four-item speech/language factor score were suboptimal. Finally, there is no certainty if the same informant provided information about the children’s autistic traits and emotional/behavioral problems at 18 and 24 months in the Q-CHAT and CBCL, as information about the identity of the informants was not collected at 18 months. However, the majority of respondents in the GUSTO study were mothers across all measures and time points.

¹Between June 2009 and September 2010, pregnant mothers over 18 years old were recruited from the two largest public birthing hospitals in the country. They were Singaporean Citizens or Permanent Residents intending to live in Singapore for the next 5 years and of Chinese, Malay, or Indian ethnicity with homogeneous parental ethnic background. Mothers who miscarried, received chemotherapy or psychotropic drugs during pregnancy or had pregnancy complications (i.e., pre-eclampsia, gestational diabetes) were excluded.

²In recognition of the constraints of conducting such detailed assessments on the full GUSTO sample and in line with the main research aims of the other GUSTO investigators, priority for inclusion in the GUSTO neurodevelopmental cohort was given to infants who had participated in previous neurodevelopmental assessments in the first weeks of life, ethnic minorities (Malays and Indians), infants who had high or low intakes of breast milk and participants who through word of mouth or interest contacted the neurocognitive team and volunteered for more detailed neuropsychological assessment.

³Detailed data on the relationship of the Q-CHAT respondent with the child were not collected at 18 months, but the majority of respondents (>80 %) were mothers across all caregiver-reported measures in the GUSTO study.