Reliability and validity of the Japanese version of the Modified Checklist for autism in toddlers (M-CHAT)

doi:10.1016/j.rasd.2010.04.016

Research in Autism Spectrum Disorders

Volume 5, Issue 1, January–March 2011, Pages 330-336

https://doi.org/10.1016/j.rasd.2010.04.016 Get rights and content

Abstract

Early detection and intervention is essential for children with autism spectrum disorders (ASD). Therefore, we examined the reliability and validity of the Japanese version of the Modified Checklist for autism in toddlers (M-CHAT), a 23-item, yes–no questionnaire regarding early autistic symptoms completed by parents of children at 18–24 months of age. Herein, the reliability of the M-CHAT was investigated for children 4–20 months of age. The M-CHAT score (the number of failed items) was found to be significantly correlated among 24 mother–father pairs (Pearson's r = .933), representing good inter-rater reliability. The test–retest reliability was satisfactory, with 22 mothers providing almost equal M-CHAT scores on two different occasions (r = .990). Significant correlations were observed between the M-CHAT score and the Childhood Autism Rating Scale-Tokyo version score in 25 two-year-old children (r = .581), indicating good concurrent validity. The M-CHAT score was significantly higher in 20 children later diagnosed with ASD compared with reference children (n = 1167), revealing sufficient discriminant validity. A short version of the M-CHAT using 9 items was proposed and effectively differentiated children with ASD from reference children. The efficacy of the Japanese version of the M-CHAT was demonstrated for first-level screening in the general population.

Introduction

Autism spectrum disorders (ASD) reportedly affect 1–2% of children (Baird et al., 2006, Baron-Cohen et al., 2009, Kawamura et al., 2008). Early detection is considered essential for children with ASD based on findings of clinical studies that have shown that early intervention subsequent to early detection can enhance their potential (Dawson et al., 2010) and lead to “optimal outcome” (Sutera et al., 2007). To date, some ASD screening tools, such as the Checklist for autism in toddlers (CHAT) (Baron-Cohen, Allen, & Gillberg, 1992), the Modified Checklist for autism in toddlers (M-CHAT) (Robins, Fein, Barton, & Green, 2001), the Early Screening of Autistic Traits Questionnaire (ESAT) (Swinkels et al., 2006), and the Baby and Infant Screen for Children with aUtIsm Traits (BISCUIT) (Matson et al., 2009) have been developed for the early detection of ASD.

Japan has established a national health check-up system that provides all children, from infants to 3-year-olds, with regular free routine check-ups. This check-up system is well organized and health check-ups at 18 and 36 months effectively determine the children's language/intellectual development. Because socio-communication abnormalities in ASD begin to manifest at 1 year of age (Kamio, Tobimatsu, & Fukui, in press), the check-up at 18 months of age appears to provide a good opportunity to detect early symptoms of ASD. The M-CHAT, a 23-item, parent-report questionnaire (Robins et al., 2001), may enhance the current system, since the checklist was developed for children 18–24 months of age and is easy to administer without increasing the burden on both the families and check-up staff.

For the aforementioned reasons, we developed the Japanese version of the M-CHAT. After a preliminary study (Kamio & Inada, 2006), with the permission of the authors, we added some illustrations (items 7, 9, 17, and 23: see M-CHAT Information, Robins, n.d.) in order to encourage caregivers to notice negative symptoms (attenuated typical development). The primary aim of this study is to establish the reliability and validity of the Japanese M-CHAT.

To the best of our knowledge, the inter-rater reliability and test–retest reliability of the M-CHAT have not been examined. This may be because most children in the targeted age range (18–24 months) are expected to pass almost all the M-CHAT items, thus, yes–no answers are skewed toward positive responses. To avoid such a “ceiling effect” in responses, inter-rater and test–retest reliabilities should be examined for children at different developmental stages. In the current study, the reliability of the Japanese version of the M-CHAT was examined for young children in the first and second years of life who are expected to fail some M-CHAT items (Inada, Kamio, & Koyama, in press).

Although satisfactory internal consistency of the M-CHAT has been reported (Kleinman et al., 2008, Robins et al., 2001), the checklist contains some dummy/buffer items (Baron-Cohen et al., 1992, Robins et al., 2001) that obscure the aim of screening from caregivers. Those items are not directly related to the behaviors of children with ASD; in other words, some items of the M-CHAT may not have face validity. As 6 items were selected from the original M-CHAT as “critical items” (Kleinman et al., 2008, Robins et al., 2001), a more simple but effective short version might be useful. Thus, a second aim of this study is to explore the critical items on the Japanese M-CHAT in order to create a short version.

Section snippets

Reliability sample

Reliability data were collected from voluntary parents recruited from local nursery schools or several communities. Inter-rater reliability data were collected from the mothers and fathers of 24 children (13 males; mean age = 10.0 months, SD = 4.3 months, range = 4–17 months) and test–retest reliability data were collected from the mothers of 22 children (12 males; mean age = 12.8 months, SD = 5.8 months, range = 4–20 months) with a mean interval of 8.3 days (range = 4–14 days). The caregivers were not

Inter-rater reliability

A significant positive correlation was observed between the M-CHAT scores (the number of failed items) of mothers and fathers (r = .933, p < .001). As shown in Table 1, kappa coefficients (κ) for inter-rater reliability were significant for 14 (κ ranges .417–1.000, mean = .712) out of 18 items. For 5 items on which the coefficient could not be calculated, the raw agreement rates were very high, exceeding .917. However, κ were not significant for the following four items: item 8 (Functional play), 14

Discussion

In this study, we demonstrated the reliability and validity of the Japanese version of the M-CHAT, identified its critical items in order to produce a short version, and reported preliminary information regarding the utility of the full/short version as a screening tool for early detection of ASD.

To the best of our knowledge, this is the first study to confirm that the M-CHAT has good inter-rater and test–retest reliabilities for infants and toddlers in the first 2 years of life. The present

Acknowledgements

This study was supported by a Research Grant from the Research Institute of Science and Technology for Society of the Japan Science and Technology Agency and by Research Grants (H19-SHOGAI-008, H19-KOKORO-006 and H20-KOKORO-004) from the Ministry of Health, Labour and Welfare of Japan. We would like to thank Ms. Shizuka Omori for her help with data collection.

References (24)

G. Baird et al.
Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: The Special Needs and Autism Project (SNAP)
Lancet
(2006)
J.L. Matson et al.
Reliability and item content of the Baby and Infant Screen for Children with aUtIsm Traits (BISCUIT): Parts 1–3
Research in Autism Spectrum Disorders
(2009)
American Psychiatric Association
Diagnostic and statistical manual of mental disorders
(2000)
S. Baron-Cohen et al.
Can autism be detected at 18 months?: The needle, the haystack, and the CHAT
British Journal of Psychiatry
(1992)
S. Baron-Cohen et al.
Prevalence of autism-spectrum conditions: UK school-based population study
British Journal of Psychiatry
(2009)
S.E. Bryson et al.
Autism spectrum disorders: Early detection, intervention, education, and psychopharmacological management
Canadian Journal of Psychiatry
(2003)
Chlebowski, C., Green, J. A., Barton, M. L., & Fein, D. (in press). Using the childhood autism rating scale to diagnose...
G. Dawson et al.
Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model
Pediatrics
(2010)
Inada, N., Kamio, Y., & Koyama, T. (in press). Developmental chronology of preverbal social behaviors in infancy using...
Y. Kamio et al.
1-sai 6-kagetsu kenshin ni okeru kouhansei hattatsu shougai no souki hakken ni tsuite no yobiteki kenkyu
Seishin Igaku
(2006)

Kamio, Y., Tobimatsu, S., & Fukui, H. (in press). Developmental disorders. In J. Decety, & J. Cacioppo (Eds.), The...

Y. Kawamura et al.

Reevaluating the incidence of pervasive developmental disorders: Impact of elevated rates of detection through implementation of an integrated system of screening in Toyota, Japan

Psychiatry and Clinical Neurosciences

(2008)

Cited by (69)

Validation of the Korean Childhood Autism Rating Scale-2
2023, Research in Autism Spectrum Disorders
We aimed to evaluate the psychometric properties of the Korean version of the Childhood Autism Rating Scale Second Edition (K-CARS2), which is composed of the Standard Version (K-CARS2-ST) and the High Functioning Version (K-CARS2-HF).
A total of 340 participants were included in this study. We assessed 145 participants (aged 2–36 years, 78.0% male) for K-CARS2-ST and 65 participants (aged 6–36 years, 81.5% male) for K-CARS2-HF, all participants were clinically diagnosed with autism spectrum disorder (ASD). We also assessed 130 participants as a control group (62 with a clinical diagnosis other than ASD and 68 without any clinical diagnosis).
The results demonstrated that the K-CARS2 showed a high degree of internal consistency, inter-rater reliability, and test-retest reliability. In terms of correlations between the instruments, the results showed significant positive correlations. Factor analysis revealed a four-factor structure for K-CARS2-ST and a three-factor structure for K-CARS2-HF. To determine the discriminant validity, a discriminant analysis for between-group comparisons was conducted with the ASD and the control groups. The sensitivity and specificity values were both 100 for K-CARS2-ST and 100 and 96.0 for K-CARS2-HF, respectively.
Cultural Adaptation and Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up in Moroccan Arabic dialect
2023, Encephale
Citation Excerpt :
The high α value of Cronbach (α=0.929) indicated sufficient internal consistency for the M -CHAT-R/F-M (Moroccan Arabic version). This statistic assessment was shown to be significantly superior to Japanese consistency with α=0.752, and slightly above the internal consistency of the Serbian and Taiwan version of M-CHAT-R/F (α = 0.91) [11–13]. Regarding the optimal cut-off point, the selection of the best cut-off point depends on the test used.
The prevalence of autism spectrum disorder is increasing worldwide, making screening and early intervention necessary. Several screening instruments have been developed in recent years. The Modified Checklist for Autism in Toddlers Revised with Follow-up (M-CHAT-R/F) is considered to be one of the specific measures designed to identify toddlers at risk for autistic spectrum disorder.
The aim of the study was to translate and adapt the original version of M-CHAT-R/F from the English to the Moroccan Arabic language.
Specialized translators and clinicians ensured forward and backward translation of the scale into Moroccan Arabic. Then, a two-stage screening of the M − CHAT-R/F − T was applied to a study sample comprised of 56 toddlers with autistic spectrum disorder (category I) and 96 toddlers with normal development (category II). “Kappa test”, “Cronbach's alpha” test, the intra class correlation coefficient, and the area under the curve were determined.
The average score results of M-CHAT-R/F were 13.12 for category I, while it was 2.24 for category II. The Cronbach's alpha coefficient of the checklist was 0.929. The kappa values ranged from k = 0.78 to k = 0.97 with a confidence interval of 95% indicating good convergence. The intra-class correlation coefficient ranged from 0.97 to 0.99, which is excellent. The area under the curve in our study was 0.988, an excellent result.
Efficiency of the Moroccan Arabic version of the MCHAT was demonstrated for screening in the general population.
Les symptômes du trouble du spectre de l’autisme (TSA) peuvent être difficiles à identifier selon l’âge et selon le degré de sévérité. Le dépistage et l’intervention précoce présentent un intérêt majeur pour la prise en charge. Plusieurs instruments de dépistage ont été développés. La liste de contrôle modifiée pour l’autisme chez les jeunes enfants, corrigée avec suivi (M-CHAT-R/F) est considérée comme l’une des outils avec une meilleure sensibilité et spécificité conçue pour dépister les enfants à risque du TSA. Au Maroc, de nombreux enfants atteints du trouble sont diagnostiqués tardivement, et aucun outil de dépistage n’ait été validé au Maroc.
L’objectif de l’étude consistait à valider et à adapter la version originale anglaise du M-CHAT-R/F à l’arabe dialectal marocain.
L’adaptation transculturelle du questionnaire du M-CHAT R/F a été achevée en plusieurs étapes, l’étude a été réalisée en collaboration avec le service d’épidémiologie de la Faculté de médecine de Fès. Nous avons mené cette étude sur une période de trois ans, de juillet 2017 à juillet 2020, la version anglaise originale du M-CHAT R/F a été traduite par deux traducteurs, indépendants et certifiés, dont la langue maternelle est l’arabe dialectal marocain et qui maîtrisent l’anglais. Après comparaison et adaptation transculturelle de l’outil par rapport au contexte médical, et éliminer toutes ambiguïté et incohérence des expressions, une rétro-traduction a été réalisée en anglais par deux autres traducteurs maîtrisant l’anglais, et qui n’étaient pas familiarisés avec la version originale du M-CHAT R/F. Ensuite, un dépistage en deux étapes du M-CHAT-R/F-T a été appliqué sur un échantillon d’étude représentatif de la population marocaine comprenant des enfants atteints de TSA (la catégorie I) et des enfants avec développement normal (la catégorie II). Les parents des 152 sujets ont été interrogés un par un, de façon successive. Tous les sujets ont bénéficié de deux passations faites par deux enquêteurs différents à 60 minutes d’intervalle. “Le test de Kappa, le test alpha de Cronbach”, le coefficient de corrélation intra-classe et l’aire sous la courbe ont été déterminés.
Les résultats du score moyen du M-CHART-R/F étaient de 13,12 pour la catégorie I, tandis qu’il était de 2,24 pour la catégorie II. Le coefficient alpha de Cronbach de la liste de contrôle était de 0,929. Les valeurs de kappa allaient de k = 0,78 à k = 0,97 avec un intervalle de confiance de 95 % indiquant une bonne convergence. Le coefficient de corrélation intra-classe était compris entre 0,97 et 0,99. L’aire sous la courbe dans notre étude était de 0,988, ce qui est très significatif.
La version arabe marocaine du M-CHART-R/F a démontré une sensibilité et une spécificité élevée dans notre étude, permettant ainsi un dépistage précoce des enfants avec risque de trouble du spectre de l’autisme dans la population marocaine.
Implementing and evaluating Social Attention and Communication Surveillance (SACS) to prospectively identify autism in very young children in Nepal
2021, Research in Developmental Disabilities
Although Autism Spectrum Disorder (ASD) can be identified as early as 12 months and diagnosed by 24 months, the mean age of diagnosis of ASD in Nepal is ∼58 months, with children missing the opportunity for early intervention.
This study implemented and evaluated a Nepali version of the Social Attention and Communication Surveillance (SACS-N) tool, to identify children between 11–30 months who are at “high likelihood” of ASD in a local community in Nepal.
Sixty Female Community Health Volunteers (FCHVs) were trained to monitor and identify the early signs of ASD using SACS-N.
Overall, 1926 children were monitored with the SACS-N, with 11 children (0.57 %) referred for further assessments at 11–15 months (n = 4), 16–21 months (n = 4), and 22–30 months (n = 3). Of these children, 10 children had a developmental disorder, including ASD and Global Developmental Delay; no information was available on one child who migrated. Hence, the positive predictive value (PPV) of SACS-N for all developmental disorders was 100 %. Of seven children attending a gold standard diagnostic/ developmental assessment, three had ASD (43 % PPV for ASD). The estimated prevalence of ASD ranged between 0.16 % to 0.26 %.
Community-based developmental monitoring of ASD and other developmental delays by FCHVs is a feasible, cost-effective and sustainable approach to promoting early identification of ASD in Nepal. Further training and awareness of autism is needed for more accurate and increased referral rates using the SACS-N, including regular supervision of FCHVs.
Improving autism screening in French-speaking countries: Validation of the Autism Discriminative Tool, a teacher-rated questionnaire for clinicians’ use
2019, Research in Autism Spectrum Disorders
Citation Excerpt :
However, we dismissed the possibility of simply translating current tests from English to French for the multiple reasons described below. First, several studies support evidence for cultural differences in the expression of autistic traits across continents and countries, jeopardising use of screeners internationally (Albores-Gallo, Roldan-Ceballos, & Villareal-Valdes, 2012; Carruthers et al., 2018; Inada, Koyama, Inokuchi, Kurada, & Kamio, 2011; Seif Eldin, Habib, & Naefal, 2008; Soto et al., 2014; Thabtah, 2017). Theoretically, this shortage may apply to instruments tailored and validated in countries such as the US, the UK or even northern Europe where cultural norms diverge from French culture.
Autism screening remains a major challenge in most French-speaking countries. Two main issues contribute to this problematic situation: unavailability of tests in French and psychometric/ methodological weaknesses of existing instruments. These shortfalls result in late and inadequate referrals to autism specialist clinics, jeopardising both children’s diagnosis and prognosis. This article describes the validation of the Autism Discriminative Tool (ADT), a teacher-rated level-two screener for children aged two to six years. It demonstrates how the ADT specific properties may help reduce actual screening challenges and improve referrals’ adequacy to tertiary autism diagnostic services.
ADT items were prospectively tested in a community-based group (n = 118), children with autism spectrum disorders (ASD; n = 90) and non-ASD children presenting mimicking conditions such as intellectual disabilities, language impairments and various psychological disorders (n = 36). Children in the clinical samples were rated by their teacher at the beginning of a diagnostic assessment process within three specialised autism clinics.
results suggest that a 26-item version performs well as a stage-two screening device, with theoretical sensitivity rate of 0.83, specificity of 0.94 and an overall correct detection rate of 86.5%. Using different cut-off scores categories, results illustrate how inadequate referrals may be avoided as 44% of non-ASD children scored negatively on the questionnaire prior to their evaluation.
based on blind teachers’ ratings, the ADT appears to be a good complementary device to help French-speaking clinicians identify preschoolers needing multidisciplinary ASD diagnostic assessment.
The Early Social Cognition Inventory - Autism Version: psychometric properties for young children with autism spectrum disorder
2024, International Journal of Developmental Disabilities
Spatiotemporal patterns of spontaneous movement in neonates are significantly linked to risk of autism spectrum disorders at 18 months old
2023, Scientific Reports

View all citing articles on Scopus

View full text

Reliability and validity of the Japanese version of the Modified Checklist for autism in toddlers (M-CHAT)

Abstract

Introduction

Section snippets

Reliability sample

Inter-rater reliability

Discussion

Acknowledgements

Lancet

Research in Autism Spectrum Disorders

Diagnostic and statistical manual of mental disorders

Can autism be detected at 18 months?: The needle, the haystack, and the CHAT

British Journal of Psychiatry

Prevalence of autism-spectrum conditions: UK school-based population study

British Journal of Psychiatry

Autism spectrum disorders: Early detection, intervention, education, and psychopharmacological management

Canadian Journal of Psychiatry

Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model

Pediatrics

1-sai 6-kagetsu kenshin ni okeru kouhansei hattatsu shougai no souki hakken ni tsuite no yobiteki kenkyu

Seishin Igaku

Reevaluating the incidence of pervasive developmental disorders: Impact of elevated rates of detection through implementation of an integrated system of screening in Toyota, Japan

Psychiatry and Clinical Neurosciences