Participants and screening

SCALES is a population study on language impairment which involved all state maintained primary schools in Surrey, England. The cohort was selected to be nationally representative using stratified random sampling (see [10] for details). Data were obtained for children who began a reception class in a Surrey school in 2011 (children’s ages ranged from 4 years and 9 months to 5 years and 10 months). The study involved a two-phase design: in the first phase, screening data from 7,267 children were obtained. In this paper, 808 children who attended special schools, or who were learning English as an additional language were excluded. In the second phase, a sub-sample from the remaining 6,459 children was selected for a more detailed clinical assessment to determine whether they had a language impairment; 636 were randomly selected (588 plus 48 children with no phrase speech) for intensive individual assessment at Year 1 and Year 3 of formal education.

Consent procedures and study protocol were developed in consultation with Surrey County Council and approved by the Research Ethics Committee at Royal Holloway, University of London, where the study was initiated. Opt-out consent was adopted for the first phase as data could be provided anonymously to the research team; 20 families opted out. In the second phase, written, informed consent for two episodes of direct assessment was obtained from the parents or legal guardians of all participants. Verbal assent was obtained from children. Prior to assessment in Year 3, families received additional information and the option to withdraw from the study; 18 families withdrew consent, five moved abroad, three could not be contacted and three provided insufficient data on the day of testing for diagnostic classification. Of the 29 children (19 males) not included in follow-up, 22 had been classified as ‘typically developing’ in Year 1 and had no evidence of language, learning or behavioural difficulties. Sixty one percent of all eligible schools took part, and 59% of all eligible children that started school that year provided data. Of the 61% of the schools that took part, 95.38% of children provided data [10].

Screening data from 7,267 children were obtained on the Children’s Communication Checklist-Short (CCC-S) which is a brief version of the CCC-2 [11], and on the Strengths and Difficulties Questionnaire (SDQ: [12]). The CCC-2 has been shown to be highly effective at discriminating children with communication difficulties from typically developing children [13]. The CCC-S contains 13 items from the CCC-2 General Communication Composite that tap language and communication skills in everyday contexts, rated on a 4-point scale. Items measure speech, morphology/syntax, semantics, and discourse skills. Teachers were asked to rate the frequency with which a range of language behaviours occur in everyday contexts on a 4-point scale: 0 = rarely/never, 1 = occasionally, 2 = regularly; 3 = frequently/always. The maximum score of 39 indicates greater disorder. The SDQ is a well-validated questionnaire which elicits teacher ratings of children’s emotions, behaviour, activity levels, peer relationships and pro-social behaviour. There are 25 attributes across the five scales which are rated on a three-point scale as ‘not true’ ‘somewhat true’ and ‘certainly true’. Twenty items from the first four scales (excluding pro-social behaviour) are summed to provide a Total Difficulties Score.

The intensive sample and test battery

In the second phase, more than 80% of the randomly selected sample took part in the intensive study (see [10] for details). The age range of our participants spans more than four years: children who were assessed in Year 1 of school were aged between 61 and 82 months. Ninety-five percent of them were reassessed in Year 3 and were aged between 85 and 111 months. Data on a range of language indices were collected measuring vocabulary, grammar and narrative skills across two modalities: speaking (expressive language) and understanding (receptive language). These are:

1. Receptive One-word Picture Vocabulary Test (ROWPVT-4; [14]). During this test the child hears a word and has to select the corresponding picture, from a choice of four. This task taps semantic memory and draws less on phonological skills than measures of expressive vocabulary. The minimum and maximum scores range from 0 to 190 with higher values indicating a better health outcome. Internal consistency for the age range assessed, 5- to 8-years, as rated by Cronbach’s Coefficient Alpha is 0.95–0.97. Test-retest reliability coefficients for raw scores is 0.97 and for standard scores is 0.91 [14].
2. Expressive One Word Picture Vocabulary Test (EOWPVT; [14]). During this test the child is asked to name objects, actions or concepts illustrated in pictures. The score on this task ranges from 0 to 190 with higher values reflecting a better outcome. Internal consistency for the age range assessed, 5- to 8-years, as rated by Cronbach’s Coefficient Alpha is 0.94–0.97. Test-retest reliability coefficients of raw and standard scores are 0.98 and 0.97 respectively [14].
3. Test of Reception of Grammar–Short Form (TROG-2; [15]). During this test children hear a sentence such as “the ball that is red is on the pencil” and are asked to select the corresponding picture out of a choice of four. If a child answers incorrectly on six consecutive items, then the test is discontinued. Scores for this task range from 0 to 40, with a higher score denoting more correct responses. As noted in the manual, split-half reliability for the TROG-2 is 0.88 and the resulting correlation suggests good internal consistency (r = 0.877; [11]).
4. Sentence repetition (SASIT E32; [16]). During this test the child is asked to repeat 32 sentences out loud. All sentences are pre-recorded and played over headphones to the child one at a time, with a break for the child to repeat the sentence in between. Each repetition is audio recorded and children are scored on whether the sentence is correct or incorrect (minimum score of 0, maximum score of 32), how many of the function words (range: 0–176) and content words (range: 0–121) are repeated correctly and whether the verb (range: 0–44) is correctly or incorrectly inflected. Higher scores on this test indicate more correct responses. The child is given two practice trials to ensure that they understood the task.
5. Assessment of Comprehension (Narrative Comprehension) and Expression (Narrative Recall) 6–11 (ACE 6–11; [17]). The child is asked to listen to a story about a monkey in a forest. The story is pre-recorded and played over headphones with accompanying pictures displayed on a laptop computer. After listening to the story, the child is asked to tell the story in their own words (Narrative Recall). The child is given a mark for each part of the story they correctly re-told. The child’s re-telling of the story is audio recorded and the score for this task ranges from 0 to 35; higher scores indicate a better outcome. According to the manual, average internal consistency coefficient (Cronbach’s Alpha) of narrative propositions for children aged 6- to 11-years is 0.73. Assessment of children’s Narrative Comprehension takes place after the Narrative Recall task; the child is asked to answer 12 comprehension questions (6 literal and 6 inference questions) about the story they had just heard. The child is scored 0 for an incorrect answer, 1 point for a partially correct response and 2 points for a correct response. The partially correct and correct responses are determined by a written guide regarding the various possible responses. The score for this task ranges from 0 to 24 with higher scores indicating a better outcome.

These tests were selected to replicate existing epidemiological findings [18] that have informed current diagnostic algorithms for language disorder in the Diagnostic and Statistical Manual of Mental Disorders (DSM5) [19]. Only children who had complete data on the 6 language indices were included in this study (n = 529 for Year 1 (however 1 child did not provide complete data on the day of the assessment); n = 499 for Year 3).

The LMS method

A thorough description of the method can be found in Cole and Green [9]. The LMS method is based on the Box-Cox transformation [20] that converts the scale raw-score to normality. The acronym LMS comes from the parameters for the Box-Cox power Lambda (L), the median (M), and the generalised coefficient of variation (S). It allows these parameters to vary smoothly with age t. The raw scores y are transformed to the variable w by: where M is the median of the untransformed measure y. This maps the median of y to w = 0. Denoting the standard deviation of w (or the coefficient of variation (CV) of y) by S, then the variable is assumed to have a standard normal distribution. Assuming that the distribution of a measure y varies with t, and that the L, M, and S parameters are read off by smoothed curves L(t), M(t), and S(t) as they vary with age then: (1) (2) Formulae (1) and (2) convert y to a skewness-corrected standardised value at each time point. By rearranging (1) and (2) we can derive a centile curve as: Where a defines the lower tail area of the centile, and z_a is the normal equivalent deviate of size a (for example, when a = 0.75 corresponding to the 75^th centile, z_a = 0.675). Based on (1) and (2), Cole and Green (1992) introduced a penalised likelihood function for the estimation of the parameters that describe the skewness, median, and CV. The degree of the penalty and hence the smoothness of the curves, are controlled by three tuning parameters, referred to as the equivalent degrees of freedom (EDF). There is one EDF for each curve, denoted by a_L, a_M, and a_S, and these three parameters are used to achieve a trade-off between goodness of fit to the data and smoothness of the curves. This latter reduces the risk of too closely following chance fluctuations, called over-fitting, but also reflects a plausible belief that while the domain of measurement may decrease as well as increase and become more or less variable over the population, it rarely develops in erratic jumps.

Standardisation

Also available as an R-package [21], here the LMSchartmaker software was used for the implementation of the LMS method [22]. LMSchartmaker reads datasets that contain age, a scale score, an optional variable for weights, and an optional ‘grouping’ variable that defines distinct target populations, most commonly the sex of the child in anthropometric studies. In this paper, we have assumed common norms for boys and girls because we do not believe it is acceptable to assume that either sex should have lower expectations of language competence, which is what sex-specific norms would suggest. In addition, epidemiological studies have found no significant sex differences in prevalence estimates for language disorder [10, 18].

As Cole and Green [9] explain, the L(t), M(t), and S(t) curves are constrained to change smoothly with age and how flexibly they can change with age is controlled by the EDFs. The higher the EDF values are, the less smooth the curves appear. Advice on choosing the optimal values for the EDF parameters, and hence the appropriate model for the data, is given in the User’s Guide to LMSchartmaker, cited at http://www.healthforallchildren.com/?product=lmschartmaker-pro. We initialised the optimisation process by specifying a_L = 0, a_M = 1, and a_S = 1. A choice of 0 for a_L implies no skewness in the distribution of the data at any given time-point, while a choice of 1 assumes that any skewness is constant across the various time-points. Values of a_L larger than 1 result in a linear (a_L = 2) and non-linear (a_L>2) change of skewness with age. With regards to the smoothness of the M(t) and S(t) curves, we set the initial values of a_M and a_S to 1 because zero values are not allowed. A choice of 1 for a_M implies no change in the median curve across time. A value of 2 results in a linear increase or decrease, a value of 3 in a quadratic change and anything over 3 in a polynomial M(t) curve. Similarly, for a_S = 1 a constant variation is implied; a_S = 2 would denote a linear change, a_S = 3 a quadratic change, and so forth.

Having chosen the optimal EDFs for L(t), M(t) and S(t), the model was then fitted to the data. For the standardisation of measures at a single time-point, we restricted the values of a_L, a_M, and a_S to a maximum of 2 in order to prevent over-fitting and achieve more sensible out-of-sample predictions. When we applied the LMS method to data from two assessments points that included an extended age span, we allowed the maximum value of a_L, a_M, and a_S to be 4.

The manual suggests one should optimise the a_M parameter first followed by the a_S parameter followed by the a_L. As a rule of thumb, a_M > a_S > a_L. To choose the appropriate model, goodness-of-fit criteria are embedded within the software. These include the Akaike Information Criterion (AIC), Global deviance (G. Deviance), Penalised deviance (P. Deviance), Schwarz Bayesian Criterion (SBC), and the Generalised Akaike Information Criterion (GAIC(3)). Pan and Cole recommend that the most stringent SBC or GAIC(3) deviance criteria should be used. In this paper we focused on values obtained from the SBC criterion and present sequential values of deviance from testing different EDFs.

After the model is chosen and fitted, the software produces several different graphs. These include plots of the original untransformed data against time, the individual L(t), M(t), S(t) curves, centile curves across time superimposed on the raw data, and the Q-test-for-fit goodness of fit plot. While there is no single formal rule as to what is best, particularly with weighted data, we focused on the minimum value of the deviance estimate, obtained by the SBC criterion, on histograms of the standardised values, as well as clinical judgement to inform our choice of model. We required, where appropriate, the deviance value obtained after fitting a model be smaller by more than 2 from the fit of the previous simpler model [23].

The LMS method allows for the construction of any centile curves and we chose to construct curves at the 3^rd, 10^th, 25^th, 50^th, 75^th, 90^th, and 97^th percentiles. Weighted and un-weighted densities of the standardised scores were obtained using STATA v.14. While the method ensures that the percentile or standard scores are always consistent across age, in the sense that the 10th percentile will never cross the 5^th percentile, over-fitting becomes evident in graphs where the centile curves show implausible variation over age.

Inverse probability weighting to the reference population

Sampling weights are used extensively in SCALES. They were constructed as the inverse of the predicted probability of a child being included in the study, so that when weighted, the estimates obtained from the sample are estimates for the entire population. Weights are particularly useful because they allow for the oversampling of the lower (or higher) tail of the distribution, where most pathological cases lie but where sampling is often insufficient. In SCALES, children identified by the CCC-S as having a higher risk of a language disorder had a higher probability of being selected to take part in the intensive assessment: a random sub-sample of 636 were drawn from the total of 6,459 children, with a higher sampling fraction for the high‐risk children (40.5% boys, 37.5% girls) versus low‐risk children (4.3% boys, 4.2% girls) and of those, 528 children had complete data on the 6 language indices and were included in this study.

Weights were constructed on a two-step basis, both for the 528 children of Year 1 and for the 499 children of Year 3 who had fully observed data on all language indices. In the first step, to construct weights for children who contributed data at Year 1, a logistic model for whether the child had been invited to participate in the second phase of the study, was fitted to the whole screened sample (n = 6,459). The covariates used were factors predictive of inclusion due to study design, such as whether a child was at high or low risk of language impairment as identified by the screen, and the number of children screened at their school. In the second step, a logistic model was fitted only to children that were selected for the second phase Year 1 assessment. Inclusion also depended on whether children had complete data on all 6 language measures. Since this reflected non-designed participation and response variation, a number of covariates were tested in a stepwise elimination process including gender, income deprivation score, size of school, percentage of girls, special education needs, free school meals, English as additional language, CCC-S score, and number of pupils taking phonics screen. The final weights were constructed as the inverse of the product of the two predicted probabilities. The same two-step process was repeated for the Follow-up assessment in Year 3 resulting in different weights.

In this paper we illustrate the use of LMS method using the CCC-S screening test and the Expressive Vocabulary test. The CCC-S was available on the whole population of 6,549 monolingual children in state-maintained schools. The CCC-S was used solely for the purposes of exploring the role of weights in our analyses, as data on this measure existed on the entire screened sample. This was achieved by visually comparing the estimated distribution of the CCC-S standard scores, obtained from an optimised model fitted to the whole screen population, against the CCC-S standard scores distribution obtained by fitting a weighted and an unweighted model with the same EDFs as the previous model, but to the intensively assessed stratified sub-sample of children only. The dataset of the entire screened sample included child ages and CCC-S raw scores only. This was uploaded into the LMSchartmaker software to initiate the process of model selection for generation of CCC-S standard scores for the entire sample. To create the sub-sample of children, we retained in the dataset participants who provided all six language indices from the intensive study. Children with low CCC-S scores, and thus good language, were markedly under-represented in this sample (by design). This was accounted for by the weights. Hence, the dataset contained values of CCC-S for children who took part in Year 1 only, along with child ages and weights for Year 1. To obtain weighted CCC-S standard scores we fed all three variables into the software and to obtain unweighted standard scores we left the weights out.

Data for the Expressive Vocabulary test existed for both measurement occasions (years 1 and 3) on the intensive sub-sample only. We used data from the Expressive Vocabulary to demonstrate the use of the LMS method when repeated measurements on the same participant existed. To create the dataset for LMSchartmaker, the Expressive vocabulary data for the two time periods were arranged in an ascending order, with values from Year 1 preceding those from Year 3. Hence, the dataset contained values of the Expressive Vocabulary at Year 1 and Year 3, children’s ages at the time of assessment, and corresponding weights for years 1 and 3. In the same way, we obtained standard scores and centile curves for the remaining five language indices, the results being presented in S1 File.

The online calculator

Based on longitudinal information from our intensive sample, and drawing upon the strengths of the LMS method, we have constructed standardised scores and centile curves for all six language indices used in the SCALES study using weights, so that the results are comparable to the entire UK population of mainstream schools. Building upon this, we developed a freely available online calculator where an age-dependent standard score for any of the six language tests, described in the Methods section, can be obtained by simply entering the raw-score and the age of a child. The calculator can be accessed at: https://ioppn.shinyapps.io/SCALES-calculator/.

Clinical implications

We anticipate that the online tool will facilitate child assessment by supplementing the clinician’s observations with clear, quantified, and comparative information on the child’s language performance relative to peers of the same age. This will aid both diagnosis and prognosis, as well as supporting regular monitoring of language progress. The production of age-dependent standard scores on six language measures can support both speech-language therapists and educators in making targeted assessments of language, identifying expected progress based on current score, and a means of identifying greater than expected progress in response to specific input. In addition, the co-standardisation of measures can facilitate comparisons between the expressive and receptive modalities of vocabulary, grammar, and syntax. Not only does the tool enable targeted monitoring of these aspects of language, it also provides the opportunity for direct comparative assessment of a child’s achievement across the different domains. For example, a clinician might determine a child is performing well at the expressive vocabulary or but relatively poorly on receptive grammar. Intervention therefore might be geared more towards improving receptive grammar. At the same time, it will also be possible to examine how improvement on receptive grammar might influence achievement on other areas of language.

Our population sample included children from across the socio-economic spectrum, though the population of Surrey is skewed towards relative affluence. A number of studies point out that groups of children from lower socio-economic strata have poorer results on language tests, which is true for other aspects of cognitive and behavioural development. Over representation of children with language disorder in areas of socio-economic disadvantage reflects complex interactions between genetic vulnerabilities and environmental circumstances. As we currently lack biological tests for language disorder, it is impossible to know what the underlying contributions to low language performance are, and clinicians should exercise caution in their interpretation of low language scores. No published, standardised tests of language or cognitive ability include separate norms for children from different socio-economic strata and whatever the cause of the language difference, children meeting our criteria for language disorder were substantially below population expectations for age on more than one test of language, and these scores were associated with functional impacts in academic success [10]. Response to appropriate language interventions may further assist in determining whether observed differences from the population mean reflect a cultural difference or a disorder.