Background
There are currently no known biomarkers for autism, and no firm biological or neurological definition of the underlying nature of the condition. Autism was initially formulated as a distinct condition based on clinical observations of children who exhibited differences in social and communicative functions as well as repetitive behaviours and interests [
1]. Since then, attempts to codify the definition of autism have resulted in several iterations of diagnostic criteria [
2]. However, the diagnostic criteria do not provide an unambiguous definition of autism since the codified criteria leave room for diverging interpretation. To apply the criteria in diagnosing autism, a clinician must have additional knowledge of the qualitative expression of different autism signs, or symptoms [
3]. For example, some signs, such as abnormal eye-contact, are also associated with other conditions, but the qualitative characteristics of the abnormalities differ between the conditions and the clinician must recognize how and when the specific quality is associated with autism rather than another condition [
3]. The clinician must also have knowledge of clinical thresholds for distinguishing autism signs from behaviours that are uncommon, but should be considered part of normal variation [
4]. A clinician’s additional knowledge comes from experience and exposure, or in other words, from encountering autism in a clinical setting or from otherwise observing and interacting with individuals with autism as well as individuals with other distinct conditions. This knowledge gained from experience cannot be easily written down or directly transferred to another person and is thus a form of implicit (or tacit) knowledge [
5]. The diagnostic practice of a clinician, i.e., who is and who is not diagnosed with autism, is guided by the codified diagnostic criteria, but these are modulated by the clinician’s implicit knowledge of and expertise in the condition.
Since clinicians’ implicit knowledge about autism cannot be fully captured in a formal definition, but rather is gained from interacting with autistic individuals, a circularity in the definition of autism arises: clinicians’ implicit knowledge influences their collective diagnostic practices which has a direct impact on the composition of the diagnosed autism population; the diagnosed population in turn forms the basis of who will be included in scientific studies about autism, and who will be seen as representatives of autism, which further contributes to clinicians’ implicit knowledge. Due to this circularity, clinicians’ implicit knowledge is central to shaping the concept of autism and to our understanding of the condition. Furthermore, the circularity can lead to changes in the understanding of autism over time even in the absence of changes to the diagnostic criteria. Whereas the explicit diagnostic criteria are codified in diagnostic manuals, the implicit knowledge cannot be observed directly. One way to gain insight into this is to question clinicians about their certainty of a given diagnosis and correlate this measure of certainty to observable characteristics in the individual being diagnosed, since the certainty of an autism diagnosis can be a measure of how closely the individual matches the clinician’s understanding of autism.
Previous studies have used diagnostic certainty, or confidence, e.g. as a proxy for symptom severity [
6]. Certainty scores have also been used to investigate whether demographic characteristics such as age, sex, or parent income [
7‐
9], cognitive or behavioural variables such as IQ or adaptive functioning [
10‐
12], or genetic factors such as
de novo mutations [
13] are associated with the diagnostic certainty of an autism diagnosis. Studies tend to find no association between certainty rating and sex [
7,
8], and findings have been mixed regarding the associations between certainty and variables such as adaptive or cognitive functioning, and age [
8,
10‐
12]. Clinical assessment is generally guided by the quantification of symptom severity, using diagnostic instruments such as the Autism Diagnostic Observation Schedule (ADOS). Certainty of an autism diagnosis has been found to be significantly but modestly correlated with ADOS scores [
11] and other measures of symptom severity [
10,
12]. This means that a substantial number of individuals with a relatively high ADOS score are not necessarily diagnosed with the highest certainty, and conversely, that some individuals may be rated with the highest certainty despite having a relatively low ADOS score. An explanation for this could be that the quality of the symptoms that are present is more relevant for diagnostic certainty than the number of symptoms. Furthermore, some ADOS items may generally be more indicative of autism than others, and thus may be more strongly associated with diagnostic certainty. For example, the presence of a few highly specific signs with the right qualitative expression could therefore result in high diagnostic certainty despite a low total number of symptoms. As such, we aim to investigate how individual items in the ADOS are associated with clinicians’ certainty rating.
Another possible explanation for the modest association between certainty and symptom severity is that other factors, not directly represented in the ADOS, may also contribute to the clinician’s certainty. However, as mentioned above, previous studies have not shown consistent patterns of associations between certainty ratings and factors such as the level of cognitive or adaptive functioning [
8,
10‐
12]. We wanted to further explore the associations of such factors as well as additional phenotypes such as language level and head circumference (HC). HC or other measures of brain size have been extensively investigated in autism with many studies finding larger heads or brains to be associated with autism [
14]. A meta-analysis [
15] investigated the percentage of autistic individuals with macrocephaly, which is defined as having a HC greater than the 97th percentile, and found that 15.7% of individuals with autism met this criterion compared to around 3% expected in the general population. Since macrocephaly during some period of development has been so strongly associated with autism in past research, this physical trait may directly or indirectly (i.e., by being related to other factors such as specific signs) impact the certainty of the clinicians performing autism assessments. We therefore wanted to further explore whether HC is associated with clinicians’ diagnostic certainty.
Many of the studies that have previously investigated diagnostic certainty have operationalized the construct as the level of confidence a clinician has that an individual is somewhere on the autism spectrum, which includes highly different phenotypic presentations, e.g. in terms of language ability (from fluent to no language) or IQ (from above or within the normal range to intellectual disability), thus introducing substantial heterogeneity. The term “the autisms” has previously been introduced to help explain the high heterogeneity, hypothesizing that multiple unknown but distinct subtypes exist within the autism clinical category that is now conceptualized as a spectrum [
16]. If such subtypes exist, it is likely that each is associated with different symptom profiles, and that individuals with different types may all be recognised as being on the autism spectrum with high certainty, but for very different reasons. For example, in a child who meets DSM-IV criteria for Autistic Disorder, speech delay may be associated with high certainty that the child is on the autism spectrum. However, for a child who meets DSM-IV criteria for Asperger Syndrome, unusual but highly developed language may conversely contribute to high certainty that this child is on the autism spectrum [
17]. Diagnostic certainty for autism spectrum disorder may thus reflect different, and sometimes opposite, deviations from typical behaviour, which is consistent with the wide heterogeneity that is accepted in the autism spectrum category. Investigations into correlates of certainty therefore run the risk of different effects negating each other resulting in an average that does not meaningfully capture why any particular included individual was diagnosed with high certainty.
To get insight into clinicians’ implicit knowledge through the investigation of diagnostic certainty, a better approach may be to focus on separate autism prototypes instead of the entire autism spectrum. A prototype represents a core presentation of a syndrome or condition, and individuals who are sufficiently similar to the prototype can be recognized by trained clinicians, presumably with higher certainty, the closer they are to the prototype [
18‐
20].
In this study we investigate diagnostic certainty based on prototypical profiles by focusing on those individuals diagnosed with Autistic Disorder as per the DSM-IV. Although previous research has indicated problems with the validity of subgroups defined in the DSM-IV, the group of individuals diagnosed with Autistic Disorder is likely less heterogeneous, and thus more representative of a single prototype, compared to those diagnosed with any autism spectrum diagnosis (i.e. including Asperger Disorder, Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS) or DSM-5 Autism Spectrum Disorder). We may thus gain insight into which features are specifically associated with the Autistic Disorder prototype, and likely obtain a stronger signal for what is considered relevant for recognizing this Autistic Disorder prototype than had we included the whole spectrum. We utilize data from the Simons Simplex Collection, which has also been used in some previous studies that have included diagnostic certainty [
7,
9,
13,
21]. These studies have examined diagnostic certainty for all individuals who have been diagnosed with an autism spectrum diagnosis, whereas we focus on those diagnosed specifically with Autistic Disorder.
Aim
In the present study, we aimed to identify the specific clinical correlates of high certainty of an Autistic Disorder diagnosis. Therefore, we investigated the following research questions: does certainty correlate with total symptom load, are there specific ADOS items that are more highly associated with certainty than others, and are other variables such as proband demographics, HC, IQ, and language level associated with certainty.
Discussion
The objective of this study was to explore factors and signs associated with clinicians’ certainty of an Autistic Disorder diagnosis to gain insight into the implicit knowledge that influences a clinician’s interpretation of diagnostic criteria and clinical decision making in general.
Certainty and ADOS items
As expected from previous research [
9‐
12], we found a modest correlation between diagnostic certainty and autism symptomatology, confirming that a substantial fraction of participants with a relatively low ADOS score were diagnosed with the highest certainty, whereas, some participants did not receive the highest certainty rating despite having relatively high ADOS scores. As mentioned previously, the modest correlation may be explained by different ADOS items having different associations with certainty such that some items contributing to the total score have little association with certainty. By investigating the associations between individual ADOS items and certainty, we confirmed that certain autism signs markedly increased the odds of being diagnosed with Autistic Disorder with the highest certainty, whereas some signs showed only minor associations with certainty, and others even showed a trend towards a negative association. This finding could suggest that particular signs have a stronger impact on how certain clinicians are in their diagnostic decision, likely reflecting that these characteristics are consistent with how the clinicians expect Autistic Disorder to appear. More items from the communication domain were significantly associated with certainty in ADOS modules corresponding to a higher level of language ability. This makes intuitive sense, as lower language ability in itself was found to be strongly associated with higher diagnostic certainty. In these individuals, the qualitative characteristics of language use are likely less important, whereas in individuals with more developed language abilities, qualitative atypicalities may have a larger influence on certainty.
The observation that some ADOS items are more associated with certainty than others may suggest that new ways of constructing assessment instruments could be investigated in the future to improve the specificity of the recognition-definition-investigation cycle [
19]. Additionally, scores on instruments such as the ADOS are traditionally based on an equal weighting of all or some items [
28] meaning that each included item equally contributes to the severity score. Some scoring algorithms (e.g., the calibrated severity score) that only include a select subset of all items have been found to identify autism with higher specificity [
29]. However, given that different items may have different associations with recognizable manifestations of autism, it is also worth considering alternative algorithms with differential weighting of items. Furthermore, it is still an open question as to whether there are interactions between different signs which could improve discrimination; for example, the presence of two items together may have a higher weight than the sum of each item presented separately.
Such considerations may be particularly relevant in relation to the specificity of an instrument since individuals with other conditions may display a substantial number of signs that may also be associated with autism. For example, Havdahl and colleagues [
30] found that the presence of behavioural or emotional problems, as well as low IQ, had a marked influence on the discriminatory threshold of many commonly used diagnostic tools such as the ADOS, suggesting issues with specificity in a complex clinical setting. It would be informative to further explore which items, individual or combined, may be solely associated with autism and which items are also commonly observed in individuals with other conditions such as ADHD or intellectual disability.
Another possible explanation for the modest correlation between the ADOS total score and diagnostic certainty is that some clinicians may score ADOS items as present based on a range of qualitative expressions of a given sign [
3], whereas only some of these expressions are recognized as autistic with high certainty. The distinction between different qualitative presentations is likely learned with experience and future research might investigate the association between qualitative variations in signs and diagnostic certainty.
Correlations between certainty, head circumference, and IQ ratio
We found that individuals diagnosed with the highest certainty had a significantly larger normalized HC than those with lower certainty ratings for all three ADOS modules. Furthermore, 85% of individuals with the largest normalized HC, i.e., individuals within the top 2.5th percentile, were rated with the highest certainty versus 64% of individuals not meeting this criterion. This could indicate that either merely presenting with a larger head than commonly expected or having characteristics that are associated with having a larger HC in the autism population may influence the certainty of the clinician. Exploring associations between the normalized HC and other variables revealed small, but significant positive correlations with several items in the ADOS. Interestingly, certain items overlapped between modules; for example, Shared Enjoyment in Interaction across all three modules, as well as Imagination/Creativity and Reciprocal Social Communication in modules 2 and 3. Most of the significant correlations between HC and ADOS items were within the social interaction, play behavior, and communication domains. In addition, many of the ADOS items in modules 2 and 3 that correlated with HC were also associated with an increased likelihood of having the highest diagnostic certainty. For example in module 2, Shared Enjoyment in Interaction, Reciprocal Social Communication, Amount of Social Overtures, and Showing, which are all from the area of social communication and interaction, were significantly associated with certainty and were also found to be associated with HC. Although previous results on an association between autism symptom presentation and HC have been inconsistent, some studies indicate that particularly social symptoms may be associated with macrocephaly in autistic individuals [
31,
32] while others link it to non-social atypicalities [
33]. The largest correlation of HC was with the verbal to non-verbal ratio (
r = -0.17), but only in module 2. Deutsch and Joseph [
34] found a similar association between macrocephaly and verbal to nonverbal discrepancy in 2003 although with a larger correlation coefficient (
r = -0.35). Interestingly, Joseph and colleagues [
35] found that school age children with an IQ profile of higher non-verbal than verbal IQ had significantly higher autism symptomatology scores within the social interaction domain. Given the associations between diagnostic certainty, HC, social symptoms, and a low verbal/nonverbal IQ ratio, it would therefore be prudent to further explore whether these characteristics are part of a specific autism presentation that is recognized by clinicians with high certainty.
Associations between certainty, language level, and age
Diagnostic certainty was associated with the age at assessment, as well as language level (ADOS module), with a significant interaction. A higher percentage of autistic children received the highest certainty rating when assessed with ADOS module 1 than those evaluated with modules 2 and 3, but the difference decreased with age. For those assessed with module 1 (no phrase speech), the percentage of high certainty was high regardless of age. For those assessed with module 2 (phrase but not fluent speech), diagnostic certainty was lower for children evaluated around three and six years old compared to children in age equivalent groups who were assessed with module 1. Interestingly, the percentage appeared to gradually reach the same high level as for module 1 for the children that are assessed at older ages. This likely reflects the fact that the absence of fluent speech becomes increasingly abnormal with age and, thus, those who are assessed with module 2 at older ages will likely be highly atypical compared to their age equivalent peers. A similar pattern was observed for those assessed with module 3, although the level of certainty was consistently slightly lower than for module 2, reflecting that a young child with highly developed language may be considered less likely to have autistic disorder.
Association between certainty and other variables
We found several significant associations between certainty and IQ, as well as adaptive and externalizing behaviours, although not consistently across ADOS modules. Associations between diagnostic certainty and other variables have been explored in previous studies [
8‐
12]. Negative associations between IQ and diagnostic certainty have been observed previously [
9,
10,
12], consistent with our findings for those assessed with ADOS modules 2 and 3. Adaptive behaviour has been found to be negatively associated with certainty in some studies [
10,
12] while others have found no association [
8]. We found a negative association between certainty and externalizing behaviour among those assessed with ADOS module 3, while no association was found with internalizing behaviour. One previous study using data from the whole autism spectrum in the SSC found weak negative associations with both externalizing and internalizing behaviours [
9], whereas another study found a positive association with internalizing behaviour and no association with externalizing behaviour [
12]. Generally, the previous studies are difficult to directly compare to our results as they operationalized certainty differently. Some previous studies have considered the certainty of the clinician’s decision regardless of whether the decision was autism or no autism. Thus, those who clearly did not meet the criteria would have had a high certainty along with those who clearly did meet the criteria. Furthermore, previous studies investigated all children meeting the criteria for an autism
spectrum diagnosis, whereas we limited our focus to the certainty of meeting the criteria for Autistic Disorder specifically. Certainty for a spectrum diagnosis may cover a broader range of signs, corresponding to the broad range of presentations that can fall within the autism spectrum, whereas certainty for an Autistic Disorder diagnosis may reflect recognition of a less variable presentation. As also mentioned by McDonnell and colleagues [
9], sample characteristics may moderate associations between clinical factors and certainty. The fact that we stratified the sample based on language level (ADOS module), which the cited previous studies did not do, thus also makes direct comparison of the results more difficult.
Limitations
The study focused on those diagnosed with Autistic Disorder, hypothesising that these individuals may be part of a subgroup corresponding to a particular prototype. The findings of this study, thus, do not describe certainty in a broader autism spectrum diagnosis. However, even the sample diagnosed specifically with Autistic Disorder contained variation, e.g. in terms of IQ, age at diagnosis, language level, and total ADOS score, and so might display some heterogeneity in terms of the factors that led to a clinician diagnosing them with higher or lower certainty. At the same time, the variation in the certainty rating among those diagnosed with Autistic Disorder was relatively low, with most individuals having certainty ratings close to the maximum value. This may have made it more difficult to detect associations between certainty and other variables. The heterogeneity of the sample is relevant for the interpretation of our results and may also have affected the magnitude of the identified effects. For example, an observed effect could be driven primarily by a smaller part of the population, but be diluted by other parts of the population that may have different mechanistic underpinnings. As such, the findings might be relevant for a small and potentially unknown subgroup, but not for most of the cohort represented in the SSC. Our finding demonstrating the correlation between HC and the verbal/nonverbal IQ ratio was only present in ADOS module 2. This highlights that it may be relevant to consider whether individuals can be stratified based on common features, such as language level or age at autism diagnosis, when investigating a heterogeneous autism population, [
18] thereby making it more likely that the individuals have something in common. Analysing the whole population may result in the identification of a very small effect that is difficult to interpret. Phenotypic
a priori stratification may decrease noise and make it more likely to identify larger effects that are relevant to the given subpopulation.
The demographic composition of the SSC may indicate a problem with representativeness, which can affect the interpretation of our findings. There was a high percentage of probands from families with a college degree, showing that the population studied had a higher level of education than that generally found in the adult US population [
36]. The percentage of non-white groups was also low, particularly in the part of the sample assessed with module 3 that comprised only 2% African Americans. Predisposing factors of autism associated with, for example, race or the level of education may explain some of these discrepancies. However, it could also reflect a selection bias with certain demographic groups having better access to assessment facilities, thus impacting the generalizability of the findings from the SSC.
Clinicians’ diagnostic certainty is a subjective rating, and so it is expected to be associated with some degree of variability. For example, some clinicians may be certain more often than others, and different clinicians may not have the same understanding of what autism looks like depending on their clinical expertise and exposure to autism. Two clinicians thus may not report the same certainty rating if they were both to assess a given individual. Such differences in how certainty is rated introduce noise and would tend to decrease the size of the observable correlations between the certainty variable and the characteristics of the autistic individuals. Thus, our results likely do not show a universal pattern of how certainty correlates with clinical factors for every clinician, but rather represent an averaged picture across the participating clinicians and indicate those factors that are most associated with certainty. Furthermore, the clinicians contributing to the SSC cohort may not be representative of all clinicians performing autism assessments.
Finally, the study is an exploratory investigation of certainty for an Autistic Disorder diagnosis, and the findings should therefore be sought replicated in future studies.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.