Freiburg Questionnaire of linguistic pragmatics (FQLP): psychometric properties based on a psychiatric sample

Asperger’s Syndrome (AS) is an autism spectrum disorder (ASD) that is characterized by significant difficulties in social interaction and nonverbal communication, and restricted and repetitive patterns of behavior and interests. According to the criteria of the ICD-10 (F84.5, [1]) AS is distinguished from early childhood autism by unimpaired speech and cognitive development. A closer look shows, however, that patients with AS have peculiarities in their understanding and use of language, well characterized by Hans Asperger [2]. In his original publication, he pointed out that while children with AS show very good abilities regarding the terminology of language and formulate correctly and precisely, they do have difficulties with respect to nonverbal communication, reading emotional and social cues in and “between” the lines of the speaker, differentiating between fact and fiction, taking into account the influence of context on a statement, and modifying communication depending on priorities [2]. In recent years, many studies have shown that people with ASD and AS show abnormalities in the pragmatic understanding and the use of language [3]-[8]. The scientific results overall are inconsistent, however, and explanations of these peculiarities have remained controversial until now [9]. Many of the described characteristics refer to an inability to respond adequately in a specific context. This is already apparent in children who often cannot select deictic expressions (I, you) correctly [10]. Grownups often show difficulties comprehending sentences with several meanings since they do not take the context into account sufficiently [3]. Problems in understanding metaphors illustrate that comprehending non-literal language also seems to pose a difficulty [11]. Peculiarities were also found in the understanding of figures of speech [12] and irony [13]. Shortcomings in the understanding of indirect language have also been shown [14]. Interpreting utterances in their social context seems to pose particular difficulties to patients with AS. They often misuse forms of politeness and are less able to narrate coherently and clearly than people without AS [15]. While the majority of authors assume a specific problem of pragmatic and/or figurative language in autism, this hypothesis is still matter of debate: Some authors argue that most of the findings mentioned above are just an artifact of a problem of general language comprehension, which was not controlled for [16],[17].

A preliminary study [18] showed, however, that people with AS are aware of differences in the way they comprehend language (irrespective of whether or not these differences are genuinely pragmatic in nature). Based on these results we assume that the self-perception of language abnormalities could be an important aspect in the assessment of AS. To date, no validated screening questionnaire for AS focusing on the self-perception of language comprehension exists. Hitherto established self-report questionnaires concern social understanding (Autism Quotient, AQ, [19]), repetitive behavior (AQ), empathy (Empathy Quotient, EQ, [20]) and systemized thinking (Systemizing Quotient, SQ, [21]). In the Social Communication Questionnaire and the Social Responsiveness Scale [22]-[24] several language related symptoms are addressed. However, there is no focus on the self-perception of the language comprehension. This study aims to fill this gap.

This study aims to investigate psychometric properties of the Freiburg Questionnaire of linguistic pragmatics (FQLP). Furthermore, we want to examine whether this questionnaire might be a useful tool in the diagnostic work up of autism spectrum disorders based on a group of diagnosed patients with AS, a convenience sample and a group of patients with other psychiatric diagnoses. Our aim is not to show whether the FQLP can serve as the sole tool to diagnose AS. Rather, our concern is to assess to what extent it can serve as a useful additional tool in the diagnostic process and as a screening instrument for AS.

Psychometric properties of the FQLP, a self-assessment questionnaire which focusses on difficulties with pragmatic speech, were explored in a sample of 57 AS patients, 66 patients with other psychiatric disorders, and a convenience sample of 56 people.

The FQLP requires a reading level typically found in people with medium education level. As 93% of the AS-group had a medium or high education level, the vast majority should have been able to read and understand our questionnaire without any difficulties.

No definite one-dimensional solution was found regarding the dimensionality of the first version of the questionnaire with 13 items. Furthermore, not all items showed sufficient factor reliability. Therefore, two items were removed. This procedure resulted in a scale with good psychometric parameters: The 11-item-FQLP is a one-dimensional construct with high internal consistency and a very high reliability, represented by a very high Cronbach’s α. Good convergent validity could also be demonstrated for the FQLP with AQ and EQ, which are well established questionnaires regarding symptoms of AS. FQLP, and AQ and EQ showed a very high statistical correlation even though they assess different aspects of the AS respectively. As hypothesized, the FQLP showed only moderate correlation with psychological distress and with symptoms of personality disorders. The correlation with education was significant, but low. Limited influence of education on self-rated pragmatic speech abilities was found. Self-perception of language comprehension measured with the FQLP is an independent construct that can clearly be distinguished from education. Discriminant validity was assessed in comparison to MWT-B, which describes vocabulary abilities, in order to clarify whether the FQLP only assesses basic linguistic (for example semantic) abilities. This resulted in no significant correlation, demonstrating that the FQLP captures a construct which is fundamentally different from vocabulary abilities. As far as criterion validity is concerned, we found significantly lower FQLP-scores in the AS-group compared to the C-group or the PD-group. On the basis of a receiver operating characteristics curve a very high diagnostic accuracy of the FQLP could be demonstrated: In comparison with the healthy population sample and the PD-group, the FQLP showed a high predictive value regarding AS.

Although the main results were surprisingly clear, there are some limitations to the study: First, because of the small sample size the result could not be cross-validated. Second, although no indication arose that age or sex had any influence on the FQLP score, it must be stated as a limitation that the groups differed in these respects. Third, the subgroups were too small to allow for a differentiated evaluation of the FQLP scores of the individual psychiatric illnesses. Therefore, reliable statements about their perception of their respective pragmatic peculiarities are not yet possible. It would make sense to particularly assess the group of schizophrenic patients in a larger group using the FQLP, since pragmatic peculiarities have repeatedly been described for this group [37]. Perhaps the FQLP questions relating to childhood and youth could be an especially helpful tool in distinguishing these two conditions. Fourth, it is difficult to judge a possible bias of the psychiatrists who diagnosed AS in the patients examined here: All patients were recruited by the Clinic of Psychiatry and Psychotherapy of the University Medical Center Freiburg, and all diagnoses were made there. Diagnoses are formed according to the ICD-10-Criteria. However, it cannot be excluded that the interest which the team has in the pragmatic peculiarities of AS patients has influenced the diagnostic process, leading perhaps to a higher percentage of patients receiving the diagnosis AS who demonstrated pragmatic language abnormalities. This may then have contributed to the fact that many AS patients reported pragmatic peculiarities in this study. According to our own judgement, this AS-sample-related selection bias has only a moderate effect if at all, because the diagnostic procedure was carried out very carefully and the questionnaires were completed independently by participants and patients without knowledge of the study hypothesis. Still a replication of our results by an independent work group is desirable. Fifth, in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5, 2013) the diagnosis of social (pragmatic) communication disorder (SCD) was newly established. Since SCD concerns a new diagnostic category, which also finds no correspondence in the ICD to date, no sufficient number of diagnosed patients is yet available who would qualify for participation in such a study. It must, however, be presumed that the FQLP as differential diagnostic instrument between AS and SCD will be less helpful, since it targets just that range of symptoms that both diseases have in common. In those cases, other instruments should be made use of. Nonetheless, this does not affect the demonstrated high sensitivity and specificity of our instrument with respect to AS in distinction to other psychiatric diseases. Where it is a clinical issue to distinguish between AS and obsessive compulsive disorder, for example, or between AS and ADHD, the FQLP can make a real contribution to the differential diagnosis. As mentioned above, care has been taken to strictly follow the NICE-guidelines in establishing a diagnosis, and pragmatic language abnormalities are not of high relevance in this process. Therefore, it is very unlikely that SCD-patients were falsely diagnosed as AS-patients.

Sixth, the questions of the FQLP implicitly require a high degree of education, since words such as “metaphor”, “ambiguity”, “rational analysis” or “intuitively” must be readily understood. The FQLP questions also demand quite a high level of self-reflection and introspection. Questions such as: “I use rational analysis to work out the meanings of metaphors” cannot be answered without introspective ability. Thus, the questionnaire might turn out to be suitable rather for high-functioning and well-educated individuals. Seventh, we did not define a cut-off point for the FQLP prior to our investigation. The cut-off criterion was retrospectively optimized and post-hoc set at the value of 32. This means that the results concerning specificity and sensitivity have to be interpreted cautiously, at least until the findings have been replicated in the context of further prospective studies. Eighth, the inclusion of subjects for the C-group using personal contacts of the authors might have introduced a bias towards high linguistic pragmatic abilities in this group. This might have led to a more significant difference between the C-group and the AS-group. Since the difference between the C-group and the PD-group – which was obviously not influenced by the mentioned bias – was comparatively low, we assume this effect to be rather small. Nineth, the nature of what the FQLP measures can be questioned. Gernsbacher and Pripas-Kapit [16] argue that the deficits in pragmatic language which are found in autism could also be a consequence of deficits in general language comprehension rather than of specific pragmatic impairments. If this were true, the FQLP would not evaluate pragmatic abilities, but language comprehension in general. This cannot be excluded and can be considered as a limitation of the FQLP. In particular this would imply that the name of our questionnaire may be misleading. Nevertheless this objection neither challenges the finding that the FQLP has been shown to measure a one-dimensional construct with a high internal consistency which is fundamentally different from vocabulary abilities, nor does it challenge the result that the FQLP showed a high diagnostic accuracy in differentiating AS from other psychiatric conditions.

Finally, there are a few more points which should be discussed: The one-dimensionality of the FQLP is striking in light of the fact that the items of the FQLP relate to rather different aspects of language comprehension – at least at first glance: They investigate the understanding of irony and metaphors, the strategies to decipher figurative language, the participants’ principal ideas how language should be (“In an ideal language, there would be no ambiguity of meaning”) and the extent to which participants feel they understand other persons' utterances. In our opinion, the one-dimensionality indicates that impairments in the various aspects of language comprehension assessed by the FQLP can be traced to a single underlying syndrome in the case of persons with AS.

Another interesting observation is the high convergent validity of AQ/EQ and FQLP. The results indicate that pragmatic speech abilities and the symptom ratings are highly related constructs or may even assess different aspects of the same construct. This in turn would mean that the peculiarities of pragmatic language investigated by the FQLP are a symptom of AS.

The moderate correlation between general psychopathology or symptoms of personality disorders and the FQLP results may show that pragmatic peculiarities are found also in general psychopathology and in personality disorders. However, since the non-autistic patient group, which had high scores in general psychopathology, showed only mild abnormalities in the FQLP, we assume that the constructs “psychopathology”, “personality disorders” and “pragmatic abnormalities” exist primarily independently of each other and that this moderate correlation can be explained by the frequent comorbidities of autism spectrum disorders [34].

Since no correlation resulted between the results of the FQLP and the MWT-B, pragmatic peculiarities explored by the FQLP are probably independent of vocabulary skills. One could object that tests of vocabulary like the MWT-B do not allow conclusions with respect to language comprehension [44]. For this reason, it cannot be excluded that the FQLP measures general language comprehension – and not specific pragmatic abilities. Nevertheless, it is remarkable that both the educational level and the FQLP-score of the AS-group were significantly higher than that of the PD-group. The high educational level in the AS-group argues against significant problems of general language comprehension. This indicates that problems of general language comprehension are probably not a proper explanation for the low FQLP-score in the AS-group.

The high percentage of women in our group of patients diagnosed with AS is remarkable. Older surveys assume a male to female ratio of 8:1 [45]. However, surveys focussing only on adults find more often a higher percentage of women (2:1, 3:1) [34],[35]. Lehnhardt, who found a male to female ratio of 2:1, comes up with some plausible explanations for this phenomenon [35].

The most substantial finding of this study is the remarkably high predictability of the FQLP regarding AS. As our control group consisted of people without mental disorder and patients with different psychiatric disorders this result can be seen as a clear indication that the construct examined by the FQLP is quite specific to the peculiarities of AS. Furthermore, the results of the correlational analyses of the divergent validity indicate that the FQLP measures a construct that can clearly be distinguished from general psychological distress or specificities of personality. Comparing the FQLP to other questionnaires for AS, it shows comparable or even better results: The short version of the AQ, which has 10 items, shows – depending on the cut-off-point – specificity between 73% and 89%, and sensitivity between 71% and 92% [46]. Nota bene in the aforementioned study the AS-patients were compared to a control group of healthy subjects. Even the “long” AQ with 50 items used in the screening of AS exhibited a sensitivity of 95% and a specificity of only 52% [47]. This study examined people who were already suspect of AS before the study, which explains the low specificity. The EQ demonstrated a sensitivity of 81% and a specificity of 88% [20]. The Social Communication Questionnaire (SCQ) [23] was evaluated quite well, showing a sensitivity of 89% and a specificity of 91% [22] in a quite heterogeneous study sample. Since the SCQ is a 40-item questionnaire for the parents of autistic children, it is not totally comparable to the self-assessment questionnaires for adults like AQ, EQ or FQLP. At any rate our results show that – using a retrospectively optimized cutoff criterion – sensitivity (93% at a cutting-point of 32) and specificity (85% at a cutting-point at 32) of the FQLP are well comparable to already established questionnaires used for the screening of AS. Of course, it is by no means to be expected from a short questionnaire like the FQLP that it could replace proven diagnostic instruments, let alone the clinical diagnosis. In addition, its specificity relative to social communication disorder, which was not examined, is presumably only moderate. However, since the FQLP shows a good sensitivity and specificity relative to other psychiatric illnesses while being very short (a mere 11 questions), we assume, on the basis of the results shown here, that it is very suitable as a diagnostic instrument for AS in psychiatric practice and may complement the conventional diagnostic process well.

The FQLP is a reliable, brief and valid instrument. First results regarding sensitivity and specificity are highly promising.