Background
Several anxiety scales are being employed in research and clinical practice for various reasons. Some scales, often used in research, measure specific types of anxiety (e.g., test anxiety, trait anxiety) or specific aspects of individual anxiety disorders (e.g., worry, social anxiety, specific fears) whereas other scales aim to measure a common characteristic of most, if not all, anxiety states or disorders (i.e., general anxiety) [
1]. For use in primary care practice general scales are more relevant because of their promise to detect all or most types of anxiety disorder (i.e., panic disorder, agoraphobia, social phobia, generalized anxiety disorder (GAD), posttraumatic stress disorder (PTSD), obsessive compulsive disorder (OCD) and specific phobia). Detection of anxiety disorders in primary care is important because of their prevalence and associated disability [
2]. Research has shown that general practitioners (GPs) recognize a mental health problem in most of their patients with an anxiety disorder but they have difficulty recognizing a specific anxiety disorder [
3]. A solution to this problem might be the use of a case finding instrument to distinguish between patients with high risk of having an anxiety disorder and patients with low risk. This tool must be robust to prevalence variations as GPs will use it in patient populations with various prevalence rates.
As relevant studies typically either lump different anxiety disorders together or focus on a limited number of specific anxiety disorders, there is currently a lack of evidence that available and popular anxiety scales are capable of detecting all or most types of anxiety disorder in primary care. Examples of popular anxiety scales are the Hospital Anxiety and Depression Scale (HADS) [
4], the Beck Anxiety Inventory (BAI) [
5], the anxiety scales of the Depression Anxiety Stress Scale (DASS) [
6] and the Mood and Anxiety Symptom Questionnaire (MASQ) [
7,
8], and the recently developed Generalized Anxiety Disorder scale (GAD-7) [
9,
10]. The HADS is mainly used in medical settings and appears to perform quite satisfactory [
11,
12], but it may not detect all relevant types of anxiety disorder (e.g., social phobia) [
13‐
15]. The BAI seems to be biased towards panic disorder [
16,
17] and failed to detect any anxiety disorder in some studies [
18,
19]. The anxiety scale of the DASS also seems to favour panic disorder [
20]. The anxiety scale of the MASQ was shown to be fairly good in detecting any anxiety disorder in a community sample [
21], but in higher prevalence samples the scale discriminated poorly between anxiety disorders and other or no disorders [
22,
23]. The GAD-7 appears to be a good screener for GAD, panic disorder, social anxiety disorder and PTSD in primary care [
9,
10], but in higher prevalence samples the GAD-7 performed poorly in detecting GAD [
24]. A few studies reported the failure of anxiety scales to discriminate between anxiety and depressive disorders [
21,
25,
26], which may suggest that some anxiety scales actually measure negative affect or general distress [
24].
The present study concerns the anxiety scale of the Four-Dimensional Symptom Questionnaire (4DSQ). The 4DSQ is a self-rating questionnaire comprising four scales measuring distress, depression, anxiety and somatization [
27]. The anxiety scale is composed of a collection of symptoms that are more or less specific to the various distinct anxiety disorders (see Table
1 for its items). This raises questions about the dimensionality of the anxiety scale. Is the anxiety scale unidimensional, measuring a single trait of anxiety across different groups of patients (e.g., patients with different anxiety and depressive disorders or no disorder), or is the anxiety scale multidimensional, measuring different traits of anxiety in different patient groups (e.g., panic anxiety in panic disorder patients, social anxiety in social phobia patients and general anxiety in GAD patients)? If the 4DSQ anxiety scale is multidimensional, its scores could represent different anxiety problems depending on the specific anxiety disorder involved and anxiety scores could not be compared across diagnostic groups. For instance, an anxiety score of 15 could reflect a totally different problem in a panic disorder patient than in a social phobia patient. From a practical point of view the key question is whether the 4DSQ anxiety scale is able to detect the various specific anxiety disorders equally well (e.g., whether the scale will detect social phobia as well as panic disorder). For the primary care professional it is important to know whether the 4DSQ identifies all anxiety disorders to the same extent or whether it tends to detect some disorders preferentially and miss others.
Table 1
Items of the 4DSQ anxiety scale, mean item scores for the total sample (n = 969)
a
| During the past week, − | |
21 | - Did you suffer from a vague feeling of fear? | 1.04 |
27 | - Did you feel frightened? | 0.91 |
18 | - Did you suffer from sudden fright for no reason? | 0.73 |
44 | - Were you afraid of becoming embarrassed when with other people? | 0.71 |
24 | - Did you suffer from anxiety or panic attacks? | 0.64 |
42 | - Were you afraid of anything when there was really no need for you to be afraid? (for instance animals, heights, small rooms) | 0.52 |
23 | - Did you suffer from trembling when with other people | 0.50 |
50 | - Did you have to repeat some actions a number of times before you could do something else? | 0.44 |
40 | - Did you have any fear of going out of the house alone? | 0.42 |
45 | - Did you ever feel as if you were being threatened by unknown danger? | 0.39 |
49 | - Did you have to avoid certain places because they frightened you? | 0.36 |
43 | - Were you afraid to travel on buses, streetcars/trams, subways or trains? | 0.33 |
It should be noted that the 4DSQ is not intended to be used as a screening tool in unselected consecutive patients, but rather as an assessment and case finding instrument in emotionally distressed patients. As noted above, GPs usually recognize non-specific emotional problems in patients with an anxiety disorder without recognizing that these patients actually have an anxiety disorder that needs specific treatment [
3]. The 4DSQ, as a case finding instrument, could assist GPs in separating patients with high risk of having an anxiety disorder from patients with low risk. The 4DSQ anxiety scale employs two cut-off points, based on clinical experience [
28], a lower cut-off point with a relatively high sensitivity and a higher cut-off point with a relatively high specificity. The idea is that the lower cut-off point be used to identify a group of patients (below the cut-off) with a relatively low probability of having an anxiety disorder and that the higher cut-off point be used to identify a group of patients (above the cut-off) with a relatively high probability of having an anxiety disorder. The latter group should be given priority in a subsequent clinical diagnostic workup targeted at anxiety disorder. The current cut-off points (≥8 and ≥13) are probably set too high [
29].
The present study evaluated the 4DSQ anxiety scale as a case finding tool to identify anxiety disorder and aimed to answer the following questions: (1) Is the 4DSQ anxiety scale unidimensional or multidimensional and what is the scale’s reliability? (2) To what extent does the 4DSQ anxiety scale detect each of the specific anxiety disorder types? (3) Which cut-off points are suitable to rule out or to rule in (which) anxiety disorders?
Discussion
Our results suggest that, in primary care patients, the 4DSQ anxiety scale measures a unidimensional construct. In other words, the scale seems to measure a common trait of anxiety symptoms that is present to a lesser or greater extent in various patient groups. This common trait of pathological anxiety appears to be present to a greater extent in patients with panic disorder, agoraphobia, social phobia, OCD and PTSD, and to a slightly lesser extent in patients with GAD and specific phobia. It is absent, or present to a relatively small extent, in patients with non-comorbid depressive disorders and in emotionally distressed patients without any anxiety or depressive disorder. Notwithstanding the fact that the 4DSQ anxiety scale consists of an admixture of vague anxiety symptoms (e.g., vague feeling of fear, feeling frightened) and symptoms that are more or less specific to distinct anxiety disorder types (e.g., anxiety or panic attacks, irrational specific fears, fear of public embarrassment, repeating actions, avoiding places, fear of public transport) the anxiety scale symptoms appear to work together to measure a common dimension of pathological anxiety. Although the specific anxiety disorders are conceptualized as separate disorders in DSM-IV, in our samples, the specific anxiety disorders relatively rarely occurred stand-alone as single disorders. Multiple anxiety disorders were the rule, rather than an exception. This might, in part, explain why we found the anxiety scale to be unidimensional. Additional research is needed to clarify the dimensions of anxiety symptoms and disorders.
The kind of anxiety that is measured by the 4DSQ anxiety scale was present in most patients with anxiety disorders. This finding compares favourably to existing anxiety scales. However, this anxiety was present to a slightly greater extent in patients with panic disorder, agoraphobia, social phobia, OCD or PTSD than in patients with GAD or specific phobia, and undeniably it was present to a greater extent in patients with multiple anxiety disorders than in patients with single anxiety disorders, and in comorbid anxiety-depressive disorders than in non-comorbid anxiety disorders. Still, the majority of patients with GAD or specific phobia (79%), single anxiety disorders (71%) and non-comorbid anxiety disorders (73%) scored at or above the lowest cut-off point (≥4). Nevertheless, 20-30% of these disorders scored low (<4). In contrast, 85-90% of patients with panic disorder, agoraphobia, social phobia, OCD or PTSD, multiple anxiety disorders or comorbid anxiety-depressive disorders scored ≥4. When it comes to detecting anxiety disorders in primary care patients, the 4DSQ performs better with respect to panic disorder, agoraphobia, social phobia, OCD or PTSD, multiple anxiety disorders and comorbid anxiety-depressive disorders.
A sufficiently strong association between the 4DSQ anxiety score and the presence of anxiety disorder constitutes a prerequisite for the anxiety score to be useful as a tool to detect anxiety disorder. This association depends, first of all, on the concordance of whatever the anxiety scale is measuring and what characterizes anxiety disorder (a matter of validity). In the hypothetical situation that there is 100% concordance, all patients scoring above a certain threshold on the anxiety scale would have an anxiety disorder and all patients scoring below that threshold would not. In practice, of course, the concordance is rarely 100%. In the present study there was evidence that very high anxiety scores not always implied a diagnosable anxiety disorder, and, conversely, that very low anxiety scores did not always imply the absence of anxiety disorder diagnosis. A possible reason for high anxiety scores in the absence of an anxiety disorder diagnosis might be that the patient did not fulfil all necessary criteria for a diagnosis (regarding e.g., duration, distress or disability). A possible reason for low anxiety scores in the presence of anxiety disorder might be that in some anxiety disorder cases manifest anxiety (as measured by the 4DSQ) was not a prominent feature of the disorder or was not necessarily present all the time. This happened relatively more often in cases diagnosed as GAD or specific phobia.
The observed association between the anxiety score and the diagnosis of anxiety disorder is also determined by the amount of measurement error, both in the anxiety score and in the assessment of the anxiety disorder diagnosis. Measurement error in the anxiety disorder diagnosis translates into misclassification and reduced reliability of the diagnosis. In our studies diagnostic reliability was not assessed, but typically the interrater agreement (Cohen’s kappa) of anxiety disorder diagnoses varies between 0.60 and 0.80 [
41]. A kappa of 0.70 means 70% agreement after correction for chance agreement. Considering that there is a continuity between normality and anxiety disorder, it should be realized that the risk of misclassification is greatest near the threshold of disorder.
The mean reliability of the anxiety score across the study samples was 0.90, yielding a SEM of 2 points, suggesting that the 84% confidence interval of a given observed anxiety score X was X ± 3. In other words, when the observed anxiety score was X, we could be at least 92% confident that the true score was not > (X + 3) and we could also be at least 92% confident that the true score was not < (X–3).
When performing ROC analyses, we observed wide confidence intervals and significant variability of the thresholds across the studies. This variability must be attributed to differences in prevalence and severity spectrum of the samples, and also to distributional irregularities produced by chance in relatively small samples. Combining the samples by pooling was a logical action in order to obtain more stable estimates. This way we obtained 6.5 as the best single threshold to detect panic disorder, agoraphobia, social phobia, OCD and PTSD. Yet, using this single threshold would misclassify over a quarter of patients in either group. Therefore, we chose two thresholds, one (3.5) with a relatively high sensitivity to single out patients with a relatively low probability of having panic disorder, agoraphobia, social phobia, OCD or PTSD and one threshold (9.5) with a relatively high specificity to single out patients with a relatively high probability of having panic disorder, agoraphobia, social phobia, OCD or PTSD. Note that both thresholds are 1.5 SEM away from the threshold (6.5) of panic disorder, agoraphobia, social phobia, OCD and PTSD. This implies that we can be more than 92% confident that patients with anxiety scores 0–3 do not have a true anxiety score above the threshold of panic disorder, agoraphobia, social phobia, OCD and PTSD. Conversely, we can be more than 92% confident that patients with anxiety scores 10–24 do not have a true anxiety score below the threshold of panic disorder, agoraphobia, social phobia, OCD and PTSD. The uncertainty about whether or not a patient has passed the threshold of panic disorder, agoraphobia, social phobia, OCD and PTSD has now been restricted to one third (25-42%) of all patients, who score 4–9 on the anxiety scale.
The primary care professional can use the two cut-off points of the 4DSQ anxiety scale to separate patients with mental health problems into three groups: (1) a group with high anxiety scores (10–24), (2) a group with moderate scores (4–9), and (3) a group with low scores (0–3). Patients with high anxiety scores have a relatively high probability of having one or more anxiety disorders. Importantly, a high anxiety score does not represent a clinical diagnosis in itself. In addition, as noted earlier, the 4DSQ anxiety score does not indicate which specific anxiety disorder(s) is (are) present. A clinical diagnosis should be made in the short term using clinical judgment and available clinical guidelines [
48,
49]. Given the likelihood ratio, the chance of diagnosing one or more anxiety disorders is relatively high. Moreover, patients with high anxiety scores tend to have relatively clear-cut disorders as most borderline anxiety disorders are classified into the moderate group. On the other hand, patients with low anxiety scores have a low probability of anxiety disorder and when they do have an anxiety disorder, it will often be GAD or specific phobia, or a borderline anxiety disorder. These patients do not need a diagnostic interview targeted at anxiety disorder for the time being. Probably, in this low anxiety scores group, other problems (e.g., depression, stressful life situations) are more important to address. In the middle group with moderate anxiety scores (which constituted one third of our pooled sample), the possibility of anxiety disorder has not been ruled out as the probability is about the same as the prevalence. Anxiety disorder cases in this group are relatively often just beyond the diagnostic threshold and other problems (e.g., depression, stress) might be in more need of treatment. We argue that a diagnostic workup targeted at anxiety disorder can be postponed for a few weeks while monitoring the effect of non-specific interventions (e.g., reassurance, encouragement, advice) and the passage of time. When after 3–4 weeks symptoms decline, further diagnostic workup targeted at anxiety disorder does not seem to be necessary, but when symptoms do not abate a diagnostic interview is warranted after all. In our experience, this way GPs can efficiently target their diagnostic efforts to patients with a relatively high risk of having an anxiety disorder while keeping patients with moderate risk under surveillance. We acknowledge that there is currently no firm evidence to support this strategy, but it is our impression that it works fine in the primary care setting. More research is needed in this area.
The main limitation of the present study relates to the representativeness of the datasets included. Each of the datasets had been collected for other purposes than to evaluate the measurement properties of the 4DSQ. We would have preferred a large representative sample of primary care patients with mental health problems, each extensively assessed using a standardized psychiatric interview. However, this is costly and logistically challenging. Therefore, we employed convenience datasets collected in other studies. We assumed that the psychiatric diagnoses were principally invariant across the study samples as the samples could all be considered draws from the same large pool of primary care patients with mental health problems. Due to sampling differences, a fair degree of heterogeneity across the studies was evident, but this probably represented a strength of our study instead of a weakness. Furthermore, as the 4DSQ anxiety scale demonstrated high reliability and identical measurement properties across the studies, we assumed that the operating characteristics of the scale (i.e., sensitivity and specificity) were principally the same across the studies, only varying due to sampling. Therefore, we assumed that pooling (i.e., effectively conducting a patient level meta-analysis) was the best way to obtain valid estimates for the operating characteristics of the anxiety scale.
A second limitation concerns the fact that some studies did not assess the whole range of anxiety disorders. Notably, specific phobia, OCD and PTSD were not included in three studies. We estimate that if these diagnoses would have been established with a prevalence of 10-15%, assuming that at least two thirds of these disorders would co-occur with another (already known) anxiety disorder, the total increase in anxiety disorder patients across the studies would amount to 5-10%. This would have lead to a small decrease in the anxiety scores of patients without anxiety disorder. We assume that this would not have changed the results in any substantial way. However, replication in new samples would be desirable.
A third limitation constitutes the lack of information about interrater reliability of the diagnostic interviews. We relied on the reported reliability of these standardized interviews when performed by carefully trained interviewers. However, it should be noted that low reliability (i.e., measurement error) would attenuate existing associations between the 4DSQ anxiety score and anxiety disorder diagnosis. Because measurement error usually does not correlate with anything, it is unlikely that low reliability would be responsible for false associations. In other words, the associations in this study, as expressed in areas under the ROC-curve, sensitivities, specificities and likelihood ratios, are real and provide some reassurance regarding the diagnostic reliability.
This study took place in the DSM-IV era. However, in the meantime the DSM-5 – published in May 2013 – has decided not to classify OCD and PTSD as anxiety disorders anymore [
50]. Instead OCD is included in a separate section with disorders characterized by compulsive behaviour, whereas PTSD is included in a section with disorders following traumatic or stressful events. Yet, our findings provide evidence of at least some degree of kinship between these disorders and typical anxiety disorders like panic disorder, agoraphobia and social phobia.