Validity and reliability of the Malay version of the Structured Migraine Interview (SMI) Questionnaire

Two independent translators (NNI and ZZ) performed the forward translations. Both translators are native Malay speakers with a good command of English who speak multiple dialects of the Malay language and are bicultural. The forward translated versions were evaluated by a panel of experts (NBH, SHG and SB) to further verify the conceptual, idiomatic, semantic, and cross-cultural equivalence to the English version.

The final forward translation was then back-translated into the English language by two different independent back-translators (AFAR and AHA). Both translators were unaware of the concepts and purpose of the questionnaire or the nature of the study. These translators were bicultural, native Malay speakers with multiple dialects and with good command of the English language. A special panel (NBH and SHG) reviewed the forward- and back-translated versions to produce the final draft of the Malay version of the SMI-M.

To determine face validity, the SMI-M was administered to 30 patients who have headache and are aged between 15 and 60 years. This step is essential to ensure the quality of the questionnaire and also to acquire proper feedback from patients when answering the translated questions, especially in understanding the medical terminology used. During this period, no difficulties in understanding the questions were noted. Overall, the SMI-M questionnaire was easily understood and well accepted by both the patients and the experts alike. The content validity was then established by administering the SMI-M questionnaire to an expert (SB) in the Neurology Department for further evaluation. A special panel (NHB, SHG and SB) further reviewed, discussed and modified the questionnaire based on the noted problems to construct the final SMI-M questionnaire.

Patients with headache (n = 200) who registered with the Neurology Clinic of Hospital Universiti Sains Malaysia (HUSM) between January and December 2013 were randomly screened according to the inclusion and exclusion criteria. The inclusion criteria included patients ageing between 15 and 60 years who have been diagnosed with migraine based on the criteria of the 2 ^nd edition of the ICHD (ICHD-II), 2004 [1] or who are suspected to be suffering from headache. Patients with cardiovascular disorders, neurological disorders, head injuries, ophthalmological problems, menstrual migraine or who were pregnant were excluded. The study was approved by the Universiti Sains Malaysia Research and Ethical committee (ethical no: USMKK/PPP/JEPeM [231.3.(08)]) and complies with the Declaration of Helsinki.

After the initial screening process, 157 patients having headache agreed to participate in the study (Figure 2). They were verbally informed about the purpose of the study before being asked to complete the written informed consent forms, basic socio-demographic information form and SMI-M questionnaire. Following this step, all patients were evaluated by a headache specialist during their initial visits. Overall, patients were asked to complete the SMI-M twice: the first time during their visits to the clinic (1^st compilation) and the second time at home (2^nd compilation) where the completed SMI-M questionnaires were mailed back to researchers in standard stamped envelopes seven days following the completion of the 1^st compilation. Patients were only aware of their clinical diagnosis but not the diagnosis by SMI-M. In addition, patients were also asked to continue their regular medication if prescribed by the clinician.

Kaiser-Meyer-Olkin (KMO) and the Barlett’s tests of sphericity were applied to estimate the sampling adequacy [15]. Sensitivity, specificity, misclassification rate, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio of migraine diagnosis by SMI-M were further measured to test the validity of the SMI-M.

The internal consistency of the SMI-M score was assessed using Cronbach’s alpha (α). An α value of 0.7 is considered to be “acceptable”, while an α value of 0.8 or more indicates excellent internal consistency [16]. A test-retest reliability to measure consistency across the duration (i.e., between the 1^st and 2^nd compilations) of both total and individual question scores was evaluated using Spearman correlations. A Spearman correlation is conservative, as it is not usually influenced by outliers. The percentage of concordance of the 2^nd compilation compared with the 1^st was taken as the stability of the SMI-M. Statistical analysis was conducted using SPSS version 20.0 software (New York, USA).

A total of 157 patients with headache completed the administered questionnaires relatively quickly. None of the patients had any difficulties understanding or answering any parts of the SMI-M. The mean age of the patients was 26.81 (8.33) years and the number of years of education was 14.25 (2.81) (Table 1). In this study, the majority of the patients were students (56.7%) and singletons (70.7%). The majority (63.7%) of household incomes were average (between RM 801- RM 3000). Only female patients were recruited for the study, and all of them were of the Malay race, which is the dominant ethnic group in the Kelantan state of Malaysia. The sample was adequate as indicated by a KMO value of 0.85 and a significant Barlett’s test of sphericity (p < 0.001).

Clinically, 63.7% of the patients were diagnosed as having migraine by the neurologist (SB) in contrast to 75.2%diagnosed as having migraine using the SMI-M questionnaire (Figure 3). In total, 22.9% of the patients were diagnosed as not having migraine by both the neurologist and the SMI-M questionnaire. However, 13.4% of the patients were diagnosed as having migraine using the SMI-M, and these same patients were diagnosed as not having migraine by the neurologist. The SMI-M failed to accurately diagnose three (1.9%) patients as having migraine as diagnosed by the neurologist. Therefore, the misclassification rate was 0.15, the positive predictive value was 0.82, the negative predictive value was 0.92, the positive likelihood ratio was 2.62, and the negative likelihood ratio was 0.05. In addition, the sensitivity was 0.97 and specificity was 0.63. The Kappa statistic value was 0.7 (p < 0.001) and was statistically significant (Table 2).

The Cronbach α values at the 1^st (0.93) and 2^nd compilations (0.90, p < 0.001) exceeded the 0.9 level, indicating excellent internal consistency. The Spearman’s correlations for the 1^st and 2^nd compilations were compared and ranged from 0.86 (for question 4) to 0.95 (for question 1) (Table 3). The overall concordance was calculated as the percentage of the 1^st and 2^nd compilations for each item (Table 4). The overall concordance for item 1 was very high (97%), followed by item 4 (83%), item 2 (71%) and item 3 (64%). For item 2 (moderate to severe headache accompanied by nausea and/or vomiting), approximately 85.71% of the patients changed their responses from the fourth option (Yes, 10+ times) to other options. For item 3 (moderate to severe headache accompanied by hypersensitivity to sound or light), approximately73.91% of the patients changed their responses from the fourth option (Yes, 10+ times) to other options.

Patients attending a neurology clinic are usually afflicted with severe headache associated with impairment so debilitating that a specialist’s intervention is required. During the 1^st compilation, patients were asked to report any problems related to the questionnaire. However, no difficulties in understanding and answering the SMI-M questionnaire were reported. In addition, no ongoing therapies (symptomatic or prophylactic) undertaken by the patients were altered during the course of the test-retest study to ensure that similar conditions were maintained for the patients in both compilations.

Migraine remains a clinical diagnosis based on the IHS’ migraine diagnosis criteria, which is considered to be the “gold standard”. Therefore, the findings from the SMI-M were compared with migraine diagnosis by the headache specialist in order to test the validity of SMI-M. The validity testing compared with the clinical diagnosis indicated that the SMI-M is a highly sensitive (0.97) instrument but was only moderately specific (0.63) in identifying migraine cases. Nevertheless, the sensitivity and specificity results of the SMI-M from this study were consistent with the original questionnaire.

The moderate value of kappa statistic (0.7) indicates a substantial agreement between SMI-M and the clinical diagnosis [17]. The kappa value in this study was slightly lower (0.70) than that in the original study (0.82). Previous reports suggested that regional differences may lead to variations in the clinical diagnosis. The difference in the kappa values between the original SMI and SMI-M may be due to differences in clinical practices between clinicians in the different regions [18]. Furthermore, variations between personal, organizational and systemic levels may also lead to variations in clinical practice [19]. The positive predictive value (0.82) was slightly lower than that of the original study (0.97). However, the positive predictive value depends upon the prevalence of the outcome of interest, which varies from one population to another [20]. Nevertheless, the negative predictive value in this study (0.92) was much higher than that of the original study (0.26), indicating that the SMI-M questionnaire rarely misclassified a migraine patient as a non-migraine patient [20].

The lower misclassification rate (0.15) indicates that SMI-M is not capable of labelling non-migraine patients as having migraine. A higher positive likelihood ratio of SMI-M (2.62) in this study when compared with the original SMI (2.07) suggested that there was a higher probability for a migraine patient to be correctly diagnosed using this questionnaire. A lower negative likelihood ratio of SMI-M (0.05) than the original SMI (0.22) makes SMI-M optimal in ruling out the chances of wrongly diagnosing migraine. Therefore, the SMI-M can help clinicians to decide whether a patient indeed has migraine [21].

The SMI-M has been reported to be comparable with the IDMS [9] and DMQ3 [10] but is better than the Dutch [11] and German headache questionnaires [12], which was also found in our study. For example, the sensitivity of SMI-M (0.97) was similar to that of the DMQ3 (0.99) but was higher than those of the IDMS (0.81), Dutch (0.51) and German headache questionnaires (0.63). In comparison with SMI-M, the DMQ3 contained 56 items, which can be tedious for patients to answer. In contrast, the SMI-M is very easy for patients to score when compared with the DMQ3, as it has only four items focusing specifically on migraine diagnosis. However, the IDMS questionnaire only has three items but does not contain any questions related to aura symptoms. Therefore, the SMI-M is a valid tool to be used in research for the identification of migraine in large samples where clinical interviews are less practical. In addition, the SMI-M should be applied to the local clinical settings for reconfirming the status of migraine. Nevertheless, some variations (whenever they exist) may be contributed by the study design and possibly also in the degree of strictness with which the ICHD-II criteria were applied in clinical the diagnosis.

The internal consistency of the SMI-M questionnaire was excellent for both compilations. The higher value of Cronbach’s alpha indicated a higher degree to which all items in the SMI-M questionnaire measured single uni dimensional latent constructs. This value also indicated good questionnaire reliability in diagnosing migraine among Malay patients with headache. Although the patients were aware of their neurological diagnosis, the high correlation values based on Spearman’s statistical analysis indicated excellent test-retest reliability seen with the SMI-M questionnaire. In this study, comparing the reliability of the SMI-M with the original SMI questionnaire could not be conducted, as reliability studies were not reported in the original study.

Among all of the items, item 1 yielded the highest Spearman’s correlation, which may be attributed to the simplicity of the question pertaining to the existence of recurrent headache. However, item 4 yielded the lowest Spearman’s correlation coefficient, even though the correlation coefficient was excellent. This findings may be attributed to the 7 days gap, which may have allowed some time for the development of migraine-related events that may have affected some of the responses including MA symptoms in between the two compilations.

Nearly 36% of the patients changed their response choice from the 1^st to the 2^nd compilation, which again may be attributed to the 7 day gap between the compilations. It was observed that the change of symptoms or a new migraine attack could occur in a short period, which may be sustained for long hours and may contribute to the change in scores in a short time period especially among severe migraine patients. For item 1, only 3% of the patients changed their responses, but for item 3, 36% patients changed their responses between both compilations. The duration between both compilations may have allowed precipitations or changes in the symptoms, especially if patients are hypersensitive to light and sounds, which are common stimulants for migraine as investigated in question 3. Nevertheless, gaps between 2 days and two weeks have previously been reported to be ideal for test-retest reliability studies [22].

The second part of the original SMI questionnaire that was related to symptoms was not included in the SMI-M as it is simpler, less time consuming, and focuses only on diagnosis. Moreover, removing the second part of the questionnaire did not affect its validity or reliability.

This sample was limited only to patients from the Malay race because there was no report of migraine patients from other races (Chinese, Indian and Siamese). All patients were females because no male patients with headache visited the clinic during the study period. Although care was taken with the translation and cross-cultural adaptation process to keep the originality, simplicity and clarity of the questionnaire, cross-cultural equivalence of the questionnaire is subjective and is difficult to measure precisely. Therefore, expert judgment was necessary to investigate whether the original and translated versions are equal.