In the following, under the various themes, we discuss the issues of concern and highlight how methodological shortcomings might have detrimental impact on study quality.
The population of interest
In headache research, the population of interest is usually the population of a whole country, because national data are needed for health policy, but there may be good reasons to study regions larger [
40] or smaller [
5] than a country. Depending on the aim of the study, sub-populations defined by additional characteristics may be perfectly legitimate subjects of study: specific age groups (
e.g., adults of working age, adolescents, school or pre-school children); members of groups defined by ethnicity, culture or language; workers in certain trades or professions, or university students; people with another particular disease,
etc. Headache
patient populations on the other hand, while they are easily accessed, are rarely of interest: they are highly selected and, furthermore, the criteria by which they are selected (often self-selected) are generally indeterminable. A study of such populations tells little about, and cannot be extrapolated to, either the general population or any more broadly-defined population.
Sampling, and bias avoidance
It is rarely possible to survey everyone in the population of interest, and usually necessary to choose from that population a smaller, manageable sample of people to whom access is possible. Because the intention is to generalize the data from this sample to the whole population of interest, the essential requirement is that it should remain representative of the population of interest. “Representative” means similar to the population of interest in all properties of relevance to (i.e., likely to influence) the object of measurement (here, headache prevalence and/or burden). In the context of headache, representativeness clearly encompasses age and gender, which are known to affect headache prevalence, and probably should encompass socioeconomic class, employment status, area of habitation (rural or urban), ethnicity and possibly native language and/or tribal group.
Random sampling from the entire population of interest, either simple or stratified, depends upon chance to achieve representativeness, with likelihood of success dependent on sample size. In most cases, random sampling depends on the existence of some form of overview of the population: for example, census data, population registers or a map showing all households in an area to be sampled. Studies adopting this method usually sampled by telephone: an established methodology, offering cheap and easy access. In addition, this method is possible in the absence of a population overview such as a complete telephone directory through the technique of random digit-dialling (dialling area code(s) followed randomly by as many digits as are typical for phone numbers in the area(s)) [
132]. Telephone sampling can work well in countries where telephones (land-line or mobile) are both widespread and evenly distributed across different age, gender and socioeconomic groups; otherwise, it is open to serious selection bias.
Many studies employed some form of cluster sampling, usually selecting participants from a limited number of defined geographical areas (e.g., blocks or streets, or parts of villages), themselves chosen randomly. This method is logistically efficient when selected participants are to be visited. In countries with obligatory schooling, representative samples of children of school age can be obtained by selecting all, or a random sample, of the pupils of a representative sample of schools (a form of cluster sampling).
Generally, in headache studies, only one member of a family is selected. Members of families are similar genetically, share their environment and have common lifestyles; these factors effectively reduce variance if two or more are included. Special issues therefore arise when sampling is dependent upon contacts with households rather than single persons (e.g., calling door-to-door or by household telephone without prior warning). Within a household, certain types of person are more likely to stay at home, open the door or answer the phone: selection bias will inevitably arise when the person interviewed is whoever happens to answer.
Sample size determination
Sample-size requirement depends on the prevalence of the disorder and the precision of the estimates needed. Table
2 shows the margins of error for a prevalence of 10% (taken as an example) associated with sample sizes ranging from 200 to 10,000. By increasing the sample size from 200 to 2,000, the margin of error is decreased from 4.2% to 1.3% (absolute), but the gain from sample sizes of >2,000 is relatively small.
Table 2
Margin of error (95% confidence interval) according to sample size
Margin of error (%)
| 4.2 | 2.7 | 1.9 | 1.3 | 0.8 | 0.6 |
A larger sample may be needed to estimate burden than to estimate prevalence, because burden is not distributed equally among cases: most of it is accounted for by a minority of those with the disorder. In a Swedish survey, for example, 27% of people with migraine reported 68% of all attacks [
133]. Among all people with migraine, TTH or MOH, the relatively few with MOH have the highest individual burden [
9].
Participation and non-participation
Sampling merely selects intended subjects, who become participants only when accessed and engaged (which entails procuring their willing cooperation so that the enquiry can be completed). A high level of non-participation is potentially damaging to representativeness, although how much so depends on the factors responsible for it [
134]. Non-participation results mostly from outright refusal: a key factor in headache studies is that people with headache, having a personal interest, are more likely to participate, promoting a form of selection bias that can be highly misleading. Also in headache studies, non-participation is not a constant between important subgroups (
e.g., young males are often least willing).
As an access method, visits to households permit face-to-face interviews, which is the most direct method of engagement, allowing physical examination where this is a necessary part of the enquiry. Telephone interviews are almost as direct, without allowing physical examination. Access by mail is cheap, and by e-mail even cheaper, but both methods presume the use of self-administered questionnaires (also cheap, but requiring a high degree of literacy). The lack of engagement with an interviewer provides no encouragement to respond and no opportunity for clarifications; participation rates are generally low, incomplete returns are common and selection bias is unavoidable because certain types of people are inherently less likely to reply. Summoning prospective participants to the interviewer is time-consuming for them and invites bias with regard to who are willing and have the time.
Caseness and ascertainment
In studies whose purpose is to assess headache prevalence, or describe its characteristics, case definition – what precisely is meant by “headache” (or its types and subtypes, when these are to be considered) – is of obvious and fundamental importance. The criteria of ICHD-II [
82] are currently the common language of definition, and description, of headache disorders.
Caseness must have a timeframe. This said, inconsistent terminology in headache has arisen, largely because many headache disorders are chronic but with episodic manifestations. An “active headache disorder” is, by definition, characterized by the occurrence of symptoms within the previous year [
82]. Prevalence studies that adopt this definition of a case (
i.e., an individual who reports at least one headache episode during that time) necessarily use a timeframe of 1 year and usually report the findings as “1-year prevalence”. Strictly speaking, these are estimates of the number of
current cases (
point prevalence). A different enquiry that defines a case only when symptoms are actually present (“headache now” or “headache yesterday” [
40]) also estimates
point prevalence, but of headache attacks or headache as a symptom rather than of a headache disorder. We found most studies reported 1-year prevalence. Other timeframes, especially lifetime prevalence, are relevant to particular purposes (
e.g., genetic studies). Shorter timeframes (
e.g., 3 months) are difficult to relate, whilst studies specifying no timeframe lack useful case definition.
The two-stage procedure for case ascertainment (only participants responding affirmatively to a screening question are posed the diagnostic questions) is time saving, but with a potential penalty: a negative answer to the screening question terminates further enquiry, even though it can be false.
For epidemiological purposes, diagnostic criteria are almost always built into a structured questionnaire, although this is not how diagnoses are usually made in clinic. ICHD-II criteria were not designed for epidemiological enquiry, and are not particularly well-suited to it. For several reasons, modifications of ICHD-II
are almost invariably necessary. First, their strict application would require that all participants be personally interviewed and in many cases examined by a competent clinician. In most cases, even if possible, this would be a questionable use of resources. Second, ICHD-II criteria are expressed in technical language, and must be translated for lay participants without loss or distortion of meaning. Third, certain criteria distinguishing between migraine and TTH pose particular problems in epidemiological surveys, noted empirically in studies within the Global Campaign [
3‐
5]. It has been found difficult to gather correct responses on headache duration, requiring patients to consider
untreated attacks, which they may never have or last had long ago. There are no easy lay explanations of photo- and phonophobia, which are technical concepts, and it is even more difficult to specify what degrees of photo- and phonophobia fulfil migraine criteria in ICHD-II.
The separate reporting of prevalences of migraine and probable migraine (or TTH and probable TTH), as though these were distinct entities, is highly problematic. ICHD-II sets the general rule that, when all but one criterion for disorder X are met, the diagnosis is “probable X” (
provided that not all criteria are met for another disorder Y, in which case the correct diagnosis is Y). This has an important purpose in clinical management, providing a basis for a treatment plan pending later diagnostic confirmation. In epidemiological surveys, later confirmation is not expected: initial diagnoses of probable X have no opportunity to be amended either to X or to another diagnosis. In the specific cases of migraine and TTH, diagnosis of the former depends upon the presence of specific features (
e.g., nausea, vomiting, photophobia and phonophobia, aggravation by physical activity), while diagnosis of TTH depends essentially upon the absence of these same features. By ICHD-II rules, the presence of all but one feature of migraine is not consistent with a diagnosis of TTH, and must lead to a diagnosis of probable migraine. The same is true of probable TTH. Yet there are unavoidable uncertainties in questionnaire-based diagnoses, with the consequence that, according to ICHD rules, about half of such diagnoses are probable while, in validation studies conducted in sub-groups of the same populations, fewer than 10% of expert diagnoses are probable (unreported data from Ayzenberg
et al. [
3], Yu
et al. [
4] and Rao
et al. [
5]). It is difficult to see that the concept of “probable X” (as something distinct from “X”) serves any purpose in studies of population health.
Validation of the diagnostic questionnaire
Validation proves the diagnostic capability, in the population of interest, of the diagnostic questionnaire. It gains importance when modifications of ICHD-II criteria are adopted, which is almost always necessary (see above). It must, to achieve its purpose, be performed either in a separate sample of the population of interest, selected by identical methodology to the main sample, or in a randomly or consecutively selected sub-sample of participants in the main study. Headache patients in clinic are especially unrepresentative of any population of interest in terms of their headache disorders and, probably, a range of other relevant factors. They commonly have more knowledge of headache, and perform differently from non-clinic populations when answering questionnaires because they have rehearsed their histories.
Validation is not always possible. In countries where there are no headache experts, there is no gold standard available. In such cases only, going ahead without diagnostic validation may be justifiable, because the alternative is that research of public-health importance can never be commenced. What should then be avoided is the invention of a new and untested questionnaire, rather than adopt one that has at least been used previously and validated in multiple languages and cultures.
Burden estimation
This area of enquiry is being developed by LTB in Global Campaign studies in Russia [
3], China [
4], India [
5], Pakistan [
6], Saudi Arabia, Zambia, Ethiopia, Nepal and Guatemala (not yet published). “Relevant” implies that measured burden must be attributable to headache, and not to any other cause (including comorbidity). Burden of headache has many different elements. “Comprehensiveness” requires that
all are measured if a full account is needed. However, the purpose of the study can legitimately restrict the enquiry to specific elements (for example, financial burden [
9]).
Some elements of burden are not quantifiable, but amenable only to qualitative (descriptive) analysis. In all cases of burden estimation (more so than for prevalence estimation because of the complexities of burden), the limitations of recall are an important factor in the generation of information bias.
Elements of burden
Symptom burden in common headache disorders arises from pain, and, in migraine, additionally from nausea, vomiting and photo- and/or phonophobia. While present, these symptoms may cause debility and prostration, and reduce functional ability. The consequences include inability to work. This secondary disability burden is magnified because headache is most common in people between their teens and 50-60 years of age – the productive years. Attached to disability may be a cost burden from lost pay. To the extent that this arises from lost wages, as a consequence of absenteeism, it is a relevant and important component of the burden of headache for many people. However, housewives and unemployed people may have no income to lose. In other cases, this cost is only a small part of the personal burden because it is largely borne by employers or insurers, contributing instead to the very high societal economic burden.
Headache attacks are unpleasant; people who experience them frequently worry about when the next may occur. They may identify triggers, and attempt to eliminate them by lifestyle compromise. Leisure activities may be cancelled or curtailed because of headache; when many social events have been cancelled, they are likely not to be planned in the first place. These are all elements of interictal burden, which may be sufficient to impair quality of life. The great importance of interictal burden lies in the fact that it is continuous, rather than present only during attacks occurring perhaps every 30 days. This means both that it should not be ignored and that, if over-estimated, it will greatly distort overall burden quantification.
A consequence of recurring inability to work may be decreased probability of promotion, with failure to develop full career potential. A consequence of lost school-time may be reduced career opportunities. In both cases, the result is lower pay and impaired financial security. Over a lifetime, the cumulative burden of financial losses can be substantial.
A summary measure of overall individual burden is unlikely to be comprehensive, but the concept is attractive for its simplicity. Measures of quality of life are very non-specific, but comparison with a matched non-headache group may provide a useful indicator of burden expressed in very broad terms.
Burden on others, unaffected by headache themselves, arises in several ways. Employers and work colleagues carry part of it when paid-for work is not done: either the employer pays for nothing, or colleagues take on extra duties to make up. Family and friends lose the companionship they reasonably expect, but which is not given by a person shut away in a dark room. Children may not always be looked after. Partners and other family members may inherit increased shares of chores and responsibilities. They may acquire a carer burden, called to look after the person with headache. Carers, as well as the sufferer, can lose time from work.
Health-care resource consumption, when direct treatment costs for a condition affecting a large proportion of the population are reimbursed by a state-funded health system, is a substantial contributor to
societal economic burden. But by far the greater part of the financial cost of headache is the indirect cost of absenteeism and reduced effectiveness at work [
9,
130].