Consistency and sources of divergence in recommendations on screening with questionnaires for presently experienced health problems or symptoms: a comparison of recommendations from the Canadian Task Force on Preventive Health Care, UK National Screening Committee, and US Preventive Services Task Force

Health screening involves the use of tests to identify apparently healthy people with early stage disease who do not have, or have not recognized that they have, symptoms or signs of the condition being screened. Screening is premised on the idea that early identification of asymptomatic pre-clinical disease can increase the likelihood of effective intervention and, thus, improve future health [1, 2]. Since the 1960s, when screening for breast cancer with mammography was first tested, enthusiasm for the idea that some diseases can be prevented through early detection has resulted in an explosion in the number of screening tests that have been promoted, some with evidence of benefit and others without such evidence [3].

This enthusiasm has also resulted in an expansion of the scope of screening itself. In addition to the goal of reducing risk of future ill health by detecting pre-clinical indicators of disease, the idea of screening has increasingly been applied to the use of self-report questionnaires to “screen” for existing health problems (e.g., alcohol misuse) or symptom-based syndromes (e.g., depression) that are not hidden; rather, they are experienced by patients, but not reported as health problems or observed by healthcare providers. The first example of a major national preventive care recommendation for this type of screening was the 2002 United States Preventive Services Task Force (USPSTF) recommendation for depression screening among adults in primary care [4]. Questionnaire-based screening has since been evaluated for other presently experienced health problems and symptom-based syndromes, including alcohol misuse, illicit substance use, intimate partner violence, and developmental delays in young children [5‐7].

However, screening with questionnaires for existing conditions is controversial [8, 9], and major guideline organizations have reached different conclusions about the potential benefits versus harms of some of these programs [5‐7]. Indeed, there are a number of reasons why applying a conventional test-based screening paradigm to presently experienced problems and symptoms may not improve health outcomes compared to providing patients with accurate healthcare information and appropriate assessment and intervention when problems are recognized. One such reason is that some of the conditions being screened may not necessarily be progressive. For some patients, symptoms and problems identified via self-report questionnaires reflect transitory reactions to circumstances that will resolve without intervention [8, 9]. Another is that using tests to identify and label medical conditions that patients do not otherwise recognize or report as health problems risks identifying large numbers of patients with mild conditions whose symptoms or problems may not be amenable to healthcare interventions. Finally, interventions to reduce symptoms or solve health problems are most effective when there is agreement between patients and providers on the impact of the problem and the need to address it. Such an agreement may not be present when tests are used to inform patients that they are experiencing a healthcare problem which they did not recognize as such [10].

Recommendations for screening should ideally be based on direct evidence from high-quality randomized controlled trials (RCTs) that show a sufficiently large benefit to justify the costs and harms involved in screening [1, 2, 10‐12]. RCTs designed to directly test the effectiveness of a screening program should, at a minimum, (1) randomize patients prior to the screening intervention and (2) provide similar treatment resources to patients detected with the condition or health problem in the screening and non-screening arms of the trial so as not to confound the effects of a screening program with the effects of providing different treatments. Ideally, RCTs of screening programs would also exclude patients who are already known to have the targeted condition at the time of screening, as these patients would not be screened in actual practice [11].

The objective of the present study was to examine recommendations from three major national guideline organizations, the Canadian Task Force on Preventive Health Care (CTFPHC), the United Kingdom National Screening Committee (UKNSC), and the USPSTF, to (1) document the consistency of recommendations on using questionnaires to screen for presently experienced health problems or symptom-based syndromes, (2) identify sources of divergent recommendations, and (3) determine if guideline organizations have identified any examples of direct evidence from RCTs that questionnaire-based screening programs improve health outcomes for screened patients compared to non-screened patients.

Screening for presently experienced health problems and symptom-based syndromes with self-report questionnaires has been evaluated by the CTFPHC, UKNSC, or USPSTF in the areas of alcohol misuse, depression, developmental or speech and language delays, domestic violence, and suicide risk. The CTFPHC and UKNSC have made a total of 11 recommendations against screening with self-report questionnaires and no recommendations in favor of the practice. The USPSTF, on the other hand, has made four recommendations in favor of questionnaire-based screening programs (alcohol misuse, adult depression, adolescent depression, intimate partner violence) and no recommendations against screening. In seven other cases, the USPSTF determined that there was insufficient evidence to recommend for or against the service (“I” recommendation).

The CTFPHC, UKNSC, and USPSTF all attempt to evaluate the balance between possible benefits and possible harms that would be accrued from screening programs. The methods the groups use are generally similar, although there are some differences. Both the CTFPHC and USPSTF include methods for evaluating screening pathways based on indirect evidence, such as evidence on screening test accuracy and treatment effectiveness [37, 38]. They differ, however, in that the CTFPHC uses the GRADE system [39] and makes weak or strong recommendations for or against all preventive care services it evaluates; the USPSTF, on the other hand, uses its own rating system and may make an “I” recommendation, which reflects that its members do not believe that there is sufficient evidence to make any recommendation. The UKNSC differs from both the CTFPHC and USPSTF in that it uses a list of criteria, including the availability of evidence from high-quality RCTs, to evaluate screening programs [10]. In addition, the CTFPHC and UKNSC, but not the USPSTF, consider resource use in their recommendations [10, 37, 38].

Divergences in recommendations between the USPSTF and the CTFPHC and UKNSC appear to stem from several sources. First, when recommendations diverge, the USPSTF has indicated in each case that there is at least moderate certainty that there would be at least moderate net benefit based on indirect evidence from studies of test accuracy and treatment of screen-detected symptomatic patients and, if available, potential harms of screening and treatment. The CTFPHC and UKNSC, on the other hand, have determined that those links are insufficient to establish that benefit would occur. Additionally, in the case of depression screening, the CTFPHC noted that the USPSTF relied upon RCTs of depression care management programs, which used screening tools to establish trial eligibility prior to randomization, as evidence on screening. Consistent with this, of the 13 RCTs described by the USPSTF as screening trials, only two randomized patients prior to screening and provided similar care options in patients with depression in the screen and no-screen trial arms (Table 3). Second, in divergent recommendations, the CTFPHC and UKNSC raised concerns about possible harms from screening, including overdiagnosis and overtreatment, whereas the USPSTF rated described harms as small to negligible in all recommendations in favor of screening and did not mention the possibility of overdiagnosis or overtreatment in any. Finally, cost and resource considerations were included in CTFPHC and UKNSC recommendations, but not in USPSTF recommendations.

No examples of direct RCT evidence that questionnaire-based screening improves health outcomes were described in the recommendations of the CTFPHC, UKNSC, or USPSTF. There were only six RCTs that directly tested screening interventions by randomizing patients prior to administering the screening questionnaire and providing similar management resources for patients identified as needing care in the screening and non-screening arms of the trials. In five of the trials, which evaluated whether screening for depression, developmental or speech and language delays, intimate partner violence, and suicide risk improved health compared to usual care, there were no statistically significant primary or secondary health outcomes in favor of the screening intervention.

In the sixth RCT, which tested depression screening among postpartum women in Hong Kong [14], based on outcome definitions registered prior to conducting the trial, there was one primary outcome that was statistically significant in favor of screening and one that was not. However, in the published outcome report, only the statistically significant outcome was described as a primary outcome; the non-statistically significant outcome was described as secondary [14]. As described previously [40, 41], there is concern that results from this trial may not represent what would likely occur in practice. In addition to reclassifying trial outcomes post hoc in a way that portrayed trial results as positive, rather than equivocal, the reported effect size was implausibly large. The authors randomized 231 women to be screened, of whom 55 received the low-intensity counseling treatment that was provided; 11 of 231 women in the control arm also received the treatment. The authors reported a standardized mean difference (SMD) effect size per woman screened on the Edinburgh Postnatal Depression Scale of 0.34, roughly equivalent to SMD = 1.81 for the 44 additional patients treated in the screened group compared to the control group. This reported effect per woman treated, however, is six to seven times the size of effects that are typically achieved with similar interventions in primary care settings [40, 41]. A meta-analysis of collaborative depression care treatment, for instance, reported an effect size of 0.25 SMD (N = 30 trials) [42]. Another meta-analysis of psychological treatment for adult depression in primary care reported an overall SMD effect size of 0.31 (N = 15 trials) [43]. None of the individual RCTs included in either meta-analysis approached the effect size reported per patient treated in the Hong Kong screening trial. Consistent with concerns that results from the Hong Kong trial may not be reproduced in actual practice, the only other trial of depression screening included in the present review did not find that depression screening significantly reduced the number of depression diagnoses among patients screened compared to patients not screened [13].

The USPSTF was recently criticized for relying upon indirect evidence and for not adequately considering potential harms in recommending depression screening [44]. Experts pointed out that there are numerous examples where the use of insufficient and indirect evidence has led to ineffective and harmful screening programs and argued that guideline makers should refrain from recommending new screening services based on only indirect evidence [44]. In the context of questionnaire-based screening programs, this concern is heightened because, when RCTs have directly tested these programs, they have not found evidence of health benefits. When high-quality trials are feasibly conducted, as is the case with questionnaire-based screening programs, a more conservative approach than recommending a new service without direct evidence would be to call for well-conducted RCTs.

Appropriate care that addresses patient needs, but avoids intervention without demonstrated benefit, is increasingly emphasized in healthcare planning and service delivery [45, 46]. Recognition that screening is not benign is reflected in recent recommendations for more restricted use of screening for breast [47, 48] and prostate cancer [49, 50]. Using self-report questionnaires as screening tests to identify unreported and unrecognized, but presently experienced, health problems and symptoms extends the boundaries of the standard screening paradigm, in which tests are used to detect hidden signs or unrecognized symptoms in order to stave off future health problems. It is possible that questionnaire-based screening might improve upon good, conscientious medical care that provides patients with information and encourages them to inquire about problems they are experiencing. Direct evidence from existing studies included in CTFPHC, UKNSC, and USPSTF recommendations, however, does not lead to this conclusion.

Without evidence that using questionnaires to search for presently experienced, unreported problems would lead to better health outcomes, the negative implications of this practice need to be carefully considered in screening recommendations, including the possibility that it would lead to overdiagnosis and overtreatment [51‐54]. Traditionally, overdiagnosis has been understood to occur when a person without symptoms is diagnosed with a condition or disease that will not lead to symptoms or early mortality and would not ever be identified without screening [51, 52]. More broadly, in the case of presently experienced problems or symptoms, overdiagnosis can occur when patients are identified with a disorder or problem that they do not experience as significantly impairing and that would not be expected to be substantively affected by medical intervention [53, 54]. This could occur in mental disorders, even when diagnostic criteria are met, such as in the presence of mild depressive symptoms that fall close to the normal range on a diagnostic spectrum [54].

Potential harms have not been well documented in questionnaire-based screening, but if screening is done, some patients who would not otherwise be exposed will experience harms. For example, individuals may be exposed to unnecessary and ineffective treatments, undesirable medication effects, the labeling of problems that may resolve on their own as medical problems, and nocebo effects from telling patients who are not otherwise specifically concerned that they have a medical problem, such as depression [10, 55].

In addition to direct harms to patients, the practice would consume scarce healthcare resources that might be better devoted to providing services to patients who clearly have health problems, including mental health problems, but who in many cases receive less than adequate care [10, 56]. Some have argued that screening with questionnaires can be done at very little cost [57], and having patients respond to questionnaires is not typically expensive. However, screening involves much more than this, including follow-up assessments to separate true from false positives, consultations to determine the best management options, and treatment and follow-up services. One study found that, when depression screening is conducted, more than 70% of visits last more than 15 minutes and 17% last more than 30 minutes compared to 42% and 6%, respectively, when screening is not done, and this only factors in the time involved in the initial screening visit, but not follow-ups and referral management, for instance [58]. The number of patients who would follow this pathway depends on the clinical setting and condition targeted. In depression, 30% or more of patients in many settings would have positive screens and would need to be evaluated, even though most of these patients would not have depression [59, 60].

By 1996, based on a conservative estimate, a typical primary care physician needed to spend 7.4 hours per day just to minimally comply with Grade A and B recommendations (moderate to high certainty of moderate or high benefit, should be offered) for preventive care from the USPSTF [61]. Since then, the number of A and B recommendations has grown, including the recommendations for questionnaire-based screening described in the present study. Physicians cannot realistically comply with all USPSTF A and B recommendations, but guidance on how to prioritize is not provided. As a result, they may determine which recommendations to offer based on their own estimation of likely benefit and harm, as well as resources required. In depression screening, a national survey found that only 4% of American primary care patients were screened for depression in 2012–2013, even though it was recommended by the USPSTF and covered by the Affordable Care Act as of 2010 [62].

There are limitations to consider in evaluating the results of the present study. First, we included only recommendations from three guideline organizations, the CTFPHC, UKNSC, and USPSTF. Although these organizations are recognized for their leadership in the area of preventive healthcare policy, these results do not necessarily apply to other organizations that make recommendations on screening. Second, we only reviewed trials included in recommendation statements and did not seek to identify other trials that may have been conducted. It is possible that there are trials of questionnaire-based screening that we did not review from other areas of screening where no recommendations have been made or from trials conducted since these recommendations were made. However, identification of any existing trials was not the objective of the present study. Rather, we sought to determine if the CTFPHC, UKNSC, or USPSTF had identified direct evidence from any questionnaire-based screening program that would support the use of indirect evidence in recommendations.

In summary, neither the CTFPHC nor the UKNSC has made any recommendations endorsing questionnaire-based screening. The USPSTF, on the other hand, has recommended questionnaire-based screening for alcohol misuse, depression in adolescents and adults, and intimate partner violence. Compared to the CTFPHC and UKNSC, the USPSTF appears to be more confident in relying upon indirect evidence, minimizes potential harms, and does not consider cost and resource utilization.