The aim of the OPEM trial was to determine whether PEMs can successfully influence FPs to improve adherence to guideline recommendations for diabetes care through treatment intensification. There were no changes (neither statistically, nor clinically) in intensification of medications (with or without including drug switches) in response to either version of the PEM, with no interaction effect. Thus, while the trial was primarily designed to test the independent effects of both the outsert and the insert, we have little reason to believe that FPs who received the combination of both PEMs would have any difference in outcome. The PEMs are mostly duplicates of each other; FPs, who already face demands on their time, are unlikely to read the insert in full after reading the brief, post-card sized summary.
Over all arms, regardless of the intervention received, we found that female physicians, those who received their medical training outside of Canada, and those who had been in practice for fewer years were more likely to engage in treatment intensification (Table
4). Nevertheless, these three effect sizes were very close to one, suggesting that, while statistically significant, the effect of the aforementioned physician characteristics on treatment intensification was minimal and is unlikely to translate into clinically meaningful effects.
Our findings are consistent with previous research [
13,
16,
31,
32], including our own previously published OPEM trials [
18,
19]. The failure of PEMs to bring about behavior change among physicians may be an indication that this KT strategy is largely ineffective and could be safely abandoned. PEM-based interventions are centered around the idea that evidence, once shared with an individual, prompts change. The negative findings from this and many other studies suggests that other barriers may exist. For example, physicians often struggle to “unlearn” old behaviors [
33,
34]. To change clinical practice, physicians must replace routine, outdated operations with new, evidence-based practices. While adding to their knowledge base can be a simpler undertaking, dismissing current practices presents a greater challenge [
33,
34]. The process of change disrupts the status quo equilibrium, causing physicians to be uncertain about practices they considered to be “certain” [
34]. Physicians included in the study had been practicing as an FP for 22 years on average (Table
1). As a result, it is possible that they may have adopted standard prescribing practices for individuals with diabetes over the years and were thus hesitant to implement a change to multiple drug classes when presented with evidence in the PEMs. Furthermore, the patients themselves may act as a barrier to PEM effectiveness. A qualitative study by Parsons et al. [
35] investigated why printed educational toolkits mailed to Ontario FPs failed to improve cardiovascular disease management among individuals with diabetes. The FPs believed that patient motivation and willingness to change behavior was the primary obstacle preventing them from adhering to the PEM recommendations [
35].
Strengths
Our trial was randomized, and, as consent was not required, also blinded with high internal validity. It was also very large, offering a precise estimate of effect, and was designed with mostly pragmatic features to align with our pragmatic (decision-making) intention, increasing its relevance to real-world decision making in Ontario and similar settings (see Additional file
1 Table
1 and Fig.
1 for PRECIS-2 table and wheel, respectively) [
36]. The PEMs were delivered in a way that is likely similar to that of a routine large scale programme: by mail to physicians alongside a journal many of them routinely received, targeting the providers who usually care for patients with diabetes in Ontario. Outcomes were measured using routinely collected electronic health information, avoiding any distortions to usual care that might arise from data collection from physicians or patients. Lastly, while prescribing recommendations were outlined in the PEMs, the choice to prescribe was ultimately left to the physician – a naturalistic approach with no restrictions on behaviour, identical to a PEM program as might be implemented in routine practice.
Limitations
Our study design cannot distinguish the point at which PEMs failed. Failure may be due to non-delivery, or the messages not being opened or read by the physicians. A 1997 survey revealed that Ontario physicians considered
informed to be a useful source of clinical information (71% of physicians reporting received
informed, and of these, 89% reported that the information was useful) [
37]. But receptiveness may not be the same as clinical influence, and 8 years later, at the time of our trial, its influence may have declined. Further, while Canadian postal delivery is reliable, physicians may not be opening their own mail, and even if they do, may not be reading
informed.
Our study is relevant only to print based educational materials delivered by mail. It is possible that physicians would have been more influenced by the PEM had it been delivered by an alternative route (e.g., fax, email). Following the shift towards electronic information as a basis for clinical decision making, electronically-delivered recommendations may have been more widely received by physicians [
38,
39]. However, these are seldom reported in the literature. A 2020 systematic review of PEMs identified only two studies comparing printed versus electronically delivered guidelines and found little to no difference in effectiveness between the two delivery strategies [
17]. That said, the quality of evidence from these studies was graded as low [
17]; thus, additional studies investigating electronically delivered guidelines may be warranted.
While the trial was designed with a pragmatic intention, it focused on higher volume clinicians treating older persons with diabetes. Only 5179 of the 10,863 eligible FPs practicing in Ontario were included, and although the excluded physicians were mostly inactive or saw few patients over 65 years, we cannot claim complete representativeness. The results are applicable to Ontario FPs who billed at least $50,000 in 2004, and who prescribed medications to at least 100 seniors (≥66 years old) in at least 10 months in 2004. It is possible that physicians who treat fewer individuals 66 and above and have less experience with diabetes care might have a different outcome.
In addition, the ODB claims database only contains prescriptions that have been reimbursed, which depends on whether they have been dispensed. The intervention effect might have been diluted by patients who received an intensified prescription but failed to get it dispensed. These individuals are likely to be balanced among the four groups as a consequence of randomization; thus, our estimate of the relative effect is unlikely to biased.
A discussion of the limitations would be incomplete without acknowledging the timeframe in which the study was conducted. The PEMs were mailed in January 2005, and the outcome measurement took place between 2005 and 2006. Estimates at the time of the trial suggested that prescriptions for all of ACE inhibitors, “other” antihypertensives, and cholesterol-lowering agents were at least 30% below guideline recommendations [
4]. One may wonder whether the results from the present study are relevant to clinical practice today. In 2015, it was found that the percent of patients aged 65 and above with diabetes in Ontario who filled a prescription for an ACE inhibitor or angiotensin II receptor blocker and a statin was approximately 60 and 72%, respectively [
40]. Further, the 2018 Diabetes Clinical Practice Guidelines still declare that diabetes-related cardiovascular complications are responsible for the greatest burden of death among individuals with diabetes and recommend treatment with appropriate medications to manage these risks, including antihypertensives and cholesterol-lowering agents [
11]. Thus, the evidence-to-practice gap continues to persist. It is, however, worth noting that, while the recommended target drug classes remain the same, some of the medications that were included as part of treatment intensification in the trial, for example atherosclerosis inhibiting agents, are not recommended today. Therefore, one must interpret the results within the timeframe of the study.