Development and evaluation of a blended educational programme for general practitioners’ trainers to stimulate proactive HIV testing

From November 2013 to April 2014, 265 GP trainers at the Academic Medical Center, University of Amsterdam (AMC-UvA), were invited to participate in the programme. To become a GP, medical graduates must complete 3 years of specialty training linked with an experienced GP, the GP trainer. GP trainers complete a 4–5 year training programme, with 8 days of refresher training required every year. GP trainers are trained to perform a learning conversation with their resident during their 4-year training program as a trainer. Evidence-based practice guidelines for GP trainers are often referenced during this training [9]. These guidelines are developed by the Dutch College of GP trainers and are available on the internet to facilitate their use in daily practice [19].

The content of the programme was based on the Dutch STI guidelines and the accompanying e-learning package for GPs, which was released in September 2013 [9].

The STI guidelines promote more proactive HIV testing among men who have sex with men (MSM) and patients who originate from HIV endemic countries. A new provider-initiated HIV testing strategy is also briefly mentioned in the guidelines. This proactive strategy recommends offering an HIV test to individuals with HIV-related conditions, as supported by European guidelines [9, 20]. Another new provider-initiated HIV testing strategy is described in UK guidelines and is also mentioned in the additional file of the Dutch STI guidelines [10, 21‐23]. This strategy recommends a ‘routine offer of HIV testing’ to all 15–59 year olds in primary care settings where HIV prevalence exceeds 2 in 1000. This strategy is operationalized either by offering an HIV test (a) when registering new patients or (b) to anyone who undergoes a blood test, regardless of the reason.

These guidelines and their recommendations were used to formulate the goals and set-up of our blended learning programme. At the end of the programme, GP trainers were expected to have developed competences (knowledge, attitude, skills) that enabled them to diagnose HIV and other STIs (see Table 1).

Based on our learning goals for the programme and the experience of GP trainers, we also used evidence-based guidelines and multiple teaching strategies [9, 12, 13, 24, 25]. According to adult learning theory, active learner participation, multiple learning experiences and reflection on real-life experiences should be incorporated in educational strategies [25]. These were applied in our study design through e-learning, multiple meetings that focused on transferring knowledge, a ‘learning conversation’ with the GP’s own trainee, and focus groups to discuss barriers that were experienced in daily practice (see Table 2). The e-learning element was based on the STI guidelines for GPs and required about two hours. The online programme used various didactic methods, for example, interactive videos to change knowledge, attitude and behaviour concerning the STI consultation [9].

As independent learners, GP trainers were asked to read the STI guidelines and complete the e-learning package [9].

The first meeting addressed the programme’s learning goals. Clinical knowledge and communication skills were also addressed in interactive workshops. GP trainers were randomly divided into small groups (12 to 25) designed to stimulate active learner participation and expose GP trainers to new and diverse real-life experiences. A member of the Department of General Practice at the AMC and an expert from the STI guidelines committee facilitated the groups.

As home-assignment after the first meeting, GP trainers were asked to formulate quality improvement targets which they executed the next six months in daily practice and to fill in the e-learning if they had not done this before. Three months after the first meeting a reminder of their home assignments was sent by e-mail. Also, GP trainers were asked to initiate a conversation with their own trainee, which would stimulate a further transference of knowledge, problem-solving skills and motivation for further learning [24, 25].

During the second meeting, GP trainers reflected on the first meeting and their personal real-life experiences in daily practice in focus groups and discussed the barriers and facilitators they encountered in applying the new guidelines and additional provider-initiated HIV testing strategies (published elsewhere [26]). This enhanced the opportunity to discuss and translate real-life issues into their own practice. These sessions were facilitated by staff and researchers from the Department of General Practice at the AMC who acted as moderators and observers. To ensure consistency in the way the programme was taught, all experienced and trained teachers (members of the Department of General Practice at the AMC and had no role in the design of the STI guideline) of both meetings were provided with a comprehensive handbook, including training materials.

We evaluated the effect of the programme on participants’ frequency of laboratory-documented HIV testing over time. In the Netherlands, GPs collaborate with one or more laboratories, to which their patients are referred for HIV testing. We contacted every laboratory used by the participant GPs and requested information. After the initial contact, a second email was sent, including informed consent from the participating GPs, and we asked the laboratories to send us data on the total number of HIV tests for each GP over six-month periods between 2010 and 2015 (intervention group). We compared these figures with the average number of HIV tests among anonymized GPs performed in the laboratory every six months in the same period to determine whether the programme had an effect on the GP trainers’ use of laboratory-documented HIV testing (control group).

To evaluate the effects of the programme on the GP trainers’ self-reported use of HIV testing and quality improvement targets, the participants were requested to complete a questionnaire focusing on the learning goals of the programme before and three months after the programme (Additional files 1, 2, 3 and 4). GP trainers were also asked to fill in a report form to evaluate both training meetings. To ensure content validity, the questionnaires were discussed with a senior epidemiologist who is specialised in medical education and GP training (NvD), a coordinator from the GP trainers educational program at the Department of General Practice at the Academic Medical Center of the University of Amsterdam (AvdM), and a GP specialised in STI/HIV in primary care (JvB). No further validation of the questionnaires has been performed. The questionnaire contained questions on:

We assessed the six-monthly figures on HIV tests over time using a zero-inflated negative binomial regression model accounting for the large number of zeroes, over-dispersion and the clustering of tests within GPs using robust standard errors (zinb command [version 1.7.12] in STATA version 13.1 software). The effect of the educational programme was assessed using interaction terms between time (before/after programme) and programme (yes/no).

In total, 265 GP trainers were invited to join the programme and 150 attended the first meeting, with 107 giving informed consent for participation in the scientific evaluation of the programme. Seventeen GP trainers were excluded because they only participated in the first meeting. The demographic characteristics of the remaining 90 GP trainers are described in Table 3. Of these, 74/90 (82.2%) completed both questionnaires and 50% (74/150) off all GPs who attended the first meeting participated in the study. Before the start of the programme, 9.5% (7/74) of the GP trainers had completed the e-learning package, compared to 67.6% (50/74) three months after the second meeting. In addition, 79.7% (59/74) reported having had a learning conversation with their trainee after the second meeting.

Of the 90 GP trainers, 78 listed one or more laboratories (1–3) where they sent patients for HIV testing. In total, they mentioned 21 laboratories. These laboratories were approached multiple times. While we received information from seven laboratories, only four sent complete information. GPs in the intervention or control groups were only included if we had complete information from the laboratories. As a result, 11 GP trainers participating in the educational programme were included for further analysis. Some laboratories refused to participate because they had not been involved at the start of the study, or did not responded because the research was of low priority for them, or because they had difficulties retrieving this information from their database.

We found that, between 2010 and 2013, the mean annual number of laboratory-documented HIV tests decreased by 9.1% for the 624 control GPs, and by 13.0% for the 11 intervention GPs. After the programme, the annual decreases were 2.3% and 1.8% for the control and intervention groups, respectively. Before the programme, GPs in the intervention group had 50% more laboratory-documented HIV tests than the controls (IRR 1.50; 95% CI 0.79–2.85; p = 0.211). After the programme, the intervention group had twice as many laboratory-documented HIV tests as the controls (IRR 2.05; 95% CI 1.35–3.12; p = 0.001) (p_{interaction-pre- programme} = 0.198; p_{interaction-post- programme} = 0.041), (see Fig. 1).

Self-reported HIV testing behaviour showed a significant decrease after the training programme (p = 0.004). Before the programme, the HIV test was significantly more often initiated by the patient than after the programme (p = 0.001) (see Table 4). There was an increase in the frequency HIV tests initiated on the basis of symptoms or complaints related to an HIV-related infection (p = 0.017).

Of 74 GP trainers, 69 defined one or more (1–3) quality improvement targets. In addition, 179 change in behaviour goals were defined and the overall median average score on achieving their goals was 4 (1 = failure to 5 = success). Most of these goals were related to focusing more on sexual history and behaviour, STI tests and the location of testing and HIV-related conditions (see Table 5).

The proportion of GP trainers that answered the knowledge-related questions correctly was significantly higher after the programme for every question (see Table 6). Also, we compared the mean attitude scores for the related questions, but minimal to no significant differences were observed.

The total satisfaction value of both meetings within the programme was rated high, at an overall median average score of 4 (scale 1 to 5 “where 5 means strongly agree”) on both days. The relevance of each day in relation to the content of the programme and the use of mixed teaching strategies had a similar score. GP trainers were satisfied and enthusiastic about their teachers’ performance on both days, with an average of 4 to 5 (scale 1 to 5 “where 5 means strongly agree”). GP trainers mentioned that the day was interactive, informative and focused on clinical practice (as reported in the open questions section).

There is no gold standard for successful implementation of new recommendations in evidence-based guidelines [11]. It is recommended that educational strategies should be interactive and include multiple teaching strategies, for example, group discussions about evidence and setting quality improvement targets [12]. We integrated multiple teaching strategies but found only a moderate effect on GP trainers’ laboratory-documented HIV testing behaviour. This concurs with other evidence on the difficulties of changing physicians’ behaviour [11‐13]. GP trainers in our programme mentioned various barriers to and facilitators of new proactive HIV testing strategies recommended by guidelines [26]. Integrating knowledge of these barriers into the design of future interventions may help in the implementation of new proactive HIV testing strategies in daily practice [12]. Understanding the barriers hindering the adoption of new recommendations of guidelines is important and strategies should be tailored to addressing them.

Part of our study design involved active learner participation, for example, GP trainers had to formulate their own quality improvement targets [27, 28]. These goals were designed to encourage the transfer of new knowledge to daily practice by using quality improvement targets for behaviour change. We showed that a high overall median score for achieving rate of implementing GP trainers formulated intended changes.

The theory of reasoned action states that attitude and subjective norms are related (predictors) to behaviour [29]. We found no change in the majority of the attitude items. One explanation could be that the attitude items in our study were not representative of the observed behavioural change. Interestingly, the attitude item, ‘I would rather refer patients with STI symptoms to an STI clinic’, significantly decreased after the programme, which may indicate that GP trainers were more familiar with this topic.

A Dutch study showed that a multifaceted training programme using guidelines, feedback and social interaction resulted in modest improvements in test ordering by GPs [30]. Feedback on the performance of laboratory-test ordering was considered as a follow-up to our study design but was unsuccessful due to difficulties in retrieving GP trainers’ laboratory-documented HIV testing results. New efforts are being made in the Netherlands to collect this useful information. In 2014, the ‘HIV Transmission Elimination AMsterdam’ programme (H-TEAM) was set up [31]. The H-TEAM develops and implements a mix of interventions to prevent the spread of the virus and to extend HIV testing and direct linkage to care. For example, an educational programme for GPs in Amsterdam has been developed that integrates feedback on HIV test performance to stimulate proactive HIV testing.

We reported that before the programme, the GP trainers intervention group had 50% more laboratory-documented HIV tests than the controls. This might be explained by the fact that GP trainers who participated in the study had more interest in HIV prevention and were thus more likely to test patients for HIV compared to the control group.

The educational programme did have an impact on the testing behaviours of the participating GPs, but the overall downward trend in HIV tests among GPs was surprising. Even before the 2013 guidelines were established, the professional media was drawing attention to a need to increase proactive HIV testing [3, 4]. There are several reasons that might explain this counterintuitive downward trend. People from risk groups could have changed their healthcare seeking behaviour and decided to be tested for HIV at an STI clinic rather than at the GP. Another explanation is that due to the compulsory excess payments in primary care, heterosexual people are more reluctant to undertake HIV testing than before [32, 33]. Also, we hypothesize that GPs target the right group who should be tested. More research is warranted to find out the reason for this downward HIV testing trend.

One strength of this study is that we initiated a blended education programme that incorporated evidence-based practice guidelines and multiple evidence-based teaching strategies to stimulate GP trainers’ HIV testing behaviour.

Also, we tried to incorporate GPs’ laboratory-documented HIV testing to objectively measure any change in behaviour. Laboratory-documented HIV testing provides more detailed information about GPs’ actual testing behaviour compared to information gained from questionnaires, as self-reported levels of HIV testing may be biased by socially desirable answers or recall bias.

We also collected information on laboratory-documented HIV testing trends for a prolonged period and not only before and after the programme. This enabled us to reflect in greater depth on the outcomes, rather than only relying on an analysis of the questionnaires. Self-reported HIV testing in the questionnaires suggested a decrease in HIV testing. However, the programme appeared to have stabilized – at a higher level of request for laboratory-documented HIV tests – the initially stronger downward trend in the 11 GPs who participated in the programme.

One limitation of the study is that we could not collect data on laboratory-documented HIV testing for the majority of GPs in both the intervention and control groups due to lack of cooperation from the laboratories. Therefore, answering the main research question if the educational programme was effective was severely hampered. The limited results should be interpreted with caution as our findings may not be representative of all participants. It was not possible to distinguish a decline in the number of HIV tests per patient from a decline in the number of patients treated by the GP. Also, we had no insight in the stability of the population of GPs who use the four laboratories. A decrease in the number of GPs would reduce the number of patients and subsequently effect the number of HIV tests. Also, we acknowledge that GPs could have gradually switching their patients’ HIV tests to the 17 other laboratories. More detailed information needs to be collected from the laboratories, as described above, for evaluation of educational programmes. We vigorously attempted to retrieve this laboratory information but even after multiple reminders by phone and email not all laboratories agreed or responded to our requests. This is a serious drawback faced when implementing a study-design in daily reality. We recommend that laboratories need to be more involved in the design of the study. This could help bypassing some of the barriers regarding the collection of the data. Another way to evaluate these programmes is to use general practice-documented HIV testing data. As a different level of use of laboratory-documented HIV testing by GPs after the programme only offers partial evidence of GPs following the new recommendations. We would also like to know which patients are tested (for example, MSM versus non-MSM) and why (for example, provider initiated testing versus patient-initiated testing). Future research needs to collect this additional information to determine the actual impact of an educational programme on GPs’ HIV testing behaviour.