This intervention study describes and evaluates the implementation of a self-developed, practice software-based CTA tool for a research project in primary care practices. The study focuses upon the recruitment efficiency of practice staff members who are alerted on their practice computer screen as soon as they open the EPR of a possible study participant. To the best of our knowledge, this is the first study to implement and evaluate such a tool in a primary care setting in Germany. Worldwide, only few trials have reported the successful implementation of similar electronic tools [
14,
15]. While the CTA infrastructure helped to correctly identify a large population at risk with a minimal workload for the practice staff, the staff in nearly all practices missed the target to enrol 200 patients. However, the enrolment efficiency was high for those patients that were contacted by the practice staff, since the continuous reminder function allowed the practice staff to enrol patients when time and circumstances permitted. The enrolled sample was somewhat biased, but the real-time monitoring of the recruitment process helped the study centre to detect selection bias, since it was possible to track and quantify patient identification and enrolment.
Limitations
Patients whose EPRs were called up in the practice software during the recruitment period of the study were identified. Unfortunately, this does not completely coincide with the physical presence of patients in the practice during the study, since naturally EPRs are opened when information is needed and the patient may not be in the practice. Therefore, the CTA identified net sample is likely to be larger than the actual number of possible participants. As a consequence, further development of the clinical trial tool will include a documentation option to specify whether a patient is physically present or not.
The only requirements for practice staff participation in the study were willingness, an Internet connection and the use of one of the 9 most common practice software systems in Germany. We made no selection of practices according to size, patient volume, number of staff members or other criteria. We consider it advantageous, not disadvantageous, that the participating practices were heterogeneous (as GP practices in Germany are). Due to the limited number in this pilot study, the results cannot be generalised for general practice recruitment in Germany. In addition, it is not possible to draw any conclusions about the source of selection bias in the patient sample, even though we have been able to detect and quantify it using the CTA tool.
We had no control group of practices without the clinical trial tool. Therefore, we do not know how the implemented recruitment infrastructure and especially the pop-up reminder influenced recruitment behaviour and whether or not an EPR-based recruitment system is more effective than more traditional recruiting strategies in general practice. However, we know from Embi's and Rollman's prior research that EPR-based CTA systems are superior to traditional recruiting methods [
12,
15]. Moreover, the contact and enrolment rates, realized by the practice staff are rather impressive and may allow valid conclusions for the osteoporosis project.
Effects and advantages of the intervention
There are some reports in the literature of CTA systems embedded in hospital information systems, outpatient clinics or community health centres to support physicians and nurses in patient recruitment [
12,
25,
29,
30]. Our system is the first to support the practice staff in both identifying and surveying eligible patients during daily practice while recording pseudonymised data about the target population in order to monitor selection bias. In a study conducted in the Netherlands [
14], the research team used an interactive reminder tool similar to the one described here to inform the GP when an eligible patient was selected by EPR and should be asked to participate in the study. However, eligible patients needed to be selected and marked in the EPR prior to study begin. Another study [
16] with a similar technology, used filter queries to generate a list of eligible patients before study begin and provided GPs with a printed version of the list from which to pursue recruitment. In contrast to these identification procedures, the CTA tool described here operates in the background of the practice software. Therefore, a pre-selection procedure is not necessary and even newly inscribed patients matching the inclusion criteria for the study can be reliably identified.
The most important question of our study refers to the efficiency of the CTA tool used in this study. Especially 4 aspects seem to be important:
(1) The practice staff in most practices could identify an ample number of possible study participants for the specific research project (men ≥ 70, women ≥ 60). An average of 643 patients per practice met the inclusion parameters of the study. Although the registration of such a large number of patients seems to be heavy burden for the practice staff, it is noteworthy that this important step of the recruitment process, i.e. identification of the population risk, happened automatically and did not require the practice staff to exert any effort other than clicking off the reminder screen.
(2) Only one practice enrolled 200 patients into the osteoporosis survey. More importantly, many practices, especially larger practices, were poor recruiters although the CTA tool presented these practices a wealth of possible study participants to enrol. Obviously, the regular and frequent presentation of possible study participants on the screen did not stimulate the practice staff to contact them and to start the survey, but had the opposite effect. One reason may be that the clinical trial alert simply appeared too often (e.g. in practices which identified more than 1,000 patients). When the practice staff is frequently reminded about the study, the activity of disengaging or "clicking off" the reminder screen may then consume the majority of the staff's available capacity for participating in the study.
(3) When the contact level is used as the point of reference, the practice staff was able to enrol 80% of the individuals they approached about the study, which we believe to be an efficient use of local general practice staffs' limited capacities for taking part in research projects. The fact that the practice staff only personally contacted 12% of the net sample indicates that the workload in terms of the absolute number of identified patients was too high, which bears the risk of selection bias (see below).
(4) In 5,161 cases, the practice staff invested some effort in the study, i.e. opening the survey and documenting a patient's file. Of these patients, 1,526 (30%) could be enrolled into the study. In comparison, Rollman reported that only 22% of the CTA-identified patients referred from the primary care practice to an external study centre were later enrolled into the clinical study [
15]. In Embi's study [
12], the CTA intervention increased the referral rate to an external study centre tenfold in comparison to other means of identifying the population at risk. However, the enrolment rate only doubled. In our study, the practice staff completed the osteoporosis survey for nearly all patients that agreed to take part in the survey (1,526 of 1,569; 98%). The combination of a CTA-based identification tool with a practiced-based research design made it possible for the practice staff to survey a large number of patients on site without involving an external study centre.
Selection effects
One important feature of the CTA tool was the unselected presentation of all patients from the target sample as potential participants of the osteoporosis project. The CTA tool used in our study made it possible to receive real-time information about the recruitment process as well as information about age and sex of all eligible patients in the participating practices; such information is not normally available to researchers. On the basis of this information, it was possible to detect a selection bias. Older patients (80+ years) of both sexes were underrepresented in our surveys, as well as men in general. Therefore, it was possible to quantify the selection bias in terms of age and sex. Choosing and recruiting a representative, non-biased selection of participants or at least estimating the selection bias is a challenge for primary care-based research as it is essential for any generalisability of the study results [
3].
In addition to a quantifiable selection bias by demographic variables, the practice staff's limited use of CTA reminders may indicate that enrolees were selected according to other--as yet unknown and undocumented--criteria. The very low enrolment rate (<5%) in some practices may be a sign that the practice staff selected patients according to criteria outside the study protocol. For example, in the context of the present study, it might have been easier to recruit patients who are well-educated and/or communicative, patients who visit the practice during "slow" hours (i.e. disease management afternoons) or patients for whom osteoporosis is an important issue. To become aware of this dimension of selection bias is very important for further research, because clinical studies are built upon the assumption that the individuals who implement the sampling strategy are following the study protocol, i.e. that a random sample of a general practice population is truly random.
Future challenges and research
The fact that only one practice was able to survey 200 patients within the 12-month study period and that many large practices were poor recruiters will be a stimulus for us to consider the design of future projects. It would have been possible to program the CTA tool to randomly select one patient per workday and require the practice staff to survey this patient. However, past studies in general practices have shown that a rigid study protocol is very difficult for busy general practices to fulfil [
22]. Therefore, we programmed the CTA tool to over-supply the staff with possible study candidates, assuming that during the workday, one of these possibilities would pop up at a time that was convenient both for the practice staff and the patient. However, this strategy of over-supply required the practice staff to continually click off the reminder throughout the day, which (especially in large practices) consumed quite a bit of the staff's research resources, leaving little time for the "real work" of the study, i.e. surveying patients. Moreover, this strategy of over-supply may have hindered or annoyed the practice staff while performing their daily tasks. In future, it will be necessary to adapt recruitment strategies by limiting the number of patients presented on screen per day and setting reachable, individual goals such as 15% recruitment of the practice's own net sample instead of a general goal of 200 patients per practice. Such refinements can be implemented cost-effectively with only a small amount of adjustment to the CTA tool.
With regards to selection bias, it should be no problem in future applications of the CTA tool to take early measures to counteract a recognised selection bias. For example, if the study centre (or an automatic analysis algorithm) detects a selection bias towards younger patients, the CTA tool could suppress the on-screen reminder for younger patients until the selection levels reflect the net sample.
Important primary care practice conditions like finding the time, the place and the personnel to immediately conduct a 10-minute survey within the practice setting most likely had a large influence on the number of study participants. In a further step, we will systematically analyse interviews with the participating practice staff about the comfort and efficiency of the recruitment software in daily practice. Especially the experiences of larger practices with the recruitment tool need to be critically reviewed, since this tool was not able to motivate the practice staff in larger practices to meet the recruitment target. Valuable user knowledge, combined with the research team's on-going technical and logistical evaluations, should then be incorporated into the further development of this technology.