Development and validation of a multivariable prediction model for infection-related complications in patients with common infections in UK primary care and the extent of risk-based prescribing of antibiotics

Antimicrobial resistance (AMR) is one of the biggest global threats facing modern healthcare and medicine [1, 2]. A recent World Health Organization report highlighted the urgency of this problem, identifying that drug-resistant infections cause at least 700,000 deaths globally a year [3]. This number could rise to 10 million per year by 2050 if no action is taken [4‐6]. One driving factor behind the emergence and persistence of AMR is the overuse and misuse of antibiotics [7]. It is not purely a contemporary issue, as government committees in the UK discussed strategies to optimise antibiotic usage more than 20 years ago [8, 9]. Despite this, the way physicians make the decision on whether to prescribe has changed little in that time and is still largely reliant on their immediate assessment of a patient’s symptoms.

Primary care was responsible for prescribing over 80% of all antibiotics in the National Health Service (NHS) in 2017 [10]. Earlier research has examined antibiotic prescribing patterns in primary care in the UK and found that it is heterogeneous regionally and nationally [11‐13]. A recent study highlighted that substantial variability exists both within and between general practices and that there are multiple drivers behind the decision to prescribe [14]. Together, this suggests that a more evidence-based approach to decision-making for antibiotic prescribing is required to achieve better patient care. Prescribing based on an objective evaluation of a patient’s risk is a relatively new concept but is gaining popularity. For example, prescribing of statins is now guided by the QRISK algorithm [15], used to estimate a patient’s risk of developing cardiovascular disease in the following 10-year period. Applying a similar approach to antibiotic prescribing could facilitate a more targeted use of a medication that is becoming increasingly ineffective. However, to date, there are no validated risk models for this purpose.

The aim of this study was twofold: first, to develop and validate prognostic models that predict the risk of developing infection-related complications in patients who consult their general practitioner (GP) for a common infection; second, to use these models to investigate whether there is an association between the risk of an infection-related complication and the rate of receiving an antibiotic prescription. Three common infections were investigated: lower respiratory tract infections (LRTI), urinary tract infections (UTI) and upper respiratory tract infections (URTI).

This was a retrospective cohort study using data from two sources: the Clinical Practice Research Datalink (CPRD GOLD) and the Secure Anonymised Information Linkage (SAIL) databases, which made up the derivation and validation cohorts, respectively. CPRD GOLD contains longitudinal, anonymised, patient-level electronic health records (EHRs) from general practices in the UK with more than 5 million active patient records representing about 8% of the UK population [16]. SAIL contains data from approximately 80% of general practices in Wales and covers around 75% of the 3 million population [17‐19].

The EHRs included clinical diagnoses, prescribed medication, vaccination history, diagnostic testing, lifestyle information and clinical referrals, as well as patient’s age, gender, ethnicity, smoking history and body mass index (BMI). Patient-level socioeconomic information was available through linkage of the postcode of a patient’s residence to the Index of Multiple Deprivation (IMD) [20]. Patient-level IMD was aggregated based on quintiles. Antibiotic prescriptions were determined using the British National Formulary.

The derivation dataset from CPRD comprised routinely collected data from 587 general practices in England from 1 January 2000 to 31 December 2015. Patient-level data from the general practices were linked to hospitalisation data (HES for CPRD GOLD, PEDW for SAIL) containing information on the date of hospital admission and the clinical diagnoses established at and during admission (coded using ICD10 codes). Linked data were available for about half of CPRD practices which are all located in England and for all the SAIL practices. Patients were followed for 30 days after their initial GP consultation to determine if they suffered further complications as a result of their infection. The validation cohort (SAIL) covered 338 general practices in Wales from 1 January 2000 to 15 March 2017.

A total of 10.8 million incidental consultations for URTI, LRTI and UTI were analysed in this study: 8,110,530 from CPRD and 2,727,646 from SAIL. There were 6,311,321 antibiotic users (58.23%) and 4,526,855 non-antibiotic users (41.77%), with 33,067 events recorded in CPRD and SAIL combined. Comparisons between the two datasets indicate that the validation cohort had a much younger demographic (mean age CPRD = 38.98; SAIL = 29.80), although most other covariates had broadly similar values (Table 2). The disparity in mean age can be accounted for by the SAIL dataset having many more patients in the under 6 age group (e.g. in CPRD, the prevalence of URTIs was 18.6%, whereas for SAIL, it was 32.0%), which serves to reduce the average age of the population. The high level of white and unknown ethnicity is also striking. This is primarily due to the high level of unrecorded data for ethnicity, and the values are in line with other similar studies [33].

The incidence rate of events was low among non-antibiotic users, with the mean rate being 1.71 cases per 1000 person-months in the derivation cohort and 7.49 in the validation cohort. For both datasets, the incidence rate increases with age (for adults) and increasing Charlson Comorbidity Index (Table 3). Most of the hospital admissions for infection-related complications were for LRTI (CPRD, 10.28 cases per 1000 person-months; SAIL, 23.73), followed by UTI (2.09; 7.00) then URTI (1.23; 6.42).

This study developed three clinical prediction models to predict the risk of infection-related hospitalisation following a GP consultation for URTI, LRTI and UTI using the population-based CPRD and SAIL datasets. The models were successfully adjusted and updated to generalise over datasets covering England and Wales. The models were then applied to datasets containing both antibiotic and non-antibiotic users, and we observed that the decision to prescribe an antibiotic was independent of the risk of hospitalisation due to infection-related complications. Furthermore, it was found that the risk levels of patients vary significantly both across the patient cohort and by indication, which indicates that risk scores provide enough discrimination between patients to offer a viable alternative to traditional approaches to prescribing largely based on symptoms alone. Together, these two observations suggest a potential way to achieve further optimisation of antibiotic usage in primary care.

In previous work, there have been very few attempts to apply clinical risk prediction modelling to the management of infectious conditions. The prediction models that have been developed in this area have focused on specific resistance strains [34, 35] or specific age groups [36] and have had much smaller patient cohorts compared to the size of the populations considered in this study. The major strength of this work is the utilisation of two large independent population-based EHR datasets (both in terms of volume and timespan) for model development and validation. Moreover, individual risk models were developed for each infectious condition, rather than combining multiple conditions in a single model (as others have done [15, 37]).

In 2016, the King’s Fund in the UK examined the pressures of general practice and highlighted the issues faced by practitioners such as increasing workload, greater complexity of work and pressures to meet strict deadlines [38]. In this context, a clinical risk prediction model to objectively assess a patient’s risk of hospitalisation may be welcomed. Estimated risk scores could be presented to the patient, supporting a shared decision-making approach during the consultation. Additional work is needed to validate the clinical risk prediction models before they could be used in clinical practice, but this work represents the first step towards changing the way GPs assess and treat patients for multiple common infections.

Whilst some may argue that the link between antibiotic prescribing and infection-related hospitalisation is not necessarily causal, it is indisputable that antibiotics are the most effective large-scale treatment for common infections. Hence, ensuring the efficient use of antibiotics in primary care is the easiest way to manage the cases of infection-related hospitalisation downstream.

Simplified versions of the models presented here have been made available to medical professionals as an educational resource through a risk calculator tool (Fig. 4) as part of the BRIT Antibiotic Prescribing Dashboard, which is on the HSCN (Health and Social Network). Whilst the models are able to advise on which patients are at the highest risk, they do not explicitly identify which patients should or should not be offered treatment. Given that antibiotics are relatively cheap and very effective, the decision over whether to prescribe is often complicated by the fact that not treating a serious bacterial infection has a much higher cost to the individual than over-treating, leading to physicians prescribing “just in case” [39]. Patients with infections can deteriorate quickly (possibly over a matter of hours), whereas other clinical prediction models investigate conditions (e.g. cardiovascular disease or types of cancer [33, 40, 41]) that develop over a much longer period of time and allow a longer period in which to intervene.

Conversely, it cannot be automatically assumed that antibiotics should only be given to patients with a high risk of infection-related hospitalisation—clinical assessment is a crucial part of the process, and a risk-based judgement is never complete on its own. There are many additional factors that can complicate the decision-making process including whether the infection is genuine, if symptoms will improve with treatment, and potential for infection-related complications. Despite this, the calculators provide a further complementary tool and could work to counteract individual proclivities in prescribing.

Model transportability was a problem during the validation phase of the work, showing that it was difficult to find a model that generalised well to both populations. Domain validation (as done in this study) is notoriously difficult [42] and rarely attempted [43, 44] because it is not easy to account for the demographic contrast and, in this case, local infectious factors. Here, the validation cohort (SAIL) showed a much higher incidence rate of complications overall, in particular, among young children (age < 5), part of which could be attributable to the measles epidemics in South Wales in recent years [45]. Wide CIs were also observed for some groups, e.g. for patients 80+, UTI—this is likely down to the low occurrence of events for this indication, particularly for the baseline group (age 20–30), leading to low statistical power. Despite this difficulty, standard approaches were applied to account for these differences and underpin the results presented, by allowing us to develop CPRD-specific and SAIL-specific models, using the former as the foundations for the latter.

The main limitation with a study of this type is that EHRs can only provide a static snapshot of a patient’s consultation. Because of this, there is no way to fully understand the severity or type of symptoms seen by the practitioner. Here, we separated consultations into three different infections using a single umbrella term to describe many different READ codes, giving the impression that all cases of a single infection have the same level of seriousness, when, in reality, that is not the case. The vast array of codes available and variety by which the same condition can be coded adds to the complexity of conducting this type of analysis [46]. However, despite these limitations, this was the best approach because it would have been very difficult to build the prediction models for individual read codes as the incidence rate would have been too low. Our models were also limited by the selection of the predictors; in particular, some factors were not included. For instance, despite a clear regional variation in primary care antibiotic prescribing [14, 47], this aspect was not built into the prediction models as an explanatory variable. This was done because the intention was to make the model applicable to different regions and ultimately to be used in a clinical setting across the entire UK. Smoking status and BMI, which may be important predictive variables, were also omitted from the models due to large amounts of missing information in the EHRs. Finally, the data used in the study failed to capture instances of delayed prescription or cases where patients do not take (or complete) their prescribed course. These are both interesting subgroups of the main population and would possibly warrant further investigation in their own study.

A recent study investigating the drivers of antibiotic prescribing found that prescribing guidelines alone do not positively influence a change in prescribing, and suggests that more targeted interventions are needed [14]. To achieve the ambitious government target of reducing antibiotic use in the community by 15% by 2024 [48], innovative approaches are required in primary care. Whilst this study represents a start towards that goal, further work is needed including further validation of the models in new datasets and creating new models for other common infections. As well as the research step, it is crucial that this intelligence is available to practitioners to inform their decision-making. An antibiotic prescribing dashboard containing this information is being developed, with the hope of working with general practices to construct a dynamic way to integrate this into point-of-care decision support [49].

Three clinical risk prediction models were presented, capable of evaluating a patient’s risk of developing further complications as a result of their common infection. The models have been fitted and validated using two large national datasets. Examining prescribing by risk stratification highlighted the lack of relationship between a patient’s risk level and their chance of being prescribed an antibiotic; this is likely due to practitioners prescribing to symptoms but it does show a significant area where improvements could be made to tackle overprescribing of antibiotics.