Main systematic review of observational studies
We conducted a systematic review in the MEDLINE, Embase, and Cochrane Library databases in stages. The original search covered observational studies published between 1 January 2002 and 1 October 2012 and was last updated 18 September 2014. In all of the updates, we consistently used the same methodology described in Online Resource Additional file
1 (and in [
9]). Although the systematic review did not have a registered protocol, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [
10].
We included both T1 and T2 DM patients (children or adults) in our analysis. We used a consistent definition of SHE as an event with low plasma glucose levels that requires help from another person to manage. This definition has been used in numerous publications [
11‐
15] and seems to be an attractive and clinically sound choice for results used in cost-measuring studies because it directly links SHEs to resource use.
We decided to include observational studies in the systematic review in order to more accurately assess SHE rates in real-life settings. Moreover, to account for changes in clinical practice in recent years and the possible impact on treatment-related risk, only recent studies published no more than 10 years ago were included. Taking into account our updates, the earliest studies could have been published on 1 January 2002. To balance the number of studies included and the strength of evidence, we only included studies of some minimal reasonable size, which we defined as at least 100 patients in the total study, though possibly split between several groups.
We designed our review to differentiate between the following treatment regimens: insulin pumps, basal-bolus insulin therapy with long-acting insulin analogue as the basal component (BBA), basal-bolus insulin therapy with human insulin as the basal component (BBH), biphasic insulin analogue, and biphasic human insulin in T1 DM, and sulfonylurea (SU) with or without other oral drugs but excluding insulin, basal long-acting insulin analogue (BOTA), basal human insulin (BOTH), BBA, BBH, biphasic insulin analogue, and biphasic human insulin (all insulin regimens could be in combination with other antidiabetic drugs, OADs) in T2 DM. We defined basal-bolus insulin therapy as long-acting insulin analogue once or twice daily and short/ultrashort insulin at mealtime. In the main systematic review, OADs in T2 DM, especially oral antidiabetic medications different than SU, were neglected assuming that the risk of SHE is negligible. We planned to only assess risk indirectly using information on relative rates from studies searched for in a separate systematic review of secondary studies as described below.
Two authors independently conducted the selection process for relevant trials (basic search: J.P. and E.R.; 1st update: J.P. and E.R.; 2nd update: E.R. and M.J.). Protocol assumed that in the case of discrepancies between the authors, a discussion would be held until consensus was reached. We extracted the following parameters: time horizon at which hypoglycaemia was assessed, number of patients in the study group, number of hypoglycaemic episodes (absolute or mean per patient in a specified period of time, if available), and number of patients experiencing at least one SHE (if available). If one study was described in more than one manuscript, then the manuscript with the most appropriate and complete results was selected for extraction (e.g., data for a total study cohort instead of subpopulation, results presented separately for patients with T1 and T2 diabetes, or results split by insulin regimens of interest). Data from included studies were extracted by one of the reviewers and verified by the other.
We assessed the quality of the observational studies using the Newcastle-Ottawa Scale (NOS) [
16], a tool developed for case–control and cohort studies that allows the quality of non-randomized studies (its design, content, and ease of use) to be assessed. Deeks et al. [
17] pointed out that this scale is one of the two best for evaluating non-randomized interventional studies and can be used in systematic reviews as either a scale or a checklist. This tool is also mentioned in the Cochrane Handbook as a tool that can be used for assessing methodological quality or risk of bias in non-randomized studies [
18]. Thus, in our systematic review we decided to use NOS for case–control and cohort studies, while in order to assess the quality of other types of non-interventional studies we focused on the following elements: patient selection methods, methods for recording outcomes (regarding only severe hypoglycaemia), study size, and study representativeness. According to a recent systematic review of tools used to assess the quality of observational studies [
19], there are 97 tools (46 scales and 51 checklists) that can be used to evaluate observational studies, but a transparent objective assessment of the quality of observational research is still missing.
As it was a desk research, the study did not involve any human subjects (including human material or human data) and no ethical committee approval was asked for. For the same reason there was no need to collect informed consent for participation in the study.
Supplementary systematic reviews
As mentioned above, we planned to assess the risk related to OADs in T2 DM by calculating the relative rates compared to SU based on secondary studies and then impose these relative rates on the background SU-related SHE rate. Our main systematic review also yielded no observational studies for biphasic therapies in T1 DM; therefore, we decided to use similar methodology to assess the SHE rate for that treatment regimen.
In the first of the two additional systematic reviews we used the following approach. We searched MEDLINE, Embase, the Cochrane Library, and Centre for Reviews and Dissemination (CRD) to identify secondary studies (systematic reviews and meta-analyses) that can be used to estimate the relative rates (see search strategy in Additional file
1). Studies were included if the search was performed in at least two databases, including at least one of the above databases, by at least two authors (due to the need to confirm the search results) and if the search strategy was described. RCTs of T2 DM required at least one of the following: dipeptidyl peptidase-4 inhibitor, glucagon-like peptide-1 agonist, OADs such as metformin, and TZD. In addition, severe hypoglycaemia had to be defined as an episode when a patient required help from another person. We used RCTs instead of observational studies because they more commonly provide data on relative rates.
We used the following approach in the second of the additional reviews. The MEDLINE database was searched on 25 February 2015 using the search strategy presented in Additional file
1. Only systematic reviews and RCTs (also those included in systematic reviews) were eligible. Studies performed in patients with T1 DM or T1 and T2 DM (with results presented separately) had to compare patients randomized to a group receiving biphasic insulin analogues to a group receiving biphasic human insulin, a group receiving a basal-bolus insulin regimen, or a group on insulin pumps.
Data synthesis methods for the main review
The data from included studies vary in structure, i.e., some studies present the total number of episodes in a given group of patients over some time horizon, whereas others present the number of patients who experienced at least one event, and some studies present both. Studies also differ with respect to time horizon. In order to increase the flexibility of the model, we chose the Bayesian approach using the Markov Chain Monte Carlo (MCMC) estimation method implemented in JAGS using R.
We based our model on negative binomial distribution for the following reasons. Firstly, in the previous analysis we used the Poisson distribution as a natural choice to model count data that were available [
9]. However, in the present paper we decided to account for the fact that episodes tend to be concentrated in a subgroup of patients; in most of the studies, we observed that fewer patients actually experienced any episode than suggested by a Poisson distribution for a given total number of observed episodes. This observation may stem from some individual predispositions (e.g., lifestyle, genetics, etc.) and would require the introduction of a zero-inflation mechanism, i.e., the fact that many patients will have no events but some will tend to have multiple events.
Secondly, we did not have access to individual patient data (i.e., we only observed the total number of episodes in a study); thus, we needed to work with a distribution that can be aggregated to describe the number of episodes in a cohort. We also needed to be able to account for varying time horizons under plausible assumptions. Thirdly, we wanted the results of the estimation to be easily usable in further modelling, and we did not want the SHE rates to vary over time, which would require knowing the longevity of the treatment, even assuming that the time to first SHE is distributed differently than the time to subsequent SHEs and would be a possible way of introducing the zero-inflation phenomenon.
The negative binomial distribution allows for a clustering effect and can be easily mathematically expressed for groups of patients observed for varying time horizons. We decided to use a mixed fixed-effect and random-effect approach by assuming that the parameter measuring the over-dispersion is fixed across studies for a given treatment regimen while the other parameter impacting the absolute rate was random. This approach allows us to account for heterogeneity between studies, whereas taking both parameters in the random-effect approach yielded unstable results. Non-informative prior distributions were taken.
The specification and JAGS codes are given in the supplementary material (Additional file
2).
As studies usually have only one arm, the SHE rate is estimated separately for the individual classes of drugs. Importantly, the aim of the present study was not to compare drugs, which requires two-arm studies, possibly with randomization, but rather to estimate the absolute rate of SHEs for all of the drug classes separately, accounting for the tendency to prescribe various drugs to patients with varying baseline risks of SHEs (e.g., lifestyle) in clinical practice.
The median values of posterior distributions were used as point estimates. The 2.5 and 97.5 percentiles were used as limits of 95 % confidence intervals (CIs). We used 10,000 iterations as a burn-in phase and then collected every fifth of 50,000 iterations. The estimated parameters of the model allow us to assess the SHE rate (i.e., average number of events per patient-year) and the probability of a given patient suffering at least one SHE over a year.