Background
Methods
Study design
Search strategy
Eligibility criteria
Study screening and selection
Data extraction
Situations | Type of bias that might arise |
---|---|
a. Follow-up starts after eligibility criteria completion and treatment assignment. This situation happens when the follow-up starts after eligible individuals have started the treatment. The follow-up time is left-truncated, and individuals who experience early outcomes after starting treatment are not captured. | Prevalent user bias |
b. Follow-up starts at eligibility but after treatment assignment. This situation happens when the follow-up starts after individuals have started the treatments, which means that the follow-up time is left-truncated. Additionally, individuals are selected based on post-treatment criteria (e.g., individuals have no outcome that occurred before the start of the follow-up). | Prevalent user bias and selection bias due to post-treatment eligibility |
c. Follow-up starts before treatment assignment and eligibility. This situation happens when individuals need to meet the eligibility criteria after the follow-up has started and individuals have started treatments. For example, patients have to receive at least 2 consecutive prescriptions of treatment to be included in the analysis, but follow-up starts from the first prescription. Those who have an outcome within this time are excluded from the analysis leading to immortal time bias, or those who stop treatment after the first prescription are excluded leading to selection bias. | Immortal time bias and selection bias due to post-treatment eligibility |
d. Follow-up starts at eligibility, but treatment is assigned later. This situation happens in two cases: 1) When there is a grace period, a period from when individuals meet the eligibility to when they start treatments. For example, a study compares no antibiotic use with initiation of antibiotic use within 7 days since diagnosis of urinary tract infection. If an individual starts antibiotics on day 7, it means that they have survived for 7 days leading to immortal time bias. 2) When individuals have to use the treatment for a given period to be classified in the exposed group. For example, individuals have to fill three consecutive prescriptions of aspirin to be classified as an aspirin user group, and non-aspirin users, otherwise. This also leads to immortal time bias. Another issue that might arise from this situation is the risk of misclassification of treatment. For instance, in the example of initiating antibiotics within 7 days since diagnosis, if the individual has an outcome before day 7 and has not started the antibiotic, we are uncertain to classify her/him to the no-antibiotic user or antibiotic user. | Immortal time bias and misclassification of treatment |
Data synthesis
Data sharing
Patient involvement
Results
Study characteristics
Research transparency practices
N = 77 (%) | |
---|---|
Name of journal | |
- NEJM | 3 (4) |
- The Lancet | 2 (3) |
- JAMA | 13 (17) |
- Annals of Internal Medicine | 9 (11) |
- BMJ | 14 (18) |
- PLoS Medicine | 8 (11) |
- Circulation | 7 (9) |
- European Heart Journal | 4 (5) |
- Journal of the American College of Cardiology | 15 (19) |
Location of corresponding authors | |
- North America | 40 (52) |
- Europe | 25 (32) |
- Asia | 10 (13) |
- North American and Europe | 1 (1) |
- International | 1 (1) |
Study design | |
- Cohort study | 67 (87) |
- Not clearly reported | 10 (13) |
Treatment evaluated | |
- Pharmacological treatment | 53 (69) |
- Non-pharmacological treatment | 23 (30) |
- Both | 1 (1) |
Comparator | |
- Active comparator | 49 (63) |
- Usual care | 17 (22) |
- No treatment | 11 (14) |
Median sample size [min–max] | 24,000 [9100–80,000] |
Data sourcea | |
- Registry | 34 (44) |
- Electronic health record | 17 (22) |
- Health administration data | 14 (18) |
- Health insurance claims data | 20 (26) |
- Others | 11 (14) |
Funding source | |
- Not for profit | 43 (56) |
- For profit | 7 (9) |
- Both | 12 (16) |
- No funding | 5 (6) |
- Unclear | 10 (13) |
Research transparency practices | |
- Using a reporting guideline | 7 (9) |
- Code and algorithm used to classify exposures provided in supplementary documents | 57 (74) |
- Code and algorithm used to classify outcomes provided in supplementary documents | 60 (78) |
- A statement to provide data upon request | 10 (13) |
Reporting essential information of the target trial
Reporting of essential information | N = 77 (%) |
---|---|
Study characteristics | |
- Specification of the target trial | 2 (3) |
- Using a diagram to illustrate study design | 14 (18) |
Eligibility criteria | |
- Inclusion criteria for the study | 76 (99) |
- Post-baseline events in inclusion criteria (e.g., use of treatment, no follow-up data) | 9 (12) |
- Exclusion of individuals with contraindications for interventions evaluated | 9 (12) |
Adjustment for confounders | |
- Propensity score | 60 (70) |
- Inverse probability weighting | 10 (12) |
- Multivariable regression | 15 (17) |
- Instrumental variable | 2 (2) |
Outcome | |
- Primary outcome reported | 77 (100) |
Causal contrast of interests | |
- Intention-to-treat effect | 11 (14) |
- Per-protocol effect | 6 (8) |
- Both | 10 (13) |
- Not specified | 50 (65) |
Key time point of the target trial | |
- Time point of the start of follow-up | 72 (94) |
- Time point of eligibility criteria | 72 (94) |
- Time point of treatment assignment | 68 (88) |
- All three time points | 63 (81) |
Risk of bias due to failure of specifying a target trial
Author, year | Patients (n) | Intervention | Comparator | Outcomes | Time points of eligibility | Times points of treatment assignment | Start of follow-up | Situations of failure of emulating a target trial | Type of bias might arise | Solution described by authors |
---|---|---|---|---|---|---|---|---|---|---|
Converse 2019 [19] | 111 | Angiotensin II inhibitors after continuous-flow left ventricular assist devices implant | Usual care | Gastrointestinal bleeding | 30 days after operation to implant continuous-flow left ventricular assist devices | Within 30 days since operation | 30 days after operation | b. Follow-up starts at eligibility but after treatment initiation. | Prevalent user bias, selection bias due to post-treatment eligibility | No solution described. |
Skriver 2019 [20] | 29,136 | Low-dose aspirin | Usual care | Prostate cancer mortality | 1 year after prostate cancer diagnosis | Within 1 year since diagnosis | 1 year after prostate cancer diagnosis | b. Follow-up starts at eligibility but after treatment initiation. | Prevalent user bias, selection bias due to post-treatment eligibility | No solution described. |
Friberg 2019 [21] | 47,492 | Oral anticoagulant | No treatment | Diagnosis of dementia, ischemic stroke, intracranial bleeding | At the time of diagnosis of atrial fibrillation | 6 months before the start of follow-up | 30 days after the diagnosis of atrial fibrillation | b. Follow-up starts at eligibility but after treatment initiation. | Prevalent user bias, selection bias due to post-treatment eligibility | No solution described. |
Brauer 2019 [23] | 424,996 | Trazodone | Other antidepressants | Diagnosis of dementia | At the second prescription of antidepressant | At the first prescription of antidepressant | At the first prescription of antidepressant | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | No solution described. |
Xie 2019 [24] | 214,467 | Proton pump inhibitors | H2 blockers | All-cause mortality | 180 days after treatment group assignment | At the first prescription of either PPI or H2 blocker | At the first prescription of either PPI or H2 blocker | c. Follow-up starts at treatment initiation but before eligibility. | Selection bias due to post-treatment eligibility | No solution described. |
Brown 2019 [25] | 1555 | Immunomodulatory disease-modifying therapies | No treatment | Disease progression | 6 months after treatment commencement | Date of treatment commencement | Date of treatment commencement | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | Exposure was considered as time-dependent variable in all analysis to adjust for immortal time bias, but no solution described for selection bias due to post-treatment eligibility. |
Kim 2019 [26] | 1.0705 | Statin | Statin + fenofibrate | Cardiovascular events | 3 months after fenofibrate initiation | Date of fenofibrate initiation | Date of fenofibrate initiation | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | No solution described. |
Axtell 2019 [30] | 3276 | Surgery | Medical therapy | All-cause mortality | The first echocardiographic diagnosis | Time of surgery | The first echocardiographic diagnosis | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | Time-dependent propensity score matching and allocate time before surgery to the control group. |
Gharbi 2019 [31] | 157,264 | Antibiotic | No treatment | Bloodstream infection, hospital admission, and all-cause mortality | Date of urinary tract infection diagnosis | Within 7 days since diagnosis | Date of urinary tract infection diagnosis | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | No solution described. |
Gray 2019 [32] | 9653 | Chemotherapy | Usual care | All-cause mortality and breast cancer mortality | Date of diagnosis | Date of chemotherapy commencement not reported but likely to be after the date of diagnosis | Date of diagnosis | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | No solution described. |
van Rein 2019 [33] | 272,315 | Antithrombotic therapy | No treatment | Major bleeding | The date of their atrial fibrillation diagnosis | Treatment starts at any time after diagnosis | The date of their atrial fibrillation diagnosis | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | Exposure was treated as time-dependent variable in Cox regression. |
Mahévas 2020 [34] | 173 | Hydroxychloroquine | Usual care | Survival without transfer to ICU | Admission to hospital | Within 48 h since admission | Admission to hospital | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | Patients from the control group who reached the primary outcome during the grace period were randomly assigned to one of the two groups, given that their observational data were compatible with both groups at the time of the event. |
Rosenberg 2020 [35] | 1438 | Hydroxychloroquine with or without azithromycin | Usual care | In-hospital mortality | At admission to hospital | Start treatment at any time during hospitalization | 24 h after admission | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | No solution described. |
Geleris 2020 [38] | 1446 | Hydroxychloroquine | Usual care | Composite of intubation and death | 24 h after arrival at the emergency department | Start treatment before or any time after hospitalization | 24 h after arrival at the emergency department | b. Follow-up starts at eligibility but after treatment initiation for prevalent user group. d. Follow-up start at eligibility, but treatment is assigned later for new user group. | Prevalent user bias and immortal time bias | No solution described. |
Jorge 2019 [36] | 9659 | Renal transplant | Usual care | All-cause mortality | The initial date of entry into the waitlist | Surgery at any time after being the waitlist | The initial date of entry onto the waitlist | d. Follow-up start at eligibility, but treatment is assigned later. | Immortal time bias and misclassification of treatment | Exposure was considered as time-dependent variable in all analysis. |
Rea 2018 [28] | 44,534 | Two drug therapy antihypertension | Monotherapy antihypertension | Cardiovascular events | 1 year after the dispensing of treatment | The first day of drug dispensing | The first day of drug dispensing | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | No solution described. |
Lin 2018 [27] | 6558 | Low-dose of rivaroxaban | Standard dose of rivaroxaban | Major bleeding events | Patients refilled prescription more than once since the start of rivaroxaban | The first prescription of rivaroxaban | The first prescription of rivaroxaban | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | A sensitivity analysis was performed to include patients who did not refill their prescription after the first one. |
Siontis 2018 [22] | 25,523 | Apixaban or switching from warfarin to apixaban | Warfarin | Ischemic stroke, major bleeding events, and death | Diagnosis of atrial fibrillation | The date of the initial anticoagulation prescription or the date of apixaban prescription if patient switched from warfarin to apixaban | The date of the initial anticoagulation prescription or the date of apixaban prescription if patient switched from warfarin to apixaban | b. Follow-up starts at eligibility but after treatment initiation. | Selection bias due to post-treatment eligibility | No solution described. |
Ramos 2018 [29] | 46864 | Statin | No treatment | Incidences of atherosclerotic cardiovascular diseases and all-cause mortality | The second invoice of statin | The first invoice of statin | The first invoice of statin | c. Follow-up starts at treatment initiation but before eligibility. | Immortal time bias and selection bias due to post-treatment eligibility | No solution described. |
Discussion
Situations | Possible solutions |
---|---|
a. Follow-up starts after eligibility criteria completion and treatment assignment which leads to prevalent user bias. | Select new users [12]. |
b. Follow-up starts at eligibility but after treatment assignment which leads to prevalent user bias and selection bias due to post-treatment eligibility. | Select new users and ensure that individuals are not selected by an event that happens after the follow-up starts [12]. |
c. Follow-up starts before treatment assignment and eligibility which leads to immortal time bias and selection bias due to post-treatment eligibility. | Keep all individuals who start the treatment since the start of follow-up, create an exact copy of the population, assign them to one of the intervention groups from the start of the follow-up, and censor when they start to deviate from assigned treatment [39]. One strategy which is often used to account for immortal time bias in literature is to consider exposure as a time-dependent variable. However, this strategy is not adequate to address the risk of selection bias due to post-treatment eligibility, as an uncensored group might not be exchangeable with the censored group [3]. |
d. Follow-up starts at eligibility, but treatment is assigned later which leads to immortal time bias and misclassification of treatment. | 1) Randomly assign individuals to one of the treatment strategies [12]. 2) Create an exact copy of the population, assign them to one of the intervention groups from the start of the follow-up, and censor when they start to deviate from assigned treatment [39]. One strategy which is often used to account for immortal time bias is to consider exposure as a time-dependent variable. However, this strategy is inadequate to address the risk of misclassification, because if individuals have outcomes during the grace period, we are uncertain which intervention group they should be classified into. |
Guiding question | Explanation |
---|---|
1. When does the follow-up start? | - Check if the authors report the start of follow-up. It might be called the baseline, index date, and time zero. |
2. When do individuals complete eligibility? | - Check if authors report when individuals should complete eligibility. |
2.a. Can individuals be eligible at multiple times? | - Check if individuals could be eligible at multiple times and whether authors used a strategy to overcome this: (1) choose a single eligible time and (2) choose all eligible times and conduct a sequence of trials at each eligible time. |
2.b. Is there any post-baseline event (i.e., an event after the follow-up starts) in the eligibility criteria? | - Check if any events after the start of follow-up are listed in the eligibility criteria, e.g., complete 2 consecutive prescriptions or no outcome for the first 2 months after the start of follow-up. |
3. When are individuals assigned to an exposed or non-exposed group? | - Check if the authors report clearly when individuals are classified as exposed or non-exposed group. |
3a. Do individuals have to use treatment for a given period to be classified as an exposed group? | - Check if individuals have to use treatment for a given period, e.g., complete 2 consecutive prescriptions to be classified as exposed and non-exposed, if not, start the treatment or complete only 1 prescription. |
3.b. Is there a grace period? | - Check if individuals can start the treatment sometime after the start of follow-up and eligibility. |