| Retrospective medical record review and physician notification form (PNF) (survey). | Test type: 11 blood tests and 3 screening tests (mammography, Papanicolaou smear, Fecal occult blood) commonly performed in the OP setting with clinically significant results (values defined by physician consultation). | Varied systems across practices: 14 with no EMR (paper based progress notes and test results), 5 with EMR (for outpatient progress notes and test results), 4 with partial EMR (electronic progress notes or test results but not both). | Rate of apparent failures to inform or document was 7.1% (135 failures divided by1889 abnormal results) (Range 0-26.2%) | Patient outcomes not examined |
Population: 5,434 randomly selected patients aged 50-69 years who had been seen 90-360 days prior to the date of the review (between June 2005 and February 2006) in 19 community-based and 4 academic medical centre primary care practices (n = 23). | Indicators: Apparent failure to inform defined as abnormal results for which the reviewer could not find evidence within the medical record that the patient had been informed within the defined time interval (usually 90 days). The PNF was mailed to confirm failure to inform. | Use of a partial EMR was associated with higher failure rates for follow-up compared to not having an EMR (OR, 1.92; P = 0.03) or with having a total EMR (OR, 2.37; P = 0.007). |
Failure to document defined as apparent failure to inform for which the physician stated in the PNF that the patient had been informed but this had not been documented. |
Failure to inform defined as apparent failures to inform for which the physician did not return the PNF or stated on the form that the patient had not been informed. |
| Retrospective medical record review | Test type: positive fecal occult blood test (FOBT) | Centralised nurse based notification system. A designated nurse reviewed all electronic medical records of positive FOBT cases from each preceding month. The nurse then provided a monthly spreadsheet to notify appropriate clinic managers of cases with no indication of follow-up. The clinic managers then notified the patients’ physician. | 15% (63/423) of positive FOBT cases notified by the nurse lacked a documented follow-up plan within 4 weeks of the positive result. | In seventeen of the 30 cases which had follow-up plans delayed beyond 4 weeks, 2 had colorectal cancers found 215 and 233 days after the initial positive FOBT (1 included T3N0M0 rectal cancer and 1 included T4N0M0 transverse colon cancer) |
Population: 423 positive FOBT were identified for a 10 month period (December 1 2003 to September 30 2004) by the laboratory service of the medical centre of a 347 bed tertiary care facility of the Department of Veteran Affairs | Indicator: Outcomes of positive FOBT cases which had no documented follow-up plan and no explanation for this (such as ‘patient declined’ or ‘patient opted to go elsewhere’). | Full electronic medical record system with access to patient medical data from all VA medical centers and clinics. | 15/63 (24%) had no appropriate follow-up plans; 30/63 (48%) had follow-up plans delayed beyond 4 weeks; 13/63 (21%) had plans within 4 weeks which were not documented; 5/63 (8%) had colonoscopy not indicated (example patient with known active Crohn’s disease with colonoscopy 105 days prior). |
% reported per test and per patient (one test per patient) |
| Retrospective medical record review | Test type: Abnormal results from 4 tests: pap smears; mammograms; international normalized ration (INR); prostate-specific antigen (PSA) (called high risk abnormal results) | Paper logs of abnormal test results are maintained by staff at each community health clinic as an internal record of tracking patients with abnormal results. | 34% (116/344) of abnormal results did not have documentation that that appropriate follow-up had occurred: 11% (9/82) for mammography; 26% (16/61) for INR; 45% (47/105) for Pap smears; 46% (44/96) for PSA. | Not examined |
Population: 344 abnormal results from 11 clinics (non random selection) within a large urban multisite federally qualified community health center. Abnormal results were identified in two ways: patients with abnormal results from 2004 to 2007 using administrative databases and, secondly manual review of paper logs of abnormal results maintained at family health centres from January 2007 to August 2008. | Indicator: No documentation that the result was: filed in the medical record, the provider signed and responded to the result, the patient was notified of the result, and the appropriate follow-up occurred in a timely manner (or patient refusal for follow-up). Timely follow-up for each test type clearly defined in study (example, pap smear – colposcopy performed within 3 months; mammogram – diagnostic procedure performed within 3 months). | Manual medical records used in each community health clinic. | Range of failure to follow-up between clinics was 13% to 59%. |
% reported per test | The last step of the results management process, ensuring appropriate follow-up, accounted for most of the errors in follow-up. |
For patients receiving follow-up care 49% of the time the follow-up did not occur in a timely manner |
| Retrospective electronic medical record audit | Test type: Suspected malignancy in radiology examinations (including conventional radiology, sonography, CT, MRI, barium exams, excretory urography, myelography, angiography and excluding mammography). | A semi-automated process involving electronic, paper and telephone reporting was employed for the coding and review of potentially malignant findings. | Provider unaware of imaging finding in 4/395 (1.0%) of primary medicine clinic patients with suspected malignancy | 5 of the 8 patients (IP or OP) where the provider was unaware of the findings had a final diagnosis of cancer. |
Population: 395 possible malignant cases between April 2003 and March 2004 in ER/urgent care, primary care medicine, non-primary care medicine and surgery in a University affiliated hospital. | Indicator: No documentation of follow-up in electronic medical record. | If a malignancy was found the radiologist contacted the referring clinician or appropriate member of the clinical team by telephone or rarely by secure email. The radiologist documented the contact in the report. Radiologist also assigned a code 8 to the report to signify possible malignancy and documented this in the electronic record. On a weekly basis the cancer registrar (a nurse practitioner) retrieved a list of all suspected malignancy patients from the database of all oncology patients at the hospital. The registrar monitored the electronic clinical record (patient notes, patient appointments, additional radiology or pathology visits scheduled) for documentation of appropriate follow-up. The registrar notified the hospital’s tumour board and/or ordering provider. Then the responsibility for tracking additional patient evaluation was delegated to the nurse practitioner in charge of active cancer cases. | In all referring services (IPs and OPs) provider unaware of imaging findings in 8/395 (2.0%) code 8 cases. |
% reported per patient (each patient had only one test) with analysis separated based on referring service. | The type of imaging test did not predict appropriate follow-up (p = 0.18). |
In the primary medicine service, no difference was seen between physicians and nurse practitioners in appropriate documentation of follow-up (p = 0.61). |
| Retrospective medical record review | Test type: Abnormal fasting plasma glucose levels or oral glucose tolerance tests (OGGTs) (Diabetes screening) | Not stated in paper but use of electronic laboratory database and electronic medical records. | 125/202 (62%) of abnormal results received no appropriate follow-up | Those who received appropriate follow-up were more likely to have been scheduled for follow-up appointments than those who did not (92% versus 66%, P = 0.001) and were more likely to have kept the scheduled follow-up appointment (90% versus 58%, P = 0.001) |
Population: 8,286 patients 45 years and over who had continuous enrolment in the M-CARE between 1.1.1998 and 30 June 2001. 5,752 (69%) were screened for diabetes over this three year period with 202 (4%) people having an abnormal result. | Indicator: No documentation of recognition of the abnormal screening in the medical record, including a comment about the result being abnormal, that is, a diagnosis of impaired fasting glucose, impaired glucose tolerance or diabetes referral to a dietician and/or performance of a definitive diagnostic test (fasting glucose, OGTT or HbA 1c) within 6 months of screening. |
M-CARE health maintenance organisation members assigned to the University of Michigan Health System (UMHS) for primary care (184 primary care physicians in 22 locations). M-CARE is a member managed care organisation wholly owned by the UMHS. | % reported per test |
| Retrospective analysis of merged databases: managed care administrative database; billing data from the MCO; and the DUMC laboratory database (96% of Duke Managed Care laboratory tests). Review of medical records of selected patients. | Test type: Abnormal glycemic testing. Laboratory tests used were plasma glucose tests and haemoglobin A1c (HbA1c) measurements. Abnormal defined by any one of three criteria: HbA1c 7.0% or higher, plasma glucose 200 mg/dL or higher, or plasma glucose 126 mg/dL or higher if the laboratory value was denoted as fasting in the database. | Not stated in the paper but use of electronic laboratory database and manual medical records. | At least 9% (when restricted to findings that are most suggestive of diabetes (high HbA1c or 2 high plasma glucose tests) and as many as 18% who have a positive lab. test suggesting diabetes have no evidence in the billing data or medical record that the condition was recognised. | Not examined |
Population: All patients over 30 years enrolled in the MCO of Duke University Medical Centre between April 1996 and March 1999. 1426 patients during this period had a laboratory test result suggestive of diabetes | Indicator: Number of patients with abnormal glycemic test results suggestive of diabetes who had no evidence of recognised diabetes documented in their medical record or included in the billing database. | 1426 patients had laboratory tests suggestive of diabetes. |
A patient had recognised diabetes if they had the appropriate ICD-9 code in the administrative database or if diabetes was mentioned in their medical record. They identified those without ICD-9 codes as candidates for medical record review and then reviewed a 30% random sample of those charts for mention of the word diabetes or DM in the progress notes. | 1122/1426 (79%) had ICD-9 codes for diabetes in the database. |
% reported per patient | 36/304 without ICD-9 codes had mention of diabetes on medical record review. |
| Retrospective medical record review. | Test type: Abnormal laboratory and imaging | Detailed description of different ways each step of the results management process was undertaken in each of the four family medicine offices, 1 of which had an EHR. Most processes were manual (phone, fax, written response) involving physicians and other staff. One practice had no physician acknowledgement or response to result nor patient notification nor documentation nor follow-up process. | 42/160 (26%)abnormal laboratory or radiology tests had no documentation in the chart of the clinicians response to the abnormal result | Not examined |
Used qualitative methods to explore test management practices in each of the 4 family medicine offices. | Indicators: | Examples of patient notification of results included: | 17/160 (10%) of abnormal radiology or laboratory results had no clinician signature; 58/102 (57%) had no documentation that advice, recommendations or information were given to the patient about the abnormal result. |
Population: 4 family medicine offices in the Ohio region. 25 random charts from each office (n = 100) that contained laboratory or imaging results in a 12 month period. 160 test results reviewed. | a) no documentation of clinician signature on result; or | • Copy of test result mailed to patient |
b) no documentation in the chart of clinicians’ response to the result. | • Standard test result checklist mailed to patient |
% reported by test type | • Letter or note written to patient |
• Physician calls patient |
• Staff calls patient |
• At a future patient visit |
• Patient not notified |
Examples of varied processes listed from the four family medicine offices for the follow-up of results needing further care step were: |
• notes made to self by physicians; |
• notes made to self by staff; |
• ticker file maintained by staff or physician; |
• registry of single test type (e.g. Pap smears); |
• no follow-up. |
| Retrospective medical record review | Test type: Clinically significant abnormal laboratory and imaging | 4 paper based and 4 EMR systems. A total of 274 results were managed by an EMR. Results managed with an EMR were more often in the right place in the chart (100% versus 98%), had more clinician signatures (100% versus 86%), interpretations (73% versus 64%) and patient notifications (80% versus 66%) documented. |
EMR systems: 36% (21/88) of abnormal results had no follow-up plan documented in an EMR. | Not examined |
Population: 8 purposefully chosen Family Medicine Offices to provide variation around geographic location (rural, suburban, urban), practice size, patient insurance status, technology level (EMR, no EMR), and residency program (program, no program). | Indicator: No documentation in chart of follow-up plans. | Four offices had written protocols and/or adhered to office-wide practices in two or three test result management steps (patient notification, clinician signature and test tracking). The other four offices had only one or no steps with written protocols or results management practices that were well followed. No offices had standardized processes for documenting interpretation of test results or follow-up for abnormal results. |
Paper based systems: 60% (49/82) of abnormal results had no follow-up plan documented in the manual system |
25 charts at each office (n = 200) that contained laboratory or imaging results in a 12 month period. Total of 461 test results reviewed in the 200 charts in the 8 offices. | % reported by test type (commonly grouped tests (complete blood counts, metabolic profile etc) were considered a single test | For the subset of abnormal results (n = 170) 64% of results managed with an EMR had a follow-up plan documented, compared to 40% for paper managed results. |
| Retrospective medical record review after data linkage of patients who received colorectal cancer screening in the VA quality improvement program linked with the VA Electronic Outpatient Clinic File. | Test type: Positive fecal occult blood test (FOBT) | Not stated in the paper however, VA have a quality improvement program which monitors rates of preventative care across a broad range of clinical domains, including colorectal cancer screening and VA has had electronic medical record system since 1985. | Of those patients with positive fecal occult blood tests (n = 313) 41% (128/313) did not receive follow-up testing. | Not examined |
Population: All patients aged over 52 years sampled from a QI review for colorectal cancer screening attending a primary clinic in a VA hospital between 1st October 2001 to 30th September 2002. | Indicator: Patients had no follow-up if one of two criteria were not met: 1) documentation of colonoscopy or barium enema that was dated after the positive FOBT or 2) if the patient was referred for colonoscopy. |
| Retrospective review of malpractice claims | Test type: abnormal diagnostic or laboratory test result | Not stated in paper | 23/181 (13%) cases of missed diagnoses were due to ‘responsible provider did not receive diagnostic or laboratory test results’. | Of the 23 cases where the responsible provider did not receive the test result, 17 cases were for missed cancer diagnoses. |
Population: 181 closed malpractice claims involving diagnostic errors in the ambulatory setting which resulted in patient harm between 1984 and 2004 | Indicator: Evidence of missed diagnosis resulting from provider not receiving abnormal diagnostic or laboratory test results abstracted from documentation in claim file from each insurer and background information from relevant medical record | In 22/181 (12%) of cases, the diagnostic or laboratory test results were not transmitted to the patient | Of the 22 cases where the test result was not transmitted to the patient, 15 were for missed cancer diagnoses. |
4 malpractice insurance companies (north eastern, south western, western United States) insuring approximately 21,000 physicians, 46 acute care hospitals and 390 outpatient facilities: including a variety of primary care and outpatient specialty practices. |
| Retrospective medical record review | Test type: abnormal value for glucose test for diabetes | The practice has an EMR which includes appointments, test orders, laboratory results and progress notes. At the time of the study there was no systematic screening program for diabetes and the EMR did not have specific flags for episodes of diabetes screening. | 466 glucose tests performed and 65/466 were abnormal. The proportion of abnormal values depended on the cut-off: 65 (14%) ≥101 mg/dl; of these, 26 (6%) ≥110 mg/dl; and 15 (3%) ≥125 mg/dl | Not examined |
Population: 301 randomly selected patients over 20 years of age with no known diabetes who received care at the study site over a one year period with a minimum of 2 further visits in the next 3 years | Indicator: No documented follow-up for abnormal value of glucose test for diabetes, or no indication in record of subsequent action in response to abnormal result | For glucose values: |
Study site was a large outpatient general internal medicine practice affiliated with an academic health centre (New York USA) (Cornell Univ.) | 101-109 mg/dl, 100% (24/24) had no follow-up; 110-125 mg/dl, 57% (15/26) had no follow-up: ≥126 mg/dl, 46% (7/15) had no follow-up |
| Retrospective medical record review | Test type: Laboratory tests pending at discharge (for example pleural fluid cytology, thyroid function tests, stool culture, HIV test) | Not stated in paper but use of separate manual inpatient and outpatient medical records and outpatient PCPs had no access to the patients’ discharge summaries. | 7/86 (8%) (95% CI, 2 to 14) of all participants and 41.2% of participants with tests pending at discharge had at least 1 test follow-up error | Not examined |
Population: 86 patients hospitalised in one medicine service who were subsequently seen by their primary care physicians in the affiliated outpatient practice within 2 months of discharge | Indicator: Laboratory test was pending at discharge and was not acknowledged (ie: documented) in the outpatient chart by the outpatient primary care physician | No association between test follow-up errors and rehospitalisation |
Participants selected by retrospective record review of 366 randomly selected patients chosen from 2139 discharged from 950 bed urban teaching hospital (Mount Sinai School of Medicine, New York, USA) between July 2000 and June 2001. | % reported per patient |
| Retrospective medical record review | Test type: abnormal serum potassium | OP practice has a paper based chart system and laboratory test results are retrieved on a computerised information system via computers located in each patient care room. Paper copies of test results are placed in attending preceptors’ mail-boxes within 1-7 days of the results being available. All results are reviewed by attending preceptors who are expected to act on abnormal test results. Critical lab results are communicated from the lab to a nurse in the primary care practice. The nurse then communicates the critical lab result to an attending physician. | No documented evidence of follow-up at all in 11/109 cases of hyperkalaemia (10.1%). | Not examined. |
Population: 86 patients with abnormal potassium levels (hyperkalaemia or hypo) seen at a large hospital based general internal medicine/primary care practice (Mount Sinai Medical Centre, New York USA) | Indicator: No documented response /action to abnormal serum potassium result in the medical record or computerised laboratory system | 25% of patients repeat potassium levels were not performed until they were seen at routine follow-up visits or when they visited the clinic for problems unrelated to hyperkalemia such as medication refills. |
Between September 2003 and August 2004. | % reported per test and per patient |
| Retrospective medical record review | Test type: abnormal serum potassium | Paper based chart system with results of laboratory tests retrievable via the computerised information system. Critical lab results are directly communicated from the lab to physicians in the practice. | After 30 days 26/259 elevated serum potassium tests (10%) had no follow-up. | Overall 10.8% of the episodes of hyperkalemia resulted in hospitalisations, and another 8.5% of episodes resulted in ED visits. There were no deaths resulting from the episodes of hyperkalemia. |
Population: 190 adult patients with elevated serum potassium levels over a 4-year period (January 2002 to December 2005) seen at a large primary care practice at a 1100 bed tertiary care academic teaching hospital (Mount Sinai School of Medicine, New York USA) | Indicator: No documented attempt to contact patient for follow-up of abnormal result, or no documentation that subsequent patient visit addresses the elevated serum potassium result, or no documented repeat serum potassium testing | After 30 days 18% of the cases had no repeat testing. |
259 serum potassium tests that were non-haemolyzed and non-contaminated samples with values ≥6.0 mEq/L |
| Prospective longitudinal study involving medical record review and patient survey | Test type: “Marginally abnormal” mammograms for which short term follow-up imaging is required in 3-6 months | Not stated in paper. | 45/126 (36%) of women with abnormal mammogram requiring short-term follow-up did not receive appropriate or timely follow-up. | No adverse effects for those who did not received appropriate follow-up (one who did receive appropriate follow-up was diagnosed with breast cancer during the course of the study) |
Population: 126 (out of 181 with abnormal mammogram requiring 6 month follow-up) English or Spanish speaking women with abnormal mammograms (between June 1996 and June 1997) requiring short term (6 months) follow-up imaging from ten academically affiliated ambulatory medical practices in the Boston metropolitan area. The practices were diverse in location, structure and degree of academic affiliation. | Indicator: no documentation in their medical record that they had received a follow-up mammogram, a surgical consult or a breast biopsy within 7 months of the index abnormal mammogram. | Key finding was that documentation of follow-up plan by the treating doctor was associated with the appropriate follow-up of test results (adjusted odds ration 2.79; 95% CI, 1.11 to 6.98; P = 0.029) |
| Retrospective medical record review plus telephone contact with providers | Test type: Critical imaging results (abnormal radiographs, CT scans, MRI scans, mammograms, ultrasonograms) | VA uses CPOE which relies on computerised notification of abnormal test results (alerts) displayed prominently through a view alert window displayed in the EMR every time the provider logs on. | 45/1,017 abnormal result alerts (4%) were completely lost to follow-up (after checking with provider if they knew of result) 4 weeks after the date of the study. | Not examined |
Population: 1,017 electronically transmitted alerts for abnormal imaging results over an 83 day study period from patients attending multispecialty ambulatory clinics of a Veterans Affairs medical centre (Houston, Texas USA) between March 2 2006 and May 28 2006. | Indicator: No documented evidence of response to abnormal imaging results in patient chart, and no provider awareness of abnormal imaging results upon telephone contact with provider. Failure to acknowledge an alert was based entirely on the provider clicking on the alert in the electronic View Alert system in the EMR. | With the exception of life threatening findings, communicated via telephone, the radiologist alerts the provider electronically to the presence of ‘significant unexpected findings’ using codes specific to the institution. When the provider clicks on the alert this indicates that the provider received the imaging result and this is the only mechanism for the EMR to record the providers’ acknowledgement. However, providers can ignore alerts and bypass them. | Providers failed to acknowledge 368/1,017 (0.36%) transmitted alerts (ie: failed to click on the alert in the EMR). |
| Retrospective medical record review plus telephone contact with providers | Test type: Abnormal imaging results (abnormal radiographs, CT scans, MRI scans, mammograms, ultrasonograms) | The Computerised Patient Record System (the EMR at VA) includes a notification system (the View Alert system) for alerting clinicians to clinically significant events such as abnormal diagnostic test results. When the PCP is out of the office, they can assign covering surrogate HCPs to receive their alerts. When a trainee, a subspecialist, a surrogate or a substitute HCP orders a test, a second alert is automatically transmitted to the PCP (dual-alert communication). In this study the HCP who received an alert was considered responsible for timely follow-up (if 2 HCPs were alerted, they were both responsible for timely follow-up). HCPs have the option of bypassing or ignoring the View Alert window. | For all alerts (acknowledged and unacknowledged) no evidence of documented follow-up action was found in 131/1196 (11%) of the alerts. | In the 92 cases that did not receive timely follow-up, more than one fourth (n = 26 (28%)) led to a new diagnosis being established. Cancer was determined to be a new diagnosis in 11 of these cases. |
Population: Of 123,638 imaging studies, 1,196 (0.9%) generated alerts for abnormal imaging results from November 2007 to June 2008 from patients attending multispecialty ambulatory clinics of a Veterans Affairs medical centre (Houston, Texas USA) | Indicator: Alerts lacking electronic acknowledgement—Tracking software determined if the alert was acknowledged (that is, opened for viewing within 2 weeks of transmission). | Staff radiologists electronically code abnormal imaging that requires action as alerts which are then transmitted to the View Alert window. Alerts stay active in the window for 2 weeks | For all alerts (acknowledged and unacknowledged) timely follow-up was lacking in 92/1196 (7.7%) of all alerts. |
Alerts lacking timely follow-up: Medical records were reviewed for unacknowledged electronic alerts within 4 weeks of transmission to determine if there was any documented response to the alert (defined as any documented evidence of follow-up action). Providers were telephoned to exclude failure to document. | 217/1196 (18.1%) alerts were unacknowledged |
| Retrospective medical record review plus telephone contact with providers | Test type: Abnormal laboratory tests which generated a ‘high priority’ mandatory (provider does not have the option to turn off the alert) automated notification to the provider without concomitant verbal notification. 4 laboratory test alerts met these criteria: haemoglobin > = 15; positive hep C antibody; prostate specific antigen > = 15; thyroid stimulating hormone (TSH) > = 15 | The Computerised Patient Record System (the EMR at VA) includes a notification system (the View Alert system) for alerting clinicians to clinically significant events such as abnormal diagnostic test results. | Timely follow-up (within 30 days) was lacking in 79/1163 (6.8%) (79 out of all alerts) and was statistically not different for acknowledged and unacknowledged alerts. | Not reported |
Population: Between May and December 2008, 78,158 tests (Hb A1c, Hep c, TSH, PSA) \were performed of which 1163 (1.48%) were transmitted as alerts. Study conducted in a large multispecialty ambulatory clinic of VA Medical Center and 5 satellite clinics located in Southeast Texas. | Indicator: Alerts lacking electronic acknowledgement—alert tracking system determined whether the alert was acknowledged (provider clicked on and opened the message) within 2 weeks of transmission. | Timely lack of follow-up for specific tests: HCV Ab (Elisa) 13.2%; PSA 0.9%; TSH 6.3%; Haemoglobin A1C 6.9%. |
Alerts lacking timely follow-up: Within 30 days of result transmission, record review and provider contact determined follow-up actions (e.g. Ordering a follow-up test, prescribing or changing treatment, contacting the patient) | 10.2% of alerts (119/1163) were unacknowledged. |
| Retrospective medical record review | Test type: Abnormal thyroid stimulating hormone (TSH) for monitoring patients with hypothyroidism on levothyroxine medication | Not stated in paper | 37/96 patients (39%) had no appropriate surveillance response within 2 months to high or low TSH results. | 24 patients (7%) experienced an adverse drug event (ADE) during the study period and 12 patients (3%) developed an ADE related to levothyroxine and 6 of these were secondary to insufficient supplementation |
Population: 363 outpatients from a large tertiary care hospital receiving levothyroxine therapy over a 1-year period – 2000 to 2001(randomly selected) | Indicator: No appropriate surveillance response within 2 months of a newly abnormal surveillance result |