Background
Methods
Search strategy
Inclusion and exclusion criteria
Study selection
Data extraction
-
General system features;
-
Clinician-system interaction features;
-
Communication content features;
-
Auxiliary features;
-
Guidelines features.
CATEGORY | FEATURE | EXPLANATION | PROPORTION |
---|---|---|---|
General system features
|
Presence of networks *
| User has access to recommendation in computer terminals, available at several workstations in the hospital. | 0.20 |
Type of suggestion *
| Recommendation is provided in different ways including reminders of overdue health care tasks, alerts of critical values, prompts for various active care issues. | 0.78 | |
Conflict of interest *
| Software designer or producer is involved in the design of study. | 0.38 | |
Degree of automation *
| User automatically receives prompts (complete automation) instead of active initiation of the system by user (incomplete automation). | 0.80 | |
Clinician-system interaction feature
|
Automatic provision of recommendation in paper version as part of clinician workflow **
| Recommendations printed on paper forms and attached to patient charts by clinical support staff, so that clinicians do not need to look for the computer advice. | 0.29 |
Automatic provision of recommendation in electronic version as part of clinician workflow **
| Electronic recommendations linked to patient charts display automatically to clinicians when a clinician accesses the database. | 0.82 | |
Data updating via network *
| Data of patient are updated via network link to servers storing information about all contacts of patient with the hospital. | 0.33 | |
Request documentation of the reason for not following recommendation **
| The user is asked to justify the decision of disagreement with a reason such as "the patient refused" or "I disagree with the recommendation". | 0.56 | |
Provision of recommendation at time and location of decision making **
| Recommendations provided as chart reminders during an encounter, rather than as monthly reports listing all the patients in need of services. | 0.13 | |
Recommendation executed by noting agreement **
| Computerised system provides recommendations in response to an order and the user simply clicks "OK" to order the recommended tests. | 0.11 | |
Communication content features
|
Provision of a recommendation, not just an assessment **
| Systems show better actions to perform, rather than simply providing a diagnosis. | 0.11 |
Promotion of action rather than inaction **
| Systems recommend an alternative view, rather than simply recommending the order to be cancelled. | 0.11 | |
Justification of recommendation via provision of reasoning **
| Recommendation for a check justified by noting date of last exam and recommended frequency of testing. | 0.18 | |
Auxiliary features
|
Local user involvement in development process **
| Recommendation design finalised after testing preliminary versions of software (beta version) with representatives from targeted user group. | 0.09 |
Provision of recommendations to patients as well as providers **
| As well as providing chart reminders for clinicians, system generates postcards that are sent to patients to inform them of existing recommendation. | 0.18 | |
Recommendation accompanied by periodic performance feedback **
| Users are sent e-mails periodically that summarise users compliance with recommendations. | 0.02 | |
Recommendation accompanied by conventional education **
| Implementation of a recommendation is accompanied by a presentation or an appropriate explanation for following such suggestion. | 0.27 | |
User training *
| A training period is provided for users to experience the basic features of the software. | 0.22 | |
Guidelines features
|
Type of guideline*
| Recommendations are focused on preventive or treatment issues or both options. | 0.31 0.62 0.07 |
Type of condition*
| Recommendations are oriented towards acute or chronic patients or both options. | 0.16 0.60 0.24 | |
Type of intervention*
| Recommendations suggest to administrate tests or/and drugs to patients or to perform other type of intervention on them or both options. | 0.53 0.16 0.31 |
Quality assessment
-
allocation to study groups (random, 2; quasi-random, 1; selected concurrent controls, 0);
-
data analysis and presentation of results (appropriate statistical analysis and clear presentation of results, 2; inappropriate statistical analysis or unclear presentation of results, 1; inappropriate statistical analysis and unclear presentation of results, 0);
-
presence of baseline differences between the groups that were potentially linked to study outcomes (no baseline differences present or appropriate statistical adjustments made for differences, 2; baseline differences present and no statistical adjustments made, 1; baseline characteristics not reported, 0);
-
objectivity of the outcome (objective outcomes or subjective outcomes with blinded assessment, 2; subjective outcomes with no blinding but clearly defined assessment criteria, 1; subjective outcomes with no blinding and poorly defined, 0);
-
completeness of follow-up for the appropriate unit of analysis (> 90%, 2; from 80% to 90%, 1; < 80% or not described, 0).
Quality assessment | |||||||||
---|---|---|---|---|---|---|---|---|---|
Authors | Year of publication | Rivista | Study Design | Method of allocation to study group | Data analysis and results | Presence of baseline differences between groups potentially linked to study outcome | Type of outcome measure | Completeness of follow-up | Total |
Burack | 1997 | Medical Care | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Burack | 1994 | Medical Care | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Butzlaff | 2003 | Family Practice | Experimental | 2 | 1 | 2 | 0 | 2 |
7
|
Cannon | 2000 | JAMIA | Experimental | 2 | 2 | 2 | 1 | 2 |
9
|
Carton | 2002 | Clinical Radiology | Observational (time series) | 0 | 1 | 0 | 2 | 2 |
5
|
Dayton | 2000 | Medical Decision Making | Experimental | 2 | 1 | 0 | 0 | 2 |
5
|
Demakis | 2000 | JAMA | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Derose | 2005 | American Journal Manag Care | Experimental | 2 | 1 | 0 | 2 | 2 |
7
|
Dexter | 2001 | New England Journal of Medicine | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Durieux | 2000 | JAMA | Observational (time series) | 0 | 1 | 1 | 2 | 2 |
6
|
Feldman | 2005 | Health Services Research | Experimental | 1 | 1 | 2 | 1 | 2 |
7
|
Feldstein | 2006 | Journal American Geriatric Soc | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Filippi | 2003 | Diabetes Care | Experimental | 2 | 1 | 0 | 2 | 2 |
7
|
Fitzamaurice | 2000 | Arch Intern Med | Experimental | 2 | 1 | 2 | 1 | 2 |
8
|
Frank | 2004 | Australia | Experimental | 1 | 1 | 2 | 2 | 2 |
8
|
Hetlevik | 1999 | Scand J Health Care | Experimental | 2 | 1 | 2 | 2 | 1 |
8
|
Hetlevik | 2000 | Int J Technol Assess Health Care | Experimental | 2 | 1 | 2 | 2 | 0 |
7
|
Jousimaa | 2002 | Int J Technol Assess Health Care | Experimental | 2 | 1 | 2 | 2 | 1 |
8
|
Kitahata | 2003 | Clinical Infectious Disease | Observational (before and after) | 0 | 1 | 2 | 2 | 2 |
7
|
Kucher | 2005 | The New England Journal of medicine | Experimental | 2 | 1 | 0 | 2 | 2 |
7
|
Lafata | 2002 | JGIM | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Lobach | 1997 | Am J Med | Experimental | 2 | 0 | 2 | 2 | 2 |
8
|
Raebel | 2005 | Arch Intern Med | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
McCowan | 2001 | Medical Informatics | Experimental | 2 | 2 | 2 | 1 | 0 |
7
|
McMullin | 2004 | Annals of Family Medicine | Observational (retrospective cohort study) | 0 | 1 | 0 | 2 | 2 |
5
|
Medow | 2001 | Medical Decision Making | Experimental | 2 | 2 | 0 | 0 | 2 |
6
|
Meigs | 2003 | Diabetes Care | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Montgomery | 2000 | BMJ | Experimental | 2 | 2 | 2 | 1 | 1 |
8
|
Mosen | 2004 | Chest | Observational (before and after) | 0 | 2 | 2 | 2 | 2 |
8
|
Murtaugh | 2005 | Health Services Research | Experimental | 2 | 1 | 2 | 1 | 2 |
8
|
Overhage | 1996 | Arch Intern Med | Experimental | 1 | 2 | 2 | 2 | 2 |
9
|
Overhage | 1997 | JAMIA | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Poller | 1993 | J Clin Pathol | Experimental | 2 | 2 | 1 | 2 | 2 |
9
|
Rood | 2005 | JAMIA | Experimental | 2 | 2 | 1 | 2 | 2 |
9
|
Rossi | 1997 | JGIM | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Safran | 1995 | Lancet | Experimental | 0 | 2 | 2 | 2 | 2 |
8
|
Schriger | 1997 | JAMA | Observational (interrupted time series) | 1 | 1 | 2 | 2 | 2 |
8
|
Sequist | 2005 | JAMIA | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Shojonia | 1998 | JAMIA | Experimental | 2 | 2 | 0 | 2 | 2 |
8
|
Steele | 2005 | American Journal of Preventive Medicine | Experimental | 0 | 1 | 0 | 2 | 2 |
5
|
Thomas | 1999 | J Med Internet Res | Experimental | 2 | 1 | 0 | 1 | 2 |
6
|
Tierney | 2003 | JGIM | Experimental | 2 | 1 | 2 | 2 | 2 |
9
|
Turner | 1994 | Arch Intern Med | Experimental | 2 | 1 | 2 | 2 | 1 |
8
|
Williams | 1998 | Arch Fam Med | Experimental | 2 | 1 | 0 | 2 | 2 |
7
|
Zanetti | 2003 | Infection control and hospital epidemiology | Experimental | 2 | 2 | 2 | 2 | 2 |
10
|
Statistical Analysis
-
Positive Effect Prevalence, calculated as the proportion of studies showing a positive effect of CCG on the total of selected studies.
-
Negative Effect Prevalence, calculated as the proportion of studies not showing any or negative effect of CCG on the total of selected studies.
-
publication year, using 1999 (after publication of Shifman's article) as a cut-off year (1994-1999; 2000-2006);
-
design of the study: observational and experimental studies;
-
quality of the study, using 7 as cut-off score (5-7; 8-10)
Results
Variables | Countries | |||
---|---|---|---|---|
Europe | USA | Oceania | ||
Study design
| Observational | 2 (33.3%) | 4 (66.7%) | 0 (0.0%) |
Experimental | 9 (23.1%) | 29 (74.3%) | 1 (2.6%) | |
Type of patients
| Inpatient | 2 (28.6%) | 5 (71.4%) | 0 (0.0%) |
Outpatient | 4 (23.5%) | 12 (70.6%) | 1 (5.9%) | |
Guidelines receivers
| Physicians | 10(28.6%) | 24 (68.6%) | 1 (2.9%) |
Other care givers | 1 (10.0%) | 9 (90.0%) | 0 (0%) | |
Population of study
| Simulated | 1 (25.0%) | 3 (75.0%) | 0 (0.0%) |
Real | 10(24.4%) | 30 (73.2%) | 1 (3.3%) | |
Type of centres involved in the study
| Non--academic | 5 (22.7%) | 17 (77.3%) | 0 (0.0%) |
Academic | 5 (25.0%) | 15 (75.0%) | 0 (0.0%) | |
Number of centres involved in the study
| Multicentric | 7 (36.8%) | 11 (57.9%) | 1 (5.3%) |
Monocentric | 4 (15.4%) | 22 (84.6%) | 0 (0.0%) | |
Type of guideline
| Preventive | 1 (7.1%) | 12 (85.7%) | 1 (7.1%) |
Treatment | 9 (32.1%) | 19 (67.9%) | 0 (0.0%) | |
Both | 1 (33.3%) | 2 (66.7%) | 0 (0.0%) | |
Type of condition
| Acute | 2 (28.6%) | 5 (71.4) | 0 (0.0%) |
Chronic | 6 (22.2%) | 21 (77.8%) | 0 (0.0%) | |
Both | 3 (27.3%) | 7 (63.6%) | 1 (9.1%) | |
Type of intervention
| Test or/and drugs | 7 (29.2%) | 17 (70.8%) | 0 (0.0%) |
Other intervention | 1 (14.3%) | 6 (85.7%) | 0 (0.0%) | |
Both | 3 (21.4%) | 10 (71.4) | 1 (7.1%) |