Background
Methods
Development of the evidence-based medicine questionnaire
The evidence based medicine questionnaire (EBMQ)
Section | Description | No. of items | Domain | Type of data | Response options | Response combined for analysis |
---|---|---|---|---|---|---|
A | Demographic profile | 6 | NA | Nominal scale | ||
B | Frequencies in looking for medical information | 20 | NA | Nominal scale | ||
C | Knowledge regarding evidence-based medicine | 17 | Knowledge regarding information sources | 4-point Likert scalea | 1 = Unaware 2 = Aware but not used in clinical decision making 3 = Have read it but not used in clinical decision making 4 = Read and used in clinical decision making | |
16 | Knowledge regarding terms related to EBM | 5-point Likert scalea | 1 = Never heard this term before 2 = Heard of this term but do not understand what this term but would like to 3 = Do not understand this term but would like to 4 = Have some understanding of this term 5 = Understand this term well and able to explain what it means to others | 1 = Never heard and do not understand 2 = Do not understand but would like to 3 = Understand | ||
D | Practice of evidence-based medicine | 9 | Practice | 5-point Likert scalea | 1 = Strongly disagree 2 = Disagree 3 = Neither agree nor disagree 4 = Agree 5 = Strongly agree | 1 = Disagree 2 = Neutral 3 = Agree |
E | Barriers in practicing evidence-based medicine | 13 | Barriers | 5-point Likert scalea | 1 = Strongly disagree 2 = Disagree 3 = Neither agree nor disagree 4 = Agree 5 = Strongly agree | 1 = Disagree 2 = Neutral 3 = Agree |
F | Needs for evidence-based medicine | 3 | Needs | Nominal scale | ||
Total | 80 |
Validation of the evidence-based medicine questionnaire
Participants
Sample size
Procedure
Data analysis
Validity
Flesch reading ease
Exploratory factor analysis
Discriminative validity
Reliability
Internal consistency
Test-retest reliability
Results
n (%) | |
---|---|
Median age [IQR] | 32.2 [4.0] |
Female | 222 (69.4) |
Male | 98 (30.6) |
No. of participants with postgraduate qualifications | |
None | 251 (78.4) |
Diploma | 58 (18.1) |
Masters | 11 (3.4) |
Current designation | |
Medical Officer | 311 (97.2) |
Family Medicine Specialist | 9 (2.8) |
Current Work Place | |
Government health clinics | 174 (54.4) |
Private clinic | 81 (25.3) |
Government hospital | 42 (13.1) |
Othersa | 13 (4.1) |
Private hospital | 5 (1.6) |
University hospital | 5 (1.6) |
Have heard of the term “evidence-based medicine” | 319 (99.7) |
Have attended EBM courses | 222 (69.7) |
Have received formal trainings in literature search | 156 (48.8) |
Have received formal trainings in questions formulation | 121 (37.8) |
Have received formal trainings in critical appraisal | 111 (34.7) |
Have conducted research after graduating from medical school | 111 (34.7) |
Have published any article in a journal | 36 (11.3) |
Validity
Original domains | After EFA was performed | Item No. | Item | Factor 1 | Factor 2 | Factor 3 | KMO | AVE (%) | Bartlett’s test | CR |
---|---|---|---|---|---|---|---|---|---|---|
Knowledge | Evidence-based medicine websites (n = 6) | C6 | Centre of Evidence-Based Medicine (CEBM) | 0.605 | – | – | 0.834 | 43.0 | < 0.001 | 0.662 |
C10 | ACP Journal Club | 0.583 | – | – | ||||||
C5 | Database of abstracts of reviews of effectiveness (DARE) | 0.550 | – | – | ||||||
C13 | InfoClinics | 0.545 | – | – | ||||||
C2 | Bandolier (published in Oxford) | 0.495 | – | – | ||||||
C14 | Centre of Reviews & Dissertation | 0.477 | – | – | ||||||
Evidence-based journals (n = 2) | C12 | BMJ Clinical Evidence | – | 0.665 | – | 0.500 | 48.9 | < 0.001 | 0.609 | |
C3 | Evidence-Based Medicine (from BMJ publishing group) | – | 0.658 | – | ||||||
Type of studies (n = 4) | K3 | Case-control study | 0.654 | – | – | 0.692 | 49.7 | < 0.001 | 0.685 | |
K4 | Randomized controlled trial | 0.632 | – | – | ||||||
K1 | Systematic review | 0.622 | – | – | ||||||
K2 | Meta-analysis | 0.459 | – | – | ||||||
Terms related to EBM (n = 12) | K13 | Publication bias | – | 0.956 | – | 0.896 | 52.0 | < 0.001 | 0.884 | |
K11 | Confidence interval | – | 0.817 | – | ||||||
K12 | Heterogeneity | – | 0.745 | – | ||||||
K16 | Clinical effectiveness | – | 0.642 | – | ||||||
K7 | Odds ratio | – | 0.607 | – | ||||||
K8 | P-value | – | 0.589 | – | ||||||
K15 | Positive predictive value | – | 0.569 | – | ||||||
K14 | Test sensitivity and specificity | – | 0.553 | – | ||||||
K10 | Number needed to treat | – | 0.531 | – | ||||||
K9 | Level of evidence | – | 0.524 | – | ||||||
K6 | Absolute risk | – | 0.436 | – | ||||||
K5 | Relative risk | – | 0.416 | – | ||||||
Practice (n = 9) | Practice (n = 8) | P4 | EBM improves my patient care | 0.829 | – | – | 0.892 | 49.0 | < 0.001 | 0.882 |
P7 | EBM guides my clinical decision making | 0.817 | – | – | ||||||
P8 | I prefer to manage patients based on EBM | 0.759 | – | – | ||||||
P3 | Reading research papers is important to me | 0.739 | – | – | ||||||
P6 | I can implement EBM in my clinical practice | 0.727 | – | – | ||||||
P2 | I trust the findings from research studies | 0.662 | – | – | ||||||
P5 | EBM reduces my workload | 0.521 | – | – | ||||||
P1 | I support EBM | 0.456 | – | – | ||||||
Barriers (n = 13) | Access (n = 6) | B4 | I have time to practise EBM in my clinic | 0.686 | – | 0.818 | 36.8 | < 0.001 | 0.774 | |
B5 | My clinic facilities are adequate to support the practice of EBM | 0.675 | – | – | ||||||
B3 | I have time to read research papers | 0.633 | – | |||||||
B6 | Research articles are easily available to me | 0.632 | – | – | ||||||
B1 | I am able to assess the quality of research. | 0.543 | – | – | ||||||
B2 | I have access to internet to practice EBM | 0.435 | ||||||||
Patient preferences (n = 2) | B8 | My patients prefers me to practise EBM | – | 0.754 | – | |||||
B9 | My patient believes in information that is based on evidence | – | 0.754 | – | 0.500 | 56.8 | < 0.001 | 0.725 | ||
Support (n = 2) | B12 | My colleagues support the practice of EBM | – | – | 0.786 | |||||
B13 | My organization supports the practice of EBM | – | – | 0.786 | 0.500 | 61.7 | < 0.001 | 0.764 |
Discriminative validity
Item | Details of item | With EBM training (n = 222) n(%) | Without EBM training (n = 98) n(%) | Chi-square | p-value | ||||||
Unaware | Aware but not used in clinical decision making | Have read it but not used in clinical decision making | Read and used in clinical decision making | Unaware | Aware but not used in clinical decision making | Have read it but not used in clinical decision making | Read and used in clinical decision making | ||||
Knowledge Domain (Information sources related to EBM) | |||||||||||
C2 | Bandolier | 154(69.4) | 35(15.8) | 19(8.6) | 14(6.3) | 66(67.3) | 13(13.3) | 12(12.2) | 7(7.1) | 1.350 | 0.717 |
C5 | DARE | 155(18.9) | 42(18.9) | 16(7.2) | 9(4.1) | 71(72.4) | 15(15.3) | 12(12.2) | – | 6.510 | 0.089 |
C6 | CEBM | 123(56.3) | 69(31.1) | 17(7.7) | 11(5.0) | 58(59.2) | 22(22.4) | 14(14.3) | 4(4.1) | 5.074 | 0.166 |
C10 | ACP | 147(66.2) | 38(17.1) | 24(10.8) | 13(5.9) | 63(64.3) | 16(16.3) | 14(14.3) | 5(5.1) | 0.825 | 0.844 |
C13 | InfoClinics | 152(68.5) | 44(19.8) | 18(8.1) | 8(3.6) | 63(64.3) | 16(16.3) | 10(10.2) | 9(9.2) | 4.946 | 0.176 |
C14 | CRD | 175(78.8) | 31(14.0) | 15(6.8) | 1(0.5) | 75(76.5) | 12(12.2) | 9(9.2) | 2(2.0) | 2.564 | 0.464 |
C3 | EBM | 10(4.5) | 46(20.7) | 66(29.7) | 100(45.0) | 4(4.1) | 28(28.6) | 24(24.5) | 42(42.9) | 2.577 | 0.462 |
C12 | BMJ | 26(11.7) | 43(19.4) | 73(32.9) | 80(36.0) | 10(10.2) | 25(25.5) | 22(22.4) | 41(41.8) | 4.442 | 0.218 |
Item | Details of item | With EBM training (n = 222) n(%) | Without EBM training (n = 98) n(%) | Chi-square | p-value | ||||||
Never heard and do not understand | Do not understand but would like to | Understand | Never heard and do not understand | Do not understand but would like to | Understand | ||||||
Knowledge Domain (Terms related to EBM) | |||||||||||
K1 | Systematic review | 6(2.7) | 12(5.4) | 204(91.9) | 8(8.2) | 8(8.2) | 82(83.7) | 5.975 | 0.050 | ||
K2 | Meta-analysis | 10(4.5) | 14(6.3) | 198(89.2) | 6(6.1) | 14(6.3) | 198(89.2) | 16.837 | ≤ 0.001* | ||
K3 | Case-control study | 4(1.8) | 8(3.6) | 210(94.6) | 5(5.1) | 8(3.1) | 90(91.8) | 2.746 | 0.253 | ||
K4 | Randomized controlled trial | 4(1.8) | 7(3.2) | 211(95.0) | 4(4.1) | 1(1.0) | 93(94.9) | 2.651 | 0.266 | ||
K5 | Relative risk | 8(3.6) | 25(11.3) | 189(85.1) | 9(9.2) | 16(16.3) | 73(74.5) | 6.287 | 0.043* | ||
K6 | Absolute risk | 8(3.6) | 33(14.9) | 181(81.5) | 10(10.2) | 15(15.3) | 73(74.5) | 5.699 | 0.058* | ||
K7 | Odds ratio | 11(0.5) | 60(27.0) | 151(68.0) | 14(14.3) | 26(26.5) | 58(59.2) | 8.395 | 0.015* | ||
K8 | P-value | 11(5.0) | 38(17.1) | 173(77.9) | 14(14.3) | 19(19.4) | 65(66.3) | 9.004 | 0.011* | ||
K9 | Level of evidence | 7(3.2) | 30(13.5) | 185(83.3) | 9(9.2) | 19(19.4) | 70(71.4) | 7.686 | 0.021* | ||
K10 | Number needed to treat | 11(5.0) | 42(18.5) | 170(76.6) | 11(11.2) | 20(20.4) | 67(68.4) | 4.640 | 0.098* | ||
K11 | Confidence interval | 17(7.7) | 61(27.5) | 144(64.9) | 21(21.4) | 25(25.5) | 52(53.1) | 12.502 | 0.002* | ||
K12 | Heterogeneity | 21(9.5) | 74(33.3) | 127(57.2) | 22(22.4) | 35(35.7) | 41(41.8) | 11.709 | 0.003* | ||
K13 | Publication bias | 23(10.4) | 65(29.3) | 134(60.4) | 23(23.5) | 30(30.6) | 45(45.9) | 10.703 | 0.005* | ||
K14 | Test sensitivity and specificity | 4(1.8) | 25(11.3) | 193(86.9) | 11(11.2) | 13(13.3) | 74(75.5) | 14.172 | ≤ 0.001* | ||
K15 | Positive predictive value | 5(2.3) | 36(16.2) | 181(81.5) | 5(2.3) | 36(16.2) | 181(81.5) | 7.415 | 0.025* | ||
K16 | Clinical effectiveness | 10(4.5) | 48(21.6) | 164(73.9) | 16(16.3) | 19(19.4) | 63(64.3) | 12.738 | 0.002* | ||
Item | Details of item | With EBM training (n = 222) n(%) | Without EBM training (n = 98) n(%) | Chi-square | p-value | ||||||
Disagree | Neutral | Agree | Disagree | Neutral | Agree | ||||||
Practice Domain | |||||||||||
P1 | I support EBM | 1(0.5) | 8(3.6) | 213(95.9) | 2(2.0) | 6(6.1) | 90(91.8) | 2.941 | 0.230 | ||
P2 | I trust the findings from research studies | 1(0.5) | 37(16.7) | 184(82.9) | 3(3.1) | 13(13.3) | 82(83.7) | 4.216 | 0.121 | ||
P3 | Reading research papers is important to me | – | 25(11.3) | 197(88.7) | 4(4.1) | 20(20.4) | 74(75.5) | 14.511 | 0.001* | ||
P4 | EBM improves my patient care | – | 19(8.6) | 203(91.4) | 3(3.1) | 8(8.2) | 87(88.8) | 6.862 | 0.032* | ||
P5 | EBM reduces my workload | 22(9.9) | 87(39.1) | 113(50.9) | 21(9.4) | 39(17.5) | 38(17.1) | 8.838 | 0.012* | ||
P6 | I can implement EBM in my clinical practice | 2(0.9) | 23(10.3) | 197(88.7) | 3(1.3) | 16(7.2) | 79(35.5) | 4.537 | 0.103 | ||
P7 | EBM guides my clinical decision making | – | 11(5.0) | 211(95.0) | 3(3.1) | 9(9.2) | 86(87.8) | 9.130 | 0.010* | ||
P8 | I prefer to manage patients based on EBM | 2(0.9) | 36(16.2) | 184(82.9) | 3(3.1) | 25(25.5) | 70(71.4) | 6.235 | 0.044* | ||
Barriers Domain | |||||||||||
B1 | I am able to assess the quality of research. | 35(15.8) | 76(34.2) | 111(50.0) | 17(17.3) | 40(40.8) | 41(41.8) | 1.871 | 0.392 | ||
B2 | I have access to internet to practice EBM | 4(1.8) | 18(8.1) | 200(90.1) | 11(11.2) | 12(12.2) | 75(76.5) | 15.573 | < 0.001* | ||
B3 | I have time to read research papers | 25(11.3) | 93(41.9) | 104(46.8) | 22(22.4) | 39(39.8) | 37(37.8) | 7.142 | 0.028* | ||
B4 | I have time to practise EBM in my clinic | 18(8.1) | 60(27.0) | 144(64.9) | 144(64.9) | 17(17.3) | 32(32.7) | 8.545 | 0.014* | ||
B5 | My clinic facilities are adequate to support the practice of EBM | 47(20.2) | 85(38.2) | 90(40.5) | 34(15.3) | 34(15.3) | 120(54.0) | 6.935 | 0.031* | ||
B6 | Research articles are easily available to me | 50(22.5) | 71(32.0) | 101(45.5) | 40(40.8) | 29(29.6) | 29(29.6) | 12.447 | 0.002* | ||
B8 | My patients prefers me to practise EBM | 28(12.6) | 138(62.2) | 56(25.2) | 15(15.3) | 59(60.2) | 24(24.5) | 0.424 | 0.809 | ||
B9 | My patient believes in information that is based on evidence | 35(15.8) | 95(42.8) | 92(41.4) | 11(11.2) | 47(48.0) | 40(40.8) | 1.391 | 0.499 | ||
B12 | My colleagues support the practice of EBM | 13(5.9) | 84(37.8) | 125(56.3) | 12(12.2) | 36(36.7) | 50(51.0) | 3.922 | 1.141 | ||
B13 | My organization supports the practice of EBM | 12(5.4) | 72(32.4) | 138(62.2) | 8(8.2) | 33(33.7) | 57(58.2) | 1.038 | 0.595 |
Reliability
No. | Items | Test-Retest Reliability | ||||||
---|---|---|---|---|---|---|---|---|
Corrected Item-total Correlation | Cronbach’s alpha if items is deleted | Test (n = 320) | Retest (n = 184) | ICC | ||||
Mean (SD) | Median | Mean (SD) | Median | |||||
Knowledge Domain | ||||||||
C2 | Bandolier (Published in Oxford) | 0.487 | 0.811 | 1.31 (0.650) | 1.00 | 1.68 (1.003) | 1.00 | 0.567 |
C5 | Database of abstracts of reviews of effectiveness(DARE) | 0.630 | 0.778 | 1.54 (0.916) | 1.00 | 1.67 (0.922) | 1.00 | 0.485 |
C6 | Centre of Evidence-Based Medicine (CEBM) | 0.630 | 0.777 | 1.44 (0.769) | 1.00 | 1.76 (0.937) | 1.00 | 0.453 |
C10 | ACP Journal Club | 0.570 | 0.791 | 1.62 (0.844) | 1.00 | 1.55 (0.886) | 1.00 | 0.333 |
C13 | InfoClinics | 0.566 | 0.791 | 1.58 (0.907) | 1.00 | 1.63 (0.913) | 1.00 | 0.418 |
C14 | Centre of Reviews & Dissertation (CRD) | 0.650 | 0.780 | 1.52 (0.863) | 1.00 | 1.48 (0.815) | 1.00 | 0.396 |
C3 | Evidence-based medicine (EBM) | 0.492 | – | 1.52 (0.863) | 3.00 | 3.21 (0.881) | 3.00 | 0.416 |
C12 | BMJ Clinical Evidence | 0.492 | – | 3.23 (0.868) | 3.00 | 2.90 (0.997) | 3.00 | 0.379 |
K1 | Systematic review | 0.774 | 0.866 | 4.19 (0.775) | 4.00 | 4.23(0.814) | 4.00 | 0.421 |
K2 | Meta-analysis | 0.718 | 0.887 | 2.79(0.516) | 3.00 | 4.10(0.793) | 4.00 | 0.463 |
K3 | Case-control study | 0.826 | 0.848 | 2.91(0.373) | 3.00 | 4.28(0.681) | 4.00 | 0.497 |
K4 | Randomized controlled trial | 0.777 | 0.866 | 2.93(0.346) | 3.00 | 4.37(0.686) | 4.00 | 0.522 |
K5 | Relative risk | 0.747 | 0.927 | 2.77(0.535) | 3.00 | 4.04(0.741) | 3.00 | 0.450 |
K6 | Absolute risk | 0.763 | 0.926 | 2.74(0.554) | 3.00 | 4.01(0.775) | 4.00 | 0.561 |
K7 | Odds ratio | 0.742 | 0.926 | 2.58(0.634) | 3.00 | 3.82(0.822) | 4.00 | 0.506 |
K8 | P-value | 0.713 | 0.927 | 2.67(0.616) | 3.00 | 4.00(0.803) | 4.00 | 0.487 |
K9 | Level of evidence | 0.721 | 0.927 | 2.75(0.538) | 3.00 | 4.06(0.846) | 4.00 | 0.359 |
K10 | Number needed to treat | 0.676 | 0.929 | 2.67(0.599) | 3.00 | 3.95(0.943) | 4.00 | 0.528 |
K11 | Confidence interval | 0.757 | 0.926 | 2.49(0.699) | 3.00 | 3.78(0.882) | 4.00 | 0.529 |
K12 | Heterogeneity | 0.663 | 0.930 | 2.39(0.713) | 3.00 | 3.54(0.950) | 4.00 | 0.483 |
K13 | Publication bias | 0.686 | 0.929 | 2.42(0.729) | 3.00 | 3.58(0.997) | 4.00 | 0.580 |
K14 | Test sensitivity and specificity | 0.697 | 0.928 | 2.79(0.512) | 3.00 | 4.24(0.734) | 4.00 | 0.504 |
K15 | Positive predictive value | 0.707 | 0.928 | 2.74(0.522) | 3.00 | 4.06(0.861) | 4.00 | 0.503 |
K16 | Clinical effectiveness | 0.667 | 0.929 | 2.63(0.630) | 3.00 | 2.89(0.938) | 4.00 | 0.570 |
Practice Domain | ||||||||
P1 | I support EBM | 0.417 | 0.875 | 2.94 (0.279) | 3.00 | 4.43 (0.648) | 4.00 | 0.605 |
P2 | I trust the findings from research studies | 0.618 | 0.854 | 4.02 (0.683) | 4.00 | 4.09 (0.611) | 4.00 | 0.323 |
P3 | Reading research papers is important to me | 0.684 | 0.846 | 4.06 (0.687) | 4.00 | 4.06 (0.679) | 4.00 | 0.477 |
P4 | EBM improves my patient care | 0.765 | 0.838 | 4.18 (0.642) | 4.00 | 4.27 (0.626) | 4.00 | 0.301 |
P5 | EBM reduces my workload | 0.499 | 0.877 | 3.44 (0.898) | 3.00 | 3.43 (0.830) | 3.00 | 0.532 |
P6 | I can implement EBM in my clinical practice | 0.682 | 0.846 | 4.04 (0.661) | 4.00 | 3.90 (0.743) | 4.00 | 0.532 |
P7 | EBM guides my clinical decision making | 0.748 | 0.841 | 4.18 (0.607) | 4.00 | 4.10 (0.600) | 4.00 | 0.344 |
P8 | I prefer to manage patients based on EBM | 0.699 | 0.844 | 4.01 (0.713) | 4.00 | 4.02 (0.689) | 4.00 | 0.422 |
Barrier Domain | ||||||||
B1 | I am able to assess the quality of research. | 0.472 | 0.747 | 2.31 (0.736) | 2.00 | 3.34 (0.808) | 3.00 | 0.475 |
B2 | I have access to internet to practice EBM | 0.386 | 0.767 | 2.81 (0.497) | 3.00 | 3.83 (0.874) | 4.00 | 0.388 |
B3 | I have time to read research papers | 0.546 | 0.728 | 3.32 (0.803) | 3.00 | 3.29 (0.795) | 3.00 | 0.494 |
B4 | I have time to practise EBM in my clinic | 0.583 | 0.718 | 3.55 (0.810) | 4.00 | 3.45 (0.774) | 4.00 | 0.356 |
B5 | My clinic facilities are adequate to support the practice of EBM | 0.583 | 0.718 | 3.13 (0.894) | 3.00 | 3.30 (2.367) | 3.00 | 0.142 |
B6 | Research articles are easily available to me | 0.547 | 0.731 | 3.16 (0.982) | 3.00 | 3.06 (0.942) | 3.00 | 0.275 |
B8 | My patients prefers me to practise EBM | 0.569 | – | 3.14 (0.798) | 3.00 | 3.24 (0.690) | 3.00 | 0.323 |
B9 | My patient believes in information that is based on evidence | 0.569 | – | 3.29 (0.853) | 3.00 | 3.41 (0.717) | 3.00 | 0.547 |
B12 | My colleagues support the practice of EBM | 0.618 | – | 3.53 (0.795) | 4.00 | 3.53 (0.752) | 4.00 | 0.620 |
B13 | My organization supports the practice of EBM | 0.618 | – | 3.63 (0.756) | 4.00 | 3.53 (0.771) | 4.00 | 0.471 |