Psychometric analysis of the EPB implementation scale
During a psychometric assessment of the EBPIS, we determined the scale’s internal consistency and identified three underlying subscales using a factor analysis. The results indicate that the EBPIS scale can be divided into 3 subscales that assess EBP steps, including Literature Search and Critical Appraisal (α = .80), Knowledge Sharing (α = .83), and Practice Evaluation (α = .74). The mean of each subscale was less than half of the maximum score (7.9/20 Literature Search and Appraisal; 9.2/24 Knowledge Exchange; and 6.0/20 Practice Evaluation), indicating that these EBP activities are performed less than 6 times over 8 weeks. While it may be reasonable to perform some activities at this frequency, such as those on the literature search and appraisal subscale, some activities on the other subscales should be a fundamental component of using EBP with patients. For example, collecting data (i.e. use of assessments and outcome measures), evaluating a care initiative by collecting outcome data, and changing practice on the basis of this data, should be routinely performed in rehabilitation clinics. This expectation is consistent with recommendations made by the United States Institute of Medicine in their 2012 report
Best Care at a Lower Cost, which recommends that health care organizations standardize the administration of assessments (i.e. use the same assessments using standard methods) to improve care delivery, increase transparency of outcomes, strengthen public health, and generate new knowledge [
32].
The EBPIS subscales provide information about the steps in EBP, however, they do not provide information about the required steps to implement EBP or the fidelity in which they are administered. Interestingly, item 1 and 14 are most closely related to implementation of EBP, but they clustered with knowledge exchange in the analysis. Future research on the EBPIS should explore the development of another subscale that reflects implementation activities, such as frequency of using evidence or practice guidelines to guide decisions, adaptation of guidelines, and collaboration with stakeholders to implement EBPs would provide a more comprehensive understanding of steps related to implementation.
Use of evidence-based practices
In this study, we also identified rehabilitation clinicians’ self-reported use and perceptions of EBP within and between clinics in South Eastern Norway, and determined factors that may influence EBP. The EBPIS and its subscale scores indicate that common EBP activities among rehabilitation professionals in Norway are literature search, critical appraisal, and knowledge exchange. The findings in this study are similar to those of Melnyk and colleagues, who determined that nurses most commonly performed critical appraisal (31%) and had informal discussions about research (37%); however, they least commonly used evidence in practice (11%) [
25]. Our findings were also similar to a Norwegian study of nurses which identified that the EBPIS average score was 7.8/72 [
26]. In this study, Stokke and colleagues identified that the majority of EBP activities were performed in the areas of literature search, appraisal, and knowledge exchange and ~ 90% stated they did not systematically evaluate their own practice [
26]. Our findings are consistent with these and other studies, indicating that conceptual use of research is more commonly reported than instrumental use (i.e use of research evidence in clinical practice) [
25,
26,
33].
When assessing the factors associated with use of EBP, significant differences on the EBPIS and the Literature search/Appraisal and Knowledge Exchange subscales were noted between individuals with a Bachelors and Doctorate and a Master’s and Doctorate degree. No differences were noted between individuals with Bachelors and Master’s degrees or between any groups on the Practice Evaluation subscale. Other studies that examined whether differences in nurses’ use of EBP are associated with highest degree have conflicting results. One study found that those with an Associate’s degree scored lowest on the EBPIS and individuals with Doctorate degrees scored the highest [
25], whereas another identified no difference between those with and without a higher level of education [
26]. In a survey of PTs, differences in various aspects of literature search and critical appraisal were demonstrated between those with a baccalaureate as compared to a post-baccalaureate degree and between PTs with entry-level post-baccalaureate (i.e. first professional degree as a masters or doctorate) as compared to an advanced master’s or Doctorate (grouped together) [
20]. None of these studies, however, assessed differences between clinicians with Bachelor’s and Master’s degrees. Research should examine differences in practice that result from obtaining advanced degrees (i.e PhD, EdD, or DHSc). Identifying key components of a curriculum or course that may facilitate increased EPB, including evaluation and change of current practice, is equally important. Professional programs, regardless of degree level, may benefit from adding education and training related to practice evaluation.
As another mechanism of professional development, clinicians often obtain a clinical specialty designation. These data indicate that individuals with specialty designation had significantly higher scores on the EBPIS as compared to those without one. Although significant, the actual difference in total scores (31.7 points vs. 30.4 points) was minimal. Further, no differences in subscale scores were identified. Specialty certification often includes demonstration of advanced knowledge and skills in rehabilitation, therefore, this finding should be investigated further.
Educational programs and organizations that employ rehabilitation providers in Norway should consider additional activities to support clinicians in increasing the instrumental use of EBP with patients. Programs that include multi-component interventions that combine strategies and target barriers to use of a new practice have demonstrated effectiveness [
34]. An example of this type of program is the Physical therapist-drive Education for Action Knowledge translation (PEAK), which included acquiring leadership support and electronic resources, a 2-day EBP training workshop, small group work to adapt research to the local context and create a best practice list (5 months), review the best practice list, and agreement to implement it [
35]. While the baseline mean total EBPIS score was 25.5 (slightly lower than the samples studied in our project), a statistically significant improvement in score was demonstrated immediately after the project conclusion with no changes at a 6 month follow-up [
35]. The EBPIS scores statistically improved after the program, however, a chart audit indicated only one best practice behavior demonstrated a statistically significant change. The data also suggests additional support and knowledge translation processes (i.e. the Knowledge-to-Action framework), may be required to fully support implementation efforts [
18,
36].
As previously discussed, EBPs should be administered with fidelity. Survey respondents indicated there may be variation in delivery of EBPs within each clinic and throughout the region, which indicates an opportunity to improve use of EBPs or the fidelity in which they are delivered may exist. Responses did not vary by hospital with the exception of a small and significant difference between participants at 1 hospital who reported a slightly lower agreement with use of the same outcome measures locally. By streamlining the use of EBPs within and between clinics, fidelity in delivery of EBP will be improved and unwarranted practice variation may be minimized. One mechanism to promote administration of EBPs with fidelity, is through use of a learning health care system (as previously described) which integrate clinical operations, research, and patient engagement, with a robust technology infrastructure [
37]. This would facilitate monitoring of use of EBPs and facilitate robust data collection, analysis and rapid generation of practice-based evidence that has potential to improve the quality of rehabilitation. In order to build an infrastructure that supports the learning health care system, it is critical to systematically review literature, create clinical practice guidelines, adapt these guidelines for local application while defining key elements to ensure fidelity, and measure their use and impact.
Several limitations this study exist. First, we assessed the EBP perceptions of the participants. To identify actual use of EBP, a robust study design that observes practice is necessary. Additionally, other factors may contribute to EBP. Individual characteristics, such as knowledge, skills, and beliefs about of therapeutic interventions may impact adherence to practice recommendations [
38]. Organizational factors such as leadership vision, style and communication may impact adoption [
39]. Research also suggests that policy-makers, who encourage EBP through regulations, infrequently use published evidence to inform decisions [
40]. More research is needed to better understand the contribution of each of these factors to EBP in rehabilitation. The Norwegian translation of the EBPIS has been used in a previous study of Norwegian practice, however, we are unaware of the translation methods used [
26]. Additionally, 8 questions about EBPs were developed specifically for this survey, and were not previously validated. The survey was distributed to individuals who previously signed up for the email distribution list and the newsletter distributed by the Regional Center for Knowledge Translation in Rehabilitation, which may include a select group of individuals who are more interested in EBP than the general population. While social response bias may have also contributed to the outcome of the survey, respondents were aware that data would be de-identified. Given the relatively low score on the EBPIS, it is unlikely that this bias impacted the results. Lastly, various modes of survey distribution were used, therefore, we are unable to calculate an exact response rate.