What can management theories offer evidence-based practice? A comparative analysis of measurement tools for organisational context

The context of managing the knowledge base for healthcare is complex. Healthcare organizations are composed of multi-level and multi-site interlacing networks that, despite central command and control structures, have strong front-line local micro-systems involved in interpreting policy direction [1]. The nature of healthcare knowledge is characterized by proliferation of information, fragmentation, distribution, and high context dependency. Healthcare practice requires coordinated action in uncertain, rapidly changing situations, with the potential for high failure costs [2]. The public sector context includes the influence of externally imposed performance targets and multiple stakeholder influences and values, the imperative to share good practice across organisational boundaries, and a complex and diverse set of boundaries and networks [3]. Having strong mechanisms and processes for transferring information, developing shared meanings, and the political negotiation of action [4, 5] are therefore crucially important in public sector/healthcare settings, but it is not surprising that there are reports of problems in the organizational capacity of the public sector to effectively manage best practice innovation [6‐11], particularly around issues of power and politics between different professional groups [12‐17].

The development of capacity to implement evidence-based innovations is a central concept in UK government programmes in healthcare [18]. Strategies to improve evidence-based decision making in healthcare have only recently shifted emphasis away from innovation as a linear and technical process dominated by psychological and cognitive theories of individual behaviour change [19], toward organisational level interventions [20], with attention shifting toward the development of inter-organisational clinical, learning, and research networks for sharing knowledge and innovation [21‐23], and attempts to improve capacity for innovation within the public sector [24].

Organisational capacity refers to the organisation's ability to take effective action, in this context for the purpose of continually renewing and improving its healthcare practices. Absorptive and receptive capacities are theorized as important antecedents to innovation in healthcare [25]. Broadly, the concept of absorptive capacity is the organization's ability to recognise the value of new external knowledge and to assimilate it, while receptive capacity is the ability to facilitate the transfer and use of new knowledge [26‐31]. Empirical studies have identified some general antecedent conditions [32‐34], and have tested application of the concept of absorptive capacity to healthcare [35, 36], although receptive capacities are less well studied. Empirically supported features of organisational context that impact on absorptive and receptive capacities in healthcare include processes for identifying, interpreting, and sharing new knowledge; a learning organisation culture; network structures; strong leadership, vision, and management; and supportive technologies [25].

Public sector benchmarking is widely promoted as a tool for enhancing organisational capacity via a process of collaborative learning [37]. Benchmarking requires the collation and construction of best practice indicators for institutional audit and comparison. Tools are available to measure the organizational context for evidence-based healthcare practice [38‐41], and components of evidence-based practice (EBP) including implementation of organisational change [42‐45], research utilization (RU) [46], or research activity (RA) [47]. While organisational learning (OL) and knowledge management (KM) frameworks are increasingly being claimed in empirical studies in healthcare [48‐53], current approaches to assessing organisational capacity are more likely to be underpinned by diffusion of innovation or change management frameworks [54].

Nicolini and colleagues [2] draw attention to the similarity between the KM literature and the discourse on supporting knowledge translation and transfer in healthcare [55‐57], as well as between concepts of OL and the emphasis on collective reflection on practice in the UK National Health Service [58, 59], but suggest that 'ecological segregation' between these disciplines and literatures means that cross-fertilisation has not occurred to any great extent. OL and KM literatures could be fruitful sources for improving our understanding of dimensions of organizational absorptive and receptive capacity in healthcare. We therefore aimed to support the development of a metric to audit the organizational conditions for effective evidence-based change by consulting the wider OL and knowledge literatures, where the development of metrics is also identified as a major research priority [60], including the use of existing tools in healthcare [2].

Definitions of KM vary, but many include the core processes of creation or development of knowledge, its movement, transfer, or flow through the organisation, and its application or use for performance improvement or innovation [61]. Early models of KM focused on the measurement of knowledge assets and intellectual capital, with later models focusing on processes of managing knowledge in organisations, split into models where technical-rationality and information technology solutions were central and academic models focusing on human factors and transactional processes [62]. The more emergent view is of the organisation as 'milieu' or community of practice, where the focus on explanatory variables shifts away from technology towards the level of interactions between individuals, and the potential for collective learning. However, technical models and solutions are also still quite dominant in healthcare [63].

Easterby-Smith and Lyles [64] consider KM to focus on the content of the knowledge that an organisation acquires, creates, processes, and uses, and OL to focus on the process of learning from new knowledge. Nutley, Davies and Walker [54] define OL as the way organisations build and organise knowledge and routines and use the broad skills of their workforce to improve organisational performance. Early models of OL focused on cognitive-behavioural processes of learning at individual, group, and organisational levels [65‐67], and the movement of information in social or activity systems [68]. More recent practice-based theories see knowledge as embedded in culture, practice, and process, conceptualising knowing and learning as dynamic, emergent social accomplishment [69‐72]. Organisational knowledge is also seen as fragmented into specialised and localised communities of practice, 'distributed knowledge systems' [73], or networks with different interpretive frameworks [74], where competing conceptions of what constitutes legitimate knowledge can occur [75], making knowledge sharing across professional and organization boundaries problematic.

While the two perspectives of KM and OL have very different origins, Scarbrough and Swan [76] suggest that differences are mainly due to disciplinary homes and source perspectives, rather than conceptual distinctiveness. More recently, there have been calls for cognitive and practice-based theories to be integrated in explanatory theories of how practices are constituted, and the practicalities of how socially shared knowledge operates [77, 78]. Similarly, there have been calls for integrative conceptual frameworks for OL and knowledge[79, 80], with learning increasingly defined in terms of knowledge processes [81, 82].

Practice models have their limitations, particularly in relation to weaknesses in explaining how knowledge is contested and legitimated [83]. In a policy context that requires clinical decisions to be based on proof from externally generated research evidence, a comprehensive model for healthcare KM would need to reflect the importance of processes to verify and legitimate knowledge. Research knowledge then needs to be integrated with knowledge achieved from shared interpretation and meaning within the specific social, political, and cultural context of practice, and with the personal values-based knowledge of both the individual professional and the patient [84]. Much public sector innovation also originates from practice and practitioners, as well as external scientific knowledge [85, 86]. New understandings generated from practice then require re-externalising into explicit and shared formal statements and procedures, so that actions can be defended in a public system of accountability.

Our own preference is for a perspective where multiple forms of knowledge are recognised, and where emphasis is placed on processes of validating and warranting knowledge claims. Attention needs also be directed towards the interrelationship between organisational structures of knowledge governance, such as leadership, incentive and reward structures, or the allocation of authority and decision rights, and the conditions for individual agency [87‐89]. Our own focus is therefore on identifying the organizational conditions that are perceived to support or hinder organizational absorptive or receptive capacities, as a basis for practical action by individuals.

The indicators for supportive organisational conditions are to be developed by extracting items from existing tools, as in previous tools developed to measure OL capability [90]. Existing tools are used because indicators are already empirically supported, operationalised, and easily identified and compared, and because our primary focus is one of utility for practice [91], by specifying 'the different behavioural and organisational conditions under which knowledge can be managed effectively' [92p ix]. Measurement tools that were based on reviews of the literature in the respective fields of KM and learning organisations were chosen as comparison sources to assess the comprehensiveness of the current tools in healthcare, and to improve the delineation of the social and human aspects of EBP in healthcare. If this preliminary stage proves fruitful in highlighting the utility of widening the pool for benchmark items, future work aims to compare the source literatures for confirming empirical evidence, with further work to test the validity and reliability of the benchmark items.

A structured literature review was undertaken to collate measurement tools for organisational context from the domains of research use or RA in healthcare, or for KM or OL in the management or organisational science literature.

Thirty tools were identified and appraised [see Additional file 1]. Based on the inclusion criteria for tool development and testing, 18 tools with 649 items in total were selected. These are listed in Table 2, with information on development and psychometric testing [see Additional File 2] The number of the tool from Table 2 will be used in subsequent tables.

In total, 261 items related to organisational context were extracted from the measurement tools. For two tools [105, 106], the full text of each item was not available, so the names of the categories of measurement for which results were reported were used as items, e.g., organisational climate for change.

The importance of understanding context has been reiterated by the High Level Clinical Effectiveness group [18]. This project was developed in response to perceived limitations in the conceptualisation and measurement of organisational context for EBP in healthcare. We wanted to move away from the rather narrow focus on RU and change management to include wider process and practice-based perspectives from the KM and OL literature. Our analysis of existing measurement tools has confirmed differences in emphasis across the domains. Measurement tools for RA and RU focus more on access to new information, leadership, and resources for change, and less on recognizing, valuing, and building shared knowledge. This is congruent with the culture of 'rationality, verticality, and control' [[6] p660] in healthcare, but the lack of attention to social context may be one reason why attempts to improve practice by influencing the behaviour of individual practitioners have variable results [111].

The emphasis in KM and OL tools on shared vision, learning culture, and sharing existing knowledge reflects a more socially mediated view of knowledge. If it is groups and networks that generate the meaning and value to be attached to evidence, organisational efforts to improve EBP would need to do more to shift towards supporting horizontal knowledge transfer. Networks have emerged as a recent UK government strategy for moving health research into action by creating clusters that break down disciplinary, sectoral, and geographic boundaries, but communication structures alone are unlikely to be successful for knowledge transfer across specialized domains [4, 112] without additional mechanisms to support the transfer of practice and process knowledge [73].

Since this search was conducted, three additional tools to measure organisational context in quality improvement related areas have been reported [113‐115]. Each of these tools has strengths, including attributes such as feedback that are not included in our model, but none have comprehensive coverage of all of the attributes identified in this study.

A potential weakness in using existing tools as sources in this study is that they might not reflect the latest theories and concepts, because tool development tends to lag behind conceptual development. This might result in inadvertent bias towards earlier more technical models, and we acknowledge that the existing tools do largely adopt a structuralist perspective. While the items contained in the existing measurement tools can only ever provide a rather simplistic reflection of complex phenomena, we felt that including them was better than not attempting to express them at all. However, while new perspectives are worth investigating, the unquestioning interdisciplinary transfer of theory also needs care. Compared with the public sector, there are differences in the types of problems, the availability of information and resources, and the motivations for evidence uptake and use. KM theory supposes an identified knowledge need, scarce information, and a workforce motivated by external incentive in a resource-rich environment. EBP on the other hand requires compliance with externally produced information for predominantly intrinsic reward, with high innovation costs in a resource-limited environment. A number of studies have identified some of the difficulties of knowledge sharing in the public sector [1, 6, 11, 16, 48, 116, 117]. Organisational theory may not transfer well into healthcare if EBP is viewed as a process of social and political control to promote compliance with centrally derived policy, rather than a generative process to make best use of available knowledge.

Assessing organisational absorptive and receptive capacity with the aim of improving organizational conditions is postulated as a first step in supporting a research informed decision-making culture. Foss [87] suggests the emergence of a new approach referred to as knowledge governance: the management of the mechanisms that mediate between the micro-processes of individual knowledge and the outcomes of organisational performance. But what would this mean in practice? The kinds of support which KM and OL tools include as standard, but that are not well reflected in existing tools to measure context in healthcare, would include effort to detect and support emergent and existing communities of practice; encourage and reward individuals and groups to ask questions; discuss and share ideas across knowledge communities; and support the progression, testing, and adopting of new ideas by embedding them in systems and processes.

The processes by which individual- and group-level knowledge are collated into organisational level capability to improve care are less clear. If social networks of individuals are to be facilitated to undertake repeated, ongoing, and routine uptake of evidence within their daily practice, we also need to extend our thinking even further toward considering the organisational contextual features that would support the collective sense-making processes of key knowledge workers.