Decision maker perceptions of resource allocation processes in Canadian health care organizations: a national survey

Ninety-two (92) replies were retained for analysis: 88 complete and 4 substantially complete (at least 80%, or 18 of the main 22 questions answered in full) – 80 in English and 12 in French. We achieved 34.4% of the ideal target (92/267), or 37.7% of the adjusted target (92/244); response rate among all managers contacted across the four waves of data collection (n=410) was 22.4%.

At least one reply was received from every province or territory (see Table 1). Geography is a potential proxy for many organizational features, such as socio-economic or political context, or institutional aspects of the healthcare system that in Canada can vary across jurisdictions due to the provinces’ constitutional authority over healthcare services. However, we do not have enough responses to carry out meaningful sub-analyses on this dimension.

Sixty out of 89 organizations are represented by at least one respondent (see Table 1). Among respondents, slightly over one-half indicated that their organization’s total annual budget was $500 million or less; 30% reported a budget of greater than $1 billion. The majority of respondents indicated that their budgets had increased over the last three years, whereas about one-fifth of respondents indicated a stable trend and a small proportion (8%) indicated that their organization’s budget had contracted. Based on the researchers’ assessment of population and geographic location, we would suggest that 22% of respondents came from primarily urban organizations (average population of just over 800,000), and 32% from primarily rural or remote organizations (average population of slightly less than 55,000); the remainder of cases would be classed as mixed or indeterminate.

Fifty one percent (51%) of respondents self-identified as carrying out an Operational role; 28% were in Planning portfolios and 21% in Finance roles. About 55% of respondents had an educational background in medicine or another health profession. Responsibilities were related to respondents’ educational background: 80% of Operations, 50% of Planning, but none of Finance senior management team members reported doing their major training in medicine or other health professions. As expected, respondents appear to be quite an experienced group with many years of work in senior management roles, largely within the health system. The mean number of years in senior management was 11.8 (standard deviation 7.0), with a mean of 10.8 years specifically in the health sector (standard deviation 6.9). Respondents on average had been employed by their current organization, or any of its direct predecessors, for 10.6 years (standard deviation 8.9).

About one-half of respondents indicated that their resource allocation process was best described as what we would label a formal one, while about one-quarter deemed it historical and one-quarter felt that resource allocation processes in their organizations were driven primarily by political factors. (See Table 2 for definitions of the different types, developed by the authors for this study—these were presented to respondents in the form of short scenarios).

We then asked respondents to identify whether or not their organization-wide resource allocation process involved up to 11 elements – identified from the literature – which might be characterized as formalization (See Figure 1). As a group, those who indicated that their organization used primarily a formal process selected on average a larger number of these features: 3.8 compared to 2.8. This suggests that practice within organizations with a ‘formal’ process in fact may differ from that in other organizations. However, it may be that, due to the sequencing of these questions, respondents who stated they used a formal process may have felt compelled to choose more features as justification for this claim.

The above said, none of the features put forward to suggest formalization were reported by even a majority of respondents. The most common feature, dedicated technical support for resource allocation, was reported by 44% of managers. Somewhat surprisingly, respondents from larger organizations reported less use of formal processes; in organizations with budgets greater than $1 billion, 69% of respondents identified historical or political factors as predominant, while in smaller organizations the majority of respondents (59%) cited formal or rational processes as most characteristic. No pattern was observed for role, budget trend, or turnover variables.

Overall, 89% of respondents agreed that resource allocation decisions were not only business ones, i.e., solely financial or budget matters, but matters of ethical concern. 70% of respondents either agreed or strongly agreed with the statement: “Our organization-wide resource allocation process is fair”. Sixteen percent disagreed while 15% were neutral (“neither agree nor disagree”) (see Figure 2). Interestingly, respondents do not appear to think that other stakeholders are equally confident about the fairness of the process. Forty-two percent felt that other key stakeholders would agree that the process is fair, while one-quarter expected that outside observers would consider the organization-wide resource allocation process to be unfair.

As described above, we used the A4R framework to embed questions about concrete resource allocation practices which might be seen as normatively fair. Relevance: Among respondents to this survey, 90% agreed that alignment with vision and mission was a criterion which they used in resource allocation decisions, and 65% said the same about congruence between options and the organization’s corporate values. Publicity: Respondents to the survey were asked if decisions and their rationale were communicated to internal and external audiences. Seventy-seven (77%) agreed or strongly agreed that their organization had a plan to communicate results to internal stakeholders, while 48% gave the same rating in respect to the public and other external stakeholders. Enforcement: One survey item asked whether or not participants felt that “agreed-upon process rules and decisions are followed through on and enforced”. 66% of respondents indicated that this was the case; 23% were neutral and 11% disagreed. However, only 21% of respondents indicated that their organizations evaluated either the process or the outcomes of their resource allocation system. Appeals: Only 24 of 92 participants (26%) stated that there was a formal process by which resource allocation decisions might be reviewed or appealed. Empowerment: Among respondents, one-half (50%) agreed or strongly agreed that “affected stakeholders have a formal opportunity to provide input into the decisions being made”; 24% disagreed.

We also consider some of the organizational and individual factors which might account for respondents’ perceptions of process fairness: primary role and type of process (formal or other). Across each of the three primary roles (finance, operations, or planning), approximately two-thirds of survey respondents ‘agreed’ or ‘strongly agreed’ that their process was fair; i.e., there is no obvious pattern here. By contrast, whether or not respondents agreed or disagreed that their process was fair appears to differ depending on whether or not they indicated that formal processes, or historical or political processes, were predominant (see Table 3).

Respondents were asked to give an overall rating of their organization-wide resource allocation process, on a scale of 1 to 5, from very poor to very good. Just over 50% rated the process as good or very good, while 37% rated it as fair. Eleven percent (11%) gave a poor rating, while no one stated that the process was ‘very poor’. The proportion of respondents giving good or poor ratings appears to vary depending upon whether they considered a formal process, or historical and political factors, to be the driving force behind organization-wide priority setting (see Table 3). Respondent role and senior leadership turnover showed no obvious pattern. Those from organizations with $1B+ budgets more commonly gave their resource allocation ratings a process of poor or fair than did those with smaller budgets; this seems consistent with the earlier reported results about the effect of organization size.

Our survey asked about the presence of a range of potential enablers of effective resource allocation, as identified in the literature, e.g., [29] (See Table 4). Note that most respondents either agreed or strongly agreed that their organizations possessed both a learning culture and strong leadership, though we were unable to explore in depth what these concepts might mean. For each case, we calculated a cumulative or total enabler score: Score +1 for presence of enabler; 0 for “neither agree nor disagree”, -1 for absence of enabler. The range of possible scores is then −12 to +12 (number of individual responses valid on all items=85). Briefly, scores ranged from +7 to −5 (median, +3), with 14% of respondents reporting net negative scores on this measure. It appears strongly suggested by scatterplot data that respondents who identified a greater number of enablers more commonly gave higher overall scores to their current resource allocation procedures (see Figure 3). [The graph groups all respondents by their score, -5 to +7, and then presents the average overall rating given by each group].

Both academic opinion, and many assessments of what decision makers view as the ‘ideal’, support the idea of more formalized resource allocation processes in the health sector [16, 20, 37‐39]. In our survey, about half of respondents claimed to observe formal processes of resource allocation at the senior management level. Most indicated that these were self-developed, not directly derived from models in the literature (Figure 1). This is admirable initiative perhaps, yet suggests the existence of a knowledge translation gap, whether that is due to lack of opportunities for researcher-decision maker interaction, or the failure of academics to ‘push’ or managers to ‘pull’ relevant research [51]. Fewer respondents identified the presence of the elements of what we would consider main features of formalization, as derived from the literature. Perhaps there is social desirability bias—respondents feel that their organizations should make decisions on this basis. Or perhaps decision makers’ understanding of a formal allocation process contains element other than those so far identified in the literature; we could not have anticipated or asked about the presence of such features. Further research is required.

We also need to note what seems like a counter-intuitive relationship between organization size and type of process identified in this data. We anticipated that large organizations would more likely have formalized resource allocation processes; the results do not confirm this. It may be that large organizations are exposed to more diverse opinions and more open political contestation around decisions. Another possible explanation could be that it is easier to obtain consensus among stakeholders in smaller organizations to implement a particular form of resource allocation, including formal processes if desired. On a positive note, we might take this to mean that there are no systemic barriers which prevent organizations of any size from being able to establish practices that enable high performance in resource allocation.

Our respondents recognized the ethical nature of their resource allocation practice; only 11% agreed that they were simply making business decisions. This contrasts with more than 40% in a US survey (from which we derived this question) [52]. It was notable that respondents, taken as a whole, seemed to suspect that outside observers would perceive their organization-wide resource allocation process as less fair than they themselves do. Further data would be required as to what may account for this discrepancy. It might for instance be mediated by views about the effectiveness of organizational communication. Recent research with healthcare organizations in one Canadian province has found a strong link between perceived fairness and perceived transparency [41]. There is some evidence that decision makers may hold different preferences in regard to priority areas for spending than members of the public or healthcare providers [53, 54] which might influence what the groups believe to be fair or unfair about the processes currently being used.

The use of A4R elements as survey items provides a way of checking respondent claims about overall fairness against the presence or absence of specific features which are often considered to exemplify fair process. If adherence to A4R principles constitutes fair process (which is an assumption rather than a given), then we should be somewhat cautious in accepting the self-reported claims about fairness made by our respondents. That is, claims about fairness may not be supported based on the extent to which features thought by researchers to facilitate fairness are actually reported as present by these respondents and described above. Some literature has suggested that decision makers are less attached to the importance of formal appeals mechanisms than to other aspects of A4R [55]. Our data appear consistent with this. This leaves interesting questions as to whether it is the attitudes and beliefs of decision makers, or the theory of what makes ethical practice, which needs to change in order to bring about alignment here.

We seem to have received generally positive responses from our survey participants (assuming that one takes a grading of ‘fair’ to be a positive response). Decision makers no doubt are working hard to do their best, within constraints. But very few nominated themselves as ‘high performers’, indicating awareness of much work yet to do. The findings related to barriers and enablers support the importance of factors previously identified in the priority setting literature as contributing to or restricting effective organizational performance in resource allocation [29].

As with many surveys, it is possible to question the representativeness of the respondents and the generalizability of responses. However the broad range of responses from many different organizations across the country gives us some degree of confidence that we have tapped a diverse set of informants. Response rate is not out of ordinary for an email survey [56]; the large-scale UK surveys noted above had response rates of 23% [26] and 53% [27]. Given that informants are very busy executives who were approached without advance notice, we are not unhappy with the return. We presume (but cannot be sure) that response was provided by the intended informant, and not delegated to someone else.

A further potential limitation is that we certainly have not measured all the relevant variables. The strength and quality of organizational leadership, for one, may account for whether or not organizations choose to implement formal resource allocation processes, the nature of such processes, their fairness, and perceptions among team members of success. We included one question which touched on this (Table 4) -- most respondents reported that their senior management teams displayed strong leadership. However, we know that leadership is a multi-dimensional concept, though we were unable to disaggregate it here. Reeleder et al. identify some of the needed skills as creating relationships, managing networks, building supportive coalitions, and mobilizing support [57]. Dickinson et al. suggest that among these functions “a number .. tend to be weak, or absent, from leadership skills of health care professionals” [58].

In addition, internal consistency in responses from respondents from the same organization is perhaps best described as ‘fair’ (for ordinal questions, inter-rater reliability calculated with Spearman’s rank correlation coefficient averaged 0.580; for nominal questions, Cohen’s kappa averaged 0.274). This is not necessarily surprising, as previous research has noted the members of senior executive teams can have strikingly different perceptions of organizational processes [59]. Role within the organization may lead to different experiences and vantage points to account for this –though we tested it in our study and did not find it to be an explanatory factor. A host of other factors may also be at play. We only included senior managers in this survey. We might well expect senior managers as a group to believe, for instance, in the fairness of processes which they may have helped craft. Those in mid-level management positions may not necessarily perceive organizational resource allocation processes and results in the same terms, but further research is needed to address this.