The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review

Economically-focused literature were those articles whose specific focus was on either studying or discussing QI economic benefits. This made up 15 articles of the 68 QI literature. Amongst these were four conceptual literature, (three of which were business case discussions) [22, 36, 81, 82], four QI evaluation frameworks [83‐86], two systematic reviews of economic evaluations [87, 88], one article discussed cost effectiveness of QI programmes [89], one article calculated an organisation’s QI related cost-savings [90], and two were economic evaluations [91, 92]. One article discussed cost–benefit analysis [93]. Of the total 15 articles, ROI was a specific subject of only four [81, 84, 87, 91].

There were three economic evaluations [89, 91, 92], and two systemic reviews of QI economic evaluations [87, 88]. By the virtue of their study foci, their measure of ROI was the monetary. These authors saw savings as an important QI outcome, however also discussed a wide range of QI benefits. For example, Crawley-stout et al. [91] considered internal outcomes (e.g., cost-reduction, productivity, and time savings) and external benefits (e.g., patient costs and carer time). Crawley-stout et al. described ROI as a performance measure used to evaluate investment efficiency in financial terms. de la Perrele et al. reported a lack of QI economic evaluations in their review. They concluded that collaboratives are potentially cost-saving. However, they found that studies used variable methods to assess cost and effectiveness, and that studies did not report negative findings. They recommended that future research should include societal perspectives of costs and savings [88]. Banke-Thomas et al. stated that SROI (a societal version of ROI) can be used across healthcare [87]. However, there were challenges with inadequate skills for ROI evaluation, lack of credible financial proxies, a lack of consensus on; who to include as beneficiaries, how to account for counterfactual and appropriate study-time horizon [87].

These made up 53 [94‐147] of the selected 68 articles. These articles included QI effectiveness, process, and impact evaluations as well as discussions of QI achievements over time e.g., [94, 110, 116, 117, 130, 138]. These articles did not focus on ROI or economic measures, but nonetheless highlighted financial outcomes as important benefits for consideration. Some QI implementation studies assessed their implementation costs [109, 140, 145], as part of their study reporting guidelines [78, 148]. Authors here also discussed improving QI effectiveness determinants such as staff and safety culture development. Authors discussed or mentioned financial value or benefit, financial returns or outcomes, cost savings, reduction, containment, and economic impact, as well as productivity, efficiency, value, and benefits. Of these, cost-saving was the most frequently used term. These articles considered ROI as one of many organisational outcomes [103, 130, 135].

There were three QI evaluation frameworks [103, 104, 124]. These frameworks also considered various elements of organisational benefits. Chow-Chua and Goh [103] combined existing organisational performance tools; the Singapore Quality Award (SQA) model (modelled after Baldrige Award) and balanced scorecard (BSC) to develop a performance and quality improvement evaluation framework for hospitals. Four strategic components were seen as crucial: the drivers of QI (e.g., leadership), internal performance, knowledge management, and QI outcomes. McLees et al.’s [124] framework for QI in public health was described as a performance management tool, and focused on two key constructs: efficiency and effectiveness. Ciarniene et al.’s [104] framework envisioned broad value creation through QI.

Morganti et al. [126] remarked that there is a lack of an agreed concept of QI success, and by extension, ROI. This was seen in how authors gave priority to certain outcomes. Van den Heuwel et al. [141] for an example, referred to quality improvement as business improvement, viewed ROI quantitatively, and expected quality improvement to be a valuable “side effect” of value improvement (often a euphemism financial improvement). Alternatively, others proclaimed to value patient safety and quality first, and saw financial matters as the valuable “side-effect” of QI [85]. Hunter et al. [116] considered “cost savings or increased efficiency “helpful by-products” (p. 129).

Swensen et al. [86] QI business case discussion stated that their QI investment decisions were never based purely on positive ROIs but on broader qualitative considerations. A similar view was held by O’Sullivan et al. [130] and Shah and Course (2020). Bailit and Dyer [83], advocated for broad business cases that embrace different rationales for QI investment. Fischer and Duncan [36] stated that some interventions are purely designed to produce health outcomes. They also called for a broader QI outcome definition that acknowledges the utility of differing projects and value for all stakeholders. The review also indicated that even failed goal attainment can be useful in providing insights and legacies like building capacities and safety cultures [94, 110, 116, 117, 130, 138, 146].

Overall, financial outcomes were not the primary or commonly sought goal or addressed outcome. However, it was seen as directly or indirectly significant by the majority of the authors. The perception that QI is an expense used for revenue generation was seen to be due to faulty assumptions by some authors [86]. For example, authors suggested that profit-seeking through QI first emerged as an optional strategy to increase revenue and market-shares by for-profit healthcare organisations [22, 83, 86]. However, grey areas on views existed and views appear to have shifted towards integrating or emphasising non-monetary outcomes over time. The literature also agreed that QI does not always save cost, and financial outcomes are not the only organisational objectives [22, 36, 82, 83, 89, 93]. These literatures portrayed ROI as any value or benefit from QI for various stakeholders.

The reviewed literature illustrated five main ROI uses related to QI: ROI as 1) a strategic business case development tool, 2) an investment performance measure, 3) a comparative evaluation tool, 4) a cost management tool, and 5) a performance management tool. ROI was also used to create fiscal awareness [81]. Some of this ROI use was similar to commercial ROI use. Various concepts were used to denote a return-on-investment. These concepts were used in relation to changes and improvements in various organisational outcomes including patients, staff, financial, and overall organisational development. Both economic and non-economic focused literature used almost identical concepts to denote an investment and a return as seen in Fig. 3.

However, profit, revenue, and market share were mainly found in the few economic focused literature. The majority of literature put greater emphasis on non-monetary QI benefits. This indicates that although different logics were applied in the conceptualisation of healthcare QI-ROI, the dominant logic was that of health and social care and not economics or markets.

At face value, there were two broad QI-ROI philosophies: the economic and the healthcare ROI philosophies. However, our synthesis indicated a merging meaning of ROI from different schools of thought. Through markets logic, the economic philosophy perspective views outcomes in terms of their tangible, quantitative, or financial offerings. The economic philosophy is related to managerial logics as managers’ roles require them to monitor organisational financial performance. Alternatively, the healthcare perspective suggested a more dulled though increasing financial focus. Healthcare logics (medical, societal) tended to view ROI qualitatively, primarily from a patient and staff perspectives but also wider internal and external stakeholders. The integrated literature indicated that healthcare leaders value these perspectives too. Table 4 illustrates this finding.