Results
The analysis identified interacting complexities in the TORPEDO programme which played out differently in different sites and settings. These are presented under the NASSS domains below. Direct quotes from the new primary dataset of ex post interviews are labelled “ex post interview [number]”; quotes from the original TORPEDO dataset are labelled with the original coding notation (e.g. 2282-005, with the first four digits indicating the original study site number and the last three digits indicating the participant number).
The condition
HealthTracker was designed for use in two kinds of patient: those who already had cardiovascular disease and those (usually asymptomatic) who were potentially at high risk of developing it. Established cardiovascular disease is well characterised, and guidelines for its management are relatively uncontested and widely accepted. The evidence base on managing cardiovascular risk is more complex. It is skewed towards a white European and North American population (especially the US Framingham study, on which the HealthTracker algorithm was partially based). Furthermore, since cardiovascular risk is a continuous variable influenced by multiple risk factors such as blood pressure and cholesterol levels, HealthTracker could not offer an unambiguous binary categorisation of patients into “high risk” or “low risk”.
In Australia (as elsewhere), cardiovascular disease is strongly patterned by socio-demographic factors: it is commoner in those who are poor, those with low health literacy, and in Aboriginal people. Such individuals are more likely to have comorbidities such as diabetes or mental health conditions (“High risk is a sort of multifarious set of component conditions”—ex post interview 2). They may also have cultural beliefs and practices that affect their ability and willingness to understand the risk communication and comply with preventive treatment.
The risk communication tool in HealthTracker worked well for many patients (“I found the patient education information just great, it was just wonderful.”—Nurse in TORPEDO study, 2368-005). But it assumed that patients would be able to understand a visual representation of quantitative risk and make a rational decision to alter their lifestyle based on it. This was not always the case, partly because of numeracy (“I find that they’re [absolute risk percentage] harder to explain to the patient with a number. So we need to go back and look at how to translate into the number needed to treat. But that’s very hard concept for the patient to understand at the moment too.”—GP in TORPEDO study, 2290-001), and partly because of competing priorities in complex lives (“[HealthTracker] works for people that have structured lives … but some patients are much less organised, and they have social and other medical problems … which interfere with their ability … to accept and to seek out systematic care”—ex post interview 4).
The TORPEDO findings confirmed that cultural habits die hard and familiar folk models of illness and risk may over-ride less familiar epidemiological ones [
34]. A 66-year-old patient with a family history of cardiovascular disease and adverse clinical and lifestyle factors commented:
... it’s no good saying we’ll change your lifestyle. I’m 66 years old. I have a lifestyle, you know. I’m not an alcoholic. I don’t over-drink. I don’t you know I don’t overeat. I’m just, just a big bloke. Look, I didn’t walk out of there thinking, oh, I’m only going to eat salad and, you know, drink water for the rest of my life.—Interview with patient in TORPEDO study [
31]
The “no symptoms, no problem” mindset helps explain why patients with established cardiovascular disease appeared to engage better than those flagged for primary prevention (“I have this difficulty convincing patients that they should be on medication when the indication is only based on high risk … ‘Doctor, but I haven’t got the problem now so why do you want to give me the medication’ and, of course, medications are not cheap”—GP in TORPEDO study, 2308-001).
The technology
HealthTracker had many attractive design features (“[GPs] loved it, … loved the traffic light [which] was simple, [and] loved seeing the graphs, looking at the heart age over time”—ex post interview 1). Many valued the way it structured care (“HealthTracker reminds me what needs to be followed, to be checked and followed. So it wasn’t so much telling me what the guidelines are, it was telling me what I needed to do to ensure that their health, everything’s been covered”—GP in TORPEDO study, 2303-001). But GPs described technical glitches (such as when data on the patient’s record did not appear in the HealthTracker viewer) that were frustrating and interfered with their use of the technology in real time. Many found themselves regularly on the phone to the helpdesk. A major concern was that “apps that would just chew up memory, make the EMR [electronic medical record] run slowly; people said, ‘I don’t want to have anything to do with this thing, because it’s making my existing work flow worse’” (ex post interview 2).
Despite its visual appeal, “the user design wasn’t very good” (ex post interview 3), and in retrospect, findings suggest that it was not fit for purpose. It was not easy to integrate HealthTracker into existing workflows and practices for quantifying risk, advising patients, and prescribing medication. HealthTracker appeared in a side bar on the GP’s screen with pop-up prompts, “and sometimes prompts would go up, [or] wouldn’t; some of them wouldn’t see it because they would [mistakenly] shut it off, [and] … would say, oh, it’s gone, I don’t know where it is.” (ex post interview 1). Because of these technical imperfections, GPs participating in the TORPEDO study soon divided into highly motivated and/or technically adept ones, who persisted with the technology, and the rest, who gave up on it (“We tried to fix it and it didn’t work, then I just stopped doing it, yeah. We never knew why, I don’t know if it is the software because we tried many times … It’s never worked.”—part-time GP in TORPEDO study, 2290-003).
HealthTracker’s inbuilt algorithms foregrounded “hard” risk data (biometrics, family history) at the expense of “softer” data (e.g. on personal and cultural context) that could have informed a more individualised approach to care [
27]. This was a conscious design feature, but it helps explain why, glitches aside, different GPs had very different levels of use of the tool (see domain 4).
The TORPEDO project team sought to facilitate adoption by ensuring from the outset that HealthTracker was able to integrate with more than one electronic record system. Whilst it covered only two such systems, they amounted to 80% of the Australian market. But the integration was only one way: “nothing from HealthTracker populated into the EMR; [only] the reverse occurred” (ex post interview 3). This meant that the risk score and management plan did not automatically populate the patient’s record—a feature that contributed to clinicians’ experience of “clunkiness”.
The value proposition
HealthTracker was developed in a university setting by publicly funded research. There were two implicit potential models for introducing it into Australian primary care. The technology could be sold directly to GP practices or paid for by or through government entities, e.g. Primary Health Networks (PHNs), which are the organisations responsible for planning and commissioning primary care services, one for each of 31 geographically defined locations across Australia; and Medicare, which is the publicly funded universal healthcare system in Australia. The value proposition varied accordingly. The Australian government is prioritising digital health initiatives (see “
The wider system” section). To government as a third-party payer, the potential value of
HealthTracker would be “quality of care, [a] better performing health system, reduced inefficiency, better use of medicines, … reduction of morbidity and mortality, and no unintended safety consequences” (ex post interview 2). In addition, there was hope among the TORPEDO team that
HealthTracker would support a shift towards a more prevention-oriented healthcare system. One researcher commented that the Australian primary healthcare system is designed to be “… reactive, not proactive, and what we’re trying to do [by introducing HealthTracker] is to graft on some extra things that make it more proactive” (ex post interview 4).
The TORPEDO team also anticipated that the value to Primary Health Networks (which at the time were known as Medicare Locals) would be in the form of improved workflow and easier audit and performance management within GP practices. A modelled cost effectiveness analysis showed a small but statistically significant reduction in clinical risk factors within a PHN population based on the TORPEDO trial data, suggesting a small economic benefit from preventing CVD events (paper submitted). The economic evaluation showed that if HealthTracker were to be scaled up to a larger population, the intervention has potential to prevent major CVD events at under AU$50,000 per event averted. However, at the PHN level, investment decisions for commissioning similar interventions based on cost-effectiveness analyses are scant.
The heavy burden of preventable and costly cardiovascular disease, particularly in Aboriginal communities, made the value proposition particularly compelling for community leaders. One researcher recalled, when recruiting the Aboriginal Community-Controlled Health Services to the TORPEDO trial, “it was just so pressing how – every one of the [community] board members, either themselves or relatives, knows someone who’s died of heart disease, or stroke, or diabetes, or kidney diseases; it’s just absolutely everywhere” (ex post interview 2).
Some individual GPs shared this perspective, viewing HealthTracker in positive value terms as supporting better (more proactive) care and making it easier and quicker to follow evidence-based guidelines and monitor their own performance. They felt it could potentially save them time “because it got all sorts of information out of the medical record and told you what otherwise you have to go hunting for” (ex post interview 5). Because HealthTracker synthesised several guidelines so as to streamline decision-making in patients with multi-morbidity, it saved considerable time sourcing individual guidelines.
But this would generate value only for GPs who were committed for professional reasons to delivering guideline-informed care, since
HealthTracker increased overall consultation length [
33]. The conversation triggered by the risk visualisation tool could sometimes be lengthy (“the thing is, it’s not time to run the programme, it’s time to actually chat to the patient. So if you’re going to go through all this, you’ve got to be prepared to have a good 10 minute chat with the patient because you actually want to engage them and help them understand where they’re at and make a difference and that’s the time.”—GP in TORPEDO study, 2282-005). Since the Australian payment system predominantly rewards GPs on a fee-for-service basis rather than (say) incorporating a pay-for-performance scheme (as in the UK Quality and Outcomes Framework [
35]), the technology could be viewed as having negative financial value for GPs, especially given the many technical “bugs”, which could be time-consuming to resolve.
The value of HealthTracker to patients was complex and varied for different individuals and communities. For example, only a minority of patients valued the focus on prevention and future health gain: “patients who are not at high risk, who are motivated and got high health literacy, are the minority, [while] the majority of patients at high risk have got multiple problems and need much more hands-on working” (ex post interview 4). And some GPs had commented that using HealthTracker would increase their professional status in the eyes of current or potential patients—“that you’re a 21st century doctor and you’re doing the right thing” (ex post interview 4).
However, patients’ main priority when choosing a GP was not always quality of care delivered. For example, in ethnically diverse areas of Sydney, “[a large proportion] of the GPs consult in a language other than English, people find the same language, same culture GPs, … that’s what people are looking for, they’re not necessarily looking for them following guidelines” (ex post interview 4). Some patients were driven predominantly by material needs. The Australian copayment system meant that out-of-pocket payments for a GP consultation could be $30–$50, which might place negative financial value on additional medication and GP appointments triggered by a HealthTracker focused consultation.
The intended adopters
In the TORPEDO study, at least 1 GP responded to a survey in 21 of the 30 intervention sites; of these, fewer than one third said they used
HealthTracker for more than half of eligible patients, even though most expressed positive attitudes to the technology (e.g. they considered it easy to use, valued the data it generated, and felt it helped improve the quality of care) [
32].
Their reasons for limited adoption were complex; they included technical issues described in domain 2. Those aside, HealthTracker’s potential to prompt the screening of asymptomatic patients for cardiovascular disease risk was “a low hanging fruit that most GPs were happy to engage with and could see that was an important thing to do” (ex post interview 2). This partly explains why the intervention arm of the TORPEDO RCT showed significant improvement in process measures (measuring and documenting risk factors). But despite this, there was no significant improvement in prescribing preventive drugs in the TORPEDO trial. At the level of individual consultations, GPs may have been taking account of the (often complex) socio-cultural factors described in domain 1 when judging whether to use HealthTracker at all and (if they did) whether to follow the algorithm’s recommendations.
Limited adoption of
HealthTracker was also, TORPEDO researchers hypothesised, because there was a mismatch between the recently published recommendations for primary prevention of cardiovascular disease inscribed in the software (based on formal guidelines) and more intuitive prevailing assumptions about what was good practice (based on collectively shared practical wisdom known as
mindlines [
36]). For example, GPs may have withheld medications because of anticipation of poor adherence or history of non-adherence. Also, negative media reports about statins at the time of the study [
37] may have made some GPs more cautious, especially when managing patients who were high risk but without established disease. TORPEDO data showed that whilst statin prescription increased among those with a diagnosis of cardiovascular disease, it fell for those without such a diagnosis: “I think there was a huge drop in the prescription for statin … Lipitor came into the news around that time. … And I quickly had a look and realised that yes, I did reduce the prescription of the statin … [and] the prescription for high blood pressure group may have dropped at the same time, not just the statin.” (GP in TORPEDO study, 2290-001).
HealthTracker also appeared to exert what one researcher called “psychic costs” in the form of anxiety induced by a red light which alerted GPs to recommendations they did not follow [
38]. GPs felt they were being marked down and expressed along the following lines: “don’t tell me to do something when I’ve made an active decision in discussion with my patient to not do it, don’t keep giving me a red traffic light” (ex post interview 3). In contrast, a GP researcher who was part of the TORPEDO team said: “I found it very useful, every time I saw a patient, I’d open the
HealthTracker and have a quick squizz, and make sure that there were no red indicators anywhere” (ex post interview 5).
The organisation(s)
Not all practices invited to participate in the TORPEDO study chose to do so. And among the studies that participated in the trial, 15 declined to participate in the post-trial study because of the following: the service was closing or moving (4 practices), concerns that
HealthTracker would slow down their computer system (3), limited resources (3), changing to an incompatible electronic record system (3), already using another cardiovascular risk tool (1), and lack of interest (1) [
32]. In other words, the practices which declined to participate may have had significant organisational-level issues to report, and findings from the practices which
did participate may not reflect all those issues.
There was wide variation in participating practices’ underlying capacity to innovate. Technical infrastructure was sometimes poor, increasing the likelihood of technical crashes (“some practices don’t tend to change their hardware very often, or let it upgrade very often, so you’re trying to run sophisticated new software on older machines”—ex post interview 5). Some larger GP practices and Aboriginal Community-Controlled Health Services (ACCHSs) had “been engaged in quality improvement work very strategically for about 15 years [and] already had an operational structure that they could weave [HealthTracker] into” (ex post interview 2). In some, there was a dedicated individual focused on audit and quality improvement (“we can report to them that, you know, for example only 30% of the high-risk patients are being prescribed with triple therapy and they go, whoa.”—Health information officer (ACCHS) in TORPEDO study, 2282-001). Notably, some of the more confident larger practices sought a high degree of autonomy over how and when HealthTracker was used.
Larger practices sometimes also had an on-site IT support person or technically adept practice manager who could troubleshoot problems and coordinate remotely with the developers. At the other end of the spectrum were small, poorly resourced practices, who “had less experience doing this sort of thing, [and] probably needed a bit of arm twisting to sign up” (ex post interview 2). In extreme cases, the practice was not even able to install the software. More commonly, a “series of cascading negative things [could] then lead to complete abandonment”.
Whilst practice size was to some extent a proxy for capacity to innovate, the latter was also influenced by the practice’s governance structure [
39,
40]. In small one- or two-doctor practices, decision-making was generally very streamlined. In a typical GP practice, quality improvement is commissioned by Primary Health Networks (PHNs) and practices are facilitated to conduct audits of their electronic medical records and provide de-identified data to the PHN. Each PHN is governed by a board, but there are hundreds of GP practices within a PHN region. Thus, the owner of a small practice was a GP who was essentially the CEO and the provider, such that “once you’ve engaged the principal or principals, and if they’re taken with the idea, then they’ll just do it” (ex post interview 4). This also explained why small practices could sometimes (albeit relatively rarely) overcome capacity disadvantages (“I’m probably taking about 90% of the data cleaning here, in this surgery.”—GP in TORPEDO study, 2290-001).
In larger organisations, several levels of governance were involved. In ACCHSs, for example, there were three tiers of decision-makers: “[The first tier is] community elected broad members, … the next tier is about senior management support for it, that’s the CEO and their senior level staff, and then the next tier is the providers or clinicians. …. We wouldn’t be able to work with any service without having all three of those processes in place” (ex post interview 2). Whilst strategic-level actors tended to make decisions on the basis of population disease burden and likely long-term benefit, operational-level actors appeared to be more concerned about short-term costs and workload implications and the factors discussed in domain 4.
Larger GP practices required greater coordination and aligned governance structures to facilitate the organisational change that was necessary for adoption, and this depended on competing priorities and staff continuity (especially in training practices with a high turnover of registrars). There was sometimes a mismatch of priorities between the “entrepreneur” GP (or, occasionally, a practice manager), who made the decision to sign up for the trial and embraced the technology with enthusiasm, and other staff (fellow GPs and most practice managers) whose engagement was often much lower. As the TORPEDO researchers discovered, “when you sign on a [large] GP practice … usually agreed by the lead GP who may be enthusiastic about intervention, … it really needs all the GPs to be committed and want to use it” (ex post interview 3).
Larger practices had a more diverse and distributed workforce. Potentially, this could reduce the cost of adoption of
HealthTracker, for example, if nurses rather than doctors undertook the risk assessment (as happens routinely in the UK [
41]). But large practices typically have a clear division of labour (with formal job descriptions, for example), so optimal embedding of new technologies may require revision of roles and routines and regular retraining. In some cases,
HealthTracker work could not be sustained if a key member of administrative staff was absent. Given the high staff turnover in larger practices, community health workers (e.g. Aboriginal health workers, who already undertook some screening and health education tasks) could potentially “spend more time explaining to [patients] what it [the HealthTracker data] was all about, talking to them about lifestyle changes, their medication, why they need to be on them, how they could continue taking them and supporting them to do that” (ex post interview 4). Unfortunately, use of
HealthTracker could not be easily incorporated into community health workers’ role in some ACCHSs for several reasons including lack of access privileges, low health worker confidence in use of computers, perceived time constraints, low GP confidence in health workers, and governance issues (“they weren’t given the green light by the head of the board”—ex post interview 1).
Variation in capacity to innovate (a phenomenon we have documented previously in GP practices involved in complex intervention trials [
42]) raised the question of whether and how much to support each GP practice to implement
HealthTracker during the TORPEDO trial and subsequent real-world implementation. This was partly for cost reasons (“it would have taken an extra couple of years [of planning] and another million dollars or something; it’s not cheap to do this kind of stuff”—ex post interview 4) and partly because of concerns that too much external support would limit the external validity of the findings. For these reasons, TORPEDO researchers decided to implement the intervention in a more or less standardised way.
Some of these findings, based on the NASSS framework, are resonant with those of an earlier theorisation using normalisation process theory, which identified four key influences on the routinisation of
HealthTracker in participating practices: organisational mission and history (e.g. strategic investment to promote a culture of quality improvement), organisational leadership (e.g. ability to energise staff), team environment (e.g. extent to which team members with different skill sets worked in complementary ways), and technical features of the tool (covered in domain 2) [
33].
The wider system
HealthTracker was not classed as a medical device so did not require regulatory approval. Technology vendors saw regulation as a two-edged sword. On the one hand, lack of regulatory hurdles meant that it was easier to get them to market. On the other hand, achieving regulatory approval, had it been required, would have given the vendor an advantage over competitors.
The TORPEDO team was keen to create an institutional environment that would promote the use of HealthTracker by GP practices. They sought to position HealthTracker nationally so that it could generate revenue for GPs and GP practices in the future.
For example, they sought to maximise the chance that professional bodies supported and endorsed its use: “We made a decision very early on that that we would just use [existing] guidelines, whether or not we agreed with the guidelines” (ex post interview 3). This strategy was based on the assumption that if the guidelines emanated from professional societies, most physicians would accept them as reasonable. They had anticipated a potential scale-up platform through the Royal Australian College of General Practitioners (RACGP) and had selected the technology developer because of its existing relationship with RACGP (“we were somewhat lured into the attraction of working with them [the developers], because they’d signed this partnership with the College of GPs … to make this software available to all 20,000 members of the College of GPs”—ex post interview 2). However, RACGP subsequently discontinued this partnership because of negative feedback from its members, especially in relation to the tool slowing down practice systems. Even though RACGP had a long history of endorsing clinical practice guidelines, they did not endorse HealthTracker to their members. This was partly because “when it comes to endorsing software, that’s a relatively new space for them; [they] approached it like a guideline, … and missed the point that we weren’t trying to create a new guideline; we were trying to implement existing guidelines” (ex post interview 2).
By targeting an institutional level higher than professional organisations (i.e. government), the TORPEDO team sought to alter the rules that govern recognition and reimbursement of the use of software in delivering health services more broadly. The team had initially sought to list the use of HealthTracker on the Medicare Benefits Schedule (MBS), the government-subsidised health services, given Australia’s fee-for-service remuneration model for GPs. But this approach stalled initially: “we put in a submission to the federal government only to be told eventually that from a legislative viewpoint, MBS items can’t be attached to software” (ex post interview 3).
PHNs have the mandate to facilitate quality improvement programmes as part of their work, with dedicated staff to support that work, though such programmes do not tend to be focused on particular technologies. The TORPEDO researchers hoped to use the results of the trial “to drive the decision-making process a little bit more rationally” (ex post interview 2). This was particularly important at the time, given the absence in Australia of other quality incentives to promote proactive care for people at risk of cardiovascular disease. Without such incentive programmes, or the ability to bill patients or insurers for using HealthTracker and similar software, the chances of widespread adoption and scale-up of HealthTracker are probably limited.
The TORPEDO researchers built inter-organisational communication and networking into the study design. It is well established that complex innovation in healthcare is facilitated when different organisations communicate with one another, share experiences, and resources, and progress a shared vision of what they are collectively trying to achieve—perhaps using the quality improvement collaborative model [
17]. As Dixon-Woods et al. found, inter-organisational communication and collaboration conveys strong normative pressure to engage with the programme and improve performance to match that of others [
24].
The Australian Primary Care Collaborative (APCC) had been established in 2005; it involved over 4000 health professionals from over 2000 services across the country, with a principal goal of improving access and chronic disease care [
43]. This initiative was running in parallel with the TORPEDO study and achieved some improvements in quality of care and clinical outcomes [
43]. The TORPEDO team worked with the APCC group, using the APCC web platform for reporting peer-ranked data, and running joint workshops and webinars aimed at GP practices and ACCHSs. But the uptake of these efforts was variable and restricted to GP practices that were already experienced in the quality improvement collaborative approach [
33]. Those practices aside, inter-organisational communication and networking was limited. Some of the TORPEDO team reflected on the tension between the RCT design (assumed to be a controlled experiment of a fixed intervention) and the more iterative approach encouraged in quality improvement:
there’s always the challenges of the RCT design, the side of you that you sort of test fixed ingredients or pills, and you don’t change things, adapt things over time. So I think if we had a different kind of design in evaluating this, it would have been more of that kind of cyclical adaptation over time, constantly reiterating, modifying our intervention, potentially taking it into different areas as we started to build a sort of community of practice, and I think all of those things are as important—ex post interview 2
Adaptation over time
The TORPEDO study began in 2008, so this analysis allowed us to assess how HealthTracker, and the organisations seeking to support its use, had evolved and adapted over time. As noted in the previous section, a desire to keep the intervention fixed to meet the standards of the RCT design existed in tension with the need to make local adaptations to improve its embedding.
One challenge for practices was maintaining staff skills in the face of high turnover or flagging commitment. GPs who used HealthTracker only sporadically tended to forget the content of the training. Some practices found that it was necessary to retain “someone on the ground who is familiar with the tool inside out and with the IT infrastructure, who can coordinate with the developers” (ex post interview 1). Such support implies a recurrent cost, to be borne by Primary Health Networks or GP practices (or, within the context of the study, by the TORPEDO research group). Another factor that reduced sustainability of HealthTracker was the limited ability of the software vendor to respond technical difficulties by adapting the technology. It took around 2 years after the TORPEDO implementation study ended for them to release the next generation of the software (which GPs claimed still had “bugs”). This lack of agility had a negative impact on adoption. The TORPEDO team subsequently moved the development of HealthTracker in-house to a technology spinoff of their host research institute.
The limited interoperability of HealthTracker with other technical systems (see domain 2) was viewed by TORPEDO researchers as problematic in the context of more integrated clinical workflows within primary care and a national policy decision to increase interoperability between primary, secondary, and tertiary care. Some researchers felt that to make the technology more sustainable, it would need to develop the functionality to exchange information between systems rather than simply calculate and visualise risk. They considered that unless HealthTracker becomes fully integrated into the electronic record, it will inevitably have to compete with other third-party add-ons, as “ … one player in a very congested space, competing for that crowded real estate on the GP’s screen” (ex post interview 2). To address this challenge, researchers suggested expanding the number of conditions for which HealthTracker could be used: “if HealthTracker is … for just one condition, you might get a few people to use for a little while, but … if it could be developed for a whole range of interventions that might be sustainable … [for example if it had] multiple uses … like a Swiss Army Knife, … so that it looked the same and did similar things” (ex post interview 4). The counter-argument is that additional functionality would increase both technical and operational complexity and likely generate new problems elsewhere in the system.
An opportunity recently emerged to adjust financial incentives. In 2018, the entire Medicare Benefits System programme was undergoing a review (commenced in 2015), and an application for listing (not specific to HealthTracker) was made to create item numbers around performing risk assessment and management. A similar submission was recently also made to the Medical Services Advisory Committee, which advises the Australian government on which new medical services should receive public funding. As of April 2019, interim MBS items (to be reviewed over the next 2 years) have been introduced to allow GPs and non-specialist physicians to conduct a heart health check that lasts at least 20 min. This recent development has potential to shift the value proposition (see domain 4) for HealthTracker to make GPs’ use of the technology worthwhile.
Whist TORPEDO researchers were upbeat about the potential for increasing uptake of HealthTracker via such national-level levers, they acknowledged that “ … regulating clinical practice is difficult … ultimately, it’s always going to be optional, [as] the doctor can always say, I didn’t have time, I wasn’t interested, it didn’t seem like the right patient” (ex post interview 5). They also recognised that technologies generally do not have universal appeal: “Some people would [be interested], some people might not, it’s the same as almost any other thing, some practices have a spirometer and some don’t” (ex post interview 5). And that if the choice on whether to adopt HealthTracker (or not) was left to individual GPs or GP practices, uptake would likely be slow, because GPs may only realise that the technology was helpful after they had started using it. Purchase by GP practices or Primary Health Networks in such a scenario would depend on price and competing third-party software.
Two changing features of the governance structure of Australian general practice may influence adoption of HealthTracker in the future. First, it is possible that Primary Health Networks will start to provide significant direct support to GP practices to implement quality improvement initiatives, though HealthTracker may or may not be prioritised in this move. Second, with the growth of corporatised GP practice chains, more practices will have key staff such as a practice manager, IT lead, and quality improvement lead. But as the TORPEDO team found in their experience with larger GP practices, buy-in from the CEO of such corporatised practices does not guarantee that front-line clinicians will use the tool.
Another potentially positive development on the horizon is policy support for new digital health initiatives. Whilst Australia has included digital health in strategic documents since around 2005, in 2017, the first National Digital Health Strategy was released. It named several relevant goals to be achieved by 2022: (1) digitally enabled care models to improve accessibility, quality, safety, and efficiency of care; (2) workforce confidently using digital health technologies; and (3) high-quality data with a common understood meaning that can be used with confidence [
44]. However, there is still perceived to be a mismatch of investment decisions and activities needed at the organisational and adopter levels to address identified gaps in healthcare delivery and their links to improved population outcomes.
In sum, whilst there are some positive trends, there remains a high degree of uncertainty about how the fortunes of HealthTracker, both locally and nationally, will unfold in the future.
Conclusion
The NASSS framework, originally developed to explain the fortunes of health technology projects in real time, can be applied retrospectively to generate a rich, contextualised narrative of a technology-supported change effort and the numerous interacting influences on its successes, failures, and unexpected events. A NASSS-informed ex post analysis, drawing on the principles of complex systems, can supplement earlier contemporaneous evaluations to uncover emergent interactions and interdependencies that were not fully knowable or predictable at the time.
Whilst it is widely recognised that technology implementation in healthcare requires a judicious mix of “top-down” [
47], “bottom-up” [
48], and “middle-out” approaches [
49], the literature still lacks rich exemplar case studies of how such approaches may dovetail (or not) in practice. Whilst not the only way to approach complexity in technology implementation, NASSS can be used to generate multi-level accounts that incorporate the target health condition(s), the technology, the adopter system (patients, providers, managers), the organisational elements, and the broader system enablers (policy, financing, etc.). Explaining in rich detail why past programmes succeeded or failed potentially allows us to learn from history and improve the design of future programmes.
We are currently extending the NASSS framework alongside a complexity assessment tool (CAT) for use as an ex ante tool for planning, managing, and evaluating complex technology projects in health and social care. Further details of the NASSS-CAT tool are available from the corresponding author.