Analyzing the determinants to accept a virtual assistant and use cases among cancer patients: a mixed methods study

The shortage of professionals in the healthcare sector combined with the COVID-19 crisis, increasing digitalization, connectivity, and focus on patient engagement, raise the need for technological improvements in health services [1]. A virtual assistant (VA) is expected to contribute to these goals. VAs have been implemented recently in healthcare settings, though mostly experimental [2‐4]. According to Keyser et al. [5] a VA is an embodied or disembodied conversational agent that allows a user to communicate with a technical system in natural language with verbal and/or non-verbal communication characteristics through artificial intelligence (AI) [6, 7]. VA is a synonym for chatbot [5], but for clarity we’ll only use the term VA. VAs are scalable, easily accessible, and operate at low cost and at any time [8, 9]. This makes them a promising tool to increase productivity by managing routine administrative tasks, improving care delivery, and engaging patient in managing their health [10]. Some well-known examples of virtual agents are Google Assistant, Amazon Alexa and Apple’s Siri [11]. In healthcare VAs are used to book appointments, share information and even recommend care based on symptoms [12].

VAs have been analyzed in a variety of functions in healthcare and cancer treatment [13]. Firstly, they assist the treatment success, information, and education of users. Tudor Car et al. [13] enlist an overview of studies reporting on conversational agents to deliver remote services for a wide range of diseases. Among others, they are applied to educate users about sexual health, medication and general health inquiries. For instance, a smartphone VA application to optimize the monitoring of older cancer patients and increase the efficiency of the follow-up process was implemented [14]. The study found the implementation feasible in the target population (geriatric oncology, average age 83 years). Support for cancer patients was delivered through the VA Vik that provides information about breast cancer, its side effects, treatment, and practical information such as patient rights. Two other studies found that patients share surprisingly much with the VA. It improved their adherence rate, provided helpful treatment information and support [15, 16]. Likewise, iDecide, a VA delivering similar information about prostate cancer, significantly increased prostate cancer knowledge and self-efficacy in making informed decisions and using technology [17].

Secondly, VAs are used for self-diagnosis. Most are generic, but also include cancer diagnosis. For instance, two separate studies [18, 19] demonstrate how such a symptom checking VA can be developed, evaluated, and applied. In a later comparative study, the application of the above-mentioned VA [18] displayed a diagnostic accuracy similar to human doctors and safer medical triage advice [20]. Likewise, a VA with a high accuracy rate to diagnose sexually transmitted infections was developed [21], bypassing the need to visit a clinic. A widely used VA in China was used to diagnose conditions involving privacy or stigma issues [22]. Another study [23] found that patients used a symptom checker most commonly to understand the cause of their symptoms and saw it as a helpful diagnostic tool receiving useful information. Generally, a VA offering self-diagnosis requires sensitive data and has profound consequences, leading to higher legal requirements [24]. While the high accessibility, accuracy, and anonymity have led to their increased popularity, many lack the functions to assist the whole diagnostic process of a traditional medical examination [25]. With regards to cancer diseases, self-diagnosis VAs were used for genetic counselling in two studies [26, 27] that report a VA’s viability to assess hereditary cancer risk by querying participants’ family history. The systems showed high engagement, possibly reducing the data collection burden for providers and reaching a broad audience due to geographical and temporal accessibility. This is in line with similar studies proofing the usability of VAs to automate the self-anamnesis, i.e., personal medical history collection, while increasing patient’s motivation to participate [28‐31].

Thirdly, VAs improving mental health are highlighted as a separate domain due to their wide use, impact, and relevance for cancer patients, who are susceptible to depression and mood disorders [32]. VAs can improve cancer patients’ wellbeing through mental health assistance and social participation, as studies point to the little time Dutch physicians have for further communication with patients [33, 34]. Up to a third of cancer patients will develop a depression or anxiety disorder and experience problems with daily living, underlining the importance of psychosocial care [35‐37]. Nevertheless, these needs are often unmet in secondary care due to an under-identification of psychosocial problems [35]. As a result, patients lack information, which would help them cope and be more involved [38]. Similarly, in a study of Dutch colon cancer patients, respondents were unhappy with psychosocial care. Surgeons agreed that these issues received little attention due to a lack of time and expertise [39]. The COVID-19 pandemic has exacerbated this, as people living with cancer suffer from an increased risk of developing depression during COVID-19 due to feelings of isolation [40]. Moreover, Dutch breast cancer patients reported a substantial drop in emotional and social functioning due to the pandemic [41].

In a review of existing academic research on VAs in a mental health setting [42], a high overall satisfaction and potential for psychiatric use are identified. The effectiveness for patients with major depressive disorders demonstrates a VA’s feasibility with clinical populations. Patients rated the therapeutic alliance between a VA significantly higher than with a clinician [43]. Others disclosed more sensitive information due to anonymity [44]. The self-diagnosis health app Ada was an efficient diagnostic screening or help function for mental disorders in adulthood with the potential to support diagnosticians [45]. For young adults who have undergone cancer treatment, a VA delivering positive psychology skills was perceived as helpful and reduced anxiety [46]. Nevertheless, common standards of reporting and evaluation of VAs are needed for detailed comparisons [42]. Hence, more evidence for the potential of VAs to improve mental health issues and more robust results are needed [47]. A second (scoping) review on VAs in relation to mental health [48] concluded that current results on practicability, feasibility and acceptance of VAs in health care are promising, but especially these topics require more research.

The adoption of Health Information Systems depends on user perceptions [14, 49]. This implies that for VAs to realize any benefit in health care, patients need to be willing to adopt them and thus have positive perceptions towards them. Here, a holistic conceptual approach is often neglected [50]. To systematize attitudes, several technology acceptance models have been established. Venkatesh et al. [51] compared and integrated eight models resulting in the Unified Theory of Acceptance and Use of Technology (UTAUT). While it is the prevalent framework for healthcare service adoption [22] and has been applied in mobile health [52‐54] and telemedicine [50, 55] acceptance research, among others, only indirect research in the context of cancer treatment, an important domain in healthcare, was found [56]. The model consists of four determinants of intention and usage shown in Fig. 1.

First, performance expectancy relates to the beliefs that using a specific technology will help to attain gains, for instance in managing one’s health. Second, effort expectancy means the ease associated with the use of the technology. Third, social influence refers to the degree to which the participant perceives important others, i.e. the social environment, to advocate using a VA. Fourth, facilitating conditions are the degree to which someone believes organizational and technical resources are in place to assist the use of a technology [51].

A range of studies applied the UTAUT to a patient sample and analyzed electronic or mobile health technology acceptance. For instance, research on the adoption of mobile health services among the elderly [53] found that performance expectancy, effort expectancy, and social influence significantly affected behavioral intention, whereas facilitating conditions had no significant relation to intention. The authors explained this could be due to the setting of a developing country, where the elderly were dependent on their children for support. Moreover, they found that two extensions of the model, technology anxiety and resistance to change, could negatively influence intention. This is in line with previous findings that technology anxiety and resistance to change are negatively associated with the technology acceptance model equivalents of effort expectancy and performance expectancy, respectively [57].

Similarly, computer anxiety was found to act as an antecedent with a negative influence of effort expectancy [50]. The authors aimed to predict the determinants of older users’ acceptance of telehealth services with an extended UTAUT model. Performance expectancy, effort expectancy, facilitating conditions and perceived security as addition had a direct influence on intention. Additionally, the doctor’s opinion showed an indirect effect, whereas social influence was no significant predictor [50]. A recent study analyzing the factors influencing telehealth acceptance in Indonesia [58] found similar results. As above, social influence was not significantly associated with behavioral intention, whereas the doctor’s opinion affected performance expectancy and computer anxiety the effort expectancy.

UTAUT has been validated in the context of internet- and mobile-based interventions [59]. The original predictors of acceptance (performance expectancy, effort expectancy and social influence) were found to be true but did not have a moderating effect on acceptance of internet- and mobile-based interventions. Here, internet anxiety was identified as moderator and predictor. Performance expectancy was found to have the strongest influence on acceptance [59].

The notion of perceived security seems especially important considering the present setting, where patients would entrust the VA with sensitive, private information and may ask for medical advice [50]. Therefore, the UTAUT was extended with insights from trust theory to predict the uptake of an AI-based medical diagnosis support system [22]. Besides performance expectancy, initial trust was found the strongest predictor of the behavioral intention of usage. Although the sample in this study consisted of healthcare professionals limiting its transferability, it seems reasonable that trust and security-related concerns are more important in healthcare applications where risks may be more salient than increases in wellbeing. After all, the relationship between safety and innovation seems to be complementary, as discussed in many healthcare fields [60].

While testing the acceptance of telemedicine equipment, all four factors of the UTAUT (performance expectancy, effort expectancy, social influence and facilitating conditions) were found to predict behavioral intention [55]. Yet, clinicians, as well as patients, were included in the sample. Later, an extended model analyzing the acceptance of VAs in healthcare [61] was developed. It includes newly identified constructs such as privacy risk and trust. However, these findings came from interviews with students who mostly had experience in using VAs.

Considering the variety of significant factors, different samples, cultural backgrounds and researched technologies, it is difficult to derive an extended UTAUT that applies accurately to the present research subject. Cancer treatment differs from general healthcare in several points. Firstly, most people newly diagnosed with cancer are 65 years or older [62], implying possibly different priorities for technology adaption [63]. Secondly, besides known side effects, cancer therapies might accelerate the functional decline and lead to psychological distress even after treatment [14, 62]. Thus, the threatening nature of a cancer diagnosis might result in different expectations towards the scope of a digital support tool. Thirdly, though the formation of tumor specific multidisciplinary teams in many regions has improved the coordination of cancer care, alignment of efforts and communication between the various involved providers is far from perfect [14, 64].

The main conclusion from the breadth of results is that the acceptance model must be modified depending on the specific use and target population.

Many studies analyzing a VA’s acceptance in healthcare deliver mixed results and mostly use a digitally literate student sample with a risk of selection bias [65]. Moreover, a large body of research on acceptance in the mobile health context also failed to apply a theoretical base or framework to guide the identification of relevant drivers [66]. In those studies that did use the UTAUT to analyze patient acceptance, factors were of different importance in each case, and other new elements were identified. The variety of diverse and agent-dependent feedback underlines the need for a tailored design according to the targeted population [65]. Additionally, studies lacked further analysis regarding real-life applications and the actual integration [13].

To summarize, VAs are scalable, easily accessible and operate at low cost at any time [8, 9]. They have been used widely for commercial purposes and have been shown in studies to be beneficial in health care settings. New ways to reach and treat people with cancer are needed. Especially mental health needs during cancer treatment are often neglected and can be hard to fulfil [36, 67]. VAs are a way for health care providers (such as hospitals) to achieve this and acceptance of a VA is a crucial part to achieve adoption [13‐47]. Also, the need for further research on the acceptance of VAs in the actual clinical setting has been brought forward [48].

Therefore, our research applies a framework-based analysis to determine acceptance factors in the setting of cancer treatment. This exploratory research examines the acceptance factors and (value-adding) applications of a VA for cancer patients by applying a the UTAUT framework tailored to cancer patients. The research is conducted in cooperation with Maastro, a radiotherapy clinic in the Netherlands, that provides treatment to approximately 250 cancer patients per day.

This leads to the following research questions:

 

Participants of in-depth interviews were presented with exemplary interactions with a VA. The contents differed according to the three mentioned areas: (1) information and education, (2) self-diagnosis of symptoms, and (3) mental health. Such demonstrations allowed for more specific feedback and insights into the possible added value for a VA for cancer patients.

A self-administered online survey design was chosen for this research due to the multitude of identified variables influencing acceptance, the cost-effectiveness of the design and timely data collection [83].

The initial list of survey measurements was extended with questions that were based on the three additional constructs derived from the interviews. Table 4 provides an overview of the added questions per construct, the study they were derived from and its reliability.

The construct trust was measured by adapting the scale of Chandra et al. [116] from the research context of mobile payments adoption. Items for the resistance to change were adopted from a study that analyzed physician’s resistance to healthcare technology [114]. Self-efficacy was assessed using items from Zhang et al. [102], who adapted the theory of Johnston and Warkentin [117] to evaluate the acceptance of mobile health services.

Former cancer patients and radiotherapists stated various value-adding applications that a hospital can introduce for a VA. Patients considered providing treatment-related information such as treatment-logistics, common side effects and the treatment procedure itself the most promising application of a VA. They expressed mixed feelings about the use of a VA for diagnosis with some patients considering this helpful, while others regarded this a dangerous application. Radiotherapists expected VAs to reduce their workload by deploying a VA to answer common questions from patients about side effects and the treatment procedure. Furthermore, they saw the potential of a VA for acquiring (digitalized) patient reported outcome measures. Both radiotherapists and former patients expected this to offer more or faster reassurance for patients, thus reducing anxiety.

The interviews pointed to performance expectancy and effort expectancy as the key factors that drive acceptance of a VA. Former patients positively highlighted the speed, accessibility and objectivity when providing information. A user-friendly design was stated as an important factor for effort expectancy. This factor is also influenced by the self-efficacy [102] of the user. Two extensions of the research model were Trust was found an important facilitator and participant’s resistance to change as an inhibitor to accepting a VA. We extended the UTAUT model with these factors.

To quantify and validate this extended model, a survey was completed by 127 cancer patients. The strongest factors that drive the VA’s acceptance among cancer patients are performance expectancy (ß = 0.399), effort expectancy (ß = 0.258) and trust (ß = 0.210). Social influence (ß = 0.114) showed a positive though weaker influence on behavioral intention.

Overall, descriptive statistics indicated a willingness to adopt a VA for cancer treatment. The average aggregated value for behavioral intention was 3.2 measured on a 7-point scale from “strongly agree” [1] to “strongly disagree” [7]. This was more pronounced among younger respondents. A multi-group analysis did not find significant differences in the relevance of constructs among the subgroups age, gender and experience with a VA. Reasons for the insignificant differences between the subgroups might have been the partly unequal sample distribution and small sample sizes leading to insufficient power to detect differences [127].

Compared to Venkatesh et al.’s [51] study, the present model confirmed the three predictors of behavioral intention in H1, H2 and H3. As the performance expectancy construct is the strongest predictor in each of the different acceptance models [51] and specifically for internet- and mobile-based interventions [59] its high value and support for H1 was expected in our study. The weaker effect of effort expectancy compared to performance expectancy fits previous research [128] that found ease of use the secondary motivation to adopt a technology. The primary motivator is the function of the tool itself. In other words: ease of use cannot compensate for unnecessity. The lower relevance of social influence could be because older users are less receptive to societal pressure or image [129]. Furthermore, the UTAUT was initially introduced in an organizational environment where peer influence is more salient [130].

Unexpectedly, facilitating conditions showed no significant influence on behavioral intention and H4 is not supported. In the original UTAUT, facilitating conditions is considered having direct effect on user behavior instead of intention [51]. This insignificance might be due to the unawareness of the elderly about the technical and infrastructural resource requirements of a VA [53]. In addition to the existing model, the antecedent self-efficacy showed a highly significant, positive relation with effort expectancy, explaining 63% of its variance. H5 is thus supported. Therefore, the high perceived capabilities of the user were also crucial for the acceptance of a VA through an indirect effect, confirming previous research [102].

The high relevance of the factor trust underlined the sensitive issue of a VA during cancer treatment, which bears extensive consequences on people’s lives. Accordingly, trust played a central role for healthcare professionals in acceptance research [22, 131]. Also, in the present study, H6 was supported. As trust involves some degree of risk [132], patients only relied on a VA when their level of confidence about the reliability (TR1), security (TR2) and trustworthiness (TR3) is higher than the perceived risks involved. Lastly, resistance to change only had a marginally significant influence. H7 is therefore not supported. This result is unexpected, as the inhibitor was observed in the interviews and particularly salient in studies with older participants [53, 115]. A reason for the marginal significance can be related to the dominate and overwhelming effects of performance expectancy, efficacy expectancy and trust in the overall model. Another reason could be attributed to our data collection, which was partly through Facebook. This might have introduced a bias for respondents that are more open to new technological innovations. Moreover, patients may already proactively met information needs, e.g., through mobile online searches, so a VA would not have represented a major intervention.

This study contributes to technology acceptance theory by analyzing the behavioral intention to use a VA from a the perspective of cancer patients – a perspective that is often overlooked [133, 134]. This study validates the UTAUT model in cancer patients based on empirical qualitative and quantitative fieldwork insightsand extends it with two new antecedents of effort expectancy. Trust was found to be a highly factor of influence (highlighting its relevance in healthcare services), while resistance to change was not found to play an important role. Also, while the UTAUT model [51] included age, gender and experience as moderating variables, it was not tested in previous studies applying the framework, e.g., [22, 50, 53, 86]. Lastly, this article addresses the call for more conceptual approaches to acceptance research [50, 59] by analyzing attitudes towards VAs as a new service for cancer patients though such a service does not exist yet. This study attempts to forecast acceptance by quantifying perceptions rather than assessing reasons for failed initiatives.

To be accepted, a VA must be useful and provide value in demanded areas where there are frictions in the patient journey. Thus, insights from patient-oriented research should guide the design. As we found performance expectancy the strongest determinant for the acceptance of a VA, this should be the primary target when trying to increase acceptance among cancer patients. Also, performance expectancy has been found to benefit treatment for mental disorders as a primary mechanism that affects change [135]. This may be assumed to be true for cancer patients (with mental-health related issues) and therefore a relatively direct way to improve their well-being. Contrary to these findings, in our study former patients expressed their concerns about using a VA with mental health issues as they expected the ‘human component’ and empathy to be missing. Further research on this topic is adviced. Interventions to increase acceptance, e.g. educational [136] or informational [137], have been shown effective, but should be tailored to the specific target population, i.e. cancer patients.

Older people might prefer voice input [138] when using a VA, especially in the case of visual impairment [139]. However, the focus should not only lay on the attributes of a VA but also those of the users. Our study found patients will find a VA easy to use only when they have the perceived self-efficacy, leading to a need for training and close involvement in introducing a VA. In UTAUT, the normative social influence tends to encourage compliance or acceptance [140]. The social influence factor may lead to diffusion of VA usage from early, mostly younger patients to other, less affine ones. In a clinical setting, patients feel the importance that if others, in particular doctors, believe the patient should use a specific technology [141]. This suggests using doctors to advocate the uptake of a VA. It is particularly salient as older people rely on their opinion and often avoid or refer decision-making to compensate for a reduced risk tolerance [142]. Trust can be increased by incorporating feedback from patients and the patient’s council in the development and by including options such as a referral to the doctor in the interaction with a VA. An open, standard evaluation framework, as currently developed by the World Health Organization, would help build trust and select value-adding applications [143].

The above empirical results need to be evaluated against the backdrop of some limitations. Only a few former patients of the interviews received their treatment up to 11 years ago, limiting the transferability of their experience and integration of a VA into it. Moreover, although members of the patient council can provide a holistic view, they may be satisfied above average with the treatment as they are still voluntarily working with the clinic. Therefore, they might see fewer points for improvement retrospectively. Unfortunately, it was not possible to recruit more former patients of Maastro within the timeframe of the study, given the ethical and privacy restrictions that are related to clinical research. Likewise, there is a risk of selection bias as it was not possible to sample former patients representatively regarding cancer types and age groups. Instead, self-selection and snowball sampling are used. This similarly applies to doctors, limiting generalizability. Further, the content and online means of showing an illustrative example might have influenced interviewees' opinions. Although, similar interview questions about a VA were asked after only a general description of the concept. Nonetheless, further research should replicate this study with a realistic, interactive VA. Due to language barriers, some former patients may have been discouraged to participate or reluctant to elaborate. To decrease these effects, a native Dutch speaker was present in the interviews.

The sampling procedure of the quantitative research limits its external validity. The survey was mainly distributed via Facebook. Thus, only people with an account and a certain degree of digital affinity could partake, excluding particularly some elderly. Moreover, including respondents residing in a multitude of countries limits explanatory power. Yet, it was necessary to achieve a large enough sample. Here, future research may use only one nationality. Furthermore, insignificant differences between subgroups could be due to the small sample size. Future research with the ability to recruit larger samples should test whether the findings hold across subgroups.

The self-selection in online surveys carries the risk that primarily those who have intrinsic motivation and interest participate. Other facets might have been lost due to this from people more averse to these innovations, e.g., regarding the factor resistance to change which was only marginally significant. Handing out a printed version to all patients with the side effect sheet may reduce self-selection to a certain extent. Additionally, the research was cross-sectional conducted in a short period. A longitudinal study accompanying the introduction will provide added value by measuring the actual use behavior omitted in this study and the potential change of constructs such as performance expectancy and effort expectancy over time.Moreover, trust formation can be split and analyzed at different levels [144]. Therefore, further studies should assess the antecedents of trust. For instance, it may test whether a VA promoted as “approved by doctors” or “developed by a provider with years of experience” creates more trust.