MHealth and perceived quality of care delivery: a conceptual model and validation

The mobile health (mHealth) market has experienced a significant growth since 2011, with the global mHealth market estimated to grow to about $60 billion by 2020. Lee [1] argued that that many hospitals are spending vast amount of money for implementing mHealth solutions and expecting that users (i.e. physicians) will employ the technology to improve the quality of care patients receive at the point-of-care. Concurrently, hospitals face increasing demands to participate in a wide range of quality improvement activities [2, 3] by streamlining their processes in order to deliver high quality and safe care [4, 5]. Instant real-time access to data at the point-of-care is causing a paradigm shift in how physicians deliver healthcare, making services more streamlined and cost effective [6].

Despite the wide endorsement and support for the implementation of mHealth, Rahurkar [7] argue that empirical evidence surrounding the benefits of Information Technology (IT) in health care remains to be firmly established. The limited evidence for the performance of mHealth technologies may be attributable to a lack of appropriate evaluation frameworks [8‐10], with Boudreaux et al., [11] arguing that there exists no evalutation method which is mHealth specific. While some attempts have been made by various researchers [12‐16] to evaluate mHealth, this work has predominantly focused on reviewing and reporting the adoption of mHealth technologies at early phases of implementation. The lack of evaluation across the mHealth field, primarily in post-adoption stages of implementation, may be perceived as a major weakness of the mHealth domain [9, 10]. Therefore, for mHealth to be truly valuable and have a positive impact on quality of healthcare delivery, the IT artefact must be utilized post-adoption [17, 18]. Furthermore, Goodhue and Thompson [19] argue that the capabilities of the IT must match the tasks that the user must perform. Serrano and Karahanna [20] argue that the role of the user in influencing successful task performance has not been explored in-depth within a healthcare setting. Indeed, there has been a dearth of research focused on exploring an mHealth specific evaluation framework [11, 18] in a post-adoptive scenario, specifically in a hospital setting [21].

Task-Technology Fit (TTF) reveals the association between IT and individual performance and is based on the premise of ‘fit’. ‘Fit’ has been widely utilized and is associated with performance. For example, MIT’s 90’s model is underpinned by the theory of fit [31, 32] and argues that fit contributes to high performance [31‐33]. Optimal performance occurs only if there is a tight fit (alignment) among the domains of strategy, structure, management processes, individual roles and skills and technology [31].

TTF operates at an individual level of analysis with the position that IT is more likely to have a positive impact on individual performance and be utilized if the capabilities of the IT match the tasks that the user must perform [19, 34]. It consists of five constructs namely, task characteristics, technology characteristics, task-technology fit, performance impacts and utilization. TTF theory as a theoretical lens for understanding the performance of electronic health (eHealth) technologies has previously been explored [35]. Specifically, Chiasson et al. [35] answers the call for research by Furneauz [36] to understand the effect of user performance on utilization and the association between the use of effective technology and user performance. They found TTF to be a useful theory for exploring IT in healthcare and illustrated the positive association between use and performance.

However, in the context of performance, the association between the utilization of a technology and the PQoC (as a performance outcome) has not been investigated. It is noted [4, 37] that there is scant empirical evidence of the impact which healthcare information systems have on the quality of care. Towards deriving an understanding of same, a conceptual model is developed to explore physicians’ view which mHealth has on PQoC. Specifically, a conceptual model is developed (based on the technology to Performance chain model) to focus on PQoC as the dependent variable.

To test the conceptual model (Fig. 1) an observational quasi-experimental one group posttest-only design was employed. The self-reported data was collected by employing a survey and distributed through the internal hospital channels to physicians who adopted iPads for their daily activities. The online survey questionnaire was constructed using indicators already validated in existing research (see Additional file 1). The survey instrument was piloted to ensure content and construct validity. Over 20 medical experts (professionals who work in healthcare delivery and actively utilize mHealth as a part of their daily work related activities) in the US and Ireland participated in the pilot study. Following this, the questionnaire was refined before launching the survey.

Data was collected over a four-month period, in 2012, from physicians in an acute care context within The Ottawa Hospital, Ontario, Canada. The Ottawa Hospital (TOH) made over 3000 iPads, incorporating a mobile Electronic Medical Record application, available for use by physicians. TOH is one hospital spanning three sites (Civic, Riverside, and General Campuses) and has been using mHealth at the point-of-care for numerous years. The survey was distributed via email (an URL link) to physicians in TOH, Canada. No hard copies were distributed to the potential participants since all members of the TOH staff had Internet access.

One major types of bias that is often associated with web surveys is common method variance bias [63]. To overcome the former, several reversed-scored items were used to reduce sign rating problems associated with Likert scales [63]. See Additional file 1 for a description of the items used as part of the questionnaire.

Participants using mHealth within their daily practices for 6 months or more was eligible for the study. A total of 157 responses were obtained from various physicians via the administration of an online survey (871 physicians in total at the time of data collection), yielding a response rate of 18%. Not all of the 871 physicians were available/active during the data collection period of time, which may have impacted the response rate. Noteworthy, it is argued that physicians are often a group with low survey response rates [64]. Nevertheless, 157 responses were cleaned for missing values and 102 complete responses were used for data analysis. While research [65] argues that missing values affects directory of taking the decision, G*Power (version 3.1.2) was used to conduct power analysis and to establish whether the sample size is sufficient. The analysis revealed a power value close to one indicating that the sample size of 102 is sufficient as per Cohen [66]. A key benefit of using Partial Least Square (PLS)- Structural Equation Modeling (SEM) is that it accommodates the use of small sample sizes giving that the ten [10] times rules is met. The 10 times rule depicts that a sample size should be equal to 10 times the largest number of formative indicators used to measure a particular construct, or 10 times the largest number of paths directed at a construct in the model [66]. Our sample size satisfies both requirements.

Structural Equation Modeling (SEM) was used for the hypotheses testing and data analysis. The PLS (SEM) approach, which uses component-based estimation was chosen since it allows simultaneous examination of both the measurement and the structural models. The measurement (outer) model portrays the relationships between a construct and its associated measurement items whereas the structural (inner) model represents direct and indirect unobservable relationships among constructs [67, 68]. In addition, the PLS approach, in contrast to covariance-based SEM, allows testing of the relationships in the model with less restrictive requirements and relatively small sample sizes. PLS is also considered very appropriate for exploratory studies and for testing theories at earlier stages of development [69], and it is highly suitable for prediction-oriented research [70].

The evaluation process of the PLS path model with mixed formative and reflective constructs involves two steps [70‐75]. Step 1 involves the testing of the quality of the measurement (outer) models of first-order constructs used in the first stage as well as other endogenous and exogenous constructs. After this we assess the appropriateness of the high order constructs. The research model includes two second-order constructs TTF - task technology fit and Utilization (Fig. 2). We followed the recommendations provided by Becker et al. [76] for repeated indicators, two-stages, and hybrid approaches for estimation hierarchical second-order constructs. As Step 1 was successful and latent constructs were found to be reliable and valid, Step 2, which necessitates the assessment of the structural (inner) model, was conducted [70, 75]. SmartPLS 3.2.6 was employed for the PLS model assessment.

After excluding 55 responses from the 157 received, 102 surveys were usable for data analysis. This cohort represented 59 males and 43 females, with ages ranging from 18 to 25 years (n = 2); 26–40 years (n = 58); 41–55 years (n = 25); 56–65 years (n = 14) and > 65 years (n = 3). Attending Physicians accounted for 53% of the responses (n = 54), with Residents/Fellows completing the remainder of 47%.

The reflective measurement constructs of the research model were assessed according to the established criteria to assess PLS models with reflective constructs (e.g., [67, 70, 73]). The research model includes six first-order reflective constructs of higher-order construct Task-Technology Fit, three first-order reflective constructs of a second-order construct Utilization, one exogenous reflective construct Self-Efficacy, and one endogenous PQoC (Fig. 1). Hence 10 (ten) reflective measurement models were assessed for reliability and validity. Table 1 demonstrates Individual reliability of the indicators - the magnitudes of all indicators are above this lower limit of 0.707, with the lowest value of 0.731 and majority of values above 0.8.

Table 1 illustrates that all manifest items are reliable for all reflective constructs, demonstrating internal consistency reliability since both parameters (i) Cronbach’s α and parameter (ii) composite reliability have high values (all values are above 0.752), with the required value being above 0.7 [70]. The validity test of the reflective constructs examines the convergent validity and the discriminant validity. Average Variance Extracted (AVE) for all constructs is higher than 0.5, which indicates sufficient convergent validity (Table 1).

We tested discriminant validity with three approaches: Fornell-Larker criterion analysis (See Table 2), cross-loadings (see Additional file 2), and assessing the heterotrait-monotrait ratio (HTMT) of the correlations (See Table 3).

The results of all tests confirm that the manifest variables (indicators) presented in the research model are reliable and valid.

While the TTF theory has been studied in health domains [20, 77] and even variant model has been suggested such as inclusion of self-efficacy in the model [78, 79] and feed-forward chain in the TTF theory [35], a dearth of research focuses on the impact on PQoC [20]. Towards addressing this shortcoming in existing research, this study examines the impact of mHealth on the PQoC in a post-adoptive scenario. The conceptual model was developed and empirically tested (Fig. 2). The model explains 64% of the PQoC. Furthermore, it also explains 63 and 68% respectively of the endogenous constructs (TTF and utilization). TTF was found to be the dominant construct in explaining the variance of PQoC. We can infer that in a post-adoptive scenario, TTF becomes fundamental (and a very important mediator) for PQoC. Realizing how an organization can improve TTF will lead to better PQoC.

A systematic review [80] on mHealth adoption by healthcare professionals found that perceived usefulness and ease of use, design and technical concerns, cost, time, privacy and security issues, familiarity with the technology, risk-benefit assessment, and interaction with others (colleagues, patients, and management) are the main factors to providers’ adoption behavior. Our research corroborates these findings. However, towards understanding how to improve TTF, our model tested the main components of TTF and found that physicians should keep investing in learning and training, regardless of the stages of technology adoption. Learning and Training was found to be the most critical factor in the formation of TTF. Training and implementation management, as part of the Normalisation Process Theory (NPT), is argued [81] to promote the successful implementation and integration of interventions into routine work. Timeliness and Ease of Use were found to be the second and third factors in the formation of TTF. The medical domain is one which is continuously evolving, necessitating physicians to continuously learn. By utilizing mHealth to keep up to date with the latest clinical/medical protocols, physicians are continuously striving to improve quality of care.

Research confirms that self-efficacy plays an important role for IT utilization (cf. [55, 56, 82, 83]). In a post-adoptive scenario, the findings reveal that self-efficacy is critical for utilization, and this construct has the highest direct and total effect on utilization. Interpreting this finding, there is an ongoing requirement to continuously enhance an individual’s skillset for using mHealth. The findings reveal that self-efficacy has a small effect on TTF, which is unsurprising given the fact that the mHealth is already embedded in a physicians’ work practices. In the formation of TTF, the findings further reveal that technological characteristics dominate followed by task characteristics.

In the conceptual model (Fig. 2), the alignment between task and technology (TTF) has an impact upon use. Interestingly, our findings reveal that (1) the technological characteristics of mHealth and (2) healthcare physician’s work practices have no direct impact upon utilization. Towards explaining these findings, one should consider the context of the study. Data was gathered at a post-adoptive stage, meaning that mHealth had been continuously used over an extended period of time resulting in mHealth being embedded in physicians’ work practices. It was found that there is no direct impact of task characteristics and technology characteristics on mHealth utilization, although we hypothesized these impacts. However, both these constructs have significant indirect effects (TTF is a mediator) and total effects on utilization. Therefore, at the most advanced stages of utilization direct impacts are not relevant, but TTF becomes a very important mediator.

A user’s behavior can range from stagnation in utilizing IT features to total integration of the IT in his/her work domain [84‐86]. Therefore, it is important that the features/functionalities of mHealth can be adapted easily to reflect the true but constant changing working nature of physicians to complete any given task within hospitals. MHealth containing electronic pharmacopoeias (i.e. drug information), medical calculations, guideline information and administrative tasks have been identified as the most useful resources by physicians, nurses and other clinical staff [87].

In the context of understanding utilization and its constituent parts, this article decomposes utilization into routine, feature, and value-adding use; the three of which have not been collectively examined in explaining utilization. By breaking utilization into these three constituent parts, we were able to identify that routine use and value-adding use are the major utilization factors for mHealth when delivering healthcare services at the point of care. At the earliest stages of adoption routine use can be very critical. However, at a post-adoption stage, routine use, while statistically significant, is the least important factor in mHealth utilization; feature use takes the leading role. An explanation for this is that as system usage becomes repetitive and habitual, routine use emerges. This confirms existing research [53]. In the context of value-adding use, physicians take advantage of the current artefact by exploring features which they as individuals are less familiar with. This potentially enables mHealth to be used by physicians in novel ways/for unanticipated emerging patient problems. Therefore, in post-adoptive scenarios physicians may employ different features to cope with changing working requirements.

This article answers calls for the development of a specific mHealth evaluation framework which is scant in existing literature [8‐11]. In meeting this request, we have also addressed the dearth of research examining mHealth in a post-adoptive scenario and its impact upon Perceived Quality of Care Delivery (PQoC).

MHealth presents healthcare organisations with a significant amount of opportunity which benefits healthcare professionals and patients alike. This study informs hospitals and software vendors as to the performance of mHealth by clearly demonstrating that physicians using mHealth at the point-of-care enhances their PQoC a patient receives. As the availability of mHealth continues to increase, we call that all mHealth should be reviewed by clinical experts in order to safeguard the quality of care patients receive.

This study also contributes to the practitioner community by highlighting the importance of adapting mHealth to adhere to users work practices, without unnecessary disruption to the use of the service. Changes in work practices within healthcare environments are often dictated by external forces (e.g. pharmaceutical society introduces new guidelines for dispensing drugs). For PQoC to remain constant, it is imperative that mHealth continuously evolves and adapts to changing work practices and that mHealth be designed with work practices in mind. Indeed, our findings reveal that once technology is embedded, technology characteristics are a secondary consideration for physicians.

Although this research achieved its objective, the results of this study should be interpreted in the context of its limitations. First, this model was examined from a healthcare physician perspective. While a healthcare physician population was appropriate for this study, the conceptual model (see Fig. 1) could be tested across a wide cohort of medical professions (e.g. nurses, physiotherapists, dieticians and, pharmacists). Such context extensions are argued ([88]) p.103) to be “part of on-going efforts to provide generalised measures of TTF constructs”. Additionally, healthcare services are often delivered across different levels (e.g. primary, secondary and territory) and scenarios (e.g. preventive care, urgent care, emergency care, home health, and long-term care) ( [89] p.66). As a result, we further urge future research to also consider these domains. Moreover, individuals use mobile technology, especially smartphones, for both hedonic and utilitarian purposes [90]. Building from this, future research should examine medical professionals who use smartphones which are consumed for both work and personal purposes. Although rich data was obtained from participants in the study to develop and validate the conceptual model, future research could conduct similar empirical work with a larger study population. This will further validate the research model.