How do patients value and prioritize patient portal functionalities and usage factors? A conjoint analysis study with chronically ill patients

This study used a conjoint analysis questionnaire. ‘Conjoint analysis’ is a survey-based statistical technique mainly used in market research that helps determine how people value different attributes of a product [15, 16]. The objective of conjoint analysis is to determine what combination of a limited number of attributes is most influential on a respondent’s choice or decision making. We used a discrete choice experiment conjoint analysis in our study, since is the most common type of conjoint analysis used in health economics, outcomes research, and health services research [15]. This type of conjoint analysis consists of two steps: 1) a choice experiment with respondents and 2) a statistical analysis. In the choice experiment, the attributes are used to describe a certain product and can consist of one or more levels. Different fictive profiles, possible variations of the product, can be created by combining various attributes and levels, which are then shown to a respondent to determine which combination he/she prefers. For instance, a product can be a tablet that is described by various attributes and levels of those attributes: price (levels $100, $200), screen size (levels 8.9, 10.1) and battery length (levels 14 h, 9 h). Fictive profiles can be presented to a respondent, such as:

The respondent can choose option A, B or C in the trade-off process whether he/she would buy that tablet. This process is repeated for various fictional profiles. In the second step of the conjoint analysis, the statistical analysis of all respondents’ choices for all presented profiles, the relative importance of different attributes and the trade-offs between these attributes are statistically determined [15‐17]. In other questionnaire based methods to measure respondents’ preferences, such as rating or interest questions, it is often not measurable to accurately value how much a certain attribute would influence a respondents’ choice in preferring one attribute above the other [18, 19]. However, choice-based conjoint analysis does determine which individual attributes and levels are favored over others. Conjoint analysis is gaining popularity in the health care setting [17], where it recently has been used to assess patients’ preferences regarding pharmacological treatment for bipolar depression [20] and to examine how older adults rate and identify the importance of healthcare seeking and utilization aspects in the United States [21].

We conducted a comprehensive literature review (Additional files 1 and 2) in which 42 factors were initially found influencing the use of patient portals by patients. Based on this review, four experts in healthcare discussed recurrent as well as meaningful factors that could be used as an attribute and defined the attributes and levels to be used in the conjoint analysis. Seven attributes were constructed for our conjoint analysis, shown in Table 1, each consisting of three assigned levels. The fictional profiles of patient portals presented in our questionnaire were generated using the orthogonal main effects plan: instead of presenting all possible combinations (levels^attributes = 3⁷ = 2187), the smallest manageable combination of profiles to test with respondents are presented - knowing that the statistical analysis will be able to balance how often a specific level is presented to a respondent. By means of 18 profiles our respondents were asked to choose their preferences for patient portals. Additional questions were asked to gain sight on respondents’ demographic characteristics, health status and experience with patient portal use. A potential bias of the questionnaire could have been that respondents would not be able to envision a patient portal by the items or wordings chosen in the questionnaire. For this reason the questionnaire was validated by means of cognitive interviews with six people representative for the target group. During these interviews, eight unique problems were identified. All issues were addressed by changing several formulations and the visual design of the questionnaire. The questionnaire is shown in Additional file 3.

We recruited respondents by sending the questionnaire to patient panels of two Dutch patient associations: the Heart Council (HVG) and the Lung Fund (LF). Data was collected in the months April and May 2017 by means of the tool spidox.net . Members of the panels consisted of chronically ill patients with a cardiovascular disease and patients with a lung disease. Of the total chronically ill patient population in The Netherlands, 16% are cardiovascular patients and 11% are patients with a lung disease [22‐24]. Members of the HVG and LF patient panels were therefore representative for the Dutch chronically ill population and all respondents were eligible for data analysis. Respondents were only excluded if 1) the questionnaire was not completed; 2) response time was under four minutes; 3) response time was above 60 min and 4) data entries were possible spam since they had the same IP address. To calculate the minimum required sample size of this study, we used the recommendations of Orme regarding sample size determination on choice experiment conjoint analysis [25]. Orme recommends a formula for this sample size determination: \( \frac{n\ast t\ast a}{c}\le 500. \)In this formula, n is the number of minimum respondents needed, t is the number of profiles presented to the respondents (18 in our study), a is the number of options to choose from per profile (3 in our study) and c is the highest level per attribute (3 in our study). The number 500 is the threshold for representing the main-effect level of interest in the statistical analysis, yet Orme explains this number is intended to be a minimum threshold. We chose to increase that threshold, in order to be certain of sufficient representations per main-effect level. We therefore used the following formula: \( \frac{n\ast 18\ast 3}{3}\le 2000 \). This resulted in a minimum required sample size of 111 respondents. Standard to conjoint analysis, a conditional logistic regression was used to analyze the data. The data analysis was performed using RStudio version 1.1.383, using packages support.CEs(0.4–1), survival (2.41–3), lmtest(0.9–35), plyr(1.8.4).

Tables 3 and 4 show the overall results of the conjoint analysis, including all attributes and differences between the levels of the attributes. The most prominent and significant result is that respondents prefer to access a patient portal via a laptop or desktop above using a smartphone or tablet. Second, they prioritized to ask questions about medical data or about earlier visits to the provider via the portal. Thirdly, they dislike a delay of two weeks of their information shown in a portal as compared to direct publication of information, yet this is not a significant difference.

The results of this study indicate that aging characteristics influence patient portal preferences, especially regarding technical aspects, such as access and login means. Our respondents state a preference to access a patient portal using a laptop or desktop, rather than using a tablet or a smartphone. Nevertheless, respondents aged younger than 65 years old, the majority being between 45 and 64 years old, were less negative about using a tablet as an access means than the overall respondent group. A possible explanation for older respondents’ access preference may be that the older adult and elderly target group had experience with inadequate user-interface designs of portals on small screens. Portals ─ both web-based and native app versions ─ have complex navigation structures. However, to suit the cognitive capacities of older patients and prevent usability problems, navigation complexities should be minimized [26, 27]. Furthermore, irrelevant information and cluttered presentation of (medical) information on smaller screens of tablets and smartphones inhibit older patients in reading and interpreting this information [28, 29]. A recent study has showed that older people increasingly do show interest in using tablets, yet they have concerns about the process of learning how to use such devices. They further worry about unclear instructions and support during that learning process [30]. Patient portal developers should take advantage of the older user interest in these devices. Albeit, in further development of mobile versions of patient portals, current knowledge on aging barriers influencing the experienced usability of mobile user-interface design [27] and portal functionalities should be taken into account.

Our analysis surprisingly showed that especially the elderly respondents preferred using a solely username and password as a login means to a patient portal. They preferred this above the more secure methods, called two factor authentication (2FA). Privacy and security are important aspects discussed in literature on patient preferences on patient portals [31‐34] and the 2FA method is often mandated by governments to ensure privacy and security. The 2FA method is yet complex to use and often leads to usability problems experienced by older patients [35, 36]. We encourage software engineers in the field of privacy and security together with usability experts to rethink login means to patient portals in order to create a secure as well as a user-friendly login means. They can explore the opportunities for biometric authentication for example. In doing so, it is important to take the challenges of biometric authentication into account in relation to physical effects of older adults’ medical complications, such as cataracts and stroke [36]. An improved login means addressing both privacy and security as well as experienced ease of use by older patients, will likely strengthen their engagement in using patient portals.

Respondents in our study were negative about a built-in publication delay of two weeks of their medical information. Nevertheless, our results show no significant difference in preferences between the options of ‘instant publication’ versus ‘publication after new information has been explained by a healthcare provider’. Previous studies evaluating the publication of medical information in patient portals show inconsistent results on whether publication empowers patients or if publication might harm patients when information is shown without mediating physician input [37, 38]. This is especially discussed within the perspective of publication of test results [37, 38]. Our study provides a strong indication that chronically ill patients do not prefer a delay in publication of their medical information in a patient portal. We therefore advise against such a delay feature in the implementation of portals of which chronically ill patients are the main user group. We further want to affirm the need for customization of medical information publication, where settings can be changed for each individual patient based on his/her preferences in obtaining medical information with or without mediating input from a physician.

The customizability of medical content in a portal is further emphasized in relation to the terminology used to publish this content in a portal. Most respondents prefer a summary of the medical information in laymen’s terms, presenting less but more understandable information, above a complete uncensored medical file. Our study therefore supports the idea that patients experience difficulties in understanding the medical information and jargon published in patient portals [31‐33, 39‐41]. In developing patient portals, we advise to consider customized features in which the provider can manually edit the content before publication. Another possible solution, better suited to the high workload of providers, is to automatically transform medicals terminology standards into laymen terms [42].

In our study, the option for using features to contact the healthcare provider is seen as a main priority by the respondents, which is in line with other studies claiming that such a functionality is an important facilitator for patient engagement [41, 43]. Patient-provider communication via a patient portal can yet be ineffective due to the absence or late replies from their physician [44]. This combination of findings suggest the need for further research on what patients define as a prompt response for various types of questions and how providers’ workflows can allow for such prompt responses on questions asked by patients via a portal.

An interesting finding of our study is that respondents showed a strong preference to use a portal with medical information from multiple providers, possibly working in various healthcare organizations. Especially experienced portal users preferred this, whereas male respondents were more in favor of obtaining medical information from one provider. Our results further show no preference for a digital interoperable export functionality and most respondents are interested in the option to create printable overviews such as in Word or PDF. Negative experiences with the cumbersome tasks of distributing health information across different centers, without any benefit of portals supporting this process, may be a possible explanation for this finding. This finding further shows that for chronically ill patients to gain more benefits from portals, the portal landscape needs to transform from ‘silos’ to an integrated ‘ecosystem’ of actors. In addition, it can be assumed that chronically ill patients currently manage their data across different healthcare centers by printing the data whereas they do want to have a holistic overview of their medical data across these centers in the future. Yet, they do not like to spend much effort in manually exporting digital information from one portal to another. In transforming portal silos to an integrated ecosystem, it is thus important to create systems that minimize the tasks of patients in creating a complete multicenter overview of their health data. A first step to achieve this is to technically facilitate the sharing of medical record information across centers by adopting interoperability standards, such as Fast Healthcare Interoperability Resources (FHIR) in the development of (future) digital environments for patients to access, manage and share their medical data [45].

This study has several limitations. First, respondents were part of a patient panel; we can therefore assume they have a high interest in aspects related to their health and disease management. Consequently, the findings of this study cannot be extrapolated to patients with a (very) low health status and/or interest in their health. These patients might have different preferences regarding patient portal design and use. In spite of this limitation, since our study is based on a large sample of respondents from two different chronically ill patient groups, this study certainly adds to our understanding of possible coherence of portal functionalities and usage factors in relation to disease management from the perspective of chronically ill patients. Furthermore, despite of the interest of the patient panels’ members in health and disease management, the majority of the respondents had no experience with patient portal use and were older people. Aspects that make them representatives of the Dutch chronically ill patient group. Second, it is possible that some words or formulations in the questionnaire were misunderstood by respondents or that respondents who had no prior experience in using a patient portal could not envision a portal based on the wording of the questionnaire. This problem was limited by validating the questionnaire before using it in practice. Third, the underlying model for a choice-based conjoint experiment is nonlinear due to the modeling using a logit function. The variance-covariance matrix which is used to generate the design is dependent on betas. Since we had no prior knowledge about these betas we chose to assume linearity just as rating-based methods and developed the choice sets using heuristics. In order to gain a more optimal design a pre-test or assuming a prior distribution would have improved the outcomes and statistical efficiency.