Introduction
Internet interventions have a large potential for public health impact. Increases in use of the Internet and mobile devices worldwide have made such interventions increasingly common. The initial efficacy of the Internet intervention approach compared with non-web-based interventions has been demonstrated for numerous health behaviors and chronic conditions [14, 22] and in mental health [19]. A review of 24 randomized studies found that computer- or Internet-based interventions that combine health information with social, decision, or behavior change support significantly changed patient knowledge, perceived social support, and key behavioral and clinical outcomes when compared with non-web-based control programs [14]. Effect sizes reported in a meta-analysis of Internet interventions compared with controls ranged from −0.01 to 0.75, but results were consistently more favorable for those participants assigned to the Internet interventions [22]. A similar meta-analytic review in mental health showed that, for 12 studies involving over 2,000 subjects, the effect sizes for Internet interventions ranged from 0.00 to 1.5 with average effect sizes in the medium range of 0.2 to 0.5 [19]. These reviews demonstrate that Internet interventions are producing favorable behavior change outcomes.
One of the most commonly cited reasons for choosing to investigate Internet interventions is reducing health services and delivery costs [5] as interventions delivered over the Internet are likely to cost less than treatments requiring frequent contact with health care professionals. Even if Internet programs are found to be equally effective or even less effective than programs delivered via a traditional mode (e.g., face to face, phone), their relatively low delivery cost could result in Internet programs being more cost effective. Researchers are beginning to examine issues related to cost of Internet-based interventions and comparing them to costs for traditional delivery mechanisms. This information is critical for policymakers and stakeholders to prioritize scarce public health and health care resources. It is common in early stages of research to recommend that future studies of Internet interventions report the costs and cost effectiveness associated with an intervention. A comprehensive economic analysis requires substantial time and proficiency in health economics and may be beyond the scope of initial efficacy or pilot investigations. However, economic analysis is a feasible and essential outcome for this field as it emerges from efficacy to effectiveness research and is particularly appropriate for translational research [3].
Anzeige
The purpose of this paper is to provide a brief review of the literature on the cost effectiveness of Internet interventions and to discuss specific considerations for conducting economic analyses of Internet interventions.
Review of Cost-Effectiveness Studies
A search for internet interventions using the terms “internet,” “online,” “web-based,” “cost-effectiveness,” “return on investment,” “ROI,” “cost benefit,” “business case,” “economic analysis,” “cost,” “intervention,” “treatment,” and “program” was conducted in PubMed with dates from 1995 through 2008. Few studies were published prior to 1995 using the Internet as a means of intervention delivery. The search resulted in a total of 420 unduplicated published papers. Studies included for further detailed review met the following criteria: (1) published in a peer-reviewed journal, (2) reported on an intervention or treatment program, (3) used the internet as one mode of intervention delivery, (4) reported cost measures associated with intervention delivery, and (5) were published in English. We excluded any published citations that did not explicitly state at least some of the cost components of the intervention or report on the cost effectiveness or cost benefit of the internet intervention relative to a control or alternative intervention. In addition, this review excluded computer applications or other telemedicine interventions that were not delivered via Internet (see comprehensive review by Kaltenthaler [10] of economic evaluations of computer-based treatment for depression and anxiety). Of the 420 articles identified in the initial search, only eight met these criteria in full and are summarized here.
Two studies conducted incremental cost-effectiveness analyses. A study from the UK assessed incremental cost effectiveness of an Internet-based obesity management program vs. usual care [11]. Little detail was provided on the costs quantified in this study; however, the researchers did include the costs of program development when calculating the incremental cost-effectiveness ratios. As discussed below, these “sunk” costs are generally excluded from cost-effectiveness analyses because they are not expected to be repeated were the intervention to be adopted on a broader scale. Due to lack of differences in outcome measures between intervention and control groups and higher costs of the Internet-based program (partly driven by the inclusion of the development costs) vs. usual care, the Internet-based program was deemed not to be a cost-effective approach for obesity management.
A study from Australia conducted by Mihalopoulos and colleagues [12] assessed incremental cost effectiveness of an online intervention targeting panic disorder supplemented with support from a general practitioner or a psychologist compared with usual care. The analysis was conducted from the health care system perspective. Program costs included labor costs for medical staff (general practitioners or psychologists), cost of the online software, start-up and development costs of the software, and time costs. The analysis was conducted under assumptions that the intervention was available to all eligible people and was perfectly adhered to. The authors reported incremental cost-effectiveness results (as compared with usual care) ranging from $3,200 per disability-adjusted life year (DALY) when supported by a general practitioner to $4,300 per DALY when supported by a psychologist. However, these results require perfect adherence, which may be unlikely to occur in practice.
Anzeige
In a study conducted in Germany, Runge and colleagues [15] reported a cost–benefit analysis (assigning monetary values to both costs and outcomes) of a standardized patient management program (SPMP) aimed at improving health outcomes of asthma patients aged 8–16 as compared to usual care. The authors also assessed the cost benefit of adding an Internet-based education program to the SPMP. Program costs were determined as the amount that insurance companies cover for asthma patient education programs plus Internet access fees. To monetize program benefits, the researchers calculated the average asthma-related costs associated with physician visits, emergency department visits, hospital stays, medication, transportation to and from medical care visits, and work loss during the 6-month periods pre- and post-intervention (extrapolated to 1-year pre and post periods). The reductions in these medical and nonmedical costs were defined as the savings associated with the program. Program benefits were then quantified as the difference in savings between control and intervention groups. Over a 1-year period, the SPMP plus the Internet program had a benefit–cost ratio of 0.79 from a healthcare system perspective and 0.70 from a societal perspective (that also includes participant costs) as compared with usual care, indicating that program costs exceeded the benefits. However, as compared with SPMP alone, the addition of the Internet component resulted in an incremental benefit–cost ratio of 3.65 from a healthcare perspective and 3.04 from a societal perspective, indicating that the additional benefit associated with the Internet component exceeded the additional cost to provide it. It should be noted that patients in the Internet intervention arm were not randomized but rather self-selected, thus potentially biasing the results.
Two other studies attempted to monetize both program costs and benefits to quantify a return on investment (ROI). A study from the UK assessed ROI of a multicomponent worksite health promotion program where access to a web portal was one of the program components (in addition to printed materials and face-to-face seminars and workshops) [13]. Reported annual program costs were $138 per eligible employee; however, no details were provided on the derivation of these costs. The authors monetized changes in absenteeism and presenteeism using participant salaries and estimated that the program yielded a positive ROI of $1.9 in savings due to increased productivity for each $1 invested when focusing on absenteeism only and $6.19 for each dollar when including both reduced absenteeism and improved presenteeism. These high ROI estimates are based on the assumption of no decay in effectiveness over time and no employee turnover. If either of these assumptions do not hold, then the ROI is likely to be much smaller.
Southard and colleagues [18] calculated ROI of an Internet-based cardiac rehabilitation program as compared with usual care. Program costs totaled $453 per participant relative to usual care and included labor costs for nurses, overhead and administrative costs, and a 6-month subscription to the Internet program. Program savings were estimated using differences in medical costs between intervention and control groups associated with cardiovascular-related emergency room visits and hospitalizations. The authors estimated an ROI of 2.13 for this Internet-based program ($2.13 in savings for each dollar invested in the program). This study included one of the more comprehensive analyses reported in the literature we identified in this review.
Two studies, one from the USA and one from Sweden, reported to have assessed cost–outcome measures of Internet interventions targeting headache patients [1, 20]. However, the cost measures estimated in these studies were not assigned monetary values; instead, cost efficiency was calculated as the percent improvement in headache disability index divided by an estimate of average therapist time spent with each patient. Another study from Sweden that compared an Internet-based self-help program vs. standard face-to-face group therapy for the treatment of tinnitus distress reported a similar measure of cost effectiveness where average therapist time spent with each participant was divided by the change in the outcome measure assessed by questionnaire [9]. The authors of these three studies concluded that the Internet-based programs were more cost efficacious than face-to-face treatments delivered by a therapist. However, these findings are inconclusive as they do not include all measures of costs or effectiveness from either the healthcare system or societal perspective. The only cost captured in these studies was the therapist time (not quantified in dollar value) without any considerations for other costs that the Internet-based program incurred.
Despite a growing body of outcome literature on internet interventions, this review reveals that only a limited number of studies so far have attempted to incorporate economic endpoints into the analyses, and most have significant shortcomings. This lack of detail on the cost and cost effectiveness of internet interventions relative to traditional delivery methods makes it extremely difficult for policy makers to determine whether or not these interventions are a good use of health care resources. This lack of cost data is likely a reflection of the early stage of research for many papers published on Internet interventions during the review period. As the field now moves to effectiveness studies, it is important for cost-effectiveness data to be collected to guide further development and research areas as well as potential health care and dissemination decisions.
Specific Cost Considerations for Internet Interventions
Several helpful references and primers on cost-effectiveness analysis in health care and public health have been previously published [2, 4, 7, 8, 16, 17, 23]. We encourage researchers to consult these resources before planning, designing, and implementing economic evaluations of Internet-based programs as these resources provide guidelines and recommendations that will improve comparability and quality of studies.
However, in addition to the previously published guidelines, researchers should bear in mind unique aspects of Internet-based interventions that may affect the conduct of economic evaluations of these types of interventions. Specifically, in this section, we present considerations for cost-effectiveness analysis of Internet interventions in the hopes that the next generation of internet intervention research will incorporate economic analyses.
Study Perspective
The “perspective” of the study refers to the viewpoint the study will take to quantify costs and outcomes. Identifying the study perspective is an important early step because it dictates the types of costs and outcomes that must be quantified and included in the cost-effectiveness study. The US panel on cost effectiveness recommends conducting cost-effectiveness analyses from the societal perspective [16], which captures and quantifies the value of all resources used for an intervention incurred by the intervention provider and participants. However, oftentimes a more narrow payer perspective is relevant, especially if participant burden is equal across interventions or is not part of the decision of which intervention(s) to adopt. This perspective is also more straightforward in many cases because of the difficulty involved with quantifying participants’ time and resources devoted to a particular intervention.
Anzeige
For an Internet-based intervention, the most obvious opportunity cost from a participant perspective is the time a participant spends in front of the computer logged on to an Internet intervention. It may be possible to track the actual time that participants spend going through the online intervention modules or estimate expected time that it should take for someone to complete the online activities. However, the time spent in front of the computer is often only a small part of the time it takes to “do the behaviors” needed to change the health outcome targeted by the intervention. In the case of obesity management, as an example, the additional time an individual spends engaged in physical activity, reading nutrition facts labels, writing down foods in a food log, or working through behavioral exercises designed to teach skills necessary for long-term weight management, would be relevant costs to capture when considering participants’ time costs. When assessing the intervention from a participant perspective, these costs are important to capture as they often help explain why promising interventions are unsuccessful and/or have very low reach. One approach to capturing these costs is for participants to keep a log of time spent in front of the computer and in activities recommended by the program. Participants can be also asked to estimate the average time (e.g., weekly) spent on program activities. The dollar value can then be calculated by multiplying the time estimates by the median hourly wage of a population similar to program participants (this wage represents the opportunity costs of their time).
Costs to Develop the Intervention
Intervention development costs are the costs required to set up the intervention. They include costs to develop specific components of the program and capital investments that can be used for the length of the program. These costs are referred to by economists as “sunk costs” because they are not expected to recur if the program were to be expanded. In an Internet intervention, these costs are often very time and resource intensive, and the largest portion of these sunk costs are likely to be programming costs and the cost of content experts but can also include costs related to registering domain names, licensing pertinent data bases (e.g., food databases for nutrition) or other software used for the intervention (e.g., chat software etc.) or for making hardware purchases (e.g., development servers).
Sunk costs are not relevant for cost-effectiveness analyses because they would not need to be repeated if the intervention were adopted on a broader scale. For example, when a managed care plan considers whether or not to include a new drug on their formulary, what matters is how much they have to pay for the drug and what the potential benefits are, not how many billions of dollars were spent developing the drug. The same goes for internet interventions; it is marginal (future) costs that matter, not sunk (prior) costs that would not need to be repeated. However, technology-based systems likely need to be enhanced and updated to reflect current technology trends. Thus, there is likely a useful life for an Internet intervention after which some updating of the program will be needed. This may be superficial, such as updating the look and feel of the system, but often new features and programming that bring the system up to current user expectations may require more extensive reprogramming which are much more costly. These future costs should be included in the cost-effectiveness analysis. As the field progresses, it will also be important to quantify costs separately for select features of the intervention as each feature requires additional costs to develop and maintain and the increase in effectiveness may or may not be worth the additional costs. Finally, to help others who may be considering developing internet-based interventions in the future, we recommend reporting both the sunk and nonsunk costs even though the latter will not be used in the cost-effectiveness analysis.
Program Dissemination Costs
Ongoing program costs (those that are incurred as long as the intervention is being implemented) can be separated into fixed costs and variable costs. Fixed costs do not change if additional program participants are enrolled in the intervention. Examples of such costs for an Internet-based program include costs associated with server maintenance and data storage (although once the number of participants reaches a certain number, data storage requirements may increase). Variable costs change if additional participants are enrolled in the program. Examples of variable costs are labor costs if a human is involved in supporting the intervention such as emailing participants or providing personalized feedback or telephone support for promoting behavior change or adherence to the intervention.
Anzeige
Because a large portion of total implementation costs required to deliver an Internet-based program on an ongoing basis are fixed costs that do not vary (in total) based on the number of participants, the cost of delivering the program per patient will be determined largely by the number of patients enrolled. As a result, reporting per patient cost using the number of participants from a clinical study or demonstration project may be an underestimate and will not accurately reflect the true per participant cost of an Internet-based program once it is disseminated to a wider audience. Ideally, researchers should estimate the likely take-up rate for the Internet-based intervention and use that to estimate average per patient cost of the program. These estimates of participation rates can be used to calculate per patient intervention costs under various assumptions of program dissemination and take-up.
Average vs. Incremental Cost-Effectiveness Ratios
When conducting cost-effectiveness analysis, researchers can assess average or incremental cost effectiveness of a program. In an average cost-effectiveness analysis, a program is compared with the status quo or a “do-nothing” alternative. In an incremental cost-effectiveness analysis, one program or strategy is compared relative to its alternative. Internet-based interventions are often proposed as alternatives or adjunctive to other methods of delivery (face to face, phone, etc); therefore, it is important to establish an appropriate comparison base and construct incremental cost-effectiveness ratios for these types of programs. This will require that costs and benefits of alternative strategies are also captured and quantified in addition to the Internet-based program. In addition to comparing Internet interventions with other non-Internet-based comparisons, larger factorial designs where various types of Internet interventions are compared are becoming more common [21]. In such designs, Internet programs including a variety of features are compared for effectiveness (e.g., automated vs. human feedback, stand alone vs. phone calls to promote adherence) and can also be compared for incremental cost effectiveness as long as the costs of each key feature are separately tracked.
Outcome Variable for Cost Effectiveness
Internet-based interventions, as with traditional interventions, will ultimately focus on a variety of behaviors. Many cost-effectiveness studies present data on the costs associated with a unit change in the main outcome of interest. For example, weight loss studies often focus on the costs per pound or kilogram lost. Although informative, if possible, these metrics should be transformed into a metric that can also be compared across all interventions, regardless of the behavior targeted. Traditional metrics include life years saved or quality-adjusted life years saved (QALY) or even dollars (ROI or cost–benefit analysis). Including these more universal metrics would allow, for example, a weight loss intervention to be compared to an intervention that targets migraines or any other behavior. Although there is no hard and fast rule, it is often stated that interventions that cost less than $50,000 per QALY are a good value [6]. Converting pounds lost or reductions in migraines to QALYs has its own challenges and may require additional modeling and assumptions. Researchers should consider quantifying this metric if possible to allow decision makers to weigh the costs and benefits of all possible interventions aimed at improving health using a common framework.
Summary
This brief review demonstrates the scant publication of economic analysis of Internet interventions in this early stage of the field. The initial efficacy of this form of treatment delivery has been established in mental health and behavioral medicine. It is now imperative to move the field forward to begin to capture and report on the costs and cost effectiveness of Internet-based interventions. This information will be critical for health care decision makers, public health, and government in allocating scarce health care and research funds.
Anzeige
Acknowledgements
The University of North Carolina at Chapel Hill’s Office of the Vice Chancellor for Research and Economic Development provided partial support for open access publication.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.