Background
Skin cancer is the most common and one of the most preventable forms of cancer in the United States [
1]. An increasing number of effective interventions for the primary prevention of skin cancer are available and recommended; however, few of them have been systematically disseminated and implemented [
2]. Furthermore, little is known about the barriers and facilitators to the implementation of effective interventions for the primary prevention of skin cancer [
3]. These issues are addressed by the field of implementation research.
Implementation research studies the processes and factors that are associated with and lead to the widespread use and the successful integration of an evidence-based intervention [
4]. Implementation of evidence-based interventions most likely occurs in stages and is defined as the process of putting to use an intervention within a specific setting (
e.g., a school or worksite) [
4,
5]. The quality of implementation can be characterized by the degree to which the intervention is carried out in a new setting as prescribed by the original protocol (
i.e., fidelity) [
6,
7]. Implementation fidelity has been shown to determine the success of the implemented intervention by influencing the relationship between the intervention and the intended outcomes [
8,
9].
A number of factors influence the speed and extent of implementation of evidence-based interventions, including individual-level and setting-level adopter characteristics, contextual factors, intensity of the intervention, and characteristics of the intervention [
9,
10]. Characteristics of individuals that influence the implementation include background characteristics (
e.g., education), attitude toward the intervention, self-efficacy and motivation to implement the intervention, and position within the setting/organization [
9]. Attributes of the adopting setting that appear to influence implementation include the setting size, perceived complexity, formalization, and organizational and service system factors (
e.g., characteristics and style of the leadership, attitude toward the intervention, and administrative and financial support and resources available for the implementation of the intervention) [
9,
11].
Contextual variables refer to the broader physical, political, social, economic, and historical factors relevant to the implementation [
12]. The intensity of the intervention can be characterized by the requisite level of training and technical assistance and the quality of information and materials (
i.e., tailoring) received by the adopters before and during the implementation [
9]. Finally, the perceived characteristics of the intervention affect implementation: these may include relative advantage, compatibility, observability, trialbility, and complexity [
4].
Although the role of these factors is well described in the literature [
10,
13], little research has been done on identifying their relative contributions to the implementation of effective skin cancer prevention interventions. A recent systematic review of the implementation literature found only three skin cancer prevention dissemination and implementation studies published between 1971 and 2008 (excluding the one described and used in this paper) [
3,
14‐
16]. The results from these studies regarding factors influencing the implementation process were mixed. Furthermore, these studies did not discuss potential influential factors systematically, did not include a large number of possible predictors, and did not account for the hierarchical structure of these influences (
i.e., individuals nested within settings). To achieve widespread cancer control, a better understanding is needed of the characteristics that contribute to the successful implementation of effective skin cancer prevention interventions [
17].
The analysis reported here addressed an ancillary aim of the Pool Cool Diffusion Trial and assessed the relative contributions of lifeguard background characteristics, sun protective attitudes, sun protective behaviors, pool characteristics, and treatment group to the implementation of a widely disseminated skin cancer prevention program by lifeguards.
Context
Pool Cool is a multi-component educational and environmental sun safety intervention conducted at swimming pools [
18]. Pool Cool was tested in an efficacy trial and found to be effective in improving children's sun protection behaviors, sun safety environments at swimming pool, and reducing sunburns among lifeguards [
18,
19]. Furthermore, a dose-response relationship was observed between the number of lessons and activities that children were exposed to and their sun protection habits [
18].
The efficacy trial was followed by a pilot dissemination study and a larger randomized diffusion trial, the Pool Cool Diffusion Trial. The analysis described in this paper used data from the Pool Cool Diffusion Trial. The Pool Cool Diffusion Trial applied constructs from the social cognitive theory, the diffusion of innovations theory, and theories of organizational change [
20], and was designed to evaluate two strategies for the dissemination of Pool Cool. The two dissemination strategies tested in the trial were the basic and enhanced delivery methods (
i.e., treatment groups). The enhanced group pools received a more intensive, theory-based dissemination intervention, including additional sun safety incentives, more environmental resources, and technical assistance (motivational and reinforcing strategies) in addition to the standard intervention components. More specifically, pools in the basic group received a Pool Cool Toolkit and program training that were similar to the ones used in the original pilot study and efficacy trial [
18]. Enhanced pools received the same information and materials as the pools in the basic group plus additional sun-safety resources, including Pool Cool incentive items (hats, UV sensitive stickers, water bottles,
et al.), additional sun-safety signs, and possibly a shade structure. Pools in the enhanced group were also given booklets entitled, 'How to Make Pool Cool More Effective' and 'The Pool Cool Guide to Sustainability' - a guide that includes suggestions and methods for securing continued funding and support, including developing partnerships with local organizations to continue the program after the end of the research study. Enhanced pools also participated in a 'Frequent Applier' program that earned raffle points as incentives to encourage maximum participation in the program. Raffled items included extra Pool Cool incentive items (hats, lanyards, pens,
et al.), extra gallons of sunscreen, and shade structures. Field coordinators representing pools from the enhanced group also participated in two to three additional conference calls each summer were actively engaged in discussions regarding program maintenance and sustainability that were not discussed with field coordinators responsible for basic pools.
The Pool Cool Diffusion Trial was conducted across four calendar years for two consecutive cohorts of three years each, starting in 2003 and 2004 at swimming pools in 28 metropolitan areas across the United States. Pools were recruited in cooperation with the National Recreation and Park Association (NRPA) using multiple methods: NRPA web site notices, NRPA email list-serves, conference displays, and targeted advertisements in aquatic magazines and NRPA newsletters. Metro regions were required to have at least a minimum population size of 100,000 and at least four outdoor swimming pools willing to participate. Recruited pools were both public (city, county, military,
et al.) and private (YMCA, country club,
et al.). Pools were required to be outdoors, to offer swim lessons to children five to ten years of age, and to be large enough to recruit at least 20 parents to fill out surveys. Lifeguards were not specifically recruited but participated based on their employment at a given study pool. The intervention components, theoretical foundations and examples for each construct, data collection procedures, and findings from the main randomized controlled trial are described in more detail elsewhere [
20‐
23]. The analysis presented in this paper addresses an ancillary aim of the Pool Cool Diffusion Trial that is different from the aims of the main randomized controlled trial.
Discussion
This study used multilevel methods to evaluate the relative contributions of lifeguard-level and setting-level adopter characteristics and treatment group to the implementation of an effective and widely disseminated skin cancer prevention intervention. Several individual-level (lifeguard characteristics) and setting-level (pool characteristics and treatment group) factors were found to be significantly associated with implementation. The most important predictor of implementation was the number of weekly visitors (inverse association) at the pool, closely followed by enhanced treatment group (positive association).
A common measure of the quality and success of implementation is the degree of implementation [
8]. In the context of this study, the degree of implementation was measured by a composite score calculated based on the level of implementation of Pool Cool intervention components by lifeguards, on a scale ranging from 0 to 10. The mean value on this scale was four (SD = 2 in 2004 and 3 in 2005) in both years (2004 and 2005) indicating moderate implementation for most lifeguards. The individual items that were implemented most often were the ones that indicated whether the lifeguard used sunscreen, received sunscreen sample or a message pen, taught the Pool Cool sun safety lessons, and knew the location of and used the Pool Cool's Leader's Guide. These are considered main components at the core of the Pool Cool program [
23].
The intraclass correlation for pools in these data was relatively high (35% in 2004 and 34% in 2005), which underscores the usefulness of a multilevel approach in analyzing the data. It also indicates that about 35% of variance in implementation is explained by level 2 characteristics.
All three lifeguard-level domains significantly contributed to the variance in implementation. Education was the most important level 1 predictor of implementation, suggesting that lifeguards with at least some college education were more likely to implement Pool Cool than lifeguards with a high school education or less. This finding is consistent with conclusions from previous studies showing higher levels of education to higher implementation levels among the adopters [
6,
13,
31].
The adopters' positive attitude toward and their self-efficacy to implement an intervention have been shown to increase the likelihood of successful implementation of evidence-based interventions [
9,
32,
33]. Furthermore, previous implementation research in the physical activity literature found that if the delivery agents themselves practiced the health behavior promoted by the intervention, they were more likely to successfully implement the program [
34‐
37]. In this study, both lifeguard sun protection-related attitudes and sun protection-related behaviors significantly explained variance in implementation, although the individual predictors of sun protective barriers and norms had nonsignificant coefficient estimates. This instability might explain the unexpected, positive relationship between sun protective barriers and implementation.
Six level 2 predictors were included in the final model (number of weekly pool visitors, intervention intensity, latitude, pool location, sun safety and/or skin cancer prevention programs, and sun safety programs and policies), three of which (weekly pool visitors, sun safety environments and policies, and intervention intensity) showed consistent direction of effect and statistical significance across the two years.
The most important predictor of implementation in the final model was the number of weekly pool visitors. In this study, an inverse relationship was observed between the number of weekly pool visitors and the level of implementation for Pool Cool by lifeguards. This variable is a proxy for the size of the pool and might influence implementation fidelity in a number of ways. The most likely explanation for the inverse correlation between the number of weekly pool visitors and implementation is that because pools received the same amount of intervention materials regardless of their size, implementation might have been more limited in larger pools where lifeguards had to share the same amount of resources for more visitors. This explanation suggests that, to increase implementation of the intervention, the amount of intervention materials provided for the pools should be proportional to the number of visitors the pools serve.
There is a growing agreement among researchers and practitioners that more innovative and active approaches enhance the implementation of effective interventions [
36,
38‐
40]. More intensive implementation strategies include but are not limited to tailoring and packaging of the intervention materials in a user-friendly way, enhancing organizational capacity, establishing systems and rewards for implementation, providing training and technical assistance to adopters, and conducting and reporting evaluation of implementation efforts [
9,
16,
33,
41‐
43]. For example, a study by Mueller and colleagues [
44] that evaluated the effectiveness of different strategies for the dissemination of evaluation results on tobacco control programs to program stakeholders found that multi-modal and more active approaches to dissemination increased the usefulness and further dissemination of the evaluation results. Furthermore, previous implementation research studies of skin cancer prevention found mixed results on the effect of intensity of intervention [
14‐
16]. For example, Schofield and colleagues were assessing two strategies for the dissemination of a sun-protection policy in primary and secondary schools in New South Wales, and found that more intensive implementation strategies were more effective in primary schools but not in secondary schools [
14]. In a study conducted by Buller and colleagues using web-based strategies to disseminate a sun safety curriculum to elementary schools and child care facilities, intensity of the intervention (basic versus enhanced website) did not seem to influence the online purchase of the program [
15]. Finally, Lewis and colleagues disseminated a sun safety program to zoological parks and found that more intense implementation strategies resulted in only marginally significant improvement in short-term implementation for certain components of their intervention and no difference was observed for long-term implementation when compared to the basic implementation approach [
16].
In our analysis, treatment group was the second most important predictor of implementation levels. Lifeguards at pools that were randomized to the enhanced treatment group implemented the intervention more than did pools that received the basic treatment. Similar results were found for each subscale of the dependent variable in a post hoc analysis. These findings reinforced the role of more active, multi-component strategies in successful implementation.
Although there were more nonsignificant variables at level 2 (pool characteristics) in 2005 than in 2004, the final models across these two years were consistent. Overall, the patterns in the 2005 final model were similar to the findings from the 2004 analysis and the replication analysis confirmed the robustness of weekly pool visitors and intervention intensity as important predictors of implementation of Pool Cool.
To our knowledge, this is the first skin cancer prevention implementation study using clustered randomized controlled design, including a large number of potential influencing factors and accounting for their multilevel nature. Furthermore, the large sample size and use of two years worth of data with replicate analyses make the findings from this study a robust addition to the existing implementation research literature.
Several limitations of this study should be acknowledged. First, close to 50% of baseline respondents in 2004 and 40% of baseline respondents in 2005 were excluded from the final analysis due to inability to identify the matching follow-up survey responses. During data management, efforts were made to include as much data as possible and to compare baseline information for included and excluded surveys. In order to keep the lifeguard surveys brief, lifeguard perceptions of the intervention characteristics were not measured in the Pool Cool Diffusion Trial. However, extensive information was already available on the acceptability of the Pool Cool program and on the program-related factors that contributed to the implementation of the intervention (
e.g., ease of program implementation, compatibility of program with swim lessons, comments about major program components) from the pilot study, the efficacy trial, and the process evaluation of the Pool Cool Main Trial and the pilot study of the Pool Cool Diffusion Trial (results are reported elsewhere) [
18,
45]. Finally, Pool Cool is a multi-component intervention, and it is not possible to separate out the effects of influencing factors on different components. However, the health behavior literature suggests that in the context of complex, multi-component interventions, the measurement of implementation fidelity should focus on the functions and process of the intervention rather than on the individual components [
46].
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
BAR carried out data management, analysis of the data including multilevel modeling, interpretation of data, and created the first draft of the manuscript. EN was involved with the management of data, participated in the analysis and interpretation of data, and provided revisions on the content of the manuscript. TE coordinated the original data collection and was involved with the data management. ADD was involved with the data analysis (with a special focus on multilevel modeling) and participated in the interpretation of data. She also provided revisions on the content of the manuscript. RCB was involved with the initial conception and design of the analysis and was involved with the data analysis and interpretation and provided revisions on the content of the manuscript. KG led the original conception, design, and acquisition of the data for the Pool Cool Diffusion Trial, supervised the data management and analysis, and participated in the interpretation of data. She also provided revisions on the content of the manuscript. All authors read and approved the final manuscript.