Design preferences for global scale: a mixed-methods study of “glocalization” of an animated, video-based health communication intervention

In 2018, we created a video-based maternal child health (MCH) communication intervention in collaboration with the South African National Department of Health. The intervention was aligned with and intended to amplify their existing, text based, national MCH education program, called The Road to Health. Our shared goal was to design and produce a series of educational videos, intended to optimally engage a broad spectrum of mothers, fathers and caregivers across this culturally diverse country [14]. In designing this intervention, we aimed to overcome four key challenges using existing health communication approaches and digital solutions:

A major goal during the design of the pilot intervention, was to enhance emotional appeal and character identification, highlighted as important strategies in the behavior change, communication science, and education literature [18‐23]. Following the launch of the intervention [24], we received requests to adapt it for use in other global regions. This raised the important question of whether or not the design prototypes would resonate across global audiences.

There is a shortage of research on visual design preferences for health communication across global audiences. In particular, mixed-methods approaches would help to provide both a broad sense of global design preferences, as well as a more nuanced understanding of those preferences. In this study, we summarize feedback from global learners on a set of animation design prototypes developed for diverse global audiences.

In this study, we used a mixed-methods, sequential explanatory study design with a qualitative descriptive approach [25, 26]. The study consisted of a quantitative survey of preferences, followed by qualitative in-depth interviews. We used the results of the quantitative survey to a) inform the selection of interview participants and b) develop the interview guide for the second phase of the study (see Additional file 2).

We applied a Human Centered Design (HCD) approach to develop eight animation prototypes (see Fig. 1) [14]. HCD involves a process of rapid prototyping, iterative cycles of feedback from target audiences and re-design by the development team [27]. The earliest formative feedback, shaping the development of the prototypes [14], was gathered in South Africa in 2018, through our interactions with the South African National Department of Health and a large community-based maternal child health organization engaged in community health promotion. The number of prototypes and their primary health messages were pragmatic decisions based upon the availability of resources, including the expertise of four artists who designed the prototypes according to our stated goals, as well as the real-world health messaging needs of our collaborating health organizations.

We recruited participants for the quantitative survey of preferences from a cohort of learners enrolled in a global, massive open online course, Stanford Introduction to Food and Health [28]. The first author is lead instructor for this course, offered on Coursera, a massive open online learning platform serving more that 35 million learners worldwide. Learners enrolled in the Stanford Food and Health course represent more than 95 countries and enroll in cohorts, with a new cohort beginning every 2 weeks. In April 2019, we gauged interest in participating in a research study, among the most recent cohort of course completers. All learners who expressed interest in participating in research on health communication (n = 642) were invited to complete the preferences survey, where the 8 design prototypes were presented (see Fig. 1). We obtained informed consent digitally via an information sheet, sent to learners by email. Participants were not offered any financial or other incentives for their involvement. Three hundred and thirty participants, from 73 countries, completed the quantitative survey of preferences. We grouped participants by 9 global regions, using the World Health Organization’s definition of 6 global regions and sub-dividing three of these regions (the Americas, Europe and the Western Pacific region) to help further contextualize participants’ responses. The demographic characteristics of our participants are described in Table 1.

Respondents indicated preferences for the animation prototypes in several ways. For each prototype, participants were asked to express whether or not they expected the prototype to resonate with people in their country as well as ranking the prototypes according to their own preferences. Participants were then invited to sort prototypes into categories of would, might, or would not resonate with a global audience and finally, participants rated all prototypes on a 6-point scale.

From the survey responses, we calculated the percentage of respondents characterizing each prototype as “would resonate within my country” as well as the percentage of respondents that ranked each prototype as a) their preferred prototype or b) within their top two preferred prototypes. We then analyzed the survey responses by global region, examining which prototypes were characterized as first, second or third choices for both global and local resonance. Finally, we calculated the average ratings of each prototype on a 6-point scale.

Survey respondents were also invited to share unstructured feedback on each prototype, using an open-feedback comment box embedded within the survey. This qualitative data, collected as part of the quantitative survey, was then analyzed by the research team through open coding. Codes that emerged from the data were organized into themes that were used to inform interview guide development for the in-depth interviews.

A qualitative descriptive approach was used to inform the qualitative portion of this study [26, 29]. The qualitative descriptive method has been used to capture and describe broad perspectives on health interventions, often as the qualitative component of a mixed-methods study [30]. Qualitative description is a naturalistic approach, appropriate for this study as it recognizes the subjective nature of participants’ preferences. Rather than claiming to advance theoretical or conceptual understanding, our aim is to contribute to quality improvements in future animated, global health communication interventions [26, 30].

Telephone interviews are gaining popularity as a data collection method in qualitative descriptive research, especially as they can support maximum variation purposive sampling and help researchers examine the perspectives of international participants [30].

A majority (n = 224, 68%) of survey respondents indicated a willingness to participate in a follow-up in-depth telephone interview. From those who volunteered, maximum variation purposive sampling [31] was used to select 20 respondents from whom we could elicit the widest variety of perspectives (across geographic location, gender, age, years of education, and style rating patterns). Maximum variation sampling has been described as particularly useful in qualitative descriptive research, by Sandelowski and colleagues (2010) [29], as it acknowledges a breadth of diverse perspectives and experiences.

Interview participants were shown short video animations incorporating the animation prototypes they had previously ranked in the online survey. Because the video prototypes were excerpts from real-world MCH interventions, created with collaborating health organizations, the exact health messages varied somewhat across prototypes, including MCH topics such as breastfeeding, early child nutrition and alcohol avoidance in pregnancy. Participants received a single-page reference document containing the eight prototypes and their corresponding names, as presented in the survey. They were then asked to reference the video samples and the prototypes during interviews, which occurred over a two-month period following completion of the online survey. Participants were asked to describe why they felt drawn to or put off by certain prototypes, they were also asked to provide thoughts on characteristics that would be important for globally scalable content.

Interviews were conducted by a co-investigator who is a full-time staff member at Stanford’s Digital Medic South Africa program, a health education and evaluation team based in Cape Town, South Africa. We conducted regular debriefings with the co-investigator who conducted all 20 interviews, as recommended in the literature [32]. This allowed the research team to gain early insights into emerging themes and supported our process of substantive coding. As suggested by McMahon & Winch (2018) [32] debriefings also allowed us to rapidly share emerging data within our production teams and with collaborating stakeholders, thereby shaping the developmental trajectory of ongoing health communication projects.

A combination of conventional content and thematic analysis was used to analyze the interview data, with the intention of keeping the analysis accessible and generating findings that would be readily usable by future creators of animated health communication content. Following each interview, audio-recordings were uploaded and transcribed. Through inductive category development, interviews were coded by two co-investigators (RPC and MA). After reading through the transcripts for the preliminary general analysis, the transcripts were read a second time, with the goal of identifying and highlighting words and phrases that captured key concepts. The co-investigators then made notes of their initial thoughts and impressions with the goal of developing a set of preliminary codes. Specific quotations relevant to each code were collected and organized in a shared Google Drive document. In this way, the research team identified and documented the emerging set of codes, then grouped these into themes, relevant to the development of animated health communication videos for global learners. Rather than assess or strive for high inter-rater reliability that would allow each co-investigator to code entirely independently, we used a process involving constant comparative analysis to independently identify, then verify emerging codes and then organize these into themes. Comparing notes revealed that similar codes were being identified by both co-investigators and validated efforts to ensure that the emerging codes and themes originated from the data, rather than the investigators endeavoring to fit the data into pre-determined categories.

As themes emerged from the early interviews, the interview guide was adapted to include questions focused on these themes. For the purposes of our study, we defined data saturation, in line with prior research [33], as the point at which we had gathered enough information for the study to be replicable and when further coding was no longer feasible (ie: no new codes emerged).

In order to establish trustworthiness, as described by Lincoln and Guba (1985) [34] we consulted with a qualitative researcher at the Heidelberg Institute of Global Health, who reviewed and provided comments on the overall adequacy of this approach. We also created a Google Drive folder containing all audio recording of the interviews, transcripts, data analysis notes, and analytic memos, to establish credibility. In order to ensure transferability, we used purposive sampling and made detailed contextual notes for each respondent.

Most respondents indicated preferences for three or four of the eight design prototypes when asked which of these they felt would be most likely to resonate globally. Across the design prototypes, Mercury was most commonly preferred, with 62% of respondents categorizing it as “would resonate very well for global learners” and 46% ranking it as the first or second most-preferred design prototype. Earth was the second most preferred, with 60% of respondents categorizing it as globally resonant and 32% ranking it as the first or second most-preferred (see Fig. 2).

When assessed by region, Mercury and Earth were frequently ranked within the top three most preferred design prototypes. This preference was consistent for both local and global video interventions across respondents who lived in the USA & Canada, Western Europe, Latin America, the Middle East & North Africa, South Asia and Oceania. Respondents living in East Asia did not commonly include Mercury in their top three preference rankings and those in Sub-Saharan Africa tended to rank Saturn higher than Mercury, (see Fig. 3). These results were similar to those found when analyzing data by the World Health Organization’s global regions, a schema that defines fewer (6 rather than 9) global regions [35].

Of note, respondents in some regions indicated different preference rankings locally compared with the design prototypes they expected to resonate globally. For example, in East Asia and Latin America, respondents frequently preferred Neptune, but did not anticipate it to resonate globally.

Similarly, when asked to categorize the design prototypes by how likely respondents felt they would resonate locally and globally, Mercury and Earth were most frequently included (68 and 67%, respectively). Six of the designs (Mercury, Earth, Saturn, Neptune, Venus and Uranus) were included by more than 50%. Respondents were also able to “dislike” designs they felt would not be preferred by local learners in their regions. Venus was disliked by 34% of respondents. Both Jupiter and Mars were disliked by more than 50% of respondents.

The star rating exercise produced similar results, with Mercury, Earth and Saturn receiving the highest star ratings, and Mars and Jupiter the lowest. A minority of respondents (14%) opted to provide no response to the star rating exercise (see Fig. 4).

Several questions were consistently explored across all interviews. We asked respondents for their views on a variety of considerations applied to the design of animated content for global audiences. Within this context, respondents were probed regarding visual characteristics that either drew participants to certain characters or repelled them, as well as preferences regarding characters, backgrounds/settings and color choices.

Probing questions posed in the original structured interview guide elicited several themes that provided a more nuanced understanding of the design prototype preferences captured through the survey feedback. During the open coding process, several new themes emerged, including codes that related to the use of a narrative approach to animated video health communication and qualities that make a narrated voiceover credible and engaging. Additional emerging themes included optimal video length, ideal number of learning objectives, as well as the portrayal of family roles in health communication videos. After noting these emerging themes, we adapted the interview guide to specifically probe on these topic areas. In this way, the interview guide remained a living document, responsive to feedback from previous interviews. The investigators agreed that saturation had been reached when interviews 15–20 revealed no new codes, in line with definitions in the literature on qualitative in-depth interviews [36, 37]. Summaries of codes and themes that emerged from the data are presented in Table 2.