Comparison of two statistical indicators in communicating epidemiological results to the population: a randomized study in a high environmental risk area of Italy

Risk communication is an important but difficult task, which poses numerous challenges for scientists, health practitioners and policy makers. Research has investigated how risk and benefit judgement in different fields may differ between experts and lay people [1, 2], however this discrepancy does not necessarily imply lay people’s higher tendency to misunderstanding or misperceptions [3]. Indeed, it may reflect different perspectives and judgements as a result of several factors influencing the risk evaluation process, including personal attitudes, general knowledge on the subject, media portrayals of hazards and risks, public perceptions, social roles and networks [3‐5].

The formatting of the information seems to be an important factor in risk communication. Several studies have investigated how the use of alternative numerical and/or graphical formats to express risks and benefits related to a medical diagnosis or a treatment may influence patient’s decisions, confidence on such decision or level of concern, and two recent systematic reviews have concluded that different formats seem to have an impact on perceived magnitude of the risks [6, 7]. Sensitivity to result format seems to be an important aspect to consider also in the context of public health issues at the population level. This becomes even more important in the context of “community based participatory research” paradigms that call for a collaborative process in which the communities are involved to “co-create knowledge” [8].

In environmental epidemiology, no previous study has compared the use of different statistical formats in communicating results to the general population, despite, especially in environmental emergency contexts, communication plays a crucial role. Hence, with the present paper we aim at investigating whether the degree of concern expressed by residents of a high-risk area about epidemiological results on cancer mortality in the same area is influenced by the statistical indicator used to communicate such results [9]. In particular, we report the results of a randomized study conducted on a sample of residents in the city of Livorno (159,431 inhabitants - western coast of Tuscany), which, together with the neighboring municipality of Collesalvetti (16,791 inhabitants), is classified as a high-risk environmental site, according to the Seveso Directive [10‐12], due to the presence of a large commercial harbor and several petrochemical plants producing dangerous pollutants [13]. According to what frequently done in similar studies performed in clinical context [7], the randomized study compared a relative risk indicator with an absolute risk indicator, when used to communicate results about the health profile of the population in the area of interest (see Methods).

The main characteristics of the 340 respondents, including socio-demographic variables and other covariates included in the PS model, are reported in Additional file 4: Table A1.

The average degree of concern about overall cancer mortality (question R3) was high, equal to 8.30 (on a 1–10 scale), with standard deviation equal to 1.92. Around 91% of sample expressed a degree of concern higher than 5 (see Additional file 5: Figure A1). Notably, 8 and 10 modalities of question R3 received an unexpected amount of preference in respect to 9. Regarding question R4 (“which result is the most concerning and which one is the least concerning to you”), most of respondents declared to be particularly concerned about lung cancer mortality (62.9%), while thyroid cancer was selected as the item causing the lowest concern by 48.8% of the interviewees.

Weighting using propensity score improved covariates balance, in particular for interviewer indicator and risk attitude (see Additional file 6: Figure A2). Few units were excluded from the analysis in order to guarantee overlapping between the PS distributions under % excess and under TNH.

Tables 2 and 3 report the estimated effect of TNH-questionnaire versus % excess-questionnaire on each outcome variable (Average Causal Effect, ACE) and the weighted mean of the outcome, had all individuals been assigned to the % excess-questionnaire, considered here the control condition.

As reported in Table 2, when epidemiological results were expressed in terms of TNH, the average concern about cancer mortality due to “living in Livorno” increased by 0.31 units (p = 0.128), on a 1 to 10 scale, being the potential outcome mean under % excess equal to 8.15. Correspondently, the percentage of subjects declaring a concern larger than 5 increased by 6% (p = 0.051) under TNH, being the percentage under % excess equal to 88%.

The sensitivity analysis based on quantile regression provided results consistent with the estimated mean difference of 0.31 units arising from the main analysis (see Additional file 7: Figure A3). Analogously, using different cut-offs for the degree of concern (from 2 to 4 and from 6 to 9), the estimated probability of scores above the threshold still remained larger under TNH than under % excess, although the uncertainty of the results increased with higher cut-offs, reflecting the fact that, by increasing the cut-off, the probability of scores above the threshold progressively approached 50% (see Additional file 8: Figure A4).

Figure 1 reports the cumulative rankograms for the three causes of deaths compared in question R4 (“which result is the most concerning and which one is the least concerning to you”), under % excess and TNH. The cumulative rankogram for a specific cause of death represents the probabilities that that cause is classified among the k most worrisome ones, where k ranges from one to three. For each cumulative rankogram, we calculated also the surface under it, so-called SUCRA, which can be used to define a hierarchy among the three causes of deaths, with larger SUCRA values indicating higher degree of concern [24]. While judgement about lung cancer mortality, classified as the most worrisome item, was similar under the two indicators (SUCRA = 0.748 and 0.765 under % excess and TNH, respectively), a certain difference was observed for thyroid cancer and sexual glands cancer. The adjusted analyses confirmed these descriptive results (Table 3). No significant change in rank was observed for lung cancer mortality, while, compared with % excess, TNH caused sexual glands cancer mortality to be rated as more severe (difference between average ranks = − 0.13; p = 0.093) and thyroid gland cancer mortality to be rated as less worrisome (difference between average ranks = 0.19; p = 0.021). The adjusted SUCRA values, obtained as a simple transformation of the average ranks arising from IPTW regressions, were very similar to the unadjusted ones (Table 3).

Among people with high school diploma or higher, the probability of expressing degree of concern greater than 5 increased by 9.1% (p = 0.020) under TNH, taking % excess as reference (Table 2). Among people with intermediate school diploma there was no evidence of a difference between the two indicators. In the subgroup with higher mathematical skills, the probability of declaring a degree of concern greater than 5 increased by 7.5% (p = 0.033) under TNH, taking % excess as reference. Among people with low mathematical skills, the same ACE estimate was negative and affected by large variability (Table 2).

As shown in Fig. 2, the ranks assigned to the three causes of deaths in question R4 by the respondents with intermediate school diploma or lower mathematical skills did not depend on the statistical indicator used. On the contrary, an effect of the indicator was found among the respondents with higher educational level or higher mathematical skills, limited to the ranks assigned to sexual gland cancer and thyroid cancer (for details see Additional file 9).

In clinical context, several studies compared the use of alternative indicators of relative risk and absolute risk in communication to patients [7, 25]. This is the first study that tries to make something similar in the field of environmental epidemiology, with a randomized experiment in an area at high environmental risk. A sample of citizens was informed about epidemiological results derived from scientific work conducted on their own area. We focused on the individual degree of concern induced by communicating epidemiological results using two alternative risk indicators; investigating the level of understanding of the numerical messages by the respondents was out of the aims of our study.

We found that judgments about local risks for population health were influenced by how these risks were communicated. Specifically, a measure similar to the Number Needed to Harm, which we called Time Needed to Harm, appeared to cause slightly greater concern in citizens than % excess risk, when the overall result concerning mortality from different types of cancer was communicated.

Expressing the impact in terms of TNH led people to rank mortality from sexual glands cancer as slightly more alarming and mortality from thyroid gland cancer as slightly less alarming, as compared with expressing the same results in terms of % excess. On the contrary, no change was evident for the relative judgment about lung cancer, which was ranked as the most worrisome disease under both the experimental conditions. This result may be due to a complex mixture of factors that go beyond the numbers communicated, including role of health information campaigns and people’s knowledge, experience and personal view about severity and curability of the three diseases, and about factors than can cause them.

Educational level and numeracy influenced health risk evaluation, confirming evidence reported elsewhere [26, 27]. In particular, the observed differences between indicators were larger when the subject had higher education or better numerical skills. A possible explanation of this result is that people with higher education level/mathematical skills tend to pay more attention to numbers, as compared with people with lower educational level/mathematical skills. On the contrary, the latter group may be more influenced by other factors, for example personal experiences and views.

The role of individual experience in assigning scores or ranking diseases had emerged as relevant also during the cognitive interviews conducted to build the questionnaires: people tended to consider as more worrisome those diseases that they experienced directly or indirectly during their life, and the responses were sometimes influenced by the personal knowledge on the environmental pollution in the city and its relationship with specific diseases [28]. For this reason, even if investigating the role of individual experience of the interviewees with the diseases mentioned in the questionnaire was out of our aim, we introduced in the final questionnaires a close-ended question to investigate the possible reasons of the response to R4 (“which result is the most concerning and which one is the least concerning to you”) [9]. Only 32% of the 340 respondents declared to have replied on the basis of the numerical data presented, while the remaining 68% declared to have replied on the basis of personal knowledge or experience about the illness (37%) or on the basis of personal knowledge about pollutants released in the study area (31%).

Our experiment shows that communicating epidemiological results to the population is not a neutral task. In fact, the degree of concern induced by the presentation of results about community health, as well as the ranking of concern when comparing the results on different diseases, may depend on the risk indicator used.

Additionally, we found that the impact of using different numerical formats may vary according to individual characteristics, such as education level or mathematical skill. In particular, people having higher education level/mathematical skills seem to be more influenced by the numerical formats of the message than people with lower education level/mathematical skills, who tend to express the same degree of concern independently from the statistical indicator used in communication. This likely originates from a different ability of the message to reach different individuals: the higher the actual or perceived ability to understand numbers, the more attention is given to the numerical content of the message; the lower the actual or perceived ability to understand numbers, the higher is the role of the a priori knowledge in interpreting the message. This result supports the idea that ignoring such factors, that may hamper or facilitate communication of health risks, can lead to unequal information and, as a consequence, unequal protection/prevention across society [32]. Therefore, communication strategies shared by different actors are needed, which account for the heterogeneity of the population to whom the message is addressed.