Ecological validity of cost-effectiveness models of universal HPV vaccination: a protocol for a systematic review

Ecological validity has typically been taken to refer to whether or not one can generalise from observed behaviour in the laboratory to natural behaviours in the world [11]. In fact, ecological validity or the related idea of external validity has a long history in psychology and economics, with the issues central to this idea frequently and continuously debated [12].

The demand for ecological validity stems from the Brunswick’s concept of representative design, a construct used to assess whether an experiment or a model captures the relevant features of intended realities [13]. Using the replica model analogy [14], this can be framed by asking how well the model mimics reality on characteristics relevant to the theory. Central to representative design is the notion that any model can be thought as a sample of behaviours. For results to be generalisable, not only should attributes of participants be representative but also their expected behaviours in the specific task or situation. To do this, investigators need to specify the relevant characteristics and behaviour of the population related to the intended situation. In addition, sampling should take place on two dimensions. One involves the participants; the other concerns the situations or tasks with which the participants are confronted. Valid inferences can only be achieved by sampling in a representative manner on both dimensions.

Applied to cost-effectiveness of universal HPV vaccination, models chosen to inform public immunisation policies should take into account most of the dimensions relevant to the population (individuals susceptible to HPV infection) as well as to the specific situation (patterns of sexual mixing).

Traditionally, patterns of sexual behaviour have been characterised by focusing on the individual as unit of analysis, e.g. by measuring an individual’s number of sexual partners. However, adopting a network approach to the epidemiology of sexually transmitted infections reveals that behaviours and sexual preferences may influence the spread of infection [15]. The ecological theory of infectious disease discusses the characteristics of the individuals and the patterns of the sexual mixing relevant to HPV infection [16]. Gender, age, ethnicity, sexual activity, rate of change of sexual partners and frequency of unprotected sex are characteristics representative of individuals at risk of infection [17‐19]. Concurrent partnership and assortative mixing are relevant to the model’s representativeness of sexual mixing. Concurrent partnerships are those in which the same individual is involved in sexual partnerships occurring simultaneously, as opposed to “serial monogamy” where one partnership ends before another one starts. The proportion of concurrent partnerships effectively increases the risk of sexually transmitted infections in the population [20, 21]. The risk of infection varies according to whether partnerships are formed between people from similar (positive assortative) or different (negative assortative) prevalence and sexual activity groups. Sex with partners of the same sex (positive) [22], sex with older partners (negative) [23], sex with a foreign partner (negative) [24] and pay for sex (negative) [25] are the four behaviours which increase the heterogeneity in the population, since some susceptible individuals are more likely to acquire and transmit the infection. If the high-risk individuals mix randomly, the infection is more likely to spread [26].

Building upon the same methodological premise of ecology of infectious disease, the Natsal survey is aimed to understand the most current characteristics of sexual partners and partnerships in the general British population. Three Natsal surveys have taken place up to day: Natsal-1 in 1990–1991, Natsal-2 in 1999–2001 and Natsal-3 in 2010–2012. In the Natsal-3 [27, 28], the following five characteristics have been identified to be representative of the participants (sexually active individuals):

The following five behaviours have been identified as representative of situation (sexual mixing):

Individual characteristics and sexual mixing behaviours are relevant to the ecological validity of modelling sexually transmitted infections. For the normative outcomes of cost-effectiveness of universal HPV vaccination to be generalisable and inform public health policies, claims need to be established for the representative design of their underlying models. Ex ante, valid inferences should only be drawn by models which include all the eleven dimensions relevant to individuals and their sexual partner network. Ex post, the relative degree of representativeness of cost-effectiveness studies can be assessed by the number of relevant characteristics and behavioural patterns taken into consideration by their underlying models.

The aim of this review is to test the ecological validity of the cost-effectiveness models of universal HPV vaccination available in the literature.

Ecological validity refers to the relation between real-world phenomena and the investigation of these phenomena in experimental contexts or models. This can be framed by asking if the models provide an acceptable representation of the intended reality on all relevant dimensions. In the context of sexually transmitted diseases, patterns of sexual behaviour have been traditionally characterised by focusing on the individual as unit of analysis, e.g. by measuring an individual’s number of sexual partners. However, adopting a network approach to the epidemiology of sexually transmitted infections reveals that behaviours and sexual preferences may influence the spread of infection.

For results to be generalisable, not only should attributes of participants be representative but also their expected behaviour in the sexual mixing.

The review seeks to answer the following research questions:

The protocol is registered in the International Prospective Register of Systematic Reviews (PROSPERO) CRD42016034145, available at: http://​www.​crd.​york.​ac.​uk/​PROSPERO/​display_​record.​asp?​ID=​CRD42016034145

The current stage of the systematic review, as defined by PROSPERO, is:

2. Piloting of the study selection process.

Inclusion criteria are as follows:

Exclusion criteria:

The following bibliographic databases will be searched: MEDLINE (via PubMed); Scopus; Science Direct; EMBASE (via OVID SP); Web of Science; EconLit (via EBSCO); DARE, NHIR EED and HTA (via CRD); and CINHAL Plus. An additional search for grey literature will be conducted on Google Scholar and Open Grey. Reviews will be included to reduce the possibility of missing relevant articles.

A search strategy will be developed for each of the main databases. Additional file 2 reports details of our planned bibliographic database search strategies for MEDLINE (via PubMed); Science Direct; EMBASE (via OVID SP); Web of Science; Scopus; CRD; CINHAL Plus; EconLit (via EBSCO); Google Scholar; and Open Grey.

The PubMed “related articles” search feature will be used to reduce the risk of missing relevant articles. References of the included studies will be searched in order to identify additional relevant studies missed. A pilot of the study selection process will be conducted before initiating the systematic search of relevant articles.

Two graduate research associates (GRAs) will independently identify through database searching, screen, assess for eligibility and include in the review the relevant studies. Opinion of one of the reviewers will be sought to arrive at a consensus in case of disagreement on a study for inclusion.

Data will be extracted and recorded independently by two additional graduate research associates (GRAs) from the studies included in the review, according to a predefined data extraction table reported in Additional file 3.

Data extracted will be clustered and prioritised: the main outcomes will report the inputs to the demographic and epidemiological model, while additional outcomes will refer to basic inputs to the cost-effectiveness valuation.

Monetary values in different currencies will be transformed in US dollars using purchasing power parities (PPPs). Subsequently, all the monetary values will be adjusted to 2015 dollar value using the US consumer price index (CPI). Corresponding authors of various studies may be contacted by email to clarify methods and results if the need arises.

Two external experts (a health economist and a clinical oncologist) will independently assess the risks and threats to validity relative to each of the studies included in the review. The critical appraisal of included economic evaluations will be carried out in two subsequent stages:

In case of misalignment of views, differences will be resolved in a dedicated meeting among reviewers.

Each study included in the review will be scored by the number of representative characteristics and behaviours taken into consideration (yes or no) on both dimensions. Individual study’s scores will be plotted in a 2 by 2 matrix [34] reported in Additional file 4.

The upper right quadrant represents the ideal situation, in which both individuals’ characteristics and their sexual partner networks are representatives; hence, the studies are ecologically valid. In the upper left quadrant, we have models describing representative individuals but not in a representative network. In the lower left quadrant, the network is representative but the individuals are not. Finally, in the lower right quadrant, neither the individuals nor the network are representative. Only models in the upper right quadrant provide normative economic recommendations that are fully representative, hence generalisable to inform public health policies.