Theories of change for e-health interventions targeting HIV/STIs and sexual risk, substance use and mental ill health amongst men who have sex with men: systematic review and synthesis

Our methods were guided by a protocol prospectively registered on PROSPERO in September 2018 (registration number CRD42018110317) [49].

In this paper, we report what intervention theories of change were used in eligible studies that were retrieved as part of a broader systematic review. Reports eligible for the broader review were published in 1995 or later (because e-health interventions were unavailable prior to this); focused on gay, bisexual and other men (including cisgender and transgender men) who have sex with men; and reported on e-health interventions delivered via mobile phone, internet or other electronic communication technology that aimed to provide ongoing support to populations consisting entirely or principally of MSM to prevent HIV, STIs, sexual risk behaviour, alcohol and drug use or common causes of mental ill health (anxiety or depression). We excluded e-health interventions merely facilitating one-off as opposed to ongoing support, those addressing HIV self-testing, clinic attendance or STI partner notification only, and interventions delivered by human providers via electronic media. There were no limitations by language or geographical location of studies. Reports eligible for inclusion in the theory of change synthesis described intervention theories of change, logic models or theorised mechanisms of effect.

Our search strategy, refined after piloting, covered two core concepts: men who have sex with men; and e-health. Publication date was limited from 1995 to now and no language limits or limits by study methodology were added. We searched 19 health and social science bibliographic databases, 23 October-26 November 2018. These were ProQuest Applied Social Sciences Index & Abstracts (ASSIA) (1987-current); Campbell Library (complete database); CRD Databases (complete databases); EBSCO CINAHL Plus (complete database); Wiley Cochrane Library (complete database); EPPI-Centre Database of Health Promotion Research (Bibliomap) (complete database); ProQuest Dissertations & Theses Global (1951-current); OvidSP EconLit (1886 to October 18, 2018); OvidSP Embase (1980 to 2018 week 43); OvidSP Global Health (1910 to 2018 week 41); OvidSP Health Management Information Consortium (HMIC) (1979 to July 2018); ProQuest IBSS (1951-current); Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Daily (1946 to October 22, 2018); OvidSP PsycInfo (1806 to October 2018 week 3,); Web of Science Science Citation Index Expanded (1970-present, data last updated 24 October 2018); Elsevier Scopus (complete database); OvidSP Social Policy & Practice (201807); Web of Science Social Sciences Citation Index Expanded (1970-present, data last updated 24 October 2018); ProQuest Sociological Abstracts (1952-current as of 29 October 2018). Full details of the search strings for each are available at the London School of Hygiene & Tropical Medicine’s Data Repository [50] and full details of the search conducted in the OvidSP Medline database are available in Additional file 1. We also searched three clinical trials registers for relevant ongoing and unpublished studies (ClinicalTrials.​gov; the World Health Organisation International Clinical Trials Registry Platform; the EPPI-Centre Trials Register of Promoting Health Interventions); Google for non-governmental organisation and governmental publications (limited to the first 100 results); the OpenGrey database; and references of included reports. We contacted experts to request eligible reports. Citations were uploaded to EndNote and de-duplicated. Database and trials register searches were updated 22-27 April 2020.

Two reviewers piloted an inclusion criteria worksheet by screening batches of the same 50 references, resolving disagreements by consensus. After piloting achieved an agreement rate of at least 95%, each reference was screened on title and abstract by one reviewer. A second round of screening with a comparable piloting process then focused on full study reports. Where reports provided insufficient information on the intervention to determine eligibility, we contacted study authors. Two reviewers independently extracted data from each eligible report on intervention theory of change constructs and mechanisms, the evidence presented in support of the theory of change and how the theory of change was developed. We extracted data on intervention theory of change as well as the existing scientific theories of behaviour and behaviour change that informed these.

Two reviewers independently assessed the quality of each report included in the theory of change synthesis (i.e. each “theory report”) using a tool used in our previous systematic reviews [51, 52], modified to include assessment of whether intervention mechanisms were theorised as operating differently by context [53]. Quality assessment focused on (a) the extent to which the theory of change described the path from intervention to outcomes; (b) the clarity with which theoretical constructs were defined; (c) the clarity with which causal inter-relationships between constructs were defined; (d) the extent to which the mechanisms underlying these inter-relationships were explained and (e) the extent to which the theory of change considered how mechanisms and resultant outcomes might vary by context. The two reviewers met to compare their assessments, resolving any differences through discussion. Rather than restricting the synthesis to studies judged to be of high quality, conclusions drawing on poorer quality reports were given less interpretive weight.

We undertook synthesis of author narratives describing theories of how interventions were intended to generate outcomes. We aimed first to summarise theories of change for specific interventions and then to examine whether there were one or more overarching theories of change relevant across different interventions. Theory synthesis commonly uses a meta-ethnographic approach, originally developed to synthesise findings across multiple qualitative studies [54], and now applied to theory synthesis. As originally applied to qualitative research, meta-ethnographic methods draw on primary constructs (verbatim qualitative data presented in reports of primary research) and secondary constructs (author interpretations of data presented in primary research) to develop tertiary constructs (reviewer interpretations presented in syntheses). Applied to theory synthesis, such methods draw solely on primary constructs (author descriptions of theory of change).

We initially planned to undertake line-by-line coding of theory reports in order to identify narrative themes within, and common themes across, intervention theories of change as we have done in previous theory syntheses [51, 52]. In the first stage of our analysis two reviewers piloted this approach, using data extracted from the two highest-quality studies of similar interventions [55, 56]. We applied line-by-line codes, beginning with in vivo codes which closely reflected the words used in the theory reports. We then grouped and organised codes, applying axial codes reflecting higher-order themes and met to compare and contrast the resulting coding. Because this approach did not readily capture the complex and well-described interrelationships between theoretical constructs present in the reports, we instead decided to develop a novel diagrammatic approach to theory synthesis. This methodological innovation allowed us to summarise the components of each intervention’s theory of change and the explicit and/or implied causal relationships between them, drawing on text and diagrams present in the studies. Summarising these diagrammatically also facilitated comparison and synthesis of these components and relationships across included theories of change. Specifically, we drew diagrams of theories of change based on author text and diagrams, first for each intervention and then for overarching theories of change, which applied across multiple interventions. Like the approach we had initially planned, this novel method of theory synthesis was a form of qualitative synthesis, but one that aimed to describe theories of change primarily in terms of constructs, inter-connections and interactions rather than as narrative themes. Like conventional thematic analysis, it involved an initial stage of ‘in vivo’ coding of author descriptions to identify theories of change for each intervention (but expressed diagrammatically rather than as a set of narrative themes), followed by a stage of ‘axial’ coding to explore inter-connections between in vivo coding, identifying similarities and differences across interventions to develop overarching theories of change (again expressed diagrammatically).

Where more than one publication reported on the same intervention, reviewers drew on the theory of change descriptions from all relevant reports to inform the diagram. The two reviewers then met to compare the two diagrams for each intervention and reconcile discrepancies through discussion. Drawing on the strengths of each, we developed an overall diagram of each intervention’s theory of change, which included intervention components, mediators and moderators (where discussed by authors) and intended outcomes. Where author descriptions implied but did not explicitly state inter-relationships between components of the theory of change, reviewers made inferences and noted where the diagrams were in part based on such inferences.

Finding that the theory of change approaches underpinning the interventions were not patterned by targeted outcomes, we took an inductive approach, grouping diagrams of theories of change that shared important constructs. Then, using reciprocal translation (to identify and describe similar concepts occurring across theories of change underpinning different interventions), refutational synthesis (to identify contradictory or opposing concepts occurring across theories of change) and line of argument (to synthesise distinct elements occurring across theories of change that form part of a broader whole) approaches from meta-ethnography [57], each reviewer independently analysed the diagrams within each grouping. They did this by systematically examining the constructs and the relationships between constructs presented in each intervention-specific diagram and by examining whether they recurred, appeared only once or conflicted with those depicted in other intervention-specific diagrams within the grouping. Based on their analyses, each reviewer then independently drafted one synthesised diagram for each grouping of similar intervention theories of change.

We documented each stage of this process, noting where theory of change components or relationships between these components differed between individual diagrams within the grouping; the approach taken to synthesise these components (i.e. reciprocal translation, refutational synthesis, line of argument or the exclusion of a theory of change component); and the resulting decision for the synthesised theory of change diagram. The two reviewers then met to compare their respective synthesised diagrams for each grouping, reconciling discrepancies and drawing on the strengths of each to develop a single synthesised theory of change diagram for each theory of change grouping. To demonstrate this process, Additional file 2 presents the theory of change diagrams for each individual intervention in one grouping and the resulting diagram of the synthesised theory of change for that grouping. Each synthesised theory of change was given a descriptive title inductively drawing on the central approaches of the theories of change synthesised.

In this application of meta-ethnographic methods to the synthesis of theories of change, our first-order constructs were the theory of change information described in theory reports and represented in data extraction forms; our second-order constructs (analogous to in vivo codes) were the reviewers’ interpretations of these concepts, represented in the intervention-specific theory of change diagrams and our third-order constructs (analogous to axial codes) were the higher-order interpretations, represented by the diagrams of the synthesised theories of change developed for each inductive grouping.

Our search yielded 26,044 unique references, of which 37 were included in the overall review. Of these, 33 reported on intervention theories of change (see Fig. 1).

The 33 reports included in the theory of change synthesis represent 28 unique studies and 23 interventions [55, 56, 58‐87]. Two interventions—a tailored and a non-tailored version of the Cognitive Vaccine Approach, both reported by Davidovich et al.—share a single theory of change [70], resulting in 22 unique theories of change included in the synthesis. Table 1 summarises each intervention theory of change; evidence supporting the theory of change; how the theory of change was developed and existing scientific theories on which it draws.

Agreement between independent reviewer assessments was good, ranging from agreement on none of the five quality criteria (N = 1) to 5/5 (N = 15), with agreement of 4/5 or above for more than three quarters of reports (N = 26; 79%). All disagreements on independent quality assessments were resolved by discussion. Table 2 shows quality assessment results for each theory report.

Quality varied notably across theory reports and clustered towards low and medium quality, with 14 reports (42%) meeting none or one of the assessed criteria [61, 63‐65, 76, 77, 79‐82, 84, 86‐88]; 14 (42%) meeting two or three criteria [56, 58, 59, 62, 66‐68, 71‐75, 78, 83] and only five (15%) meeting four or five criteria [55, 60, 69, 70, 85].

Quality varied across the five criteria assessed. Nearly three quarters of theory reports (N = 23; 70%) clearly defined constructs or concepts and about half described a pathway from intervention components to intended outcomes (N = 16; 48%). Slightly more than a third clearly described how theoretical constructs were inter-related (N = 12; 36%). Only four (12%) described underlying mechanisms and the same number described how mechanisms and outcomes might vary by context.

Authors cited a number of existing scientific theories informing intervention theories of change. Interventions often drew on more than one existing scientific theory. The information-motivation-behavioural skills (IMB) model and social cognitive theory were the most commonly cited, with the former informing seven interventions [55, 56, 60, 61, 64, 68, 70, 77‐80, 83, 87] and the latter informing eight [62, 63, 66, 67, 69, 72, 73, 76, 81, 84]. The former was initially developed to inform HIV prevention but has since been advanced as a model for conceptualising individual and social determinants of health behaviours more broadly [90], and the latter incorporates individual, social and structural factors shaping, and techniques for changing, health behaviours [91]. One report citing social cognitive theory also cited its predecessor [92] social learning theory [72], and one report cited the social-personal framework which the authors describe as building on social learning theory [88]. The Smartphone Self-Monitoring intervention was underpinned by the theorised role of self-monitoring in supporting self-management [85], and we noted that self-monitoring is a core construct of social cognitive theory [89].

Other scientific theories of behaviour commonly used to inform health interventions informed between one and three interventions each: the health-belief model [70, 81], which focuses on the role of individual beliefs about health problems [93]; the theory of planned behaviour [59, 69, 70], which takes into account individual and social factors and theorises that intentions and perceptions of behavioural control are the direct precursors to behaviours [94] and social support theory [81], which theorises that support or perceptions of support from people who are trusted can reduce the stress of, and improve coping with, difficult events [95]. Content for the Sexpulse intervention was informed by the sexual health model, which identifies ten components essential to healthy sexuality [96], theorising that ‘sexually healthy persons are more likely to make sexually healthy decisions’ ([82], p. 2). In addition to social cognitive theory, the Role-Playing Game intervention was informed by stress and coping theory and by the emotional learning framework [62]. In regard to bullied youth, the former posits that youths’ appraisals of their experience predict their coping strategies, with those who blame themselves, perceive little control and view bullying as a threat instead of a challenge tend to use non-productive coping strategies. The emotional learning framework specifies four health-promoting competencies, described as ‘awareness of self and others, responsible decision making, positive attitudes and values, and social interaction skills’ ([62], p. 6).

The MOTIVES intervention was rooted in behavioural economics [74], which examines how actors make decisions other than via conscious reasoning. Two interventions were underpinned by theory on interactions between cognitive and emotional factors: myDEx was informed by a ‘dual processing, cognitive-emotional decision making framework’ (p. 4), which recognises that affective states (emotions) and cognitive states (thinking) both influence decision-making, and that the former might be processed more quickly than the latter [55]. The SOLVE intervention was informed by learning from neuroscience highlighting the important role emotions play in decision-making [69].

Scientific theories of behaviour change were also cited, such as the transtheoretical model [73, 76], which maps stages and processes by which people change their behaviours [97]. The Hot and Safe M4M intervention incorporated strategies from motivational interviewing by assessing, and delivering exercises based on, participants’ ‘readiness to change’ ([68], p. 552). Two interventions were rooted in cognitive behavioural therapy (CBT), which examines inter-relationships between cognitions and behaviours and how these may be modified: the Online Mindfulness-Based Cognitive Therapy integrated mindfulness and CBT approaches [58] and the Rainbow SPARX intervention used a computer game format to deliver CBT [75].

As evidence to support intervention components and theories of change, theory reports often cited evaluations of earlier iterations of the current intervention or of similar interventions, or previous research on the scientific theories in question. Where authors discussed how the intervention theory of change was developed, these were commonly informed by formative research, existing interventions, literature on the needs of the target group, components of the scientific theories underpinning the intervention or a combination of these.

All intervention theories of change identified intended intervention components, mediators and outcomes which could be incorporated into an intervention-specific theory of change diagram except for the theory underpinning the Internet-Based Safer Sex Intervention [76], which did not identify mediators. Two theories of change also identified participant characteristics theorised to moderate the relationship between the intervention and its intended outcomes [55, 60, 74].

Our grouping of theories of change based inductively on their key constructs resulted in three groups of intervention theories of change. The largest, the ‘cognitive/skills’ theory of change grouping [55, 56, 59, 61, 65, 67, 68, 70‐73, 77‐80, 83, 84, 87, 88], was informed largely by social cognitive theory and the IMB model. The second grouping drew on two theories of change driven primarily by self-monitoring (the ‘self-monitoring’ theory of change grouping) [81, 85], and the third drew on two theories of change based on cognitive therapy approaches (the ‘cognitive therapy’ theory of change grouping) [58, 98]. Five theories of change did not fall within any of the three inductive groupings [62, 69, 74, 76, 82, 86]. The ‘cognitive therapy’ theory of change grouping comprised only mental health interventions. The other two theories of change groupings were not patterned by intended outcomes.

We drew on thirteen intervention theories of change, which varied in quality from low to high, to develop a diagram for the ‘cognitive/skills’ grouping (Fig. 2) [55, 56, 59‐61, 63‐68, 70‐73, 77‐79, 83, 84, 87, 88]. Ten of the 13 theories of change in this grouping referenced social cognitive theory [56, 63, 66, 67, 72, 73, 83, 84, 87] and/or the IMB model [55, 56, 60, 61, 64, 67, 68, 70, 71, 77‐80, 83, 87], often in combination with other scientific theories. Three referenced the theory of planned behaviour [59, 70, 88] and one referenced no existing scientific theories but shared key constructs with other theories of change in this grouping [65]. Whilst other interventions also drew on these scientific theories, the theories of change in this grouping shared as core components constructs that are key to these three scientific theories. All interventions represented in this grouping targeted sexual health outcomes, either alone or in combination with substance use or both substance use and mental health. Two interventions not included in this grouping were primarily rooted in different approaches or scientific theories but included components both overlapping with and complementing those in this grouping [74, 76]. Using the line-of-argument approach, we drew on these to augment findings within the ‘cognitive/skills’ grouping and these additions (theorised moderators and the provision of information based on assessed stage-of-change) appear in the overall diagram.

Interventions in this grouping provided information and exercises and incorporated modelling techniques. These activities were theorised to impact on mediators including knowledge, outcome expectancies, attitudes (including internalised homophobia [65, 78]), perceived norms, perceived vulnerability/risk (a construct combining via reciprocal translation those of perceived vulnerability [64, 70, 71, 77‐80] and risk perceptions [59, 65, 73]) and behavioural skills. Two interventions whose theories of change informed this grouping aimed to impact connectedness to the LGBT community [65, 78], portrayed as a mediator in the synthesised diagram though the reports were not explicit about how intervention activities aimed to engender this connectedness or how it might be related to sexual health outcomes.

We inferred that exercises would develop behavioural skills, indicating this inference by use of a dotted line in Fig. 2, though behavioural skills could be developed through other activities. For example, in the video-based Sex Positive! intervention characters modelled behaviours and this was theorised to increase self-regulatory skills [72]. Theory reports did not describe clear and recurring pathways from modelling to mechanisms of change, but we inferred (denoted by use of a dotted line) from recurring descriptions that these aimed to promote self-efficacy [72, 73].

Theory reports differed as to the activities intended to influence outcome expectancies, attitudes, perceived norms and perceived vulnerability/risk. We portray the general relationship between the intervention and these mediators in the diagram by use of a block arrow. Knowledge and behavioural skills were both described as impacting behaviour either directly or via self-efficacy, so both pathways are depicted in the synthesised diagram.

Outcome expectancies—positive and negative expectations of the results of a particular behaviour [91]—were theorised as a component of motivation in the theory of change for the WRAPP intervention [56] and this was also implied in the report for the Sex Positive! intervention [72]. The theory of change for HealthMinder highlighted outcome expectations [84] as a key mediator without discussing motivation. Given these differences, we included outcome expectancies as a mediator and made the assumption that it influences motivation in the synthesised diagram.

Motivation and self-efficacy both featured prominently in this grouping’s intervention theories of change. Whilst some reports suggested that motivation and self-efficacy influence each other in one direction or the other [56, 68], most did not address their inter-relationship, either portraying the constructs as independent [78, 79] or including only one [70, 72, 73, 84]. The synthesised theory of change diagram therefore portrays both constructs as impacting behaviour independently via a line-of-argument synthesis.

Whilst some intervention theories of change included either motivation [59, 65] or intentions [56, 67, 68, 83, 87], those that included both variably portrayed these constructs as influencing each other in one direction or the other [55, 60, 78] or both combined into one construct [61, 70]. We have therefore combined these constructs via reciprocal translation in the synthesised diagram.

We interpreted the construct of perceived behavioural control [55, 70] to be similar to the concept of self-efficacy because the latter was variously described as including an ‘ability to refuse to have anal sex if a condom was unavailable’ ([56], p. 7), ‘confidence in practicing safer sex behaviours’ ([77], p. 3003) and/or the ‘ability to avoid the situational temptation to have unprotected sex’ ([77], p. 3003). Self-efficacy and perceived control were therefore merged via reciprocal translation in the synthesised theory of change diagram.

Theorised moderators are shown at the top right of Fig. 2. In a few cases, theory reports discuss how these might operate [55, 74]. Bauermeister et al. theorise that MSM’s ability to enact sexual risk reduction behaviours might vary by partner type (casual encounter, romantic interest or friend with benefits) and that MSM experiencing stressors related to being a sexual minority, experiencing psychological distress or using alcohol or drugs might have less behavioural control, limiting their ability to ‘regulate their affective motivations’ (p. 5) and enact sexual risk reduction behaviours [55]. Linnemayr et al. theorise that participants in the MOTIVES intervention (which is not included in this grouping but augments its findings via line-of-argument synthesis) who are more acculturated to the USA and can more easily communicate with healthcare providers might face fewer barriers to HIV testing, thus benefiting less from the intervention [74].

Finally, some interventions in this grouping [68, 77] and some from outside of it [74, 76] aimed to boost intervention impact, for example, by recruiting participants at a critical point for behaviour change [77]; delivering content based on the participant’s assessed stage-of-change [68, 76]; or using prize lotteries to incentivise knowledge retention [74].

We defined a grouping focused on theorised ‘self-monitoring’ (Fig. 3) by synthesising theories of change underpinning Smartphone Self-Monitoring [85], an intervention addressing all three health outcomes examined in our review, with a theory report assessed as of high quality, and the TXT-Auto intervention [81], a sexual health and substance use intervention with content tailored by the user’s risk profile, with a theory of change description assessed as of medium quality. The synthesised theory of change posits that self-monitoring via frequent behavioural surveys prompts the user to reflect on their behaviour and reinforces the desired behaviour, which increases behavioural regulation and leads to behaviour change.

Both interventions in this grouping asked the user questions about their behaviour to prompt self-monitoring, and both were driven by self-monitoring as a key mechanism. Frequency of self-monitoring varied by study and outcome, ranging from four times per day [85] to weekly [81]. The theory report for Smartphone Self-Monitoring suggested the need for users to first establish criteria for their desired behaviour, such as personal norms or standards, against which they can assess their actual behaviour [85]. However, there was no evidence that this was a separate stage of the intervention so this stage does not appear in the synthesised diagram. The synthesised theory of change posits that behavioural questions result in self-monitoring, which prompts reflection in terms of pre-established criteria. This reflection is theorised to result in either self-reward or self-critique, generating enhanced self-regulation, a theoretical mediator of behaviour change.

Whilst both theory reports included in this grouping suggest processes of change are more complex than depicted in this synthesised theory, neither specified further self-monitoring mechanisms prospectively. The theory of change underpinning TXT-Auto suggested the intervention worked via two non-intersecting pathways, one captured in the ‘social/skills’ synthesised theory of change and one featuring self-monitoring. Arguing that pathways from self-monitoring to behaviour change are underdeveloped, Swendeman et al.’s research drew on qualitative data to further explicate the theory of change for Smartphone Self-Monitoring [85].

We drew on theories of change underpinning Online Mindfulness-Based Cognitive Therapy [58] and Rainbow SPARX [75], a computer game intervention, to develop a ‘cognitive therapy’ grouping (Fig. 4). Descriptions of both were assessed as of medium quality. These interventions were the only two in this review that targeted mental health outcomes alone, and both drew on cognitive therapy techniques to reduce depression and improve mental health.

Activities in the synthesised theory of change diagram include CBT approaches (forming the core of the Rainbow SPARX intervention) [75] and mindfulness-based cognitive therapy, which combines mindfulness with cognitive behavioural techniques [58]. Whilst we do not detail all activities comprising each of these approaches in the synthesised theory of change diagram, we include key components of each: behavioural training (CBT), an emphasis on prioritising ‘being’ over ‘doing’ (mindfulness-based cognitive therapy) and relaxation training (both approaches).

Theory of change descriptions for both interventions referred to recognising, paying attention to or developing awareness of thoughts, feelings and situations, with Rainbow SPARX [75] then challenging negative cognitions and Online Mindfulness-Based Cognitive Therapy focusing on accepting and ‘letting go’ of them [58]. We use differently shaded block arrows in Fig. 4 to delineate these distinct pathways. Both sought to reduce internalised homophobia and mitigate its effects on health [58, 75]. The report for Rainbow SPARX suggested this was done by providing information to promote young people spending more time with those who accepted them and reducing exposure to homophobic bullying [75], whilst the report for Online Mindfulness-Based Cognitive Therapy did not make clear how this would be achieved [58]. Descriptions of both interventions’ theories of change suggest that ultimately the reframing of distressing emotions is theorised as a key mechanism for improving mental health.

Whilst informed by the sexual health model, the theory of change underpinning the Sexpulse intervention did not specify components or mechanisms and so could not be synthesised [82, 86]. Four other interventions, with theory reports ranging from low to high quality, did not fit within any of the three inductive theory of change groupings [62, 69, 74, 76]. These interventions cut across the targeted health outcomes. Whilst some components of their theories of change overlapped with, or complemented, the above groupings, theories of change for these interventions featured elements that differentiated them from those included in the groupings. The MOTIVES [74] intervention was based on behavioural economics, and the SOLVE intervention [69] was driven by the theorised role of emotions in decision-making, featuring ‘shame associated with sexual stigma’ (p. 3) as a key mediator. Drawing on social cognitive theory, stress and coping theory and the social and emotional learning framework, the Role-Playing Game aimed to improve mental health and reduce substance use via use of productive coping strategies with a focus on help-seeking [62]. Informed by the transtheoretical model, the theory report for the Internet-Based Safer Sex Intervention [76] suggested that the intervention was primarily based on a stages-of-change approach, tailoring content based on responses to monthly sexual behaviour surveys. The MOTIVES intervention [74] and Internet-Based Safer Sex Intervention [76] theory reports augmented the ‘cognitive/skills’ synthesised theory of change diagram.

For this review, we synthesised the theories of change underpinning e-health interventions targeting HIV/STIs, sexual risk behaviour, substance use and mental ill health amongst MSM. We developed a novel approach to do so by using diagrams rather than narrative themes to summarise theories of change within and across interventions.

Whilst we have previously conducted reviews synthesising intervention theories of change using line-by-line coding of descriptive text [51, 52], we found that this approach did not capture the often precisely described and complex inter-relationships between theoretical constructs presented in the body of literature for this review. Theories of change included in our past reviews, which addressed the integration of health and academic education [52] and positive youth development interventions [51], were relatively simple and either not significantly informed by existing scientific theories [52] or informed by theories that are not typically portrayed visually [51]. In contrast, theories of change underpinning the e-health interventions included in this review were more complex, more explicitly theorised and largely drew on existing scientific theories which are typically conceptualised in terms of diagrams indicating relationships between their theoretical constructs, with constructs widely recognised and understood and therefore not always discussed at length. Whilst thematic analysis is a good way of rendering explicit what is implicit, it is less appropriate where the literature itself is more explicit. We therefore developed a novel method of theory of change synthesis in which we created diagrammatic summative logic models of intervention theories of change. By inductively grouping these models according to their core constructs and using meta-ethnographic approaches, we identified three emergent theoretical approaches underpinning the included interventions and we created synthesised models of each approach. We have thus synthesised theories of change underlying interventions with similar approaches. These summarise and integrate existing theories of change rather than providing a novel overarching theory of change for such interventions.

Social cognitive theory and the IMB model featured most prominently in intervention theories of change and informed a small majority. Whilst reports cited a number of other existing scientific theories, each informed only a few interventions at most. In the ‘cognitive/skills’ synthesised grouping, based on theory of change descriptions assessed as ranging from low to high quality, information and exercises are theorised to influence behavioural skills and a range of distal cognitive mediators theorised to influence behaviour via motivation/intention and self-efficacy/perceived control. The synthesised theory of change suggests that particular strategies can boost intervention impact, whilst other factors relating to the participant or (for sexual health interventions) their partner moderate impact.

Though represented in fewer interventions, two other distinct theories of change groupings emerged. The ‘self-monitoring’ grouping, based on theory of change descriptions assessed as of low and high quality, focuses more narrowly on the role of self-monitoring in triggering reflection, self-reward/critique and behavioural self-regulation. Theory reports suggest this synthesised diagram represents the basic components of more complex cognitive pathways [85] and can sit alongside other distinct mechanisms of behaviour change in intervention theories of change [81]. Underpinning interventions focused solely on mental health and based on theory of change descriptions assessed as of medium quality, the ‘cognitive therapy’ grouping is rooted in cognitive therapy techniques which can be augmented by mindfulness. In this approach, activities promote awareness and recognition of the participant’s thoughts, feelings and situations and, via either challenging or accepting negative cognitions, aim to reframe negative emotions to improve mental health.

Theories of change included in this review were informed by scientific theories that have been associated with greater impacts in reviews of e-health interventions specifically, including the transtheoretical model [41, 45] and the theory of planned behaviour [45], though these approaches underpinned a minority of interventions. Several also featured modelling and self-monitoring, behaviour change techniques [99] which some evidence suggests might be effective in e-health interventions [45, 100].

As is common with e-health behavioural interventions [45, 48], the theories of change underpinning interventions in this review tended to rely on individually focused psychological theories of behaviour or behaviour change. Whilst these are unlikely to address structural factors contributing to the syndemic such as marginalisation, homophobia and discrimination [19, 101], the accessibility and anonymity of e-health interventions offer one approach to reducing barriers to service access stemming from stigma and discrimination [26] and might form an important element of a broader mix of interventions addressing individual and structural factors.

Furthermore, we found that included interventions drawing on multiple theoretical approaches [55, 58, 65, 69, 75, 78] took into account sexual minority-related stressors. One accounted for such stressors as a theorised moderator of intervention impact [55], whilst others aimed to increase connectedness to the LGBT community [65, 78] and reduce internalised homophobia [58, 65, 75, 78] and associated shame [69] to reduce sexual risk [65, 69, 78] and improve mental health [58, 75]. However, theory reports included little discussion of the relationships between these constructs, limiting our ability to explicate their roles in the synthesised theories of change.

It was notable that some existing scientific theories that informed theories of change were theories of behaviour change (for example, CBT theory) whilst others were theories of behaviour and its determinants (for example, the health belief model). Authors generally did not report drawing on scientific theories oriented more explicitly towards selecting strategies for enacting behaviour change, such as the Behaviour Change Wheel [102] or the Elaboration Likelihood Model [103]. Intervention developers might usefully draw on such models.

Our synthesis aimed to develop overarching theories of change for different sub-types of e-health interventions to address HIV/STIs and sexual risk, substance use and mental ill health amongst MSM, which we hope will help inform future interventions. Intervention developers might also draw on existing scientific theories which aim to integrate existing scientific theories of behaviour and behaviour change, such as the PRIME and COM-B models [102, 104].

Our synthesis is limited by the quality of the existing theory reports, which did not often describe clear pathways from activities to intended outcomes. In some cases, reviewers inferred relationships between theory of change constructs (denoting assumptions in the theory of change diagrams), and in others the relationships between specific activities, mediators and outcomes could not be determined. Whilst our approach to theory of change synthesis enabled us to systematically explore constructs and the relationships between them across intervention theories of change, synthesised diagrams do not capture aspects that theory reports suggest influenced theories of change where their role was not clear enough to be included in intervention-specific diagrams.

We did not assess the parsimony of theories of change, because we have found in past reviews that this is very difficult to consistently operationalise as a criterion of quality assessment; however, this is an important feature of theories of change. We also did not aim to systematically assess the evidence base for each of the scientific theories underpinning the intervention theories of change, because this was outside the scope of feasibility for this review and would require assessing not only the evidence for the scientific theory but also the evidence for the application of that theory to the outcomes targeted in the theories of change it underpins.