Background
Methods
Literature review in order to identify potential standards
International expert panel
e-Delphi exercise to identify and prioritise standards
Open round
First scoring round
Second scoring round
Quantitative analysis of scoring
Qualitative analysis of free-text comments
Results
Literature review initial list of potential standards
Checklist section | Suggested checklist item |
---|---|
Introduction | |
Background | Report the care or quality gap (e.g. failure to implement guidelines) that the intervention seeks to address [29]. |
Methods | |
Setting | |
Report readiness to change (also those not eager to change) [29]. | |
How did the setting enable or hinder the implementation? [10] | |
Participants | |
Intervention | Provide detailed description of intervention strategies: |
What is the relationship of components to the theory discussed above? [10] | |
Describe the intervention received by control group (not just describe as ‘usual care’) [31]. | |
The intervention must be delivered as part of routine care [11]. | |
Analysis | Describe measurements used to assess implementation effectiveness [29]. |
Results | |
Participant flow | Report the process by which eligible patients were invited to participate (e.g. number invited and timescale) and any exclusions [4]. |
Setting | Report characteristics of the setting to enable assessment of representativeness [4]. |
Intervention | Cost of the intervention [4]. |
Report any intervention feasibility, acceptability, and potential sustainability [28]. | |
Discussion | |
Reflections | |
Report any lessons learned from the implementation process [10]. | |
Report any barriers or facilitators for implementing in routine care [9]. | |
Discuss the relationship between adaptations/modifications and treatment outcomes, implementation outcomes and adverse events [28]. |
International expert panel
e-Delphi exercise to identify and prioritise standards
Open round
Consensus (scoring rounds)
Section | Standard | Consensus (% agreement with scores 7, 8 or 9) |
---|---|---|
Title and abstract | There should be a structured abstract which clearly states aim, study design, setting, population, intervention, outcomes, conclusion and implications. | 95% |
Introduction | Identify the aspect of care that the new service being implemented aims to address (e.g. implementing a guideline recommendation or evidence-based management). | 90% |
Critically report the evidence underpinning the new service to be implemented: (e.g. phase III randomised controlled trials, systematic review, guideline recommendations). | 100% | |
Describe the rationale for the new service design. | 95% | |
Report the implementation strategy used and its underpinning theory. | 84% | |
Clearly define the aims of the study, differentiating between implementation (process) objectives and effectiveness (clinical) objectives. | 100% | |
Method (setting) | Describe the study setting (including health service, personnel involved, patient and public involvement, demography of patients, etc.). | 100% |
Give year(s) during which the new service was implemented (i.e. planned, initiated and actively developed) and followed up. | 95% | |
Methods (the new service) | Describe the new service (e.g. components/content, frequency, duration, intensity, mode of delivery, materials used) with advice on accessing additional detailed information. Use of a standardised checklist (e.g. TIDieR) is recommended. | 100% |
Describe the professional backgrounds, roles and training requirements of the personnel involved in delivering the intervention with advice on accessing additional detailed information. | 84% | |
Define the core components of the intervention, and the processes for assessing fidelity to this core content, and what, if any, local adaptation was allowed. | 100% | |
Describe the intervention received by control/comparator group not simply stating ‘usual care’. | 95% | |
Methods (population) | Describe sites invited/excluded with reasons. | 100% |
Describe the population targeted by the intervention and any eligibility criteria. | 100% | |
Report method by which patients are referred to or access the new service. | 100% | |
Methods (randomisation) | Description of randomisation (or if not randomised how comparator group was selected). | 95% |
Methods (data) | Describe outcome measurements (specifically describing any that are at population level) distinguishing between process and clinical outcomes and health economic data. | 100% |
Describe data collection processes (specifically including methods of extracting routine data). | 100% | |
Describe any processes for quality assurance (especially for use of routine data). | 84% | |
Methods (analysis) | Describe power calculation and rationale for sample size. | 100% |
Describe methods of statistical analysis (with reasons for that choice including approach to clustering, handling of missing data, intention to treat analysis, and adjustment for confounders, etc.). | 100% | |
Specify a priori sub-group analyses (e.g. between different sites in a multicentre study, different clinical or demographic populations). | 95% | |
Results (population) | Report the number of sites approached, reasons for non-participation and characteristics of participating sites. | 89% |
Report the total eligible population (e.g. number of people with the relevant condition registered with the practice, or eligible for a service), number approached and any exclusions. | 100% | |
Report participation rate among the eligible population, compare characteristics with the eligible population as a whole and describe any known reasons for non-participation. | 95% | |
Report compliance with/attrition from the service as a process outcome. | 95% | |
Include a CONSORT diagram (modified as necessary) to illustrate the recruitment of sites, provision of service to patients and any sub-groups. | 84% | |
Results (fidelity) | Report fidelity to the core components of the planned intervention (including, in multicentre studies, in the different settings). | 100% |
Report any modifications or adaptations to the new service during the course of the study. | 100% | |
Results (outcomes) | Report outcomes for the whole eligible population before an analysis of any sub-groups. | 100% |
Report process and clinical outcomes. | 100% | |
If relevant, report impact on use of health service resources (and ideally cost of the intervention). | 84% | |
Report any unintended consequences or adverse effects. | 100% | |
Discussion | Interpret findings in the light of the general body of literature and consider implications for health care services (including issues of generalizability, transferability, strategies for facilitating and normalising into routine care). | 100% |
General | Include statement(s) on regulatory approvals (including, as appropriate, ethical approval, confidential use of routine data, governance approval), trial/study registration, funding and conflicts of interest. | 89% |
Section | Standard | Consensus (% agreement with scores 7, 8 or 9) |
---|---|---|
Title and abstract | The title (or abstract if word count of title precludes) should include a description of the methodology (e.g. phase IV implementation study, cluster randomised implementation trial, interrupted time series, before and after, stepped wedge study). | 79% |
Introduction | Include a description of the wider health care/policy/commercial context. | 58% |
Describe any pilot implementation work and the conclusions from that work. | 63% | |
Methods (the new service) | What is the relation of components of the intervention to the rationale for the new service design and/or theory underpinning implementation discussed above? | 30% |
Define role of the researchers in design and implementation. | 79% | |
Methods (population) | If applicable, describe any consent required (which should be to the new service and not to research). | 53% |
Describe recruitment of any sub-groups recruited for additional research tasks (e.g. questionnaire completion, physiological measures, detailed record analysis). | 47% | |
Results (population) | Report details of any subgroups recruited to specific research tasks (e.g. questionnaire completion, physiological testing) as opposed to the clinical service. Compare characteristics of any sub-groups to the whole eligible population. | 74% |
Discussion (population) | Include a structured abstract (for example including summary of findings, strengths and limitations, comparison with other studies, conclusions and implications). | 58% |
Reflect on the processes of implementing the service, barriers or facilitators and lessons learned. | 79% | |
How did the setting enable or hinder the implementation of the new service? | 79% | |
How was the new service was implemented highlighting (if relevant) variations between sites and over time and the impact on treatment outcomes and unintended consequences? | 74% |
Over-arching issues raised by the expert panel
Dual aims of implementation and effectiveness
‘This should clarify the “effectiveness aims” from the “implementation aims.”’ [Open round comment]
‘Analyses must examine impacts/outcomes as well as processes’ [Open round comment]
‘[Cost-effectiveness] is important for (governmental) health agencies, funding agencies, insurance companies, but also for healthcare centres and health researchers themselves’ [Open round comment]
Balancing the need between detailed descriptions and the risk of overload
‘Sorry, lots of essentials in my response. Hard to say much shouldn't be there really, good suggestions for a reporting standard’ [First scoring round comment]
‘Only concerns - all individual items are fine - is the overall cumulative burden compared to journal space usually available’ [Open round comment]
Fidelity to the intervention vs. adaptation to the new service
‘Also the concept of fidelity and implicit demand to not report variations - which we know happen everywhere’. [Open round comment]
‘It is inevitable that changes will be made to the service and so the assumption should be that changes will occur and that these need to be described’ [Open round comment]
‘Need to allow for change in intervention over time as well as local adaptability - these [questions] assume new service is fixed in aspic’ [First scoring round comment]
‘Components of the new strategy may be part of the “usual care” given in one centre but not in another.’ [Second scoring round comment]
Overlap with other reporting guidelines
‘Perhaps better to defer investigators to these existing guidelines when methods of study (RCT, observational) overlap with existing guidelines’ [Open round comment]
‘A review and compilation of the relevant CONSORT statements and extensions should be used to expand the starting list’ [Open round comment]