Background
Aim
How CERQual was developed
Making an overall CERQual assessment of confidence and creating a Summary of Qualitative Findings table
Moving from a review finding to a summary of a review finding
How much detail should a summary of a review finding include?
• You should make the summary as explicit as possible: for example, a hypothesized connection or pathway between factors should not be implied but should be described clearly • The level of detail that you include in a summary of a review finding will vary according to the nature of the finding and the style agreed by the review team: o Providing detailed contextual information in a summary of a review finding (e.g., ‘In three studies from Japan, the UK and Ghana, young women reported that…’) may give the impression that the finding does not represent a broader phenomenon and is not transferable (see [8] and Table 3). It may be more appropriate to include this descriptive information in the column of the SoQF table that lists the studies contributing to the review finding o You should only include geographic or other specifiers in a summary of a review finding if these form part of the explanation of the finding; are needed to understand the finding or the group or context to which the finding relates (for example, ‘Health care providers reported that…’ or ‘Young men from rural areas experienced….’); or are critical to addressing the review question [8]. • Where a review is commissioned in support of a guideline, you may want to write the SoQF table to take into account the information needed to make a recommendation. For example, the summaries of review findings included in the SoQF table may focus on issues related to the acceptability and feasibility of the interventions being considered in the guideline and any important implementation considerations [25]
How should the summaries of review findings be ordered and presented?
• You can use titles and subheadings in the SoQF table to flag larger groupings of review findings • You can use a broader introductory or summary sentence, or a one line header, at the start of a summary of a review finding to help users quickly understand the gist of the finding |
Example | Original summary of a review finding |
Explanation of concerns with how the review finding has been written
| Improved summary of a review finding |
---|---|---|---|
1 | Lay health workers in child health studies in Angola, Zambia and Zimbabwe, and supervisors from these studies, as well as two studies in HIV/AIDS clinics in South Africa, expressed concern about the lay health workers’ workload and the distances they had to cover. Lay health workers in Angola, South Africa, Zambia and Zimbabwe sometimes found it difficult to carry out all of their tasks because of this. |
Unnecessary contextual information included in the summary of a review finding
| Lay health workers and supervisors expressed concern about the lay health workers’ workload and the distances they had to cover, and lay health workers sometimes found it difficult to carry out all of their tasks because of this. |
2 | Some parents wanted less information about childhood vaccination while other parents wanted a larger amount of information and/or information at a larger number of timepoints. Acceptance of vaccination varied among these parents. |
The summary review of a finding is not as explicit as possible—the hypothesised connection between the amount of information that parents would like to receive and their acceptance of vaccination is implied, but not spelled out
| The amount of information that parents would like to receive about childhood vaccination is linked to their acceptance of vaccination. Parents who had accepted vaccination as necessary typically wanted less information. Parents who questioned or refused vaccinations typically wanted a larger amount of information and / or information at a larger number of timepoints. |
3 | Women reported being beaten by health workers. |
This summary of a review finding provides insufficient detail to understand the situation
| Women reported experiencing physical force by health providers during childbirth, In some cases, women reported specific acts of violence committed against them during childbirth, but women often referred to these experiences in a general sense and alluded to beatings, aggression, physical abuse, a rough touch, and use of extreme force. Pinching, hitting and slapping (either with an open hand or an instrument) were the most commonly reported specific acts of violence. |
Determining the review findings to which CERQual should be applied
Making an overall CERQual assessment of confidence in each individual review finding
-
No or very minor concerns regarding [methodological limitations/coherence/adequacy/relevance] that are unlikely to reduce confidence in the review finding
-
Minor concerns regarding [methodological limitations/coherence/adequacy/relevance] that may reduce confidence in the review finding [concerns to be described]
-
Moderate concerns regarding [methodological limitations/coherence/adequacy/relevance] that will probably reduce confidence in the review finding [concerns to be described]
-
Serious concerns regarding [methodological limitations/coherence/adequacy/relevance] that are very likely to reduce confidence in the review finding [concerns to be described]
Summary of review finding | Studies contributing to the review finding | Methodological limitations | Coherence | Adequacy | Relevancea |
CERQual assessment of confidence in the evidence
| Explanation of CERQual assessment |
---|---|---|---|---|---|---|---|
1. Use of force: Women across the world reported experiencing physical force by health providers during childbirth. In some cases, women reported specific acts of violence committed against them during childbirth, but women often referred to these experiences in a general sense and alluded to beatings, aggression, physical abuse, a rough touch and use of extreme force. Pinching, hitting and slapping, either with an open hand or an instrument were the most commonly reported specific acts of physical violence. | 6, 9, 10, 13, 21, 61, 67, 68, 73, 75, 77, 80, 84, 86, 87, 91, 96, 97 | Moderate methodological limitations (6 studies with minor, 6 studies with moderate (unclear recruitment and sampling), and 3 studies with serious methodological limitations (unclear reflexivity, insufficiently rigorous data analysis)) | No or very minor concerns about coherence | No or very minor concerns about adequacy | Minor concerns about relevance (5 studies with direct relevance, 8 studies with partial relevance, and 1 study with unclear relevance. 15 studies total from 10 countries, including 1 high income, 2 middle income and 7 low income countries. Geographical spread: 2 studies in Asia, 1 study in Europe, 1 study in LAC, 1 study in MENA, 1 study in South America, and 8 studies from sub-Saharan Africa.) | High confidence | 15 studies with moderate methodological limitations. Data from 10 countries across all geographical regions, but predominantly sub-Saharan Africa. No or very minor concerns about coherence and adequacy. |
2. Physical restraint: Women in Tanzania and Brazil reported physical restraint during childbirth through the use of bed restraints and mouth gags. | 86, 97 | Moderate methodological limitations (1 study with minor and 1 study with serious methodological limitations (inappropriate research design, no reflexivity, unclear ethical considerations)) | Minor concerns about coherence (Some concerns about the fit between the data from primary studies and the review finding) | Serious concerns about adequacy (2 studies in total with limited, thin data) | Moderate concerns about relevance (2 studies with partial relevance from 2 countries) | Very low confidence | Two studies (Tanzania and Brazil) with moderate methodological limitations. Limited, thin data from 2 countries. Minor concerns about coherence but limited data available. |
Summary of review finding | Studies contributing to the review finding | Methodological limitations | Coherence | Adequacy | Relevancea |
CERQual assessment of confidence in the evidence
| Explanation of CERQual assessment |
---|---|---|---|---|---|---|---|
1. While regular salaries were not part of many programmes, other monetary and non-monetary incentives, including payment to cover out-of-pocket expenses and “work tools” such as bicycles, uniforms or identity badges, were greatly appreciated by lay health workers. | 2, 5, 11, 12, 22, 29 | Minor methodological limitations (five studies with minor and one study with moderate methodological limitations (unclear recruitment and sampling strategy, no reflexivity)) | Minor concerns about coherence (some concerns about the fit between the data from primary studies and the review finding) | Minor concerns about adequacy (six studies that together offered moderately rich data) | Minor concerns about relevance (studies of lay health worker programmes from three continents and including a fairly wide range of different clients and health issues) | Moderate confidence | Minor concerns regarding methodological limitations, relevance, coherence and adequacy. |
2. Some unsalaried lay health workers expressed a strong wish for regular payment. | 5, 13 | Minor methodological limitations (both studies had minor methodological limitations (unclear sampling strategy, no reflexivity)) | Minor concerns about coherence (some concerns about the fit between the data from primary studies and the review finding) | Serious concerns about adequacy (only two studies, both offering thin data) | Moderate concerns about relevance (partial relevance as the studies were from only two settings, both of which were in Africa) | Low confidence | Moderate concerns regarding relevance and serious concerns regarding adequacy of data. |
3. Lay health workers, particularly those working in urban settings, reported difficulties maintaining personal safety when working in dangerous settings or at night. | 3, 15, 16, 25, 31 | Moderate methodological limitations (2 studies with minor methodological limitations, 2 studies with moderate methodological limitations (unclear ethical considerations and unclear statement of research aims) | Minor concerns about coherence (some concerns about the fit between the data from primary studies and the review finding) | Moderate concerns about adequacy (studies offered very thin data) | Minor concerns about relevance (studies of lay health worker programmes across three continents but for a limited range of health issues) | Moderate confidence | Moderate methodological limitations and moderate concerns regarding adequacy of data. |
• Each overall CERQual assessment should ideally be made through discussion among the review authors. This process may also involve consulting with an expert group for a synthesis • Using the CERQual Evidence Profile, look across the assessments you have made for each CERQual component. Note particularly any components for which you have serious concerns • Decide whether you will ‘rate down’ (i.e., lower the level of confidence in the review finding) at all for the concerns identified and, if so, whether you will rate down by one or two levels. When making this overall assessment, consider the following: - Typically, the overall assessment of confidence should be rated down by at least one level for each component for which you have identified serious concerns - Where concerns in relation to a component are minor or moderate, it may not be necessary to rate down. However, if there are a number of such concerns, it may be appropriate to rate down by one level to represent two or more of these concerns • When making a judgement on whether to ‘rate down’, also consider the following: - To some extent, the importance of concerns regarding a CERQual component needs to be judged in relation to the review finding. For instance, where a finding represents ‘mid-level’ theory regarding a phenomenon, it may be important that this is backed by considerable data and that the fit between the data from the primary studies and the review finding is clear. Concerns regarding adequacy of data and coherence may therefore be particularly critical in making an overall CERQual assessment for this finding - The data contributing to a review finding may come from studies that are assessed to have different levels of concern in relation to a CERQual component. This variation can be captured in three ways: (1) make a judgment that captures the highest level of concern for the component; (2) make a judgment that captures the lowest level of concern for the component; or (3) make a judgment that captures the “middle ground” for the component. • For example, a synthesis of parents’ views and experiences of communication for childhood vaccination includes a finding that parents felt that the information that they received about vaccination was unbalanced or one sided. This finding was based on three ethnographic studies, two assessed as having no or very minor concerns and one assessed as having moderate concerns regarding methodological limitations; and two focus group studies, both assessed as having moderate concerns regarding methodological limitations. The synthesis authors decide to give an umbrella assessment of ‘moderate concerns’ for the methodological limitations component - The initial assessment of coherence may prompt changes in the way that a review finding is conceptualised and described if, for example, it becomes clear that the finding would make more sense as two separate findings. Where this occurs, the assessments for all of the CERQual components may then need to be revisited for the finding/s • Recommended standard phrases for describing the assessment for each CERQual component and the overall assessment are provided in Additional file 4 • You should strive to be consistent across review findings in a synthesis in assessing the extent of concerns regarding each CERQual component. Consistency across syntheses is harder to achieve and it is more important to be explicit and transparent regarding judgements so that the rationale for these are clear to users |
Creating a CERQual Evidence Profile and a Summary of Qualitative Findings (SoQF) table
Objective: To synthesise qualitative and quantitative evidence on the mistreatment of women during childbirth in health facilities. Perspective: Experiences and attitudes of stakeholders in any country about the mistreatment of women during childbirth | |||
---|---|---|---|
Summary of review finding | Studies contributing to the review finding |
CERQual assessment of confidence in the evidence
| Explanation of CERQual assessment |
1. Use of force: Women across the world reported experiencing physical force by health providers during childbirth. In some cases, women reported specific acts of violence committed against them during childbirth, but women often referred to these experiences in a general sense and alluded to beatings, aggression, physical abuse, a rough touch and use of extreme force. Pinching, hitting and slapping, either with an open hand or an instrument were the most commonly reported specific acts of physical violence. | 6, 9, 10, 13, 21, 61, 67, 68, 73, 75, 77, 80, 84, 86, 87, 91, 96, 97 | High confidence | 15 studies with moderate methodological limitations. Data from 10 countries across all geographical regions, but predominantly sub-Saharan Africa. No or very minor concerns about coherence and adequacy. |
2. Physical restraint: Women reported physical restraint during childbirth through the use of bed restraints and mouth gags. | 86, 97 | Very low confidence | Two studies (Tanzania and Brazil) with moderate methodological limitations. Limited, thin data from 2 countries. Minor concerns about coherence but limited data available. |
Objective: To identify, appraise and synthesise qualitative research evidence on the barriers and facilitators to the implementation of lay health worker programmes for maternal and child health Perspective: Experiences and attitudes of stakeholders about lay health worker programmes in any country Included programmes: Programmes that were delivered in a primary or community healthcare setting; that intend to improve maternal or child health; and that had used any type of lay health worker, including community health workers, village health workers, birth attendants, peer counsellors, nutrition workers and home visitors | |||
---|---|---|---|
Summary of review finding | Studies contributing to the review finding |
CERQual assessment of confidence in the evidence
| Explanation of CERQual assessment |
1. While regular salaries were not part of many programmes, other monetary and non-monetary incentives, including payment to cover out-of-pocket expenses and ‘work tools’ such as bicycles, uniforms or identity badges, were greatly appreciated by lay health workers. | 2, 5, 11, 12, 22, 29 | Moderate | Minor concerns regarding methodological limitations, relevance, coherence and adequacy. |
2. Some unsalaried lay health workers expressed a strong wish for regular payment. | 5, 13 | Low | Moderate concerns regarding relevance and serious concerns regarding adequacy of data. |
3. Lay health workers, particularly those working in urban settings, reported difficulties maintaining personal safety when working in dangerous settings or at night. | 3, 15, 16, 25, 31 | Moderate | Moderate methodological limitations and moderate concerns regarding adequacy of data. |