Learning Club intervention
The Learning Clubs intervention is a community-based structured program which is delivered to women from when they are less than 20 gestational weeks pregnant, to one year after birth.
The cluster randomised controlled trial of the Learning Clubs intervention will follow women from early pregnancy (baseline survey), and, with their children, to two years after birth (in three follow-up surveys). All women recruited from 84 communes (half the communes will be randomly allocated by an independent statistician to each trial arm) will receive the current maternal and child health standard of care. Women in the intervention group will receive the Learning Clubs intervention in addition to the standard of care. Women with cognitive and serious physical disabilities who are not able to implement the intervention practices will be ineligible for the study.
The number of clusters and overall sample size were calculated using the clustersampsi module in Stata, Version 13 (StataCorp LP, College Station, Texas, USA). In order to detect a difference in the primary outcome (Bayley Scales of Infant and Toddler Development cognitive development score < 1 SD at 2 years of age) of 15% in the control arm and 8% in the Learning Club intervention arm (with 80% statistical power and a significance level of 0.05; intra-cluster correlation coefficient [ICC] = 0.03), a total of 1,008 pregnant women from 84 clusters is needed [
13].
Process evaluation design
The process evaluation will use multiple methods to collect qualitative and quantitative data. Small group discussions, semi-structured interviews, case-studies and independent observations will be used to collect qualitative data. Quantitative data will be sought through self-reported questionnaires and secondary analyses of routinely collected data.
Purposive sampling methods will be used to recruit informants for the qualitative components of the evaluation. Informants will include stakeholders from commune, provincial and national levels of government, including Health Department officers, project implementation officers, commune health workers, early childhood education workers, Women’s Union representatives, and members of the community. We will select participants to ensure diversity in terms of sector represented, extent of experience with implementing the Learning Clubs, and geographic distribution within Ha Nam province. We will continue to recruit stakeholders until we have reached informational redundancy.
At least 504 women will be recruited from communes assigned to the intervention arm and invited to participate. Attendance lists (women, their partners and other family members) are being kept by facilitators at each Learning Club meeting. These will enable us to calculate participation rates by total number of groups attended and by session topic.
Timing of data collection
Data will be collected at several time points. The process evaluation will be conducted alongside the four survey rounds of the outcome evaluation (at baseline, when the women are less than 20 weeks gestation; at Follow-up 1 when they are in late pregnancy; Follow-up 2–1 year after birth, and Follow-up 3–2 years after birth).
Data collection methods
Domain 1 - Context
The aim of this component is to evaluate variation in current maternal and child health programs, quality of usual maternal and child health services, contamination from any other relevant programs and the local support among the 84 communes in the intervention and control arms.
Current maternal and child health services in each commune will be assessed through obtaining information about 11 core maternal and child health indicators, including two mortality indicators, one indicator, stunting, among children under five years of age and eight coverage indicators. These are the ones recommended by the World Health Organization (WHO), the United Nations’ International Children's Emergency Fund (UNICEF), Countdown to 2015, and the Health Metrics Network to monitor maternal and child health, and child mortality in the world’s high-burden and low-income countries [
18]. Primary data sources for these indicators will be the vital registration and health facility reports of the 84 communes in Ha Nam province in 2018, 2019 and 2020 (see Additional file
1).
Quality of usual maternal and child health care services
Quality of existing maternal and child health services will be assessed through indicators in three domains: maternal, newborn and general health. These indicators were proposed by WHO to evaluate the quality of maternal and new born health care provided by primary health facilities. Information about these indicators will facilitate comparison of data in this project with international data [
19]. The data source for this component will be mainly routine health facility reports (see Additional file
2).
Contamination
A data collection form will be developed and distributed to the Women’s Union, provincial government authorities, commune health stations, key social mass organizations in both control and intervention groups to collect information about any related maternal and child health activities such as training, other interventions, new services and information sources that may affect the outcomes of the Learning Clubs. This form will be distributed to project communes at every survey round and completed forms will be posted to the Research and Training Centre for Community Development (RTCCD) in prepaid reply envelopes. In addition, the team will investigate cases of sharing information between the intervention and control groups.
Local support
Frequency and active involvement of local authorities and social organizations in maternal and child health activities will be measured to assess local support at each commune. This information will be gathered by a form developed by the process evaluation team. Information sources are the Women’s Union and their related partners. The support may include access to meeting rooms, computer equipment, the involvement of mass social organizations (Youth Union, Famer’s Union) in advocacy activities, financial support for Learning Club staff, or provision of other necessary equipment. Types of support and frequency of access, activities, or use will be assessed.
Other socio-economic characteristics of the commune
In addition information, such as population size, commune health facilities and human resources, number and nature of factories or construction sites located in the commune and neighboring areas, and environmental pollution sources, will be obtained at baseline and during the Learning Clubs intervention. These factors may result in common illnesses among children and mothers and affect their health.
Domain 2 - Implementation
This domain will evaluate the implementation aspect of the Learning Club intervention, especially in terms of the quality (fidelity), quantity (dose) and its alignment with the current strategic context in Ha Nam and in Vietnam.
Intervention fidelity and dose
There are different definitions of “fidelity” [
20‐
25]. Commonly, it refers to the degree to which an intervention is implemented as planned. In this study, “fidelity” is defined as the quality of the intervention delivered, while “dose” is the quantity of the intervention such as frequency, duration, and coverage [
26,
27]. Structurally, the Learning Club intervention content can be divided into two periods: during pregnancy, and when the infants are 0–1 year old. After each period, workshops will be conducted among all facilitators to obtain information about their experience in operating the clubs and in delivering the intervention contents. During these workshops, semi-structured interviews and small group discussions with the club facilitators will be conducted to identify any modification in delivering the intervention, in terms of the content and dose, and to understand reasons for these modifications, if any.
In addition, during the club meetings, a team of provincial trainers will undertake routine supportive supervision and observations of Learning Club meetings. A supervision checklist will be developed for this team to record any variations and report any changes observed in the meetings.
Alignment with strategic context
In order to prepare for the scaling up phase, an important element to be considered is the appropriateness of the intervention for the local context. This is defined as the relevance and perceived fit of the intervention in maternal and child health policies and interventions [
28]. At Baseline and Follow-up 3, secondary data collection and semi-structured interviews with key stakeholders at national, provincial and commune levels of the education, health and labor sectors and Women’s Union will be held. Information about the strategies, regulations and vision in terms of maternal and child care in the local communes will be sought. The purpose of this component is to identify relevant government strategies, which may enable the integration of the intervention into in national programs. It is estimated that around 10 interviews will be undertaken at each survey round.
Domain 3 - Mechanisms of impact
Participation rate
At each Learning Club’s meeting, a participation form will be used by the facilitators to record the number of people attending the session. The form and (with participant consent) a photo of the meeting will be sent to the project officers. Weekly and monthly reports will be prepared by the project officers to establish participation rates. These reports will be submitted to the project coordinator, the process evaluation team and the research advisory group.
Acceptability
Small group discussions with Learning Club participants will be held at Follow up 1 (F1), Follow up 2 (F2) and Follow up 3 (F3) to explore their experiences of and views about attending Learning Clubs sessions; advantages, difficulties and unexpected outcomes of participation will also be sought. Participants in these discussions will be recruited using maximum variation purposive sampling. This type of sampling design allows the recruitment of outliers whose experiences are diverse to maximize the variety of perspectives among people contributing data. Therefore, Learning Club participants who are more and less able to apply the Club content in self-care and care of their infants will be included to explore underlying factors that might influence the outcomes.
Process mediators, including changes in facilitators’ knowledge about early childhood development, and skills and confidence in Club operations; participants’ changes in knowledge and skills in providing care for their child, sharing of paid and unpaid work with an intimate partner and quality of family relationships; and support from family members to the club’s mothers will be collected at F1, F2 and F3 through small group discussions and a self-administered questionnaire.
Domain 4 - National integration
National integration steps
The process evaluation officer will work closely with the project implementation team to document all steps taken to ensure that the Learning Club content is harmonized with existing national programs. Information about steps taken in the preparation, implementation and post-intervention phases, to engage local authorities, relevant stakeholders and policymakers in the project, will be gathered using a project diary. This component will provide descriptions of purposeful steps that the project team has conducted to inform policy makers and advocate for the intervention.
Barriers and enablers
Semi-structured interviews and small group discussions will be organized to identify the barriers and enablers for national integration of the Learning Clubs, from each stakeholder’s perspective. Key representatives from government agencies (Women’s Union, health, education and labor sectors) at national, provincial and commune levels will be invited to participate. The views of local and international non-government and United Nations agencies, including the World Health Organization, UNICEF, and the United Nations Population Fund (UNFPA), about the intervention will be also be sought. It is estimated that about 25 interviews will be conducted in total by the process evaluation team.
Data management and analysis
Consent will be sought from participants for all small group discussions and semi-structured interviews to be audio-recorded. Data will be transcribed by experienced researchers in Vietnamese and then translated into English so that investigators in this bi-cultural team can co-contribute to analyses. Data will be uploaded to a secured cloud-based storage system in Monash University, Australia. Only authorized researchers will have access to the data.
Transcripts will be entered into the NVivo version 11.0 and data will be analyzed thematically. Frameworks for coding of data will be established and revised as new themes emerge. Themes reflecting the context, implementation, and mechanism of impacts will be used in the interpretation of the impacts of the Learning Clubs on ultimate and intermediate outcomes. They may also be used to inform post-hoc secondary analysis of the outcomes, for instance, to examine whether contamination by other relevant projects may diminish the impacts of the Learning Clubs on infants’ cognitive development.
Domain 1 – Context
Data collected from Domain 1 will yield a comprehensive map and description of the context and influencing factors of maternal and child health in the 84 participating communes. These will enable differences in outcomes between control and intervention arms to be understood in this context. The four key contextual factors to be considered will be maternal and child health status, quality of maternal and child health services, local support and social and economic characteristics. The indicators of the maternal and child health status, the quality of related usual care services will be analyzed and compared to other settings using the guidelines developed by the World Health Organization and its partners [
18,
19].
Domain 2 – Implementation
The fidelity and dose of the intervention will be categorized into high, medium and low levels of fidelity and dose. This provides evidence of the variation in delivery of the intervention, especially in terms of what aspects of the intervention have been delivered, how well they have been delivered, and reasons for any modifications made. These modifications will reflect what the Learning Club facilitators needed to do to fit the intervention to the real situations in their communes. It will help the project team to revise the content and structure of the intervention to meet the capacity of the facilitators in rural areas. In addition, the alignment with the strategic context will enable mapping and adjusting the Learning Club intervention to existing policies and regulations related to maternal and child health in Ha Nam. It will enable the project team and policymakers to identify relevant national programs that the project may contribute to.
Domain 3 – Mechanism of impact
Domain 2 provides information on the acceptability to the facilitators of operating the Learning Clubs, and Domain 3 ascertains the accessibility of and acceptability to the target audience. The participation rate reflects the involvement of mothers, their intimate partners and other family members in the Club meetings. Qualitative information from the small-group discussions will inform revisions of the intervention, in terms of content and format, if required, to meet participants’ needs more effectively. It will inform the dissemination strategy for the final scaling-up phase, if this is indicated. Moreover, information about the process mediators will contribute to ascertaining whether the underlying causal assumptions of the intervention do or do not lead to differences in the outcomes between trial arms.
Case studies will be developed to explain the causal relationship between the input and the outcome conditions. The qualitative data will inform revisions, if necessary, to the theory of change.
Domain 4 – National integration
This component provides a detailed description of the integration procedure into existing relevant national programs. It will also provide the structure and rationale for each step of the implementation and evaluation of the intervention. This evidence for policy makers in Vietnam will enable the production of guidelines for integrating community-based initiatives into the government health and social protection systems.
Integration of process evaluation and outcomes findings
In addition to assessing the primary and secondary outcomes, this process evaluation is one of the two evaluations to be conducted alongside the main cluster randomized trial, the other is an economic evaluation. There is still debate among researchers about how to integrate process evaluation data into outcome and cost-effectiveness evaluations. In this project, after the final data collection is completed, the process evaluation data will be combined with the other evaluations to enable a comprehensive picture of the effectiveness of this community-based intervention to be prepared. Quantitative process data will be integrated into the analysis of outcomes and cost-effetiveness using regression models and path analysis. Qualitative components will provide insights to explain the causal pathways of changes in the outcomes.
Blinding
The process evaluation will be conducted independently by two experienced health researchers who are bilingual in English and Vietnamese. These researchers will be able to conduct interviews and analyze data in both languages and will not be involved in implementing any project activities. As one of the aims of the Learning Clubs project is to establish whether the Learning Clubs intervention is effective under real-world conditions, the process evaluation team will not provide feedback to the Learning Club project staff, project stakeholders, and project advisory group or exchange relevant information that may result in changes to the project. Information obtained from the process evaluation will be taken into account in the final analysis and interpretation of the results of the main trial and will be published separately.
Dissemination strategy
The data will be used to write peer-reviewed publications, workshop presentations and technical reports. In addition, it will be included in the policy brief to advocate for integrating the intervention nationally. No personal information will be provided in any documents released from the project.