A questionnaire to measure women’s experiences with pregnancy, birth and postnatal care: instrument development and assessment following a national survey in Norway

Development of the PreMaPEQ followed an established procedure used previously by NOKC for developing data-collection tools and routines for conducting surveys in new target groups. The initial literature search in 2009 did not identify any instruments that met the specifications of this study [3], but did provide some information about relevant topics. Hence, a development process was commenced [4], the first step of which was to set up a reference group including service users, authorities and clinicians. The purpose of this group was to collect comments and views during the development process from various stakeholders representing important expertise. The group met three times and discussed questionnaire contents and inclusion criteria. The second step was to explore what is important for people in this situation by interviewing women and their partners with recent experiences with the services. The interviews were semi-structured, individual, and face to face, and the answers underwent conventional content analysis [5]. The interviewees varied with regard to age, parity and ethnicity, and the information they provided was highly consistent with findings from the literature search. The third step was to construct the questionnaire itself. Four sections were constructed, each with a specific colour to reflect the different phases of the health-care course; that is, pregnancy control with green headings, birth with red headings, postnatal hospital stay with orange headings, and finally follow-up in the community health clinic (helsestasjon in Norwegian) with violet headings. The items (i.e. questions) in the questionnaire asked whether desirable properties or behaviours were present, with the intention of collecting a description that was concrete and factual rather than being judgmental [6]. Among the many possible response formats [7] most of the questions were answered on the following five-point ordered response scale: 1 = not at all, 2 = to a small extent, 3 = to some extent, 4 = to a large extent and 5 = to a very large extent. The alternative answer of “not applicable” (NA) was also allowed where it was important to discriminate between user missing and answers that were skipped because the respondent had not used the service in question. This five-point scale performed better that a ten-point scale in a previous study, and was considered more suitable for assessing patient experiences [8]. Consequently, the former has been chosen to be consistently applied in NOKC’s surveys, making it possible to compare over time and, to some extent between different groups of health care users. In the fourth step, the questionnaire was administered to 18 women, who were then interviewed to evaluate whether its structure, questions and response formats were performing as intended [9]. The fifth step was a pilot test that was conducted in a university hospital, in which all women aged 16 years old or older who gave birth over a 2-month period were included (births in which death occurred were excluded). The pilot was conducted as a small version of the large survey to come, to provide an opportunity to detect flaws and weaknesses in the study in time to correct them [10]. This pilot test also provided the opportunity to test and compare the efficiency and cost of two different data-collection routines, for which reason the sample was randomly divided into two groups. The sixth step involved revising the questionnaire in accordance with the findings of the pilot test, and then evaluating it by administering it to 13 women followed by interviewing them to evaluate that final version [11]. Technical revisions were made to capture the various combinations of services the women had used, to cover the diversity in a best possible way. The last question asked whether the respondent would be willing to answer a new questionnaire after a short interval so that the stability of the results could be tested (i.e. test-retest).

The printed version of the questionnaire consisted of 145 items on 16 pages. The contents were generic, so that unusual but still relatively frequent events were left out, such as unplanned births at home or during transport. The women may have used many different combinations of services, both public and private, and not all variations could be captured by the questionnaire.

The final questionnaire was translated into English for facilitating responses from immigrants. The translations were carried out according to recommended procedures [12]. We had no knowledge of the linguistic abilities of the potential respondents, so the invitation was to all otherwise eligible women. See Additional file 1 for the printed version in English.

The same procedure was used to construct a questionnaire targeting the women’s partners; however, that questionnaire was not assessed in this paper.

The focus of this nationwide survey was the entire care course from the first visit in prenatal services, through birth and postnatal care in birth institutions, and finally to the follow-up in community health clinics. The financing and delivery of prenatal monitoring and postnatal follow-up in health clinics is the responsibility of the municipality of residence of the individual women, while the birth institution—be it in a large hospital or a local maternity clinic—is the responsibility of the state, via the hospital trusts.

Women who gave birth in the last quarter of 2011 in a Norwegian institution and who were 16 years old or older were included. Experiences from previous surveys led to the requirement of samples with 400 potential respondents from each hospital. The women to be included were drawn randomly from institutions with a large number of births (more than 400 during the inclusion period), and women were included consecutively from institutions with less than 400 births. The Medical Birth Registry conducted the sampling routine. Before any list of names and addresses were transferred to the Knowledge Centre, information from the National Population Registry Office was collected and any birth in which either the woman or child had died was excluded.

Potential respondents to the questionnaire were first contacted by mail about 17 weeks after the birth: they were sent a letter with information about the survey and an invitation to participate via the Internet, including a specific username and password. Two reminders were sent to non-respondents, both of which included a printed version of the questionnaire in addition to a username and password.

The names and addresses were deleted when all of the mailings were completed, and the questionnaire data were supplemented by clinical information from the Medical Birth Registry. Statistics Norway provided data regarding the country of origin for the women included in the survey and their parents.

Informed consent was considered expressed when the women had received the mailed information and submitted their response. The Regional Committee for Medical and Health Research Ethics (REK sør-øst D) approved the study.

SPSS software (version 15.0, SPSS, Chicago, IL, USA) was used to analyse the sample and variables, and for exploratory factor analysis (EFA).

Questionnaire items regarding experiences were entered into principal axis factoring analyses according to phase; that is, items pertaining to pregnancy control, birth, postnatal hospital stay and the public health clinic. EFA results were interpreted as supportive of a factor if loadings exceeded 0.40 and there were no cross-loadings. Some correlations between the factors were assumed, and the oblique rotation method of analysis was chosen [13].

In cases where the resulting factors after EFA included diverse phenomena, the factors were split and the items grouped according to the structure and process categories in Donabedian’s framework [14] or to ensure that the survey results reflected recognizable elements of care delivery. For example splitting structure (resources and organization) and process (personal relationships) or splitting information about women’s health and information about the child. We believe that this will render the survey results more useful for local improvement purposes. Items that were not included in a factor, due to poor factor loadings or high missing rate, were reported as single items. The final outline of the questionnaire is shown in Table 1.

Most single items were scored on a scale of 1–5, and the index scores were transformed linearly to a scale of 0–100.

Impression of potential differences between respondents and non-respondents were obtained by comparing those who responded to the first contact with those who responded after two reminders. In wave analyses such as this, the latter—which are the most difficult to obtain—are considered proxies for non-respondents [15].

The LISREL analysis program (version 8.8) was used to test the goodness of fit of the models [16] by confirmatory factor analysis (CFA), and was applied to further test the relationships between the manifest variables and their underlying latent constructs. The 16 scales were constructed on the basis of a combination of the theoretical structure and process categories in Donabedian’s framework, our EFA results, and to some extent the specific content of the items. Therefore the scales were not only data-driven, but also founded on a theoretical understanding underpinning the analyses. Accordingly, the CFA did not test a model based solely on the correlation of the test items but whether the measures are consistent with our understanding of the nature of the construct. Consequently, the objective was to test whether the data fitted our hypothesized model based on theory and analytic research.

It was hypothesized that there was a second-order factor structure for the instrument, with experiences with pregnancy control, the birth, the postnatal hospital stay, and the public health clinic as the lower-order factors, and care experiences in pregnancy-, birth- and postnatal care as the higher-order factors. In CFA there are two types of latent variables; endogenous and exogenous. Exogenous variables always act as independent variables, endogenous on the other hand, are variables that are influenced by other variables in the model.

Various fit indexes were used, including the root-mean-square error of approximation (RMSEA), goodness-of-fit index (GFI), comparative fit index (CFI) and incremental fit index (IFI). An RMSEA of ≤0.05 and GFI and CFI values of ≥0.90 are generally taken to indicate a good fit. IFI values range from 0 to 1, with larger values indicating a better goodness of fit.