A validation of the Postpartum Specific Anxiety Scale 12-item research short-form for use during global crises with five translations

The Coronavirus [SARS-CoV-2] or COVID-19 pandemic poses a devastating risk to the health of the global population. Amongst those thought to be most vulnerable are pregnant women and newborn infants, although guidance rapidly changed to state that pregnant women are no more vulnerable than the general population [1]. Although the growing body of evidence remains conflicting about the size of the risk to these populations, perinatal deaths have been reported [2]. This makes the perinatal period a time of increased vulnerability [3]. Whilst COVID-19 poses a serious physical health risk to those who contract the virus, there is evidence for it also affecting mental health outcomes [4, 5]. Poor mental health in relation to COVID-19 has been associated with various Government mandated restrictions, which have been enforced in an attempt to slow the spread of the virus. These include ‘quarantine’ (the enforced isolation of persons with or suspected of having the virus) [6]; ‘social distancing’ (the physical separation of persons outside of those in one’s family) [7]; ‘lockdown’ (the closure of public venues and banning of non-essential travel) [8]; and ‘shielding’ (where the most vulnerable – including pregnant women – are advised to remain at home and leave under no circumstances) [1]. Given the expected effect of the COVID-19 pandemic on mental health coupled with pregnant women and newborn infants being labelled as vulnerable groups [1], it is important to assess and understand the mental health effects in perinatal women [9]. During normal circumstances, approximately 20% of all women who give birth are thought to experience mental health problems [10]. The global pandemic is set to pose “unprecedented challenges that can significantly impact on women’s mental health” during the perinatal period [3], hence potentially driving these numbers even higher.

In 2014, the National Institute of Health and Care Excellence [NICE] requested attention to the under-detection of postpartum anxiety in recognition of the significant burden it poses [11]. Postpartum anxiety is associated with many negative maternal and infant outcomes including reduced breastfeeding [12], reduced maternal sensitivity [13], impaired bonding [14], difficult infant temperament [15], atypical neurodevelopment [16], and child emotional and behavioural problems [17]. However, general measures of anxiety are relied upon in a large majority of studies examining postpartum anxiety, but are psychometrically problematic [11, 18].

The Postpartum Specific Anxiety Scale [PSAS] examines the frequency of maternal and infant focused anxieties experienced by women across the first year of their infants’ life [19]. The 51-item measure assesses four domains of anxiety, specific to the postpartum period. Factor 1 (Maternal Competence and Attachment Anxieties) contains 15-items which address anxieties relating to maternal self-efficacy, parenting competence, and the mother-infant relationship. Factor 2 (Infant Safety and Welfare Anxieties) has 11-items which relate to fears about infant illnesses, accidents, and cot death. Factor 3 (Practical Infant Care Anxieties) includes 7-items covering anxieties which are specific to infant care such as feeding, sleeping, and general routine. Finally, Factor 4 (Psychosocial Adjustment to Motherhood) contains 18-items which address adjustment concerns since the birth of the baby about management of personal appearance, relationships and support, work and finances, and sleep. Each answer is given a score of between 1 and 4 with the maximum score being a total of 204. Initial validation of the English-language version demonstrated a score of 112 or above may be indicative of a clinical level of anxiety [19].

The predictive validity of the measure has been examined and confirmed in relation to infant feeding outcomes and behaviours [20], and maternal bonding behaviours [21]. Across both of these studies, the PSAS demonstrated stronger predictive power than a general non-childbearing measure of anxiety.

To date, initial validity and reliability has been demonstrated in one large UK sample [19], and more recently two Turkish samples [22, 23]. The English-language PSAS is currently being used throughout the UK, Canada, Australia, Ireland, Rwanda, and the USA. Translation of the PSAS has taken place in Italy, France, China, Spain, and The Netherlands, but are, as yet, unpublished. Further translations are currently ongoing in Brazil, Egypt, Germany, Greece, India, Indonesia, Iran, Iraq, Jordan, Malaysia, Portugal, and The Philippines. A further translation into Burmese (the language of Myanmar) is being undertaken by a research team in Thailand. (See Fig. 1).

Research to understand the psychological impact of COVID-19 in perinatal populations is critical in mitigating the severity of the outbreak. Rapid progress in addressing this pandemic depends upon a coherent and integrated response from researchers [24]. There have also been global calls for the mental health sciences to work in a multi-disciplinary fashion to address the possible mental health crisis which may follow the physical health pandemic [25], and where possible make addressing mental health needs an integral part of the COVID-19 response [26]. The 51-items in the PSAS take approximately 10 min for mothers to complete which, when integrated into a survey containing a battery of psychometric scales, may be burdensome, especially during the current pandemic, where specific populations may be over-sampled. Therefore, during times of crises such as the current COVID-19 global pandemic, it is desirable to use shortened measures, to reduce participant burden. Furthermore, to the authors’ knowledge, there have been no psychometric scales (including measures of perinatal mental health) validated for use during the current pandemic.

In the UK, NICE guidelines recommend psychometric measures should contain fewer than 12 items for optimal accessibility [11]. This article reports the development of a 12-item research short-form of the PSAS, validated for use in global crises [PSAS-RSF-C]. The validation of the PSAS-RSF-C, in English, acts as a nested psychometric study within a larger on-line UK survey: PRegnancy and Motherhood during COVID-19 [The PRaM Study]. As the 51-item PSAS is currently undergoing multiple translations (all at various stages of validation), we also present, within this paper, the same 12 items, in five other languages (Italian, French, Chinese, Spanish, and Dutch). By increasing the accessibility of the PSAS, validated for use during COVID-19, we aim to enable researchers the opportunity to measure maternal anxiety, rapidly and accurately, at times of global crisis.

Maternal- and infant-related demographic questions were asked at the beginning of the survey (Table 1). Specific questions were also asked on the incidence of COVID-19 in the mother and any family members. Maternal age ranged between 18 and 46 years (M = 31.69, SD = 5.15) and infant age ranged between birth and 12 weeks (M = 7.92, SD = 3.67). Women were predominantly white (95%), married (57%), university educated (64%), and professionals (52%). In addition, 140 women had a clinical diagnosis of anxiety (20%); 85 had a clinical diagnosis of depression (12%); and 28 had a clinical diagnosis of PTSD (4%). Forty-nine women believed they had COVID-19 (7%), with two of these women having been tested. Additionally, 125 women believed a family member had COVID-19 (18%), with ten of these women reporting their family member had been tested. Finally, 242 women believed their birth experience had been affected by UK Government ‘lockdown’ restrictions (34%).

Participants were recruited through on-line social media platforms via an advertisement providing a link to the Qualtrics survey platform. Prior to the main survey, an electronic information sheet and consent form were provided with a tick box to confirm consent. Upon completion of the survey, participants were provided with a full electronic debrief with signposting to relevant support information, and were entered into a £25 prize draw.

Data taken from The UK PRaM Study (N = 710) were randomly split into two samples: One for the exploratory factor analysis (n = 344), the other for the confirmatory factor analysis (n = 366).

The factor structure of the PSAS-RSF-C (Table 3) was examined using data from all the participants in Group 1 (n = 344). The parallel analysis suggested there were four factors which are consistent with the original 51-item measure. Sampling adequacy for the 12-item scale was excellent (Kaiser-Meyer-Olkin; KMO = 0.80) and Bartlett’s test of sphericity demonstrated correlations between items were large enough for PCA (χ² (66) = 284.17, p < .001). The PCA revealed four factors, which in combination explained 75% of the total variance. The UK-based PSAS Working Group [VF, SAS, JAH, JCGH, SMD, PaCh] conducted a theoretical review of the factor loadings after oblique (direct oblimin) rotation, which revealed that the factor structure of the PSAS-RSF-C was identical in nature to that of the original PSAS. Three items loaded onto each of the four factors which they belonged to in the long form: Maternal Competence and Attachment Anxieties; Infant Safety and Welfare Anxieties; Practical Infant Care Anxieties; and Psychosocial Adjustment to Motherhood. The four factors had good reliability, with McDonald’s ω ranging from .74 to .88 (see Table 3). Furthermore, the overall scale had good reliability (McDonald’s ω = .87).

The initial model was a moderate to good fit of the data (CFI = .928, TLI = .901, RMSEA = .096, SRMR = .067). Modification indices indicated a covariance should be added between three pairs of residuals (Fig. 2). As a result, the model fit improved (CFI = .973, TLI = .960, RMSEA =. 055, SRMR = .045). All items significantly loaded onto each factor (p < .001; see Fig. 2 for the standardised factor loadings). The overall scale retained good reliability (McDonald’s ω = .87).

The participants who completed all convergent measures were included in these analyses. The PSAS-RSF-C total score was significantly correlated with theoretically related measures of anxiety (i.e. EPDS-A [n = 710]; STAI-S [n = 709]), depression (i.e. EPDS [n = 710]), and parenting competence (i.e. PSoC [n = 666]) indicating good convergent validity (Table 4).

First, An independent samples t-test demonstrated that the mean PSAS-RSF-C scores for those with a self-reported clinical diagnosis of anxiety (n = 140; M = 28.35; SD = 7.45) were significantly higher than those without a self-reported clinical diagnosis (n = 567; M = 23.93; SD = 5.60) t (705) = − 7.79; p < .001. Then, to preliminarily evaluate the performance of the PSAS-RSF-C in distinguishing between those with or without a current clinical diagnosis of anxiety, a Receiver Operating Characteristic [ROC] analysis was conducted. A statistically significant ROC curve (AUC = .68; SE = .03; p < .001; 95% CI .62 to .73) revealed the optimal cut-off PSAS-RSF-C score for detecting clinical levels of anxiety was 26 out of a total of 48 with a sensitivity and specificity of .62 and .64, respectively (Fig. 3).

Translation of the PSAS follows traditional methods of psychometric scale translation [39‐42]. In brief, this requires at least three researchers to supply separate versions of the PSAS into the desired language. These translations are then given to an independent back-translator [39], who is unfamiliar with the PSAS. The back-translator will select the most eloquent translation of each item from the three translated versions [40]. This single back-translation is then checked with the UK-based PSAS Working Group, who checks the back-translation against the original PSAS for inconsistencies and intended meaning [41]. The items which make up this final back-translated version are the items which will be used for the final translated version. Inconsistencies at any stage of the translation process are discussed amongst the translating team, with irreconcilable discrepancies being referred to the UK-based PSAS Working Group [42].

We therefore have selected the same 12-items in the currently unvalidated, but translated Italian, French, Spanish, Chinese, and Dutch versions of the 51-item PSAS to form a PSAS-RSF-C for each country (PSAS-IT-RSF-C, PSAS-FR-RSF-C, PSAS-CN-RSF-C, PSAS-ES-RSF-C, and PSAS-NL-RSF-C, respectively, Table 2).

A major strength of this study is that, to the authors’ knowledge, this is the first postpartum-specific psychometric scale to be validated for use during the COVID-19 global pandemic, and in being so, is also one of the first perinatal psychometric tools to be validated in the current crisis, with others including the Pandemic-Related Pregnancy Stress Scale [47]. This means postpartum anxiety can be effectively measured during this pandemic, and similar global crises in the future. Furthermore, many psychometric studies inappropriately use factor analyses developed for interval-level data, when the psychological construct, and the measurement of it, is, in fact, ordinal in nature. A strength of this study is the use of a polychoric correlation matrix, which overcomes this common, but often statistically inappropriate practice [48]. Additionally, we report McDonald’s Omega, to appraise reliability, as opposed to Cronbach’s Alpha, which focuses on the greatest lower bounds estimate [49]. Finally, providing a validated 12-item research short-form of the PSAS means it can be more readily adopted into studies containing a large battery of tests during COVID-19 and other such global crises, without being prohibitively long. Whilst the diagnostic accuracy was found to be good in this validation study, we recommend erring on the side of caution when making clinical decisions based on this research short-form alone.

A limitation is the use of an on-line convenience sample. This recruitment, whilst pragmatic for the rapid response nature of this research, lacked sampling control. The sample were predominantly white, married women, with university education and professional occupations. There was also a high proportion of assisted vaginal births and caesarean sections, and whilst the cause of this remains unknown, it has been suggested women have delayed seeking care during the pandemic, leading to higher incidence rates of obstetric complications [50]. The psychometric properties of the PSAS-RSF-C may, therefore, vary in other populations and it should be subject to replication studies using diverse samples.

Future research efforts should be directed towards the global validation of the PSAS and the PSAS-RSF-C. Whilst the five translations of the PSAS-RSF-C presented in this paper have not yet been subjected to validation studies, validation work must be conducted to ensure the validity and reliability of both the 12-item PSAS-RSF-C and the 51-item PSAS in other languages.

Previous work examining the predictive validity of the PSAS [20, 22] consistently finds it is a more powerful predictor of perinatal outcomes than a general measure (e.g. STAI [33]). Consequently, this tool would be useful in longitudinal studies aiming to better understand the persistent mental health impact of the pandemic (and other such global crises in the future) on maternal and infant outcomes. Preliminary work demonstrates the sub-scales of the PSAS have differential effects on maternal and infant outcomes [51]. Further investigations of the sub-scales could provide greater level of detail in terms of identifying specific risk factors and mechanisms of PPA and may offer opportunities for targeted intervention.