Labour Q1 pain – poorly analysed and reported: a systematic review

Women’s experiences of pain during labour are highly variable [1] and are related to women’s overall assessments of childbirth [2]. A painful labour may have long-term consequences for women’s health and wellbeing [3]. Most women in labour require pain relief, which can be administered in either pharmacological or non-pharmacological form [4].

Comparing labour pain between different treatment groups needs consideration, both regarding when and how often pain assessments are made, and how statistical analyses have been conducted to reflect the reality of women’s pain varying during labour. Women’s pain varies both between individual women and within the same woman over time as well as between different treatment groups. The mean intensity of labour pain is closely related to the progression of labour and increases with greater cervical dilatation [5‐10]. In addition, labour pain correlates with the intensity, duration, and frequency of uterine contractions [1, 8]. Furthermore, although pain increases with time, individual progression can differ from woman to woman as a result of both biological factors and the interventions that are administered.

When pain assessments are made more than once for each woman, conclusions made on the management of labour pain will be based on repeated measures data. The most common research questions (and corresponding statistical test) for these study designs are; 1) Is there any difference in mean pain between the treatment group and the control group(s) over all time-points? (overall treatment effect between groups), 2) If so, are the differences in labour pain between groups equal or do they differ at different time-points? (interaction between treatment and time), and 3) is there a difference in pain within each group? (within group effect). In these cases, appropriate statistical methods for repeated measures data should be used (rANOVA or mixed models). This is in contrast to analyses in studies where only one time-point is of interest. In such cases, end-point analyses such as t-tests or ANOVA could be used.

The most commonly used methods to measure labour pain are verbal reports with standardised instruments such as the Visual Analogue Scale (VAS), the McGill Pain Questionnaire (MPQ), the Short-Form MPQ, verbal rating scales, or simple ordinal scales [1]. Although there are many studies that consider how labour pain should be assessed [1] in trials evaluating methods for pain relief during labour, to our knowledge there are no studies that have studied how these assessments are analysed and reported.

The purpose of this paper is to study how labour pain is analysed in studies with a repeated measures design, and to increase awareness of when and why modern statistical methods are suitable, so as to encourage their use over traditional methods. This paper will briefly review the most frequently occurring methods of repeated measures analysis for continuous outcomes when means are compared. A summary of aspects to consider when studies with an emphasis on labour pain are planned and analysed and the potential consequences of the choice of statistical method are presented in Table 1, (modified from [11, 12]).

The most commonly used study design was a randomised controlled trial (87%) with two or three experiment groups (Table 2). Analyses of labour pain were conducted including > 51 women in 66% of the studies. Measurements of labour pain were made repeatedly for each woman between 3 and 10 times in 90% of the studies, and the time intervals between each measurement was equal in 31% of the studies. Labour pain was rarely explicitly defined as a primary or secondary outcome.

A Visual Analog Scale (VAS) [19] was the most-often used tool to measure labour pain (used in 110 studies [83%]), although the description of the scale differed among the studies, or the scale was not clearly described. The maximum anchor labels of VAS were in 60% (66/110) of the studies. Among these 66 studies, we found 23 different definitions. The most commonly used anchor labels for maximum pain were “worst pain imaginable” (n = 17), “worst possible pain” (n = 10), “worst pain” (n = 10), “worst imaginable pain” (n = 9), “unbearable pain” (n = 4), and “severe pain” (n = 3). For the minimum anchor label of VAS, we found four definitions among the 67 studies where it was defined, with the most common definitions being “no pain” (n = 60) and “no pain at all” (n = 4).

Statistical methods used to compare labour pain between treatment groups are presented in Table 3. Among the 133 included studies, Mixed effect models were the most advanced statistical method found in 9 studies (7%), rANOVA in 26 studies (20%), and end-points analysis in 97 studies (73%). The most frequently used statistical methods to analyse repeated measures data were t-tests and ANOVA.

Among the 123 studies that used end-point or rANOVA to analyse labour pain, the specific number of observations used in the analysis at each time-point were stated in only 16% of the studies. In 85 of the 123 studies (69%), pain was assessed at time-points with unequal distances between them. It was not clear whether there were missing data, and/or if the measurements of labour pain were made at time-points with unequal distances in 108 of the 123 studies (88%), which used end-point analysis or rANOVA.

There was no clear presentation of how between-group comparisons were made in 34 of the 133 studies (26%). In the 46 of the 133 studies (35%) where within-group comparisons were made (i.e. comparison within a group between time-points), it was not clear how these comparisons had been made in 19 (41%) of them.

This systematic review found that studies aiming to compare at least two methods of labour pain management often use inappropriate statistical methodology, and inaccurately report how the statistical analyses were conducted. In particular:

Problems associated with the poor application and reporting of statistical methods are not new. Altman (1982) [20] noted decades ago that researchers both use the wrong techniques (wilfully or in ignorance) and use the right techniques incorrectly. Our study supports this finding, as the quality of the studies included here makes it difficult for readers to assess the value of the conclusions regarding labour pain management.

The use of more advanced statistical methods has increased in recent decades, and continues to do so [21‐23]. In general, there is a natural time lag between new statistical methods being developed and being used and applied in research. Mixed models were developed during the 1980s and implemented in statistical software programs such as SAS, STATA, SPSS and R in the following decades. In contrast to other research areas where there has been a debate on methodological aspects regarding how to analyse data that is measured repeatedly with mixed models [11], we have found no such debate in the area of labour pain research.

Thus, the prerequisites for using mixed models in the context of measuring labour pain in this study show that analyses are rarely carried out using an approach that reflects how data are collected. If and how the conclusions regarding the management of labour pain would change if appropriate analyses were conducted remains unknown.

Statements such as CONSORT [17] and STROBE [18] give recommendations on how statistical methods and results should be reported with the principle to follow; “Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results ( www.icmje.org )” [17]. In this study we did not only assess whether the statistical methods used were clearly described in the methods section (CONSORT, item 12a), but we also assessed if there was a clear agreement between the methods section and the results section, and whether the choice of statistical methods was appropriate for answering the research question in the context of repeated measured data. Our results should be interpreted with this in mind, and for clarity we would like to emphasise the following points. A clear description of the statistical methods used in the methods section does not necessarily mean that those methods were actually used, or that these methods were used appropriately. Also, a clear description of the method and clear agreement in the results section does not necessarily mean that the used method was appropriate to answer the research question.

The problem of using inappropriate statistical methods even though the statistical methods were clearly described was also quite common. For example, the methods section could describe that a t-test was used, which was also in clear agreement with the results section (which is often the case if a single method is used), but this occurred in cases where the method was not appropriate. One might argue that a “simple test” need not be presented in detail, which motivates the use of sentences such as “t-tests, rANOVA and Mann Whitney U tests were used as appropriate”. The problem is that what “appropriate use” is according to professional statisticians and applied researchers seems to differ somewhat.

Although most of the included studies were randomised and should follow the CONSORT statement for reporting, the recommendation to describe statistical methods in detail was not followed. One explanation for this lack of detail could be that not all journals have yet endorsed CONSORT, or had not endorsed CONSORT at the time of the publication of the included study. It has been shown that journal endorsement of CONSORT may benefit the completeness of the reporting of randomised trials [24].

A strength of this study is that generalizations can be made to all studies that compare the management of labour pain between groups over time with a continuous outcome, i.e. studies with a repeated measures design. In contrast to studies that investigate methodological aspects of analysis and design, we did not include studies from specific journals [21‐23, 25] or conduct our search strategy with keywords only regarding study design. Only 13% (17/133) of our included studies had the keywords “longitudinal” or “repeated measures”. Instead, we conducted our search with a subject of medical research interest, i.e. pain among women in labour, and then manually classified the studies with regard to study design, which implies that this systematic review reflects how studies in general analyse and report labour pain, not how this is done in specific journals or in studies with a specified study design based on search words such as “longitudinal” which we believe are more aware of the analysis of repeated measures [12].

Women’s perceptions of labour pain intensity vary greatly during labour, and the pattern of pain differs between women for a number of reasons, e.g. it differs between nulliparous and multiparous women and between women in early and later stages of labour [1]. Most methods for labour pain relief are safe for both mother and baby, however, their efficacy is unclear due to limited methodological quality. The way pain has been assessed differs considerably among the studies and it is not always clear which tool was used to assess pain or how the extremes of the scales were marked (anchor labels). In addition, information about pain relief at different stages of labour are also lacking [4]. The findings in our study confirm that end-points on the pain assessment scales differ considerably, which has been confirmed elsewhere [26]. High-quality systematic reviews and other synthesized research evidence compiled by HTA (Health Technology Assessment) organisations around the world, is the base for clinicians to make informed health decisions. As noted by others, to assess and analyse labour pain without considering the aspects of variability during labour, limitations in more traditional statistical methods, and without considering the fact that there are no specific recommendations with respect to anchor labels makes compiling evidence in e.g. systematic reviews challenging [4].

To improve design, analysis and reporting we recommend that authors, reviewers, and editors:

Studies aiming to compare at least two methods for labour pain management often use inappropriate statistical methods, and inaccurately report how statistical analyses were carried out. The verbal anchor descriptors of pain assessments, the time intervals between each pain assessment, and the number of pain assessments made varies largely among the studies. The statistical methods used in the analyses are often based on assumptions that are either not fulfilled or not described, or both. We recommend that authors, reviewers, and editors pay greater attention to method choice and use when designing, reviewing, and publishing studies evaluating methods for pain relief during labour.