Outcome reporting in neonates experiencing withdrawal following opioid exposure in pregnancy: a systematic review

This initiative has been prospectively registered with Core Outcome Measures in Effectiveness Trials (COMET) [24]. The complete study protocol has been published [25] and is available at https://​trialsjournal.​biomedcentral.​com/​articles/​10.​1186/​s13063-016-1666-9

Primary research studies including randomized controlled trials, clinical trials, case-controlled trials, uncontrolled trials, observation cohort studies, clinical practice guidelines, and case reports of any interventions used to manage NAS/NOWS were analyzed. For the purpose of this review, studies of pharmacological and nonpharmacological management of neonates exposed to opioids (including methadone, buprenorphine, oxycodone, prescription opioids) in utero who are diagnosed with NAS were included. We included all co-exposures based on the rationale that they commonly occur and that ICD 9 diagnostic coding does not separate withdrawals related only to opioids. All publications identified from the published search strategy between January 2007 and June 2017 were included. The rationale behind the time restriction is that a Cochrane review was published in 2009 encompassing studies reported prior to 2007 and a shift in clinical practice away from ubiquitous use of deodorized dilution of opium [26]. The steering committee felt that there was limited value in evaluating what outcome measures were used before widespread use of the Finnegan assessment tools and oral morphine treatment weaning.

The search strategy (Additional file 1) was developed in conjunction with a reference librarian at the Hospital for Sick Children on Ovid MEDLINE 1946 to present with daily update, Ovid MEDLINE in-process and other nonindexed citations. The search was also applied to EMBASE and Cochrane Central on 6 June 2019. The latest Cochrane review was published in 2010. Reference lists of four recently published systematic reviews of NAS were evaluated. In addition to the electronic search strategy completed as above, ClinicalTrials.​gov was reviewed and identified 37 studies for ongoing trials related to NAS. Bibliographies of all included studies and systematic reviews were reviewed to identify relevant articles not generated in the search. In addition, the steering committee reviewed the list of articles to ensure comprehensiveness and to provide related articles not identified by the search strategy.

Two independent reviewers (LEK and SM) screened titles and abstracts resulting from all the search strategies in EndNote X6. For studies that were deemed eligible by title and abstract, full-text articles were obtained. Full-text articles were critically reviewed independently (LEK, SM and FS) to assess eligibility. Studies published prior to 2007 were excluded as most studies focused on the use of tincture of opium which is no longer utilized for the management of NAS. Reasons for exclusions were documented. Any disagreement in study eligibility criteria was resolved through discussion and consensus or by consulting the principle investigator (LEK). Studies were excluded if they did not describe NAS health outcomes or if the full text was not available in a language mastered by our team (English, French, Spanish or Dutch). Only infants with an NAS diagnosis (irrespective of the diagnostic tool utilized) following known opioid exposure in utero were included regardless of concomitant substance exposure.

Data were extracted independently and in duplicate by two reviewers (SM and FS). Disagreements were resolved by consultation with the principal investigator (LEK). A standardized table was utilized for data extraction which included the following information: year of publication, corresponding author and contact information, study design (randomized controlled tried, cohort study, quality improvement, case series, case report), NAS intervention type (pharmacologic or nonpharmacologic), intervention group, control group, randomization, sample size, study objective in full text, method of NAS diagnosis, frequency of monitoring NAS symptoms, duration of exposure, type of maternal exposure (methadone, suboxone, buprenorphine, illicit opioids, benzodiazepines, cocaine, and so forth), study inclusion criteria, study exclusion criteria, primary outcomes, secondary outcomes, and justification for outcome choice. An outcome was included as reported if it was included in the methods, results or discussion sections. The outcome was placed under “primary outcome” if it was explicitly stated as the primary outcome in the study, it was the only outcome reported, or it was implicit in their data reporting (i.e., used in the sample size calculation). Composite outcome measures were separated to gauge the full breadth of definitions utilized in primary research studies.

Considerable heterogeneity in the terminology used to report outcomes was noted during the data extraction. Outcomes of similar themes were grouped during the data analysis process. For instance, “days of infant opioid treatment”, “length of infant methadone therapy” and “duration of infant oral morphine therapy” were all included in the outcome category “duration of pharmacotherapy for NAS”.

Outcomes reported as either primary and/or secondary were assigned to one of the four core areas plus adverse events as defined in OMERACT Filter 2.0 [27]. OMERACT is a conceptual framework to ensure that a comprehensive set of outcomes is selected to formulate a core outcome set (COS). Its core areas encompass content that is measurable in a trial that include both patient-centered and intervention-specific information. These four core areas include “death”, “life impact”, “resource use” and “pathophysiologic manifestations”. OMERACT recommends that “adverse events” should be measured within the core areas [27].

A total of 107 primary outcomes and 127 secondary outcomes were reported in the included 47 studies. Of these outcomes, 67 primary and 94 secondary outcomes, respectively, were unique ‘terms’ that were not used in any other study. The individual outcomes were mapped into outcome categories as illustrated in Table 2. For ambiguous reported outcomes, the full text of the article was reviewed to determine how it was measured to classify the outcome into an outcome category. For instance, “effectiveness of methadone for the treatment of NAS” was the primary outcome in one article [52]. The surrogate markers of “effectiveness of methadone for the treatment of NAS” were “time since need for pharmacological treatment was established”, “methadone dose in mg/kg/day”, “methadone dosing interval” and “Lipsitz scores”. These surrogate markers were separated as individual outcomes to accurately reflect specific outcome measures. Fifteen outcomes were mapped into the outcome category of “miscellaneous single outcomes”. The most frequently reported outcome category was “duration of hospital stay”, which was reported in 30 primary research studies (63.8%) as a primary or secondary outcome. Twenty-one of these primary research studies reported “duration of hospital stay” as a primary outcome measure. As noted in Table 2, there is heterogeneity in how the most commonly reported outcome (duration of hospital stay) is measured in research studies using six different definitions. Figure 2 demonstrates the distribution of primary and secondary outcome terms across individual research studies. There is also a wide range in the number of outcomes reported (1 to 12) per study. For instance, only seven (15%) of studies reported all three of the most frequently reported outcome categories (duration of hospital stay, duration of pharmacotherapy, and presence of NAS symptoms). Heterogeneity of outcome selection is reflected by the large number of primary (53/107) and secondary (76/127) outcomes that were only reported in a single study. The timing of outcome measurement was poorly reported across studies and was inconsistent. For example, neurobehavior was measured in two studies: one study reported Bayley Scale scores at 1 year and the second reported the NICU Network Neurobehavioral Scale (NNNS) at 5–7 days old and at 44 weeks postmenstrual age. Withdrawal severity scoring outcomes were reported after 1 week of treatment, six times daily, and “according to hospital policies”. Training of outcome assessors was only mentioned for seven primary outcomes and six secondary outcomes. Training protocols were only provided in one study. Outcome assessors were described for 12 primary and 14 secondary outcomes, and were mostly clinicians (physicians and/or nurses, n = 9/12 and n = 12/14, respectively). The study design may affect the outcomes reported as demonstrated in Table 3. For the most commonly utilized study designs such as retrospective cohort studies (n = 21), randomized controlled trials (n = 10) and prospective cohort studies (n = 5) there was less variability in outcomes selected compared to quality improvement studies and case reports. Cohort studies and clinical trials reported “duration of hospital stay” and “duration of pharmacotherapy” most frequently. Qualitative improvement studies also reported length of stay, which reflect the nature of the study design, targeted to reduce adverse events and resource consumption. Case series/reports, classically written to report unusual or novel occurrences, tended to report “miscellaneous single outcomes” that were not studied at a larger scale in cohort or randomized controlled trials.

One hundred and seven primary outcomes and 127 secondary outcomes were mapped to core domains defined by OMERACT filter 2.0 in Figure 3 [27]. The core area most commonly studied was “Resource Use/Economical Impact” through 72 primary outcomes and 67 secondary outcomes. This was followed by “pathophysiological manifestations” covered by 23 primary outcomes and 26 secondary outcomes and “life impact” examined by nine primary outcomes and 15 secondary outcomes. “Adverse events” were reported as three primary outcomes and 16 secondary outcomes. “Death” was not reported as a primary outcome in any study and was reported as three secondary outcomes.

The strengths of our study are the inclusion of all primary research studies, following a preregistered protocol, and methods following the PRISMA guidelines [81]. A comprehensive search of the bibliographies for the studies included and recent systematic reviews was undertaken to identify any articles missed in our search strategy. Furthermore, the steering committee reviewed the list of included articles and were invited to suggest articles that were not identified using our search strategy. The main limitations to our review are the time boundaries of 2007–2017 for the search strategy and most included studies reported data from western countries which may limit external validity. This is, however, a limitation of the literature available and we do not feel this is related to our search or selection process. In this review, we did not contact the research groups for missing information. The aim of our study was to abstract primary and secondary outcomes and, as such, we did not intend to assess the rigor to which the studies reported this information as missing data are unlikely to affect the conclusions formulated here.

The conclusions from this systematic review advocate for the formation of a COS for NAS/NOWS to establish consistently reported outcomes with standardized definitions. Other COSs and studies have demonstrated that public involvement leads to research that is comprehensive and relevant to the patients receiving evidence-based interventions [82]. Since 2010, the COMET initiative has set out to establish standardized sets of outcomes for intervention clinical trials and clinical audits [83]. Their overarching objective is to limit outcome heterogeneity, include relevant outcomes, and reduce outcome reporting bias through the development of COSs. Trials evaluating the effectiveness of a treatment should measure and report at least each outcome in the COS. However, the primary outcome should be chosen and powered to answer the research question. COS development should be comprehensive and include outcomes relevant for multiple stakeholders from patients/families, clinicians and scientists.

Recent expert reviews in the Journal of the American Medical Association [28] and the New England Journal of Medicine [3] concluded several areas of uncertainty including standardized and validated assessment tools, optimal medication treatment regimen, optimal location for weaning, long-term neurodevelopment and family outcomes in NAS/NOWS that require further research. Given the heterogeneity in outcome selection, measurement and reporting highlighted by this study, it is imperative and timely to devise a COS for NAS/NOWS. A NAS/NOWS COS will standardize research methodology to reduce research waste and enhance transparency for clinical decision-making. With a scarcity of knowledge on the long-term patient and family impact of NAS/NOWS, the COS should reflect the needs and concerns for the stakeholders that are most affected by this condition. A consensus and evidence-based NAS/NOWS COS is underway led by a multidisciplinary international steering committee. The development of this COS includes family interviews to ensure that future studies will evaluate outcome measures that are meaningful to the population it affects the most.