Severely malnourished children with a low weight-for-height have similar mortality to those with a low mid-upper-arm-circumference: II. Systematic literature review and meta-analysis

In order to exhaustively include as much evidence as possible pertaining to mortality risk of SAM children by the two anthropometric criteria, all studies were eligible for inclusion if they contained the numbers of children, less than 60 months of age, that were alive and dead when they exited the study, and who had SAM by any WHZ and any MUAC criteria, in any language and at any time.

In February 2017 we searched Pubmed, Scopus and CABI- index for papers of relevance using the search terms: (Mortality or Death or Case Fatality) and (MUAC or mid-upper-arm circumference or arm-circumference or Perimeter Brachial) and (WHZ or weight-for-height or weight-for-length or poids pour taille or PPT). Google-Scholar was also regularly consulted, particularly for papers indexed after our initial search. The search strategy is shown in Fig. 1. Papers that did not provide the numbers of children with MUAC, WHZ and outcome were not considered. Papers relating to children with primary disease unrelated to nutrition were not considered (e.g. congenital heart disease, chronic renal failure, malignancies, cystic fibrosis, etc.). Where possible the original theses, reports and supplementary information from which the paper was written were obtained as these usually gave fuller information. The data presented in our companion paper [33] were not included. One conference presentation [36] was included in the analysis. From the final 20 reports (21 datasets) containing relevant information the following data were abstracted: authors and reference: date: place: type and purpose of study: study setting, in-patient facility (IPF), out-patient treatment program (OTP) or Community cohort: the length of time the children were observed: the standards used for diagnosis of SAM by MUAC and WHZ: the age range of the children included: whether oedematous children were included or excluded: the percent of overlap between the two parameters (i.e. those fulfilling both WHZ and MUAC criteria for SAM / total SAM): the percent of missing data: the default rate: the total number of children and the number that died in the WHZ and MUAC categories. Were possible the number of children fulfilling both diagnostic criteria was obtained and these reports then formed a separate category. It was not possible to estimate the proportions of overlap for those reports where this information was not given.

The original authors analysed their data in a number of ways and some did not present results that directly compared the risk of death of SAM children diagnosed by MUAC, WHZ or both. Most analysed all-cause mortality in the whole childhood community or patient population, including children who did not have SAM by any definition. The area-under a ROC curve, logistic regression, life table, etc., including all available children, were then used in the analysis. For the present analysis the subjects that did not have SAM were not considered; this was because the objective was not to compare SAM children with non-SAM children, but to compare the relative mortality risks of children with SAM who fulfilled either the MUAC or the WHZ criterion. Therefore the original analyses were not considered. The actual numbers of children who were below the cut-off points used in the paper to define SAM who survived and those who died were used to re-analyse the data. We directly compared the case fatality rates (CFR) and relative risk (RR) of death of children with SAM by MUAC with SAM by WHZ.

There were 10 datasets where children with SAM could be divided into the following groups:

SAM by MUAC but not by WHZ criteria – designated as “S-muac”

SAM by WHZ but not by MUAC criteria – designated as “S-whz”

SAM by both MUAC and WHZ criteria – designated as “S-both”

There were 11 datasets where children with SAM by both criteria were not separately reported from those children with SAM by one or the other criterion (i.e. S-both could not be determined). In these reports children with SAM by MUAC may, or may not, also have SAM by WHZ; they are designated as “All-muac”. Similarly, those with SAM by WHZ may, or may not, also have SAM by MUAC; they are designated as “All-whz”. These papers were potentially in error because of mathematical coupling (see companion paper I [33]) and are examined as a separate group. Mathematical coupling occurs where “one variable directly or indirectly contains the whole or part of another, and the two variables are analysed using standard statistical techniques” [37, 38]. Thus, S-muac, S-whz and S-both data are mutually exclusive, but All-muac and All-whz are not mutually exclusive. There were also papers where oedematous children were incorporated into the analysis, usually with an unknown proportion of oedematous children in each group; this is known to be a major confounder [33]. Thus, where oedematous children were included in the dataset, so that marasmus could not be differentiated from marasmic-kwashiorkor the reports were analysed in a sub-category.

The data from two studies were each reported using different standards (study from Senegal in datasets 18 & 19 [39, 40] and from Democratic Republic of Congo in datasets 20 & 21 [40, 41]). Each of the duplicate datasets were included in an initial analysis by subgroup comparing the effect of using different diagnostic cut-off points to analyse the data and demonstrate the effect of using different references on the derived outcomes. The duplicated data (18 to 21) were not used in the other analyses as report 7 had incorporated the same two datasets, combined with a study from Nepal, into their analysis. As report 7 used WHO criteria, separated children with both criteria (S-both) and as far as possible excluded oedematous children the data in this report was considered to be most reliable.

Individual datasets with S-muac vs, S-whz and All-muac vs All-whz were analysed by 2 × 2 chi-squared. Where there were fewer than 5 children in any expected category the analysis was by Fisher’s exact test.

The data abstracted from the individual papers were examined by meta-analyses using MetaXL version 5.3 software [42]; the random effects model with relative risk (RR) output divided by subgroups, was used to compare the effects of potential bias. The quality effects model was used for the final analysis where the studies are grouped by whether S-both data were available or not [43]. Tests for heterogeneity were generated for each analysis by two methods (funnel plots and Doi plot). In the funnel plot, the ORs were plotted against the standard error, while in Doi plots, the ORs were plotted against z-score. To examine the potential effects of bias because of the differences in the study type, standards and inclusion criteria, the meta-analyses were repeated dividing the studies into the following sub-categories: 1) to test the biasing effect of oedema inclusion, S-muac v S-whz with and without oedematous case inclusion (i.e. omitting the 11 studies with only All-muac and All-whz information): 2) to test the effects of using different definitions of SAM, the datasets were divided by the standards and cut-off points for diagnosis: 3) to examine ascertainment and treatment bias, the studies were grouped by type of study (community cohort, in-patients treated for SAM, out-patients treated for SAM): and, 4) the difference between those that excluded children with S-both and those that incorporated them into their dataset. The assessment of study quality and potential for bias was assessed, using the criteria in table Additional file 1: Table S1. Confidence intervals are 95% for all reports and a probability of 0.05 is considered significant.

In order to examine the effects of including children suffering from both deficits (i.e. WHZ < −3Z and MUAC < 115 mm) in the comparison of mortality related to WHZ and MUAC, we calculated both the CFRs of S-whz and S-muac and for the same cohorts of SAM children when S-both was added to S-whz and S-muac to give the corresponding All-whz and All-muac CFR from the same study. This analysis could only be performed with the reports that differentiated S-both from the single deficit categories.

This is a secondary analysis of published data in the public domain. As such no ethical clearance was required.

From our empirical data [33] and other studies [59, 60] it is clear that children with oedema have a higher mortality than those that are oedema free, no matter their anthropometric status. However, the augmentation of oedema related mortality appears to be different in children who have wasting by MUAC or by WHZ.

Figure 2 shows a meta-analysis of papers 1 to 10 comparing those with and without admixture of oedematous cases. Although none of the pooled differences are significant, the first 7 studies that have excluded oedematous cases show that the RR of death is greater in S-whz than S-muac (RR, 1.12; CI, 0.75–1.68). For the 3 studies that included oedema the RR indicates that S-muac has a higher mortality risk than S-whz  (RR, 0.59; CI, 0.28–1.21).

WHO recommends that only the WHO₂₀₀₆ standards be used as the reference for WHZ and < 115 mm for MUAC diagnosis of SAM in children 6 to 59 months [61, 62]. Obsolete references were naturally used for studies published before 2006 and these have not, to our knowledge, been re-published using WHO criteria. References that are more stringent than the WHO₂₀₀₆ standards, such as the NCHS reference, should have a higher case fatality rate (CFR) and a lower case load than would be the case if the WHO standards were used. This is because the children then diagnosed as SAM will have a lower mean WHZ and thus be more severely malnourished with a higher risk of death than those included with a less stringent reference. Fig. 3 shows the cut-off weights for given heights of the references used in the various studies. Where a more lenient reference, such as the CDC₂₀₀₀ curves, is used to define SAM the additional children included in the SAM cohort will have an ameliorating effect on the CFR as they are at a lower mean risk of death, but the additional children will increase the case load. This effect can be quite profound because the absolute number of children included in the SAM category increases exponentially as the diagnostic criteria are relaxed. This is because the shape of the distribution curve of anthropometric parameters in the community is approximately Gaussian [63, 64]. Similarly, where a MUAC of < 110 mm has been used we expect a higher CFR and a lower case-load, than where < 115 mm has been used to define SAM. Thus, whether either the MUAC or WHZ criterion is more or less stringent the relative CFRs and case-loads will change reciprocally. The interpretation of the mortality rates must be judged by the references used. As shown in Fig. 3, the different reference slopes are non-linear. It is not possible to convert data obtained from one set of references to another or to properly amalgamate the data from studies that used obsolete standards.

Figure 4 shows the meta-analysis of all the datasets grouped by the standards that were used. None of the pooled differences showed a significant difference between the risk of death of children fulfilling the MUAC or the WHZ criteria. The studies that used the WHO recommended criteria have an equivalent mortality risk for WHZ and MUAC.

Unexpectedly, those studies that used the NCHS reference had the same RR as those that used the WHO standards with no difference between MUAC and WHZ. Theoretically, WHZ should have had a higher mortality because of the shape of the cut-off for NCHS -3Z; but, as the WHO and NCHS reference lines cross each other at about 73 cm height the net effect will depend upon the height (age) distribution of the children in the studies. The lack of difference between WHO and NCHS could also be due to confounding by inclusion of oedematous children in each of the NCHS studies as well as some of the WHO studies. Nevertheless, there is no net difference and for this reason we have included studies that used the NCHS reference in our main meta-analysis.

The two datasets (a single report) that used the CDC2000 criteria [40] and reduced the cut-off for MUAC to < 110 mm (CDC2000/110) strongly favoured a higher risk of death for those admitted by MUAC; this was expected and almost reached significance. Report 18 and 19, from Senegal, used exactly the same original data, therefore these reports can be directly compared. When NCHS and MUAC< 115 mm (NCHS/115) was used All-whz has a much higher mortality than All-muac (RR, 1.60; CI, 0.93–2.77); whereas, when CDC2000/110 was used All-muac had a much higher risk than All-whz which just reaches significance (RR, 0.59; CI, 0.34–1.00). The same data were also used in reports 20 and 21 from DRC; again there is a substantial difference between the RRs when using the two different anthropometric references. Savadogo et al. [58], in the “other” group, used an obsolete MUAC-for-age formula to calculate All-muac Z-scores [65] and then divided the results into tertiles for analysis; minus 4Z using this standard increases from 103 mm to 111 mm between 6 and 59 months of age. The mean Z score of Savadogo’s children on admission was − 4.59Z which for a 12 to 24 month old child equates to about 100 mm. It is clear that the MUAC standard for these children was very low which would account for the RR of death being significantly greater for S-muac than S-whz (RR, 0.74; CI, 0.59–0.92).

In each of the other meta-analyses we have omitted these reports using CDC2000 reference and MUAC< 110 mm (# 19 & 20), the duplicated data from those reports (# 18 & 21) as well as Savadogo’s report (# 17). The data from datasets 18 to 21 are incorporated into report #7.

In-patient cohort studies are often criticised on the basis that they do not represent the children in the community and are subject to ascertainment bias [22, 30]. In Fig. 5 we sub-divide the reports by mode of treatment to determine whether this has an effect on comparison of WHZ and MUAC deaths. The in-patients had about the same mortality risk with MUAC and WHZ (RR, 0.92; CI, 0.66–1.28).

The two OTP studies are dominated by a retrospective study from Sudan (study 3) where there were very few S-muac admissions (WHZ 1486 with 13 deaths; MUAC 21 with 1 death); there was no follow-up of defaulters to distinguish death from simple non-attendance. The other report (study 4) from a secondary analysis of a prospective RCT is much more reliable and shows S-whz to have a greater, but non-significant, RR of death (RR, 1.31; CI, 0.30–5.8).

The community studies also showed a non-significant higher risk with WHZ than MUAC (RR, 1.22; CI, 0.82–1.81).

Figure 6 shows the overall analysis, omitting the duplicated studies found in the previous analyses. Here they are sub-grouped by whether S-both was incorporated or excluded from the analysis. There was no difference in the risk of death between those with a low MUAC and those with a low WHZ (RR, 0.99; CI, 0.73–1.35). The heterogeneity of the S-whz v S-muac group was low (I² = 23%) but high in the All-whz v All-muac group (I² = 80%). The statistics, sensitivity analysis, Doi and funnel plots are given in Additional file 3: Figure S1.

As there was no difference in the meta-analysis of children with single and double deficits (Fig. 6), and we had found in our empirical data that inclusion of both deficits into the S-whz and S-muac groups gave erroneous results due to mathematical coupling, we examined the difference this change in analytical procedures would cause using the studies where we had information on S-both.

The differences between CFRs of S-whz v S-muac and All-whz v All-muac in reports 1 to 10 are shown in Table 3. In two of the papers the All-whz/muac gave a larger difference in CFR than the S-whz/muac comparison; in 6 of the papers S-whz/muac CFR comparison was larger than the All-whz/muac comparison. In the other 2 papers by Lowlaavar et al. [47] and Berkley et al. [49] the direction of the difference was actually reversed (in opposite directions), so by inclusion of children with both deficits into each of the single groups contrary conclusions would have been reached. These latter studies are examples of Simpson’s paradox, an extreme form of confounding [33, 66], but inclusion of S-both into the whz v muac comparison led to erroneous results in every study.

The data from the studies included in this review are all subject to bias, some sufficiently severs to render the studies of little value in setting policy. The major problems with some of the studies are outlined in Additional file 2.

From our empirical data [33] it is clear that, in judging MUAC and WHZ as independent criteria for diagnosis of SAM, the children with both deficits have a significantly augmented mortality risk. This is confirmed by most of the studies which report the outcome of children with both deficits and have sufficient events (Table 2). Many studies only counted children with a low MUAC and added to that group those that also had a low WHZ (S-both), and their deaths, without separating those with double defects (datasets 11 to 21).

This incorporation of exactly the same data into both groups, and then comparing the groups, results in a phenomenon termed mathematical coupling [37, 38] that causes the analytical results to be in error. In other words some of the children were compared with themselves. The meta-analysis did not show any difference between these two types of selection of children to include in an analysis. We therefore examined the papers where we could assess the extent to which this may have affected the data (Table 3) and found that CFRs were different in each case, some with greater augmentation of WHZ and others of MUAC; two of the studies even reversed the relative magnitude of the WHZ and MUAC mortality rates, a phenomenon termed Simpson’s paradox [66]. The degree and direction of the changes depend upon the relative mortality in the data common to both groups and the relative size of the three groups. These factors were different in each of the studies shown in Table 3.

The report by Lowlaavar [47] is particularly illustrative of the errors that can occur. There were no deaths at all in the children with S-muac and a CFR in the S-whz children of over 6%. When the children with both a low MUAC and WHZ who have a high CFR (17%) are added to the analysis of both groups there is a dramatic reversal so that All-muac now appears to have a higher CFR than All-whz. This analytical error affected most of the studies reported, including those where single defects were also reported, because, in every report the combined data was used in their life tables, logistic (or other) regressions and generated ROC curves. As Tu et al. [66] state: “Incorrect use of statistical models might produce consistent, replicable, yet erroneous results” this seems to be the case with all the original analyses in the literature reviewed.

All the studies reviewed have limitations that are presented in the discussion and in Additional file 2. Community representative samples of children 6–59 months show that only an average of 16.5% of the children meeting the criteria for SAM have both a MUAC and WHZ below the WHO cut-off points; the rest of the SAM children have either one or the other criterion making them eligible for treatment [21]. The studies reviewed all have a higher proportion of children satisfying both criteria than are present in the community. In Table 1 we show the overlap (S-both) of children in the study, and that found in recent representative samples of the community.

The heterogeneity of the standards, the admixture of oedematous cases and the failure to account for confounding, such as TB, HIV, and non-nutrition related conditions, makes simple amalgamation of data from the different study populations problematic. Similarly, there are major co-morbidity, temporal and stochastic biases with the community studies [82]. The community studies were performed at a time when most countries had a much higher prevalence of malnutrition, all-cause mortality, and poor coverage of other public health programs such as measles vaccination, vitamin A capsule distribution, salt iodisation, insecticide impregnated bed-nets and HIV services. It is perhaps for this reason that the community studies also have a higher proportion of children satisfying both criteria than are found with recent surveys in the same areas. It is also probably the reason why the attributable fraction of death due to SAM in these older community studies was lower than expected. It is estimated that SAM and moderate acute malnutrition underlie up to half of all paediatric deaths [80].

The meta-analyses are only as good as the studies that are incorporated into the analysis. We have eliminated the most egregious of the studies, but it must be acknowledged that none of the studies are without potential bias. Therefore, unfortunately, there are no definitive unflawed data upon which to rely in order to unequivocally inform public health policy.