Techniques for Handling Missing Data in Secondary Analyses of Large Surveys

doi:10.1016/j.acap.2010.01.005

Academic Pediatrics

Volume 10, Issue 3, May–June 2010, Pages 205-210

Presented in part at the Pediatric Academic Societies' Annual Meeting, San Francisco, California, April 29–May 2 2006.

https://doi.org/10.1016/j.acap.2010.01.005 Get rights and content

Objective

Using an appropriate method to handle cases with missing data when performing secondary analyses of survey data is important to reduce bias and to reach valid conclusions for the target population. Many published secondary analyses using child health data sets do not discuss the technique employed to treat missing data or simply delete cases with missing data. Missing data may threaten statistical power by reducing sample size or, in more extreme situations, estimates derived by deleting cases with missing values may be biased, particularly if the cases with missing values are systematically different from those with complete data. The aim of this study was to determine which of 4 techniques for handling missing data most closely estimates the true model coefficient when varying proportions of cases are missing data.

Methods

We performed a simulation study to compare model coefficients when all cases had complete data and when 4 techniques for handling missing data were employed with 10%, 20%, 30%, or 40% of the cases missing data.

Results

When >10% of the cases had missing data, the reweight and multiple imputation techniques were superior to dropping cases with missing scores or hot deck imputation.

Conclusions

These findings suggest that child health researchers should use caution when analyzing survey data if a large percentage of cases have missing values. In most situations, the technique of dropping cases with missing data should be discouraged. Investigators should consider reweighting or multiple imputation if a large percentage of cases are missing data.

Section snippets

Data Source

This investigation analyzed data from the live birth component of the 1988 NMIHS (in which birth certificate data are linked to mothers' survey data) and the 1991 Longitudinal Follow-up (LF) Live Birth survey.²⁰ The 1988 NMIHS used a nationally representative sample of 9953 children born in the United State that year and linked birth certificate data to interviews of mothers. African American and low birth weight children were oversampled. The mothers of 8285 children participated in the both

Results

Figure 1, Figure 2, Figure 3, Figure 4 compare the difference in the model coefficients for the 3 independent variables (child chronic illness, child behavior problems, and maternal health) and the intercept for each of the 4 methods for handling missing data. The superior technique for handling missing data will have model coefficients as close as possible to the model coefficients with the full data (ie, the difference in the model coefficients will be close to zero and the 95% confidence

Discussion

We recommend that investigators use caution when analyzing survey data if a large percentage of values are missing for the variable of interest. When analyzing survey data with 10% or less of the values missing, there may be a slight bias by simply dropping cases with missing data (complete case analysis or listwise deletion) or using the hot deck technique, although these biases may not be severe. We found that when using the NMIHS and LF, if >10% of cases were missing a value for the variable

Acknowledgments

This work was funded by grants from The National Institute of Mental Health (1-R03-MH64060-01A1, Diane L. Langkamp, principal investigator) and the Akron Children's Hospital Foundation (Diane L. Langkamp, principal investigator).

References (28)

A.R.T. Donders et al.
Review: a gentle introduction to imputation of missing values
J Clin Epidemiol
(2006)
C.D. Croy et al.
Methods for addressing missing data in psychiatric and developmental research
J Am Acad Child Adolesc Psychiatry
(2005)
D.A. Bennett
How can I deal with missing data in my study?
Aust N Z J Public Health
(2001)
R.J.A. Little et al.
Statistical Analysis with Missing Data
(2002)
P.D. Allison
Missing Data
(2002)
J.L. Peugh et al.
Missing data in educational research: a review of reporting practices and suggestions for improvement
Rev Educ Res
(2004)
A.M. Wood et al.
Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals
Clin Trials
(2004)
D. Civic et al.
Maternal depressive symptoms and child behavior problems in a nationally representative normal birthweight sample
Matern Child Health J
(2000)
L.W. Deal et al.
Young maternal age and depressive symptoms: results from the 1988 National Maternal and Infant Health Survey
Am J Public Health
(1998)
K.D. Mandl et al.
Infant health care use and maternal depression
Arch Pediatr Adolesc Med
(1999)

S.M. Petterson et al.

Effects of poverty and maternal depression on early child development

Child Dev

(2001)

A. Chen et al.

Breastfeeding and the risk of postneonatal death in the United States

Pediatrics

(2004)

T.E. Raghunathan

What do we do with missing data? Some options for analysis of incomplete data

Annu Rev Public Health

(2004)

R.B. Hakim et al.

Effect of compliance with health supervision guidelines among US infants on emergency department visits

Arch Pediatr Adolesc Med

(2002)

Cited by (126)

Imputation of missing daily rainfall data; A comparison between artificial intelligence and statistical techniques
2023, MethodsX
Handling missing values is a critical component of the data processing in hydrological modeling. The key objective of this research is to assess statistical techniques (STs) and artificial intelligence-based techniques (AITs) for imputing missing daily rainfall values and recommend a methodology applicable to the mountainous terrain of northern Thailand. In this study, 30 years of daily rainfall data was collected from 20 rainfall stations in northern Thailand and randomly 25–35 % of data was deleted from four target stations based on Spearman correlation coefficient between the target and neighboring stations. Imputation models were developed on training and testing datasets and statistically evaluated by mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R²), and correlation coefficient (r). This study used STs, including arithmetic averaging (AA), multiple linear regression (MLR), normal-ratio (NR), nonlinear iterative partial least squares (NIPALS) algorithm, and linear interpolation was used.
- •
  STs results were compared with AITs, including long-short-term-memory recurrent neural network (LSTM-RNN), M5 model tree (M5-MT), multilayer perceptron neural networks (MLPNN), support vector regression with polynomial and radial basis function SVR-poly and SVR-RBF.
- •
  The findings revealed that MLR imputation model achieved an average MAE of 0.98, RMSE of 4.52, and R² was about 79.6 % at all target stations. On the other hand, for the M5-MT model, the average MAE was 0.91, RMSE was about 4.52, and R² was around 79.8 % compared to other STs and AITs. M5-MT was most prominent among AITs. Notably, the MLR technique stood out as a recommended approach due to its ability to deliver good estimation results while offering a transparent mechanism and not necessitating prior knowledge for model creation.
A review on missing values for main challenges and methods
2023, Information Systems
Several recent reviews summarize common missing value analysis methods. However, none of them provide a systematic and in-depth summary of the analytical challenges and solutions for dealing with missing values. For the purpose of guiding the handling of missing values, this review aims to consolidate current developments in novel missing-value research methodologies. In particular, we comprehensively investigated cutting-edge missing value solutions and methodically studied the main challenges associated with missing values analysis (missing mechanisms, missing patterns, and missing rates). Furthermore, we reviewed 63 publications that compare different strategies for deleting and imputing missing values. Then we investigated data characteristics, highlighted three main problems when analyzing missing values, and analyzed the performance of missing value solutions in these studied papers. Moreover, we conducted comprehensive experiments on 9 public datasets using typical missing value processing methods and provided a simple guided decision tree for handling missing values. Finally, we described current Research hotspots and open challenges, which give potential research topics.
Emotional Support Among Parents of Children With Adverse Childhood Experiences
2023, Journal of Pediatric Health Care
We assess the association between a child's exposure to adverse childhood experiences (ACEs) and the parent's current experiences of emotional support.
This study used pooled cross-sectional data from the National Survey of Children's Health (N = 129,988). Emotional support for the parent was classified by the presence (any emotional support, no emotional support) and type (any formal support or only informal support). All models were adjusted for relevant predisposing, enabling, and need factors.
Having two or more ACEs was associated with a higher probability of any emotional support (average marginal effect = 0.017; 95% confidence interval = 0.002–0.032) and a higher probability of formal support (average marginal effect = 0.049; 95% confidence interval = 0.028–0.069). Several individual ACEs were associated with the presence and type of emotional support.
Parents of children with higher ACEs are likelier to have emotional support, especially formal support.
Parent Ratings of Health Insurance Adequacy for Children with Emotional, Behavioral, or Developmental Problems
2023, Academic Pediatrics
Parents of children with special health care needs (CSHCN) report that private insurance is less adequate than public health coverage. Parents of CSHCN with emotional, behavioral, or developmental problems (EBDPs) may perceive private insurance to be especially inadequate due to higher need for a wider array of non-medical services and supports. This study's objective is to assess differences in parent ratings of insurance adequacy for public versus private health coverage between non-CSHCN, CSHCN, and CSHCN with EBDPs.
This study pooled publicly available data from the 2016 through 2019 National Survey of Children's Health. Multivariable fixed effects logistic regression models estimated the association between insurance type, CSCHN and EBDP status, and parent ratings of their child's insurance adequacy. Marginal effects were calculated for insurance type, CSHCN and EBDP status, and their interactions to estimate the size of the association.
Among all subgroups, consistently more parents with publicly insured children rated their insurance as adequate compared to those with private insurance. Parents of privately insured CSHCN with EBDPs rated their insurance as adequate at significantly lower rates than any other group of parents (55%)—including those with privately insured children without EBDPs (non-CSHCN= 67%; CSHCN = 63%) and all other parents with publicly insured children (non-CSHCN = 87%; CSHCN = 83%; CSHCN with EBDPs = 84%).
Future research should investigate if perceptions of insurance adequacy among families whose CSHCN has an EBDP aligns with reports of service access and unmet health care needs.
Cumulative effects of pre-pandemic vulnerabilities and pandemic-related hardship on psychological distress among pregnant individuals
2023, General Hospital Psychiatry
Our primary objective was to determine whether pre-existing vulnerabilities and resilience factors combined with objective hardship resulted in cumulative (i.e., additive) effects on psychological distress in pregnant individuals during the COVID-19 pandemic. A secondary objective was to determine whether any of the effects of pandemic-related hardship were compounded (i.e., multiplicative) by pre-existing vulnerabilities.
Data are from a prospective pregnancy cohort study, the Pregnancy During the COVID-19 Pandemic study (PdP). This cross-sectional report is based upon the initial survey collected at recruitment between April 5, 2020 and April 30, 2021. Logistic regressions were used to evaluate our objectives.
Pandemic-related hardship substantially increased the odds of scoring above the clinical cut-off on measures of anxiety and depression symptoms. Pre-existing vulnerabilities had cumulative (i.e., additive) effects on the odds of scoring above the clinical cut-off on measures of anxiety and depression symptoms. There was no evidence of compounding (i.e., multiplicative) effects. Social support had a protective effect on anxiety and depression symptoms, but government financial aid did not.
Pre-pandemic vulnerability and pandemic-related hardship had cumulative effects on psychological distress during the COVID-19 pandemic. Adequate and equitable responses to pandemics and disasters may require more intensive supports for those with multiple vulnerabilities.
Guiding principles: Effect of a science-based staff training program on knowledge and application of assistance dog training techniques
2023, Journal of Veterinary Behavior
Positive reinforcement training (PRT) is an effective and humane training technique for modifying animals’ behavior. Given the influence of assistance dog charities, the numbers of dogs trained by them and the importance of their role, it is vital that they utilize PRT when handling their dogs. This study aimed to evaluate the effect of a standardized staff training package about PRT on staff knowledge and application of dog training techniques. A sample of 363 Guide Dogs UK staff responded to an online survey. Staff who had received advanced (P = 0.003) or intermediate (P = 0.043) internal training were more likely to demonstrate knowledge of training theory than staff who had not received training. Staff were more confident in both practical (P = 0.0082) and theoretical (P = 0.0012) dog training techniques if they had received any internal training. Positive reinforcement was the most understood and applied principle with 96.4% of staff correctly answering the positive reinforcement knowledge questions and 98.2% of participants selecting positive reinforcement as a method they used to modify dog behavior. Aversive techniques (positive punishment) were less likely to be utilized by staff who had received any internal training (P < 0.001) and those who had answered learning theory questions correctly (P = 0.024). Internal training on the subject of PRT is effective at improving staff knowledge and application of ethical dog-handling practices. It is recommended that assistance dog organizations provide training on PRT to staff and stakeholders to enhance knowledge and ethical dog-handling practices thus improving the welfare of the dogs in their care.

View all citing articles on Scopus

View full text

MethodsTechniques for Handling Missing Data in Secondary Analyses of Large Surveys

Objective

Methods

Results

Conclusions

Section snippets

Data Source

Results

Discussion

Acknowledgments

J Clin Epidemiol

J Am Acad Child Adolesc Psychiatry

Aust N Z J Public Health

Statistical Analysis with Missing Data

Missing Data

Missing data in educational research: a review of reporting practices and suggestions for improvement

Rev Educ Res

Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals

Clin Trials

Maternal depressive symptoms and child behavior problems in a nationally representative normal birthweight sample

Matern Child Health J

Young maternal age and depressive symptoms: results from the 1988 National Maternal and Infant Health Survey

Am J Public Health

Infant health care use and maternal depression

Arch Pediatr Adolesc Med

Effects of poverty and maternal depression on early child development

Child Dev

Breastfeeding and the risk of postneonatal death in the United States

Pediatrics

What do we do with missing data? Some options for analysis of incomplete data

Annu Rev Public Health

Effect of compliance with health supervision guidelines among US infants on emergency department visits

Arch Pediatr Adolesc Med

Methods
Techniques for Handling Missing Data in Secondary Analyses of Large Surveys