Parents were accurate proxy reporters of urgent pediatric asthma health services—a retrospective agreement analysis

doi:10.1016/j.jclinepi.2007.02.002

Journal of Clinical Epidemiology

Volume 60, Issue 11, November 2007, Pages 1176-1183

https://doi.org/10.1016/j.jclinepi.2007.02.002 Get rights and content

Abstract

Objective

To assess agreement between parents' proxy reports of children's respiratory-related health service use and administrative data.

Study Design and Setting

A retrospective analysis of statistical agreement between clinical and claims data for reports of physician visits, emergency department (ED) visits, and hospitalizations in 545 asthmatic children recruited from sites in the greater Toronto area was conducted. Health services use data were extracted from the Ontario Health Insurance Plan and Canadian Institute for Health Information databases for each child for the interval coinciding with the proxy report for each health service type.

Results

Agreement between administrative data and respondent reports (n = 545) was substantial for hospitalizations in the past year (κ = 0.80 [0.74, 0.86]), moderate for ED visits in the past year (κ = 0.60 [0.53, 0.67]), and slight for physician visits (κ = 0.13 [0.00, 0.27]) in the past 6 months. Income, parent's education, and child quality-of-life symptom scores did not affect agreement. Agreement for ED visits was significantly higher (P < 0.05) for children who had an asthma attack in the past 6 months (κ = 0.61 [0.54, 0.68]) compared to children who did not (κ = 0.25 [0.00, 0.59]).

Conclusion

Parents of asthmatic children are reliable reporters of their child's respiratory-related urgent health services utilization.

Introduction

Accurate health services utilization data are important for epidemiologic studies [1], health services research [2], and economic evaluations [3], and these data are often collected via self-reported questionnaires or interviews [4], [5]. Given the increased use and importance of these data, it is imperative that the validity of self-reports be assessed. Several studies have investigated the validity of patient reports of health service utilization for ambulatory, homecare, and emergency department (ED) visits and hospitalizations by comparing patient reports to a second source of data such as medical records or administrative databases. Many of these validation studies have focused on older populations [6], [7] and adult populations with chronic diseases such as asthma and cystic fibrosis [5], [8].

Epidemiologic studies of children's use of health services are relevant and important. Researchers have historically examined hospital admission rates as a marker for disease morbidity and to explore changes in morbidity over time. Due to children's cognitive and developmental limitations, studies investigating children's use of health service often rely on parent proxy reports. Few studies have investigated the validity of parent reports for health services utilization by their children [9], [10], [11], [12]. Moreover, no studies have investigated the validity of parent reports for children with a chronic childhood disease. The objective of this study was to assess agreement between parents' reports of children's respiratory-related health services utilization (outpatient physician visits, ED visits, and inpatient admissions) and administrative health insurance claims data for children with asthma.

Section snippets

Participants

Approval was granted by the Hospital for Sick Children and Brampton Memorial Hospital Research Ethics Boards. Written informed consent was provided by parents/guardians and participants 16 years or older. Assent was provided by younger participants.

Data were extracted from a completed study of the effect of drug plans on outcomes in asthmatic children. Participants/parents were recruited from clinics, EDs, and primary and specialist physicians' practices. Residents of Ontario aged 1–18 years

Sample characteristics

The 545 respondents who consented to participate had higher average incomes (CAD$62,866 vs. CAD$54,120, P < 0.0001) and were more educated (received degree/diploma, 62.5% vs. 47.2%, P < 0.0001) compared to nonconsenters. Demographic data for the 545 children were collected from 459 parent proxies (parent group) and 86 adolescents (adolescent group). The majority (91%) of parent respondents were mothers. Also, most (59%) of the 545 children were male, and less than 10 years of age (73%), with an

Discussion

Agreement between administrative data and respondent reports was substantial for inpatient admissions (κ = 0.80), moderate for ED visits (κ = 0.60), and slight for outpatient physician visits (κ = 0.13) according to the Landis and Koch [17] criteria. Although there was a trend toward higher agreement between reports and administrative data for groups with higher income, higher education, and better PAQLQ symptom control scores, the differences in kappa values were not statistically significantly

Acknowledgments

We would like to thank Don DeBoer and Steven Shi of the Institute for Clinical Evaluative Sciences and Dr. Min Su, Dr. Peter Bikangaga, Dr. Milton Gold, and Dr. Teresa To for their contributions to this project. Thanks to Sara Quirk for technical assistance.

This research was funded by an operating grant from the Canadian Institutes of Health Research (Grant #77437).

References (31)

M. Kattan et al.
Cost-effectiveness of a home-based environmental intervention for inner-city children with asthma
J Allergy Clin Immunol
(2005)
R.O. Roberts et al.
Comparison of self-reported and medical health care utilization measures
J Clin Epidemiol
(1996)
W.J. Ungar et al.
Pharmacy Medication Monitoring Program Advisory Board. Health services utilization reporting in respiratory patients
J Clin Epidemiol
(1998)
P.L. Ritter et al.
Self-reports of health care utilization compared to provider reports
J Clin Epidemiol
(2001)
S. Green et al.
The collection of service utilisation data: a research note on validity
Soc Sci Med
(1979)
S.J. Teach et al.
Indoor environmental exposures among children with asthma seen in an urban emergency department
Pediatrics
(2006)
A. Forsberg et al.
Use of healthcare, patient satisfaction, and burden of care in Guillain-Barre syndrome
J Rehabil Med
(2006)
D.B. Wallihan et al.
Accuracy of self-reported health services use and patterns of care among urban older adults
Med Care
(1999)
P. Raina et al.
Agreement between self-reported and routinely collected health-care utilization data among seniors
Health Serv Res
(2002)
D.N. Guerriere et al.
Evaluation of the ambulatory and home care record: agreement between self-reports and administrative data
Int J Technol Assess Health Care
(2006)

D. D'Souza-Vazirani et al.

Validity of maternal report of acute health care use for children younger than 3 years

Arch Pediatr Adolesc Med

(2005)

C.E. Pless et al.

How well they remember: the accuracy of parent reports

Arch Pediatr Adolesc Med

(1995)

G. Canino et al.

Methodological challenges in assessing children's mental health services utilization

Ment Health Serv Res

(2002)

R. Bussing et al.

Agreement between CASA parent reports and provider records of children's ADHD services

J Behav Health Serv Res

(2003)

E.F. Juniper et al.

Measuring quality of life in children with asthma

Qual Life Res

(1996)

Cited by (23)

The economic costs of chronic pain among a cohort of treatment-seeking adolescents in the United States
2014, Journal of Pain
Citation Excerpt :
Parents completed an online version of the CSRI to collect retrospective data on economic costs incurred for their child's pain condition over the preceding 12 months. Parents were chosen as proxy reporters of their adolescents' health care utilization, as parents have been shown to be reliable informants for this purpose.32 Parents reported on the type and quantity of health services used.
The aim of this study was to assess the economic cost of chronic pain among adolescents receiving interdisciplinary pain treatment. Information was gathered from 149 adolescents (ages 10–17) presenting for evaluation and treatment at interdisciplinary pain clinics in the United States. Parents completed a validated measure of family economic attributes, the Client Service Receipt Inventory, to report on health service use and productivity losses due to their child's chronic pain retrospectively over 12 months. Health care costs were calculated by multiplying reported utilization estimates by unit visit costs from the 2010 Medical Expenditure Panel Survey. The estimated mean and median costs per participant were $11,787 and $6,770, respectively. Costs were concentrated in a small group of participants; the top 5% of those patients incurring the highest costs accounted for 30% of total costs, whereas the lower 75% of participants accounted for only 34% of costs. Total costs to society for adolescents with moderate to severe chronic pain were extrapolated to $19.5 billion annually in the United States. The cost of adolescent chronic pain presents a substantial economic burden to families and society. Future research should focus on predictors of increased health services use and costs in adolescents with chronic pain.
This cost of illness study comprehensively estimates the economic costs of chronic pain in a cohort of treatment-seeking adolescents. The primary driver of costs was direct medical costs followed by productivity losses. Because of its economic impact, policy makers should invest resources in the prevention, diagnosis, and treatment of chronic pediatric pain.
Agreement between internet-based self- and proxy-reported health care resource utilization and administrative health care claims
2012, Value in Health
Citation Excerpt :
Bhandari and Wagner [2] reported a weighted average of 60% agreement at 3 months recall, 36% agreement at 6 months recall, and 20% agreement at 12 months recall. Finally, the type of HCRU reported can also affect the level of agreement, with self-report accuracy increasing with more rare or memorable visits, including inpatient hospitalizations or visits to a specialist [16,17]. Numerous studies have evaluated the validity of self-report data relative to medical records and administrative claims data.
Although Internet-based surveys are becoming more common, little is known about agreement between administrative claims data and Internet-based survey self- and proxy-reported health care resource utilization (HCRU) data. This analysis evaluated the level of agreement between self- and proxy-reported HCRU data, as recorded through an Internet-based survey, and administrative claims–based HCRU data.
The Child and Household Influenza-Illness and Employee Function study collected self- and proxy-reported HCRU data monthly between November 2007 and May 2008. Data included the occurrence and number of visits to hospitals, emergency departments, urgent care centers, and outpatient offices for a respondent's and his or her household members' care. Administrative claims data from the MarketScan® Databases were assessed during the same time and evaluated relative to survey-based metrics. Only data for individuals with employer-sponsored health care coverage linkable to claims were included. The Kappa (κ) statistic was used to evaluate visit concordance, and the intraclass correlation coefficient was used to describe frequency consistency.
Agreement for presence of a health care visit and the number of visits were similar for self- and proxy-reported HCRU data. There was moderate to substantial agreement related to health care visit occurrence between survey-based and claims-based HCRU data for inpatient, emergency department, and office visits (κ: 0.47–0.77). There was less agreement on health care visit frequencies, with intraclass correlation coefficient values ranging from 0.14 to 0.71.
This study's agreement values suggest that Internet-based surveys are an effective method to collect self- and proxy-reported HCRU data. These results should increase confidence in the use of the Internet for evaluating disease burden.
Factors associated with disparities in emergency department use among Latino children with asthma
2012, Annals of Allergy, Asthma and Immunology
Significant disparities in emergency department (ED) rates exist for Latinos; however, few studies have investigated the factors that may account for these disparities.
To examine individual/family and health system factors among Latino and non-Latino white (NLW) children with asthma to explain disparities in ED rates.
The study was carried out in Puerto Rico (PR) and Rhode Island (RI) with the same design: a cross-sectional, observational approach with repeated measurements of selected variables.
The sample was composed of 804 children ages 7 to 15 years, with 405 from PR and 399 from RI. Latino children from both sites had significantly higher rates of ED use as compared with NLWs from RI. Regression analyses showed site, asthma control, parental reported severity of asthma and public insurance to be significantly associated with ED use.
Latino ethnicity and public insurance were among the most important factors related to frequent ED use. Revisions of the policies driving public insurance to assure better access to specialists, preventive education, and evidence-based treatment are needed. The results also suggest the need for the development of interventions in the ED that are geared toward educating families on how best to use emergency services.
Evaluation of quality of life among the caregivers of asthmatic children: The new IFABI-R questionnaire
2011, Allergologia et Immunopathologia
The health-related quality of life (HRQoL) of asthmatic children and their caregivers is correlated to management of the disease and the presence of certain morbidity indicators. The integral assessment of paediatric asthma must include the evaluation of HRQoL among the caregivers, although existing questionnaires only partially assess the dimensions of this aspect. The present study describes a new questionnaire for evaluating HRQoL among the caregivers, comprising three dimensions (functional, emotional, and socio-occupational).
The study involves two phases. A total of 81 patients between 3 and 9 years of age and their caregivers participated in the first phase, involving a qualitative and psychometric study of the preliminary version of the questionnaire (IFABI). A total of 137 patients between 2 and 17 years of age and their caregivers participated in the second phase, in which the revised version of the questionnaire (IFABI-R) was developed and subjected to psychometric evaluation.
First phase: The IFABI showed important reliability and internal consistency (Cronbach alpha=0.93), concurrent validity requiring improvement, and a scantly clear internal structure. Second phase: The IFABI-R showed important reliability and internal consistency (Cronbach alpha=0.90), adequate concurrent validity, and a three-dimensional structure whose three factors correspond to the three dimensions of the questionnaire.
The good psychometric results obtained with the IFABI-R justify its use in paediatric asthmatic patients. The questionnaire is currently being scaled, and its sensitivity to change is being assessed.
Relationship of asthma management, socioeconomic status, and medication insurance characteristics to exacerbation frequency in children with asthma
2011, Annals of Allergy, Asthma and Immunology
Less than 25% of asthmatic children are well controlled.
To identify factors associated with asthma exacerbation causing emergency department (ED) visits or hospitalizations related to health status, socioeconomic status (SES), and drug insurance.
In this retrospective cohort study, complete data were collected on 490 asthmatic children regarding demographics, SES, drug plan characteristics, health status, health resource use, and symptoms. Interview data were linked to administrative data on asthma ED visits and hospitalizations occurring in the following year. Multiple Poisson regression identified independent variables associated with ED visits or hospitalizations in the full cohort and in a subgroup with prescription drug insurance.
Younger age, previous emergency visits, nebulizer use, pet ownership, and receipt of asthma education but not an action plan were significantly associated with more frequent exacerbations. In the full cohort, children with high income adequacy had 28% fewer exacerbations than did children with low income adequacy. In the subgroup with drug insurance, girls had 26% fewer exacerbations than did boys, and children with food, drug, or insect allergies had 52% more exacerbations than did children without allergies. Children of families with annual insurance deductibles greater than $90 had 95% fewer exacerbations. Every percentage increase in the proportion of income spent out-of-pocket on asthma medications was associated with a 14% increase in exacerbations.
Asthma history, disease management factors, and SES were associated with exacerbations requiring urgent care. In families with drug plans, the magnitude of asthma medication cost-sharing as a proportion of household income, rather than income alone, was significantly associated with exacerbations.
Can an evidence-based guideline reminder card improve asthma management in the emergency department?
2010, Respiratory Medicine
Asthma is the most common chronic disease in children. Previous studies described significant variations in acute asthma management in children. This study was conducted to examine whether asthma management in the pediatric emergency department (ED) was improved through the use of an evidence-based acute asthma care guideline reminder card.
The Pediatric Acute Asthma Management Guideline (PAMG) was introduced to the ED of a pediatric tertiary care hospital in Ontario, Canada. Medical charts of 278 retrospective ED visits (January–December 2002) and 154 prospective visits (July 2003–June 2004) were reviewed to assess changes in acute asthma management such as medication treatment, asthma education, and discharge planning. Logistic and linear regressions were used to determine the effect of PAMG on asthma management in the ED. The propensity score method was used to adjust for confounding.
During the implementation of PAMG, patients who visited the ED were more likely to receive oral corticosteroids (Adjusted Odds Ratio [AOR] = 2.26, 95% CI: 1.63–3.14, p < 0.0001) and oxygen saturation reassessment before ED discharge (AOR = 2.02, 95% CI: 1.45–2.82, p < 0.0001). They also received 0.23 (95% CI: 0.03–0.44, p = 0.0283) more doses of bronchodilator in the first hour of ED stay. Improvements in asthma education and discharge planning were noted, but the changes were not statistically significant.
After the implementation of an evidence-based guideline reminder card, medication treatment for acute asthma in the ED was significantly improved; however, asthma education and discharge planning remained unchanged. Future efforts on promoting guideline-based practice in the ED should focus on these components.

View all citing articles on Scopus

View full text

Original ArticleParents were accurate proxy reporters of urgent pediatric asthma health services—a retrospective agreement analysis

Abstract

Objective

Study Design and Setting

Results

Conclusion

Introduction

Section snippets

Participants

Sample characteristics

Discussion

Acknowledgments

J Allergy Clin Immunol

J Clin Epidemiol

J Clin Epidemiol

J Clin Epidemiol

Soc Sci Med

Indoor environmental exposures among children with asthma seen in an urban emergency department

Pediatrics

Use of healthcare, patient satisfaction, and burden of care in Guillain-Barre syndrome

J Rehabil Med

Accuracy of self-reported health services use and patterns of care among urban older adults

Med Care

Agreement between self-reported and routinely collected health-care utilization data among seniors

Health Serv Res

Evaluation of the ambulatory and home care record: agreement between self-reports and administrative data

Int J Technol Assess Health Care

Validity of maternal report of acute health care use for children younger than 3 years

Arch Pediatr Adolesc Med

How well they remember: the accuracy of parent reports

Arch Pediatr Adolesc Med

Methodological challenges in assessing children's mental health services utilization

Ment Health Serv Res

Agreement between CASA parent reports and provider records of children's ADHD services

J Behav Health Serv Res

Measuring quality of life in children with asthma

Qual Life Res

Original Article
Parents were accurate proxy reporters of urgent pediatric asthma health services—a retrospective agreement analysis