Development of a validated patient-reported symptom metric for pediatric Eosinophilic Esophagitis: qualitative methods

In pediatric eosinophilic esophagitis (EoE) there can be a wide range of symptoms: from the easily recognizable presentations of food bolus impaction and dysphagia to the less obvious feeding disorders and abdominal pain [1]. Most previous efforts to develop a questionnaire to measure pediatric EoE symptoms have focused on correlations between non-validated symptom scores and histologic outcomes, specifically peak esophageal eosinophil counts [2, 3]. To date, there have been few validated instruments that capture symptoms as patient and parent proxy-reported outcomes (PROs) developed through patient focus interviews and cognitive interviews as recommended by the FDA and specific to pediatric EoE [1, 4].

Pentiuk et al. developed the non-validated Pediatric Eosinophilic Esophagitis Symptom Score (PEESS version 1.0) utilizing expert opinion and focusing on correlations between symptom scores and histologic outcomes. In their study, Pentiuk and colleagues demonstrated that subjects with untreated EoE had higher PEESS scores than treated subjects; however, symptom score and histology were only weakly correlated [5]. Aceves and colleagues have also recently developed a symptom score through the modification of a metric utilized for acid peptic disorders [3]. However, this non-validated tool was again developed through expert opinion alone and focused on correlations to histology. In Aceves's study, the total symptom score was higher among patients with EoE and gastroesophageal reflux disease (GERD) than control patients (P < 0.001). However, only symptoms of dysphagia and anorexia/early satiety were capable of significantly discriminating EoE from GERD (P < 0.01). Patients with chief complaints other than dysphagia and anorexia did not have histology findings that directly correlated with the results of the modified symptoms scale.

These prior efforts to develop a symptom score for EoE depending solely on expert opinion have not fully taken into account patient perceptions of symptom severity or response to treatment, which are increasingly important to improving health outcomes and quality of life [4]. Therefore, there is a significant need for severity indices to be developed as patient self-report and parent proxy-reported outcomes. Well designed and validated PROs have been increasingly recognized as key outcome measures for the treatment of chronic disease over the past decade. For example, the recent pediatric asthma randomized clinical trial for the medication ciclesonide utilized both patient symptoms and quality of life (QOL) metrics as key outcome measures and demonstrated improvements in QOL and symptoms scores in the treated groups relative to the placebo controls [6]. The National Institutes of Health identified PROs as key components in the clinical research "toolbox" and have launched the Patient Reported Outcomes Measurement Information System (PROMIS, http://​www.​nihpromis.​org/​default.​aspx) in an attempt to provide healthcare workers and researchers with instruments to objectively measure the disease characteristics that patients and their families deem critically important for their day-to-day health [7‐14]. It is clear that a pediatric EoE symptom score would provide a valuable new tool with which to better analyze the outcomes in pediatric EoE that are relevant to families.

As a first phase in the development process of a new PRO instrument, content validity must be established. The FDA guidelines on PRO development state that, "item generation should include input from the target patient population to establish the items that reflect the concept of interest and contribute to its evaluation." To capture outcomes from the patients' perspective, in 2009 the FDA recommended that it is important to establish content validity through patient focus groups/interviews and cognitive think aloud and cognitive debriefing protocols before evaluating other measurement properties. Basic PRO validation requires feedback from patients and their parents early in the instrument development process to determine if the instrument's item content captures the disease features that pediatric patients and parents believe are important for the instrument to measure. Content validity is supported using qualitative research methodologies to investigate whether the PRO instrument and its respective items measure the symptoms of interest from the patients' perspective. These qualitative methods are an essential part of the instrument development process from the beginning of the process, and subsequently testing other measurement properties will not replace or rectify problems with eliciting patient and parent-proxy focused outcomes from the initiation phase [7]. In 2008, Flood and colleagues described their use of cognitive interviews for a new pediatric EoE metric (Symptom Questionnaire for Eosinophilic Esophagitis) developed by experts in the EoE field for patients age 8-17 years of age and caregivers of EoE patients ages 2-7 years of age [15]. Given that the metric developed by Flood and colleagues was developed by expert opinion initially with patient input in a single cohort only, it may be limited in reflecting the patient experience.

As the primary objective of this study, we sought to identify key EoE patient self-reported and parent proxy-reported symptoms through focus interviews, to develop an EoE PRO symptom metric, and to review the patient derived metric though cognitive interviews in a separate cohort. Content validation of the Pediatric Eosinophilic Esophagitis Symptoms Severity (PEESS™ v2.0) PRO metric was supported by this iterative process.

All study protocols were reviewed and approved by the Institutional Review board at Cincinnati Children's Hospital Medical Center (CCHMC). All research described was compliant with the CCHMC ethical guidelines for clinical research and the Helsinki Declaration (http://​www.​wma.​net/​e/​policy/​b3.​htm). Informed consent was obtained from all parents and assent for children ages 8-18 years of age.

FDA guidelines on PRO development, the validated generic PedsQL™ guidelines, and the consolidated criteria for reporting qualitative research (COREQ) were used to develop the methodology for content validation [7, 13, 16].

Basic participant demographics of the focus interview and cognitive interview cohorts are provided in Table 3.

Pediatric patient self-reported and parent proxy-reported outcomes are critical components of evaluating the impact of current and planned treatments for pediatric EoE. We report a patient self-report and parent-proxy Pediatric EoE Symptom Score (PEESS™ v2.0) metric with content-validation. The key distinction between the PEESS v1.0 compared to the PEESS™ v2.0 is that the former are physician developed metrics, while the latter was developed from the words and descriptions of patients and families following FDA guidelines published in 2009.

The variety of symptoms reported by pediatric EoE patients and their families in the development of the PEESS™ v2.0 was surprising. The existing literature reports that adult EoE patients often describe varying degrees of dysphagia, whereas children often describe pain without dysphagia as their only symptom [1, 3]. Characterization of dysphagia from multiple perspectives was critical to capturing this important symptom. Patients and parent proxies were often aware that they (or their child) were experiencing dysphagia as assessed by items such as "trouble swallowing" or "food getting stuck while eating." However, the addition of items including "taking a long time to eat food" and "needing to drink a lot of water while eating food" captured the more subtle descriptions of dysphagia. Utilizing two distinct cohorts for the focus interviews and cognitive interviews also yielded invaluable information and allowed further refinement of patient self-reported and parent proxy-reported perspectives, achieving content saturation. In particular, clarification and increased reading ease of directions, addition of a combined pictorial scale and Likert scale, language modification, and changes in instrument layout were all developed from the cognitive interviews.

One particular concern regarding parent-proxy reported outcomes is that they may not be acceptable to the current FDA guidelines regarding product labeling specifications. For FDA-endorsed clinical trials that would use the PEESS™ v2.0, it may be that presently we are only able to obtain specific PRO based labeling for pediatric self-report ages 8-18 years of age. Current opinion at the FDA is that parent observer tools for children 2-7 years of age are recommended to include only symptoms clearly observable by the parent, such as: vomiting, increased time needed to eat, and drinking a large amount of liquids to swallow food. Therefore, only a subscale of questions may able to be used for clinical trials in children 2-7 years of age. However, it is also vital to recognize that FDA opinion for labeling purposes regarding PROs has changed significantly in the past several decades. It is highly possible that the current thinking on parent-proxy metrics (widely supported by many psychometricians and clinicians alike) may also change. In future studies, we will be conducting responsiveness testing of the PEESS™ v2.0 metric, further assessing the parent proxy-report metric and specific subscales. Finally, the PEESS™ v2.0 is intended for use both in and outside of the clinical trial setting. We felt that all of the symptoms important to patients and their parents as proxy are important to capture in order to reflect the true clinical picture of symptomatology.

In developing a metric for pediatric EoE, it is important to ensure that all patient areas of concern are fully elicited - a concept often termed item saturation, and a key reason why we performed our 75 patient and parent proxy interviews in two stages and two separate cohorts. Item saturation was achieved for each question and questionnaire as a whole during the cognitive interviews for each age range. Specific questions are provided in Table 2, including: "how would you change the questions words to make it easier to understand" and the overall assessment question "are there things that we forgot to ask about that you feel are important?" These questions were asked to the patients in the 8-18 year old age ranges and to the parents in the 2-18 year old age ranges. In response, the landscape questionnaire format and front page directions were changed early on, and the final draft was reviewed by patients in this format. This is an important distinction from the Flood et al. metric and the PEESS v1.0 in that items were generated by the patients in one cohort and then reviewed by a second cohort. However, the item content if the PEESS™ v2.0 is supported by the symptom overlap reported by Flood et al. and other investigators [3, 15, 18].

Another controversial area in PRO development is recall period. Some may argue that a one month recall period for the PEESS™ v2.0 is too long to accurately assess change. The use of a one month recall period has been well described by Varni and colleagues [19‐26]. In addition, the Mayo Dysphagia Questionnaire developed by Romero and colleagues was shown not to be responsive over a 14 day recall period, but responsive over a 30 day (1 month) recall period [27]. In future studies we will be conducting responsiveness testing of the PEESS™ v2.0 metric and will be assessing different recall periods - 1 week, 2 week and 1 month. One alternative to improve patient accuracy in symptom reporting by a shorter recall period and also allow for a total responsive period of 30 days would be to collect symptom scores on a weekly basis for 4 weeks, and then report a 30 day summation score. Some have also advocated for the use of daily assessments through electronic diaries. We considered using daily electronic diaries, but we have found the patient response-burden too high, and that these diaries are impractical for any but the most highly funded and closely regulated clinical trials. The PEESS™ v2.0 is intended for use both in and outside the clinical trial setting as a simple, yet effective patient-derived metric that does not cause undue burden on patients, families, clinicians, or researchers.

There are several important limitations to this study. Although the PEESS™ v2.0 metric represents an important step forward for the field of pediatric EoE, further testing and modifications will be needed. Despite item saturation, there may be additional EoE symptoms that have not been included. In addition, generalizability to non-Caucasians and to patients with co-morbidities may be a potential concern. For the PEESS™ v2.0 metric development and cognitive interviews, we felt it critical to assess patients' and families' concerns specific to EoE, not those related to other co-morbidities. In the next phase of the PEESS™ v2.0 metric development, we will test the metric construct and reliability in over 200 patients and 250 parents across a variety of demographics and co-morbidities. The fourth phase of psychometric testing will test the responsiveness of the PEESS™ v2.0 metric, key to assessing the potential performance of the PEESS™ v2.0 in clinical trials. The PEESS™ v2.0 metrics are available at http://​www.​mapi-trust.​org.

Currently, pediatric EoE is assessed using number of eosinophils per high powered field in an esophageal endoscopic biopsy specimen as the primary outcome variable. Symptoms are at best reported as secondary outcomes utilizing physician directed questions, without any attention to PROs [2, 28]. It is currently well-described that the severity of histologic inflammation in EoE as measured by tissue eosinophil counts may not directly relate to the degree of symptom severity [5]. For example, initial data in the pediatric Reslizumab (Cinquil^®) study suggests that patient symptoms and health related quality of life (HRQOL) do not correlate with histologic esophageal inflammation [29]. In a condition without known risk of malignancy or reduced life expectancy, PROs need to be equally if not more important outcome measures compared to histologic disease activity. Face and content validation of the PEESS™ v2.0 are important first steps to establishing patient self-report and parent proxy-report symptom assessments as key factors in pediatric EoE for patients, families, researchers, and care providers alike.