Development and content validation of a patient-reported endometriosis pain daily diary

The initial version of the EPDD (EPDDv1) was developed on the basis of the conceptual framework of the existing Endometriosis Pain and Bleeding Diary (EPBD) [14], available literature on endometriosis, and input from outcomes research scientists and a clinical expert from Europe (Fig. 1). In order for the EPDD to be implemented in a phase 2 clinical trial of endometriosis (TERRA; NCT01767090), cognitive interviews were conducted to test for patient comprehension of the draft items, language, instructions, and response options. Specifically, the cognitive interviews were designed to gain the patient’s perception of the survey items (ie, what the patient believes the question is asking) and comprehension of terms (ie, what specific words and phrases mean to the patient) [16]. Semi-structured, individual, face-to-face interviews were conducted with 14 women from the US with surgically confirmed endometriosis and consisted of patients completing the EPDDv1 and answering questions relating to the intent of the questions and the interpretation of specific terms. Interviewers were research scientists with training and extensive experience conducting qualitative interviews with patients across a number of therapeutic indications including endometriosis and other areas of women’s health. In preparation for these studies, the interviewers reviewed the content and purpose of semi-structured interview guide questions with the authors and conducted practice interviews with each other to become familiar with interview content and flow. Near the end of the interview, the patient’s acceptance of the electronic platform was assessed with a brief series of questions assessing ease of use, ease of response selection, acceptability of the electronic platform and any problems using the electronic platform. The information obtained from these exercises was used to inform the finalization of the EPDDv1. The EPDDv1 was then translated and linguistically validated in 12 languages (including English) to ensure conceptual equivalency of the survey items in the 10 countries that enrolled patients for the global phase 2 TERRA study. Prior to translation, all language was reviewed for any potential modification that would be required to ensure that the conceptual basis of each question would be retained. The translation and linguistic validation followed internationally accepted standards set forth by the International Society for Pharmacoeconomics and Outcomes Research and consisted of seven processes: 1) a dual forward translation was performed by two independent native-speaking translators; 2) a third native-speaking linguist assessed the translations and reconciled any discrepancies between the two translations; 3) a native speaker with proficiency in English performed a back translation to ensure that the forward translation was conceptually equivalent to the original; 4) a medical review was conducted to ensure that the survey items were appropriate for a clinical setting; 5) cognitive testing of the EPDDv1 was conducted on a small group of relevant patients with endometriosis in order to test alternative wording and to check understandability, interpretation, and cultural relevance of the translation; 6) a review of cognitive testing results compared the patients’ interpretation of the translation with the original version to highlight and amend discrepancies and finalize the translation; 7) the final step was proofreading and final review of the translation [17].

Following the launch of the phase 2 TERRA study that employed the EPDDv1, the tool was further evaluated for content validity using updated literature reviews, clinician interviews, and additional, semi-structured patient interviews. This three-step process was designed to evaluate whether items needed to be modified or deleted, or if additional items should be added to the EPDDv1. The following three steps (Fig. 1) were:

The digital audio files from the patient concept elicitation interviews were transcribed into document files and loaded into the ATLAS.ti (version 5.0) software program for coding [18]. Data coding methodology followed a three-step process. First, the coder identified each incidence of a concept expression in the transcript text. Second, the coder highlighted the actual patient quote as the assigned code name and then matched the tagged text to a code stem from the coding framework, allowing it to be grouped with other codes of similar content related to the project’s objectives. If a code stem did not exist in the coding framework, a new one was developed and the coding framework was expanded; any code stems not used were removed when the coding dictionary was finalized. Third, the final coding dictionary was the result of the full set of concept codes identified in the patient transcripts, organized by the overall structure of the coding framework so the results could be related to the project objectives. All descriptive data from the screening form were entered into SPSS (version 11.5) to generate tables of descriptive statistics.

Information obtained from these exercises resulted in the development of a conceptual disease model of endometriosis. This conceptual disease model served to facilitate an item refinement and generation meeting, during which the EPDDv1 was updated to include relevant findings from the literature review and patient and expert interviews. The outcome of the item refinement and generation meeting was the EPDDv2.

The EPDDv2 was tested in additional cognitive interviews with patients. Patients with endometriosis from the US (n = 16) and Japan (n = 15) were identified via clinical record review (US) and patient self-reporting (Japan), and subsequently recruited and screened at three US centers (Spokane, WA; Seattle, WA; New Orleans, LA) and one market research firm in Tokyo, Japan, to participate in the cognitive interviews. Patients completed the EPDDv2 and participated in face-to-face individual semi-structured interviews.

Following minor refinements made during and following the cognitive interviews, a translatability assessment was conducted to identify and rectify any major conceptual issues to ensure appropriate interpretation of the EPDDv3 and address any potential concerns regarding cross-cultural adaptation before its potential use in pivotal trials. Independent translators assessed the translatability of the 11-item EPDDv3 in 17 languages (Bulgarian, Dutch, English, French, German, Hungarian, Italian, Korean, Polish, Portuguese, Romanian, Russian, Simplified Chinese, Traditional Chinese, and Spanish [for Spain, US, and Argentina]) among 19 countries. When countries/languages are selected for the pivotal trials, the full translation and linguistic validation process would be conducted for each language. Overall, results from the translatability assessment demonstrated that the EPDDv3 is a robust tool and has utility among all translated languages. No significant issues were identified. Furthermore, additional regulatory feedback was obtained from the FDA and the EMA and a second item refinement and generation meeting was held during which the current version of the EPDD (EPDDv3) was developed.

The EPDDv1 included 18 items in six categories: pelvic pain (including dysmenorrhea and NMPP), dyspareunia and sexual activity, bleeding, hot flashes, daily activities, and rescue medication/protection. The EPDDv1 was developed to be administered electronically with a 24-h recall period. In the phase 2 TERRA trial, it was implemented with a small hand-held device.

A total of 14 patients from the US participated in the face-to-face, semi-structured cognitive interviews in three waves, with 4–5 patients in each wave. During the first wave, wording modifications were made to two instructions and 7 items of the EPDDv1. During the second wave, wording modifications were made to one item. No modifications were made during the third wave. Demographic data are presented in Table 1. Results from the Technology Ease of Use Questionnaire indicated that the participating patients found the electronic hand-held device displaying the EPDDv1 to be easy to use and well understood (Table 2).

A total of 30 peer-reviewed articles were retrieved from the search; seven articles contained qualitative data. Five types of pain were identified, and bleeding and gastrointestinal disturbance were also highlighted as being related to endometriosis.

Five clinical experts were recruited from Europe (United Kingdom, n = 1; Germany, n = 2) and Japan (n = 2) and completed a semi-structured interview. Interviews with the Japanese experts were conducted in English via typed text to minimize language misinterpretation. All five clinical experts highlighted the key symptom of endometriosis as pain; specifically, pain during menstruation, NMPP, pain during and after sexual intercourse, and pain during defecation. Pain during or after sexual intercourse was more associated with ‘deep’ pain rather than ‘superficial’ pain. A less common symptom not related to pain was bleeding. All clinical experts stated that a primary concern related to endometriosis is infertility. The most common impacts on patients’ daily lives, as perceived by clinical experts, were role functioning, especially attendance at work and school, performance of daily activities, sexual function, and partner relationships; missed work/school was consistently reported as having the greatest impact.

A total of 34 female patients were recruited from six US clinical sites and one Japanese market research firm; demographic data are presented in Table 3.

Concepts that are spontaneously expressed during the open-ended portions of the interview can be considered to be of higher importance or relevance to the patient than those expressed as a result of probing by the interviewer. Among US patients, the most common symptoms associated with endometriosis were ‘pain during menstruation’ and ‘NMPP’, both spontaneously reported by 82.4% (14 of 17) of patients who reported these symptoms (Table 4). Among Japanese patients, the most common symptoms associated with endometriosis were the same, with ‘pain during menstruation’ and ‘NMPP’ spontaneously reported by 100% (17 of 17) and 76.5% (13 of 17) of patients who reported these symptoms, respectively (Table 4).

The most common impacts associated with endometriosis were ‘problems doing physical activities’, spontaneously reported by 70.6% of US patients (n = 17) and 64.7% of Japanese patients (n = 17), and ‘having problems at work’, spontaneously reported by 58.8% of US patients and 41.2% of Japanese patients. Japanese patients also commonly reported ‘dropping other daily activities’ (41.2%) (Table 5).

Table 6 summarizes the findings from all three sources of evidence, the literature review, and the clinician and patient interviews. The findings show that there was consistency from all three sources with regard to types of pain, but less overlap with other symptoms and their impact on patients’ lives.

Analysis of data obtained from the concept evaluation exercises (ie, literature review, expert interviews, and patient interviews) informed a conceptual model of disease (Fig. 2), which served to facilitate the item refinement and generation meeting. The conceptual model of endometriosis includes symptom concepts related to pain and vaginal bleeding during defecation and gastrointestinal disturbances, and expands on impacts relating to sleep disturbances, physical functioning limitations, and social functioning limitations. During the item refinement and generation meeting, the EPDDv2 was developed based on this conceptual model to be fully comprehensive. The EPDDv2 consisted of 43 items comprising revised iterations of the original 18 items of the EPDDv1 and an additional 24 items related to pain/bleeding during defecation, gastrointestinal disturbance, and difficulties with functioning and/or sleep.

Patients were recruited and screened at three US centers (Spokane, WA; Seattle, WA; New Orleans, LA) and through a market research firm in Tokyo, Japan. The cognitive interviews were conducted in four waves in the US and three waves in Japan. Patient demographics are presented in Table 7.

The EPDDv2 items were tested in three waves of cognitive interviews in the US, and wording modifications were made to items after each wave. During Wave 1 (US), wording modifications were made to three items pertaining to pain, sexual intercourse and pain, and bleeding; no items were added or removed. During Wave 2 (US), wording modifications were made to one item, and alternate questions for nine items were proposed and tested in Wave 3. During Wave 3 (US), one item was removed from the survey, and an alternate question was proposed for one other item for further testing.

The EPDDv2 was then translated to Japanese and tested in two waves of interviews in Japan. The Japanese translation was slightly modified after each wave to ensure the accuracy and understandability of the language used. Based on Japanese patient feedback, translation modifications were necessary for the phrases ‘deep pain or penetration’, ‘spotting’, ‘social activities’, sexual ‘interest’, and ‘during the past 24 h’, which are not common terminology in Japan. Four items were evaluated and found to be better understood by patients and subsequently replaced the original items.

Following the two waves of interviews in Japan, and based on feedback from the FDA and EMA regarding the length of the survey, the EPDDv2 was reduced from 43 to 11 items consisting of five core items relating to dysmenorrhea, NMPP, and dyspareunia, and six extended items relating to sexual activity, daily activities, and use of rescue medication, yielding the EPDDv3. This 11-item EPDDv3 was evaluated in a third wave of interviews in Japan and the fourth wave of interviews in the US. During Wave 3 of the Japanese interviews, translation modifications were made to 12 items and wording modifications were made to two items. During Wave 4 of the US interviews, all patients confirmed comprehension of all items. After the final wave of interviews, the EPDDv3 was sent for translation and linguistic validation into 17 languages (including Japanese). The EPDDv3 is displayed in Table 8, and a comparison of the items included in the EPDDv1 and EPDDv2 is displayed in Table 9.