Development and initial validation of a traditional Chinese medicine symptom-specific outcome measure: a Zheng-related atopic dermatitis symptom questionnaire (ZRADSQ)

We began our work by searching the literature, both ancient and modern, for descriptions of AD Zheng- related symptoms that could inform our work. We searched two databases for the modern literature – the Chinese Bio-medicine Database (CBMdisc, 2003–2008) and the China Scientific Journals Full-text Database (CSJD, 2003–2008) – using “dermatitis/atopic” as the subject terms with all of the subheadings. The keywords used for the ancient literature search included TaiLianChuang (胎敛疮), NaiXuan (奶癣), RuXuan (乳癣), XueFengChuang (血风疮), SiWanFeng (四弯风), JinYinChuang (浸淫疮), TaiXuan (胎癣), LianMeiChuang (恋眉疮) and LianMeiChuang (炼眉疮), which were obtained from twelve textbooks or books, such as Dermatovenerology of Integrated Traditional and Western Medicine and External Medicine of Traditional Chinese Medicine. Publications up to 1919 were searched in the Encyclopaedia of Traditional Chinese Medicine (CD-ROM Updated Version, 2006).

The literatures on Zheng-related symptoms of AD that formed the basis for the item generation contained 194 articles from modern studies, textbooks, and works, and 294 paragraphs from classical literatures, including works such as “Discussion of Various Causes of Disease Periods of Five Days (Sui dynasty)”, “Golden Mirror of Medicine (Qing dynasty)”, and “Prescriptions for Universal Relief (Ming dynasty)” (see Figures 2 and 3). Items from these articles or paragraphs largely overlapped, and so we grouped them into similar themes. Only items with a frequency of more than 2% were selected.

Experts in dermatology and clinical epidemiology took part in the process of item reduction and presenting. Total of 18 items were presented, including 15 rating items, and 3 additional items that were patient-generated indexes of primary, secondary, and new occurrences of bothersome symptoms. Forty patients with AD from the Out-patient department, 2^nd Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, responded for the item reduction and initial pretesting phases. Table 1 shows the demographic characteristics of these participants. All of the participants met the Raika and Hanifin AD diagnostic criteria [10] and were aged between 7 and 25 years old. The participants used a four-point Likert-type scale to rate their AD symptoms for the 15 selected rating items, where 0 = none, 1 = mild, 2 = moderate, and 3 = severe. They were also asked to report the primary bothersome symptom in the past week, and any secondary bothersome symptoms. At the next visit, the participants were asked to report whether they had experienced any newly occurring bothersome symptoms.

We conducted descriptive analysis and tested the item discrimination coefficients for all of the rating items by dividing the participants into two groups – one group with larger item scores of more than 27%, and another group with lower item scores of less than 27% [11‐13] – and used an independent samples t-test to compare the two groups. If the result was p< 0.05, then the item was deemed to be good at discrimination; if not, then the item was deemed to be less discriminative. The item-total correlations for the rating items were also calculated (the additional items were not included). Items with a low discrimination coefficient or with a low item-total correlation were eliminated.

Base on the results of the item discrimination coefficients and the item-total correlations, one item was deleted. All 17 items (14 rating items and 3 additional items) emerging from the item reduction were answered on a four-point Likert-type scale, except for the three additional items about bothersome symptoms. The participants were asked about their symptoms in the past week, and showed ease of understanding of the items. There were no ceiling or floor effects for the 14 rating items. The instrument required less than 9 minutes (an average of 8.25 minutes) to complete.

We recruited another sample of patients with AD involved in a six-center randomized-controlled trial (ChiCTR-TRC-08000156). All of the participants met the Hanifin and Rajika AD diagnostic criteria [10], had moderate to severe AD that had lasted more than one year, and were aged between 5 and 25 years old. In this trial, there were 275 eligibility, 250 patients were included, while 30 patients did not receive intervention and 14 patients were dropped out. Among the 206 subjects who completed the whole period of intervention, 188 patients were per protocol. They completed the 17-item initial version of the Zheng-Related Atopic Dermatitis Symptom Questionnaire (ZRADSQ) and SCORAD [14]. In the nine-month duration of the trial, the participants were required to complete the ZRADSQ and SCORAD at the same time on baseline, week 4, week 8, and week 12. The ZRADSQ was self-reported by the patients or reported by their parents if they were less than seven years old.

The face validity was judged by the investigators [15]. For the rating items, confirmatory factor analysis (CFA) was used to test a model with three domains as initially designed and to eliminate items or revise the structure of the questionnaire based on a root mean square error of approximation (RMSEA) of less than or equal to 0.08 and a comparative fit index (CFI) of equal to or greater than 0.95 [16]. Using the revised questionnaire, we evaluated the convergent construct validity by calculating the Pearson’s correlation coefficients between the ZRADSQ and SCORAD. We calculated the Cronbach’s alpha values to investigate the internal consistency of the ZRADSQ domains [15], using a split-half coefficient (Unequal-length-Spearman-Brown coefficient) to evaluate the split half consistency. We considered correlations of more than 0.35 to 0.50 to be moderate, more than 0.50 to be strong, less than 0.35 to be weak, and less than 0.20 to be very weak. We used the Pearson’s correlation coefficient to take repeated measurements in a sample of 30 pretest patients (part of the sample of 40 individuals) who completed the questionnaire twice within 48 hours to evaluate the test-retest reliability of the ZRADSQ. And we used a paired t-test to calculate the discriminant validity. For the additional items, we used the standardized effect size (SES) and standardized response mean (SRM) to calculate the responsiveness characteristic of two of the additional items and the total score of the rating items. The SES and SRM were the two indexes used to judge responsiveness, with a value close to or larger than 1 indicating moderate to good responsiveness. We divided the participants into two subgroups due to their age, i.e. one subgroup with subjects equal or less than 7 years old, another larger than 7 years old. We also calculated the standards association validity coefficient, Cronbach’s alpha coefficient, split-half coefficient, paired t-test, SES, SRM respectively in these two subgroups. We used Lisrel 8.8 (Scientific Software International, Inc, Chicago) for the CFA and SPSS for Windows 17.0 for the other statistical analyses (SPSS, Inc, Chicago).

The validation study design was approved by the Ethics Committee of the 2^nd Affiliated Hospital of Guangzhou University of Chinese Medicine (i.e. Guangdong Provincial Hospital of Chinese Medicine), Guangzhou, Guangdong Province, China (Approved No.2008GL-09). The recruited participants signed an informed consent form. If the patient was younger than 18 years old, then the form was signed by the patient’s parents.

Forty participants, from the out-patient department, the 2^nd Affiliated Hospital of Guangzhou University of Chinese Medicine, completed the pretest questionnaire of 18 items. Their descriptive characteristics are shown in Table 1. The number of items was reduced to 17 through statistical analysis in the item reduction phase (see Table 2). In the discrimination coefficient analysis, the p value was less than 0.05 for all of the items.

For the 188 per protocol subjects of the trial, forty-eight percent of the participants were male. The mean age of the recruited participants was 13.57 years. Table 3 shows the general information on the participants. Among the 17 items (14 rating items and 3 additional items), the 14 rating items were introduced into the initial CFA model. Two items, Q3 (pain) and Q8 (fatigue), were eliminated due to a relatively decreasing amount of RMSEA if they were removed from the model. The final model with twelve items and three domains had 51 degrees of freedom and a chi-square value of 97.11. The RMSEA of this model was 0.07, which is less than the cutoff of 0.08, and the CFI was 0.96. All of these results indicate that the model is valid (see Figure 4). The final version of instrument, ZRADSQ, was with 12 rating items and 3 additional items.

Tables 4, 5 and 6 shows that the Pearson’s correlations between the ZRADSQ and SCORAD in total and different age groups were ranged from 0.39-0.47 (P < 0.02).

The Pearson correlation coefficient between the first and repeat measurements was 0.98 (see Table 7). The total Cronbach’s alpha coefficient was 0.84, with a range of 0.74-0.89 for the individual domains (see Table 8), and similar results were obtained in the two subgroups, i.e. one with participants equal or less than seven years old, another with those larger than seven years old (see Table 9 and Table 10 respectively). The results for the split-half coefficient of total questionnaire and the two age groups were shown in Tables 11, 12 and 13.

For both primary and secondary bothersome symptoms, the ZRADSQ and SCORAD showed significant differences between the baseline and Week 12 scores, with SES values ranging from 0.94 to 1.38 and SRM values ranging from 1.00 to 1.35. However, the SES and SRM values were smaller in those who were equal or less than 7 years old. (see Tables 14, 15 and 16).

For the primary and secondary bothersome symptoms, patients were more likely to report itching (34.73%), dry skin (21.41%), itching aggravated at night (20.10%), itching accompanied by pain (9.40%), and fidgeting (3.92%) than the other items. Only a few patients present other symptoms, such as red skin, skin are not beautiful to look at, skin erosion and skin exudation.

The scoring system of this instrument contains two parts. One is the independent VAS for the primary, secondary, and new occurrences of bothersome symptoms for the 3 additional items, range from 0.0 to 10.0, the higher the worse; the other one is a four-point Likert-type scale for the 12 rating items, values with 0, 1, 2, 3 representing None, Mild, Moderate, Severe symptoms respectively, and the sum of them is total score, higher score indicates worse status.