Reliability and validity of a questionnaire for self-assessment of complete dentures

In recent years, the National Epidemiological Survey data have revealed a decline in the proportion of edentulous people. However, given the simultaneous decline in the percentage of edentulous people and increase in the number of older people, the number of people requiring complete dentures in the United States is predicted to increase over the next 20 years [1]. Thus, the need for complete denture treatment is likely to remain substantial [2].

Patient-reported outcome measures, as recommended by the United States Food and Drug Administration, is a growing area within research [3], and the impact of treatment on patients has been attracting attention. This is also the case for prosthetic treatment: the success of complete denture treatment does not depend on the assessment by the dentist, but on patient self-assessment results. The patient self-assessment is an inexpensive and economical method with which to measure the success of the denture and can yield a large amount of data in a short period of time.

Patient satisfaction is rated using a questionnaire that is assessed via a Likert scale or the 100-mm visual analogue scale (VAS). Some patient satisfaction evaluation methods depend on only one question [4, 5], and it is likely that such evaluations may not be reliable. For example, in dissatisfied patients, it is not known whether the dissatisfaction was caused by the lower or upper denture. In addition, patient satisfaction is usually not based on one factor, but on many factors, such as taste, ease of chewing, comfort, retention, pain, fit and aesthetics. Taking those factors into consideration, other evaluation methods have been introduced that comprise several questions to permit the multidimensional evaluation of patient satisfaction [6‐9]. Limitations of these questionnaires are the lack of reliabilities and validities. On the other hand, in prosthetic treatment, one of the most widely used evaluation tools for measurements of oral health related quality of life (OHRQoL) is the Oral Health Impact Profile (OHIP). This instrument has been translated into many languages across the globe [10‐16], which has allowed for comparisons of reliability and validity between OHIP in different regions. The OHIP assesses the impact of oral conditions on the quality of life using a frequency estimation of the disruption, such as dysfunction, discomfort and disability, in daily activities. Thus, negative-oriented measures composing the OHIP may be unable to capture positive changes in patients with no negative impacts recorded at baseline because the absence of a negative does not necessarily imply a positive [17].

Consequently, it is necessary to establish a multidimensional patient self-assessment specific to dentures that can capture both positive and negative denture-related effects and have reliability and validity. We first developed a questionnaire for the self-assessment of complete dentures [18] referred to as the Patient’s Denture Assessment (PDA). The PDA was developed from an original questionnaire that is a subjective assessment about oral, physical, and psychological conditions on daily functioning. In the PDA, some question items contain quality of life similar to the OHIP, but the PDA is absolutely assessment of dentures in themselves. Moreover, the PDA is the bipolar design, which could capture both negative and positive impacts on the patients. The conceptual of framework established for the PDA is in the following. The purposes of complete denture treatment are recoveries of three major functions; chewing, speech, and aesthetics. This three functions influence acceptances and expectations for dentures, which have impacts on patients’ perceptions, consciousness and feelings of dentures. The PDA assesses these impacts of complete denture treatment on the patients’ perceptions, consciousness and feelings of dentures. From the clinical point of view, the PDA will be used for the diagnosis, prognosis, and comparing the efficacy of prosthetic treatment. For example, if the association between the PDA scores before treatment and outcomes after treatment, such as numbers of denture adjustment, length of denture survival and denture survival rate is assessed, the difficulties of treatments and longevities of dentures as prognosis will be predictable from the PDA scores at pre-treatment diagnosis. However, at this point, reliability and validity of the PDA has not been confirmed yet. Thus, the purpose of this study is to demonstrate that the PDA was a reliable and valid instrument for application in clinical settings.

Table 4 provides the mean and SD of the Before-1 and After PDA scores. It’s notable that the some question items in the ‘importance’ subscale exhibited a ‘ceiling effect’. In addition, the ‘lower denture’ and ‘expectation’ subscales had lower means for the Before-1 PDA scores than that seen with the other subscales.

In the present study, both the reliability assessed by internal consistency and test-retest reliability and the validity assessed by discriminant validity of the PDA were satisfactory. Therefore, the PDA is suitable for use in clinical settings.

The internal consistency of the PDA was assessed using two measures: average inter-item correlation and Cronbach’s α. Cronbach’s α is computed from the SDs of the question items and the SD of the total scores. As the difference between the SDs for question items of a subscale becomes larger, the Cronbach’s α of the subscale decreases [23]. In the PDA, each subscale comprised three or four question items, which often resulted in a smaller value of Cronbach’s α in each subscale. Nevertheless, five out of the six subscales presented with a value of 0.8-0.9. Only the ‘importance’ subscale presented with a low Cronbach’s α of 0.56.

It is important to note that, in this study, in describing the importance of dentures, Q20 exhibited a ‘ceiling effect’. This was because all patients had used complete dentures previously and required new dentures, thus complete dentures were important for them. For this reason, the SDs of Q20 were small for most of the patients as compared with that in the other three question items for this subscale. Moreover, the difference between the Before-1 PDA and After PDA scores mean values of Q21 or Q22 was large with wide variance. On the other hand, the difference of Q19 or Q20 was small with narrow variance. The question items of Q21 and Q22 were added after second factor analysis. Thus, the directions of the construct of Q19 or Q20 might be different from Q21and Q22. These led to a small Cronbach’s α, the average inter-item correlation and the impact in the ‘importance’ subscale. By comparison, the highest average inter-item correlation coefficient was in the ‘expectation’ subscale. It is likely that Q13 and Q15 (Table 3) deliver similar content, which may account for the highest score of the average inter-item correlation coefficients in the ‘expectation’ subscale.

The ICCs of the test-retest reliability for the ‘expectation’ and ‘importance’ subscales were low as compared with other studies [10, 11]. The reason for this discrepancy may be that the interval between the test-retest in our study was 2 months while other studies used a 1- or 2-week-interval. The ‘expectation’ subscale consisted of question items related to expectancy of improvement with new dentures and the ‘importance’ subscale consisted of question items related to the importance of dentures to patients, depending on the patients’ perception. The 2-month interval was sufficient time to enhance the relationship and trust between the patients and their operators, which may have influenced their expectation perceptions to new dentures and the importance of wearing dentures.

For the assessment of validity, there was a significant difference in the summary score and the six subscale scores before and after replacement of new dentures. Thus, it is suggested that the PDA can detect differences in patients’ self-assessment between previous and new dentures.

The ‘lower denture’ and ‘expectation’ subscales had lower means for the Before-1 PDA scores than that seen with the other subscales. This suggests that most of the participants selected for the study wore insufficient lower dentures and they had low expectations for the efficacy of their new dentures. Table 1 shows larger number of patients whose period of lower edentulousness was less than one year than upper edentulousness. Moreover, the previous study has shown that patients have a lower expectation for their new dentures when their rating for their previous dentures is low [24]. Therefore, it is suggested that shorter period of edentulousness and insufficient dentures might cause problems with lower dentures and following low expectations in the study.

In the present study, the effect size was 0.97. According to the already-described statement, it is suggested that 0.97 of the ability to detect change in the study represents high ability to detect change. The PDA after treatment was requested after the dentures had been completely adjusted, with all participants given adequate time for these adjustments. Thus, the negative effects of the previous dentures were no longer present and positive effects of the new complete dentures might explain the large effect size reported in the study.

This study demonstrated good reliability and validity of the PDA. Thus, this questionnaire could help dentists obtain a detailed understanding of the patients’ perceptions in using their dentures, which is considered valuable information for dentists when designing new dentures. In our previous study, we demonstrated that sufficient retention of lower dentures and appropriate appearance would lead to improved OHRQoL in edentulous patients who had previously been wearing insufficient lower dentures and/or dentures with an unsatisfactory appearance [18]. The PDA can be thus applied in the clinical situation. However, dentists should carefully interpret the results specific to the ‘importance’ subscale because of the ‘ceiling effect’ and the possibility of the difference direction between Q19 or Q20 and Q21 or Q22 seen in this study.

All participants in the study were recruited in a university hospital setting. Therefore, the reliability and validity of our test must be extrapolated to the general population besides patients in university hospitals. In addition, the questionnaire is presently written in Japanese only. It is recommended that the questionnaire is translated into other languages and evaluated for its reliability and validity with a view for its wider implementation in future studies.

When used among edentulous patients requiring new complete dentures, the PDA demonstrated good reliability by assessing internal consistency and test-retest reliability. In addition, the PDA demonstrated good validity by assessing discriminant validity.