Reliability and validity of cross‑culturally adapted oral health‑related quality‑of‑Life instruments for Brazilian children and adolescents: a systematic review

The COSMIN manual was used to establish the study question and to conduct the search. According to the manual, the question should include the following four key elements: 1) the construct; 2) the population(s); 3) the type of instrument(s); and 4) the measurement properties of interest. Hence, the focus question became:

What is the reliability and validity of transculturally adapted and translated questionnaires used to assess OHRQoL in Brazilian children and adolescents?

For this systematic review, studies were included based on the following criteria: validation studies and cross-cultural adaptation of OHRQoL instruments into Brazilian Portuguese, studies evaluating measurement properties of OHRQoL questionnaires in children/adolescents, and those reporting at least one of these measurement properties: reliability, internal consistency, measurement error, content validity, construct validity, criterion validity, discriminant validity, and/or convergent validity. Excluded from consideration were systematic reviews of OHRQoL measures, studies solely reporting OHRQoL assessment through instruments, the development and validation of new instruments, questionnaires consisting of a single item, and validations conducted specifically for Portuguese from Portugal.

The studies were acquired through electronic searches conducted in the PubMed/MEDLINE, Web of Science, Lilacs, VHL (BIREME), Scielo, and Embase databases. Keywords were utilized and searched within Health Sciences Descriptors (DeCs), Medical Subject Headings (MeSH), and published manuscripts focusing on oral health-related quality of life. The boolean operators AND and OR were employed alongside the following terms: quality of life, oral health quality of life, instrument, scale, questionnaire, measurement, measurement tool, psychometrics, reliability, validity, instrument validation, cross-cultural adaptation, instrument translation, Brazilian version, Brazil, Portuguese, Brazilian Portuguese. A generic search strategy was tailored to suit the specific attributes of each database, aiming to identify relevant studies for this review (Table 1). Articles and abstracts from databases were sought without language or time restrictions. Furthermore, an additional search was conducted for grey literature using Google Scholar. All included study references were reviewed to identify supplementary studies. Searches in these databases were conducted until March/2023.

The Rayyan tool (https://​rayyan.​qcri.​org/​welcome) was used in the selection of studies, duplicates identification, management, and citation of references during the development of this review [13]. The study selection process was performed by three reviewers (DWDdeO, FSL, and YGG) in two phases. In the first phase, reviewers independently identified all relevant studies through electronic search methods based on inclusion criteria applied to titles and abstracts. The full text was pre-selected for studies that appeared to meet the inclusion criteria or for which insufficient data were found in the title and abstract to make a clear decision. In the second phase, the pre-selected studies were read in full by the same researchers to define whether the study met the inclusion criteria. When necessary, the authors of the studies were contacted by email to clarify questions related to the research. Studies excluded at this or subsequent stages were recorded, along with the reasons for rejection. Observational studies that met the eligibility criteria were included in the final analysis and submitted to data synthesis. Articles identified twice or more were considered only once. Disagreements were resolved by consensus among the three reviewers. This procedure was applied at all stages. The reviewers were trained for database use before the study.

The data were qualitatively recorded to allow comparisons of the selected studies. Each researcher qualitatively evaluated the studies. Data were collected on the following items: author, year of publication, country, study design, characteristics of the participants (gender and mean age), original language of the instrument, cross-cultural adaptation process, target population, main reported results, conclusion, name of the questionnaire, acronym, generality or specificity of the instrument, method of conclusion, domains, number of items, score, period of evaluation, time of completion, availability of the questionnaire in Brazilian Portuguese, Cronbach’s alpha, internal consistency, criterion validity, construct validity, reliability, discriminant validity, general ICC value, translation, back-translation, synthesis, committee approach, pre-test and psychometric evaluation.

The psychometric properties of oral health-related quality of life questionnaires identified were then evaluated according to nine criteria: content validity, internal consistency, criterion validity, construct validity, reproducibility, responsiveness, floor, ceiling effects, and interpretability. Each scale received a positive (+), undetermined (?), or negative (-) rating for each of these measures, or a rating of 0 if no information is available. The evaluation results were presented in a table, but not using an overall score, as this gives equal importance to each psychometric property, which is not necessarily appropriate [14].

The cross-cultural adaptation process of the instruments was evaluated according to the five steps [15], namely: (1) translation, (2) back-translation, (3) committee review, (4) pre-test, and (5) re-examination of score weighting. In the first step, at least two qualified translators translated the scale from the original language into the target language. In the second step, two independent translators must translate the translated version back into the original language to ensure that the translation reflects the content of the original. The third step ideally involves a committee review to develop the penultimate version for pre-testing, and the fourth step consists of applying this version among 30–40 individuals from the target population. The final step is to re-examine the weighting of scores considering the cultural context.

The risk of bias was evaluated using the COSMIN Risk of Bias Checklist [16]. This checklist includes three parts with 10 boxes. Boxes 1 and 7 to 10 were not applicable to this systematic review. Measurement properties related to content validity (box 2), internal structure (boxes 3 to 5), and cross-cultural validity (box 6) were assessed. Each included article was assessed using “very good,” “adequate,” “doubtful,” and “inadequate” to grade the above five domains. Two reviewers (DWDdeO and FSL) independently completed this assessment of the included study, with discrepancies solved through consensus.

The certainty of evidence was assessed according to the GRADE methodology using the GRADEpro program, depending on each analyzed outcome (psychometric properties and cross-cultural adaptation). It was classified as high, moderate, low, or very low. The starting point always assumes that the pooled or overall result is of high quality. The certainty of evidence was reduced by one or two levels when risks of bias, inconsistency, imprecision, and/or indirectness were identified.

A total of 6556 articles were identified in the databases, and 1647 duplicates were removed. The manual search did not identify additional studies. In the first phase, 4879 publications were excluded. In the second phase, 11 studies were excluded (Supplement 1). Therefore, 19 articles were included in this review [17‐35] (Fig. 1).

All studies [17‐35] have a cross-sectional design and were carried out in Brazil. The mean age of participants ranged from 2 [20] to 15.42 years [22], however, two studies did not report this information [21, 32]. The number of participants ranged from 20 [32, 35] to 342 [25]. Three studies did not go through the cross-cultural adaptation process [17, 28, 29] (Table 2).

Table 3 presents the health conditions assessed by the instruments. The self-completion method [18, 23, 27‐33, 35], interviews answered by the participants themselves [19, 22, 24‐26], and interviews answered by parents [17, 20, 21, 26] were used to fill out the questionnaires. The domains/dimensions of the questionnaires were not reported in the two studies [22, 26]. The Brazilian version of the cross-culturally validated instrument was available in only six publications [18, 20, 21, 25, 32, 34].

The psychometric evaluation process, internal consistency, criterion validity, construct validity, reliability, general discriminant validity, Cronbach's alpha value, and general ICC value are presented in Table 4. The stages of the cross-cultural adaptation process; translation, back-translation, committee approach, synthesis, and pre-test were absent in three studies [17, 21, 28] (Table 5).

 The results of the risk of bias assessment are presented in Table 6. All studies were rated very good in the structural validity domain.

The certainty of the evidence was downgraded one level by risk of bias, and it was considered moderate for both psychometrics and adaptation outcomes (Table 7).