Assessment of psychometric properties of the Brazilian version of the oral anticoagulation knowledge test

Warfarin is an oral anticoagulant, coumarinic derivative used worldwide for primary and secondary prophylaxis of thromboembolism. Its efficacy and safety have been widely studied for several indications through well-conducted clinical trials. The indication of oral anticoagulation is increasing due to population aging and the consequent increase in the incidence of thromboembolic diseases [1‐5].

The management of warfarin is quite complex due to its narrow therapeutic range and wide dose–response variability. Among the interfering factors in the anticoagulant effect, we can cite the presence of genetic polymorphisms, drug and dietary interactions of potential clinical significance, as well as changes in the patient’s condition as some of the common ones [1, 6].

For proper dose management, laboratory monitoring is conducted by means of the International Normalized Ratio (INR) calculated from the prothrombin activity. The target INR is considered between 2.00 and 3.00 for most of warfarin indications. Subtherapeutic results were related to the occurrence of venous thromboembolism and higher values, to increased risk of bleeding [1]. To evaluate the quality of anticoagulant therapy, the percentage of time of INR in the therapeutic range is calculated (time within therapeutic range; TTR), the bleeding and mortality rates being significantly higher in patients with TTR values <60 %, in comparison with those presenting TTR values >75 % [7].

Studies have indicated that higher levels of knowledge regarding warfarin therapy are related to an improved stability of INR and increased TTR values [6, 8]. It is therefore essential that patients using warfarin have adequate knowledge about the therapeutic objective (indication and effectiveness), the process of use (dosage, therapeutic regimen, route of administration, and duration of treatment), safety (adverse events, precautions, contraindications, and interactions), as well as the conservation of the drug [9]. In this sense, the evaluation of the patient’s level of knowledge using validated instruments becomes necessary for identifying deficiencies and guide educational activities to create a continuous system of quality improvement of patient monitoring and safety [10].

Among the few validated tools to evaluate patients’ knowledge regarding the use of warfarin, we can highlight the Oral Anticoagulation Knowledge (OAK) Test, which was validated for use in the United States of America [8] and Malaysia [11], and showed good psychometric properties. The OAK Test was cross-culturally adapted into Brazilian Portuguese, presenting a semantic and conceptual equivalence of items in relation to the original version [12]. It is recommended that the psychometric properties of this adapted version be evaluated to ensure its validity and reliability to achieve the proposed objectives [13]. Therefore, the present study aimed at evaluating the psychometric properties of the Brazilian version of the OAK Test.

The final analysis included 201 participants under treatment with warfarin. The characteristics of the patients have been presented in Table 1. The average number of correct answers to the OAK Test was 63 % (±22) (Table 1). The analysis of the instrument’s internal consistency revealed a Kuder-Richardson coefficient of 0.818 for the total score. The exclusion of any instrument items showed small changes in the Kuder-Richardson coefficient, which ranged from 0.802 to 0.821 (Table 2).

The intraclass correlation coefficient of the total score of the OAK Test was 0.967. Table 3 shows the Cohen's Kappa coefficients of the test and retest performed, revealing temporal stability ranging between good and perfect. By using a specific evaluation form, most items of the instrument, 19 (95 %), were considered relevant for the study population. Only item 2 was regarded irrelevant to the study in question, but it was maintained in the instrument.

The validity revealed a strong positive correlation between the values of the level of knowledge measured by the OAK Test and the TTR (r_s = 0.780; p = 0.001). There was a positive and weak correlation between the OAK Test score and education (r_s = 0.191; p = 0.007) and income (r_s = 0.170; p = 0.016). Other analyses were performed and showed a negative but weak and non-significant correlation between the OAK Test score and age (r_s = −0.116; p = 0.100). There was no association between the OAK Test score and the gender of the participants (Mann–Whitney Test, p = 0.182).

This study systematically evaluated the psychometric properties of the Brazilian version of the OAK Test [12]. The results of this evaluation demonstrated adequate performance of the instrument, supporting its validation for use in Brazil.

The reliability of an instrument is related to the ability to measure an attribute accurately, over time, and the consistency and stability of the attribute measured [19]. In this sense, the Brazilian version of the OAK Test retained the reliability and validity presented by the original English instrument. Reliability was confirmed by internal consistency and test-retest measures and the content, face [12], and validities were also confirmed.

In terms of the application of the OAK Test, good internal consistency was observed as the temporal stability ranged between good and perfect. These results demonstrate that the instrument in question is reliable and has adequate temporal stability, corroborating the findings of the original study in which the instrument was developed [8], and another study that obtained similar results [11].

Through the initial hypothesis that there would be a positive correlation, of moderate to strong intensity, between the correctness levels of the OAK Test and the TTR values, the validity of the instrument was proven, since there was a strong positive correlation (r_s = 0.780). This hypothesis was also confirmed by other investigations, demonstrating that the greater the patient’s knowledge about treatment with warfarin, the better was the control of oral anticoagulation, as expressed by higher TTR values [11, 20]. We observed that formal education and income showed a weak association with the OAK Test, which is similar to the results of other studies that have not demonstrated significant associations between knowledge and these variables as well [8, 21].

By applying the OAK Test, it was possible to verify the level of knowledge of the patients treated with warfarin in a public hospital in Brazil. The average correctness of the test answers was 63 %, which is in line with that found in other studies [8, 22]; besides 71 % of patients had average or high level of knowledge. However, a similar study conducted in Malaysia identified an average of only 11.2 % of correctness [11]. Studies that used other tests also revealed differences in the level of knowledge of participants. Both low levels of knowledge [23, 24], and good levels of knowledge were observed [10, 25]. Such differences may be related to sociodemographic and clinical particularities of the patients included, the quality of anticoagulation services, or methodological differences employed in the application of knowledge tests.

In the present study, it was observed that a deficit in knowledge about the drug and dietary interactions with warfarin correlated with a correctness rate of less than 50 %. These data are in line with those of another study carried out in an anticoagulation clinic, which showed meaningful gaps in the knowledge regarding the drug and dietary interactions [22]. These findings signal the importance of implementing educational strategies to improve the understanding of these issues using approaches that can contribute to modifying patients’ behavior regarding self-care and appropriate lifestyle choices.

Most patients had TTR lower than 60 %, which is a worrisome result since the rates of bleeding and mortality are significantly higher in patients with TTR < 60 % when compared to those with TTR > 75 % [7]. Laboratorial monitoring is necessary to guide dose adjustments in order to achieve INR target. Interventions focused on INR control, and consequent TTR elevation, contribute to the increase of effectiveness of drug therapy and the minimization of adverse drug events. In this sense, the use of valid and reliable instruments allow health teams to evaluate the level of knowledge of patients and identify knowledge deficits on important aspects related to the treatment with warfarin substantiating appropriate decision for patient management [1].

Regarding the limitations of this study, it is noteworthy that, as there are no instruments aimed at evaluating patients’ knowledge about therapy with warfarin already validated for the Brazilian population, the criterion validity could not be established, as this would have required using a gold standard instrument. Added to this, the fact that although the sample size was sufficient to demonstrate the performance of the instrument, this relatively small number of individuals can also limit the ability to generalize the results for different groups of Brazilian patients on warfarin.

As another limitation of the study, OAK Test was originally designed to be self-administered by individuals with at least seven years of education. However, due to the low level of education presented by several individuals enrolled in this study, we preferred to administer the instrument by patient interview, which has extended the time of application.

However, the use of a rigorous methodological process to evaluate the psychometric properties of the Brazilian version of the OAK Test tried to maintain the reliability and validity of the original instrument in English. In addition, this instrument is useful, quick, and easy-to-apply in clinical practice, by means of individual interviews with patients with different levels of education. Based on a previous publication [12] we identified good content and face validity of the instrument. The instrument was proved to be effective in assessing what it proposes to measure, and that most of the items analyzed are relevant. This finding is in agreement with that of other studies with similar goals [8, 11]. Item 2 of the instrument was a reason of debate among the authors. Most of the population studied uses only the 5 mg warfarin tablet that belongs to the list of drugs distributed by the National Health System in Brazil, and that is not distinguishable by the color that identifies the drug concentration as it happens in the US. Thus, participants may present difficulties in understanding this question. However, we made a decision to retain Item 2 in the final version to provide a broader scope of the instrument to be used not only by public health services but also by private institutions where warfarin brands that allow patients to distinguish different drug concentrations by color may be preferably prescribed.

The results of the test may assist the stratification of patients and the improvement of planning and individualization of educational practices according to the needs of the patients. The need for further studies to check the performance of the instrument in individuals with characteristics different from the study group is indicated.

The Brazilian version of the OAK Test showed good psychometric properties and was found to be a reliable and valid instrument for evaluating the knowledge of Brazilian patients on oral anticoagulant therapy using warfarin. The results generated with the use of the instrument may be incorporated into the practice of public health care, in order to structure health education activities to ensure improvement in the knowledge of the therapy proposed, which may help increase the effectiveness and safety of the treatment.

The use of the instrument will also aid the analysis of the relationship between the knowledge of the patients and the quality of the anticoagulation treatment using warfarin, as well as the comparison of research findings between different countries. It favors, then, the communication between health professionals and patients, and contributes to the rapid identification of problems and priority needs, as well as to decision-making.

ICC, intraclass correlation coefficients; INR, international normalized ratio; OAK, oral anticoagulation knowledge; PT, prothrombin time; R_s, Spearman correlation; SD, standard deviation; SPSS, Statistical Package for Social Sciences TTR, time within therapeutic range