The design of a valid and reliable questionnaire to measure osteoporosis knowledge in women: the Osteoporosis Knowledge Assessment Tool (OKAT)

Osteoporosis is a major and growing public health problem in both sexes but particularly in women [1, 2]. It is responsible for approximately 75,000 fracture events each year in Australia in those over 60 years of age at a cost to the community of $774 M [3]. It is estimated that the proportion of women with osteoporosis increases from 15% in those aged 60 to 64 years up to 71% in those over 80 years of age. In the United States, 10 million people already have osteoporosis and 18 million more have low bone mass [4]. Physical activity and adequate calcium intake are both important for the prevention of osteoporosis [4]. There is evidence suggesting that osteoporosis knowledge is one contributor to osteoporosis preventive behaviour, though this is not a clear-cut relationship. Cross-sectional studies have varied in whether they have found an association between levels of osteoporosis knowledge and osteoporosis preventive behaviours [5‐9]. Prospective studies similarly have been conflicting with some studies demonstrating increases in osteoporosis knowledge and concurrent improvements in osteoporosis preventive behaviours [10‐12] but others demonstrating changes in knowledge but not behaviour [13, 14]. The limited research examining the ways in which osteoporosis knowledge might affect behaviour have demonstrated a relationship. In one study, causal analysis was used to demonstrate that osteoporosis knowledge was an important contributor to improving exercise and calcium intake behaviour [15]. Another study utilizing the precaution adoption process model found that women who were in the never-engaged stage of change (ie unaware of the health problem and of the precaution recommended to reduce the risk of experiencing the problem) had the lowest levels of osteoporosis knowledge and that knowledge levels tended to be higher in women at higher stages of change [16].

Few studies have reported levels of osteoporosis knowledge in random, population-based samples [13, 17‐19]. None of these have utilized validated instruments to measure osteoporosis knowledge and information available about the psychometric properties of the tools used was limited. None have specifically examined knowledge in women in the 25–44 year age range. Therefore, standardized osteoporosis knowledge levels in the general population and specifically in the 25–44 year age range remain unclear.

Measurement of osteoporosis knowledge is itself problematic and this may be part of the reason for the variation in the results of the studies described previously. There are three partially or fully validated instruments in the literature to measure osteoporosis knowledge [20‐22]. We required an instrument in which all items were relevant to the Australian population. For example, two items in the Facts on Osteoporosis Quiz [20] referred to comparisons to African-American women, which were not relevant to populations outside the USA. The planned use of the instrument was to study women who had reached peak bone mass, but were premenopausal, so the instrument needed to have been fully validated in the 25–44 year age range. As the instrument was to be administered as part of a large study, it had to be suitable for self-administration, and easy to code the results. None of the existing instruments satisfied all these criteria.

The aim of this study was therefore to describe the development of and psychometric properties of an instrument to measure osteoporosis knowledge for use in a population-based random sample of 25–44 year old women.

The study was carried out in Southern Tasmania, Australia as part of an ongoing study examining the effects of lifestyle factors on bone mineral density in women aged 25–44. The population of the region in 2000 was 194,389 of which, 28,839 were women aged between 25 and 44 years of age [23]. Subjects were randomly selected in this age range from the 2000 electoral roll. Subjects were excluded if they had previously had measurement of bone density, had thyroid disease, renal failure, malignancy or rheumatoid arthritis, had a history of hysterectomy, were on hormone replacement therapy, were pregnant or planning pregnancy within 2 years of study entry, or were lactating. Ethics approval was obtained from the Royal Hobart Hospital Ethics Committee. All subjects gave written informed consent.

Osteoporosis knowledge was measured in subjects at baseline from April to November 2000. The survey instrument was based on knowledge content common to the Osteoporosis Prevention and Self-management course (OPSMC) and an information leaflet produced by Osteoporosis Australia "Understanding Osteoporosis". The OPSMC is a chronic disease self-management course developed by the Arthritis Foundation of Victoria and utilized by Osteoporosis Australia. Osteoporosis Australia is the peak body involved in community-based osteoporosis education in Australia. The OPSMC is a small group education program that aims to increase knowledge, improve confidence and awareness and self-management of osteoporosis prevention with an emphasis on promoting appropriate lifestyle change. Educational methods include lectures, discussion, brainstorming, demonstration and small group work. The information leaflet covers information on osteoporosis including its definition, natural history of bone strength, risk factors and preventive behaviours, including physical activity and calcium intake.

The items were selected from the common material in the two interventions by a consultant rheumatologist and researcher who has specialist expertise in osteoporosis. The knowledge instrument aimed to measure a broad range of osteoporosis knowledge items that would be applicable to the Australian setting and to avoid items that would be difficult to adapt to other settings in which osteoporosis demographics might be different. It is a 20 item questionnaire, with each item having true, false and don't know options (see Additional file: 1). The analysis was performed by scoring 1 for a correct response and 0 for an incorrect or don't know response. The total score could range from 0 to 20. Face validity was also assessed using a panel of 20 people from our institution. These included research nurses, research assistants and administrative staff.

Other factors measured in the protocol were height by stadiometer (Leicester height measure, Invicta Plastics Ltd, Oadby, England) and weight by a single set of calibrated scales (Heine, Dover NH USA). Body mass index was calculated (weight/ht²). Questionnaire assessment was also made of smoking history (current/former/never), number of children, family history of osteoporosis and/or fracture, as well as fracture history in the subject, education level (4 point scale: less than grade 10, up to grade 10, completed grade12, tertiary), employment status of main financial provider in the household (employed or unemployed), hours of employment of the respondent (0, less than or equal to 20 or >20 hours per week) and marital status (6 categories).

A total of 467 women were recruited (response rate 53%). The characteristics of subjects are given in table 1. Due to small numbers in the lowest educational level and in some categories of marital status, these data are presented as three categories of education and two of marital status (married or defacto and other). The average age of participants was 37.8 years, and there was a wide spread of participants across educational levels and levels of employment. The average baseline osteoporosis knowledge score was 8.8 out of a possible 20 (s.d 3.3, range1 to 17).

Table 2 gives psychometric characteristics of the OKAT. The questionnaire had a Flesch reading ease of 45. If the word osteoporosis were removed from the questions or substituted with the term "thin bones", the reading ease rose to 65 and 68, respectively.

The index of difficulty for most items was satisfactory (between 0.12 and 0.66). Items 1 (Osteoporosis leads to an increased risk of bone fractures.), 4 (Osteoporosis is more common in men.) and 10 (Any type of physical activity is beneficial for osteoporosis.) scored above 0.75, indicating that most subjects answered these questions correctly.

The mean D-value for the questionnaire was 44%. Ferguson's sigma for the questionnaire is 0.96 which is very high.

There were no negative inter-item correlations. Items 9 (From age 50, most women can expect at least one fracture before they die.), 7 (A fall is just as important as low bone strength in causing fractures.) and 10 performed poorly when inter-item correlations were examined. These items had less than 50% of correlations with other items above 0.09. All other items had more than 50% of correlations above 0.09 and most had more than 75% and so were satisfactory. Item-total correlations are shown in table 3. Items 1, 7, 9 and 10 had an item-total correlation of less than 0.20. However, eliminating these items iteratively altered Cronbach's alpha by less than 1.5%. The changes in Cronbach's alpha resulting from elimination of each individual item in turn can be seen in table 3. For the 20 item questionnaire Cronbach's alpha was 0.69 and this increased only to 0.71 if all four items were excluded.

Principal factor analysis generated only one factor with an eigenvalue above 1 (Factor one – eigenvalue 2.3). Items loaded from 0.026 to 0.46 on this factor. Items 1, 4, 7, 9, 10, 11 and 18 loaded less than 0.3. Elimination of these items iteratively from the factor analysis did not alter the distribution of eigenvalues. There was still only one factor with an eigenvalue greater than one and the distribution of the loadings of the items on this factor was unchanged. Iterative elimination of the items with low loadings on factor analysis from item-total and Cronbach's alpha analyses caused changes of less than 1.5%. When items 1,4,9, 11 and 18 were eliminated there were decreases in alpha, rather than the desired increases.

The OKAT performed satisfactorily on virtually all components of the analysis.

The questionnaire had a lower than preferred Flesch reading ease. This was due to the use of the word osteoporosis in 12 of the 20 items. If this word was removed or substituted for by "thin bones" the Flesch reading ease became highly acceptable. As the word osteoporosis was the most accurate description of the disease about which knowledge was being measured and is widely recognised in the general population [17], its use was retained and the resulting decrease in Flesch reading ease accepted.

The questionnaire had a satisfactory index of difficulty. Item discrimination was satisfactory and Ferguson's sigma was close to the ideal value of 1.0. The three items with a high index of difficulty were retained as it was considered they related to core information about osteoporosis namely, the definition of osteoporosis, female preponderance and the physical activity requirements needed for prevention. Though factor analysis must be interpreted cautiously when analysing dichotomous variables, the fact that the analysis generated only one factor with an eigenvalue above 1 is consistent with osteoporosis knowledge being the main factor being measured by the questionnaire, and this provides some support for the construct validity of the instrument. Certainly, if more than one underlying factor had been found, we would have had cause to question construct validity.

The elimination of the items performing poorly on item-total correlation and principal factor analysis changed Cronbach's alpha by less than 1.5% and had minimal effect on the factor analysis outcome, indicating that the elimination of these items had little effect on the psychometric qualities of the questionnaire. The 20 item Cronbach's alpha of 0.69 was satisfactory, particularly given that in order to achieve high discriminatory power, a scale must include very easy items as well as very difficult items, which tends to decrease the internal consistency of the scale [26]. The elimination of individual items that performed less well did not alter the overall psychometric properties of the questionnaire and would have reduced the breadth of knowledge covered by the questionnaire ie affected content validity. These items evaluated knowledge of the meaning of osteoporosis, its prevalence, the adverse health outcomes of osteoporosis and of the physical activity requirements for prevention. Previous studies [20, 21] have described the problem of clinically pertinent items not performing ideally under psychometric testing, but still being important for the overall context of the instrument. We dealt with this problem in a similar manner to those studies. If the items contained what we considered to be core knowledge we retained them, but only after (1) examining the psychometric properties with and without the items to ensure that we did not adversely affect the reliability of the questionnaire and (2) describing the reasons for retention and the reasons for considering removing the item.

Baseline levels of osteoporosis knowledge measured using this instrument were low, with the average score of 8.8 being 44% of the possible maximum score. Most other studies of osteoporosis knowledge levels have shown low levels, but these have been in highly selected populations or have not used validated instruments to measure osteoporosis knowledge [6, 8, 10, 12, 14, 29‐32]. One study showed knowledge levels of 78% of the maximum score in 16–59 year old Norwegian women but the instrument was not validated [18]. As the average score using our instrument was low, there is scope for the instrument to be sensitive to change, which is a property that will be valuable if using the questionnaire to assess changes in knowledge with interventions. However, sensitivity to change remains to be tested.

This study has a number of potential limitations. While the sample was randomly selected, selection bias is possible due to the moderate response rate. Indeed proportion of current smokers in the sample is lower than the Tasmanian prevalence of daily smoking in females aged 25–44 in 1998 of 29% [33] and the proportion of women in married or in a de facto relationship is slightly higher than the Tasmanian proportion of 64%. However, the spread of education levels and the unemployment rate approximates the overall population figures for these variables. The range of demographics covered within the sample means that the validation has occurred over a reasonably heterogeneous group and so the questionnaire appears suitable for use in women aged 25–44 across a range of demographics. Although the osteoporosis knowledge instrument is based on the OPSMC content and the osteoporosis information leaflet, it covers a broad spectrum of osteoporosis knowledge and may be suitable for use to assess the impact of other osteoporosis educational interventions or to assess osteoporosis knowledge levels in young female Caucasian populations. The OKAT would require validation in other populations and some modifications might be necessary to reflect regional variations in osteoporosis demographics.

The OKAT for measuring osteoporosis knowledge has good psychometric properties in Australian 25–44 year old females. While it should be applicable to other Caucasian populations, this will require confirmation by further research.