Original ArticleEffects of FRAX® Model Calibration on Intervention Rates: A Simulation Study
Introduction
Although reduced bone mass is an important and easily quantifiable measurement, studies have shown that most fractures occur in individuals with a bone mineral density (BMD) T-score above the operational threshold for osteoporosis 1, 2, 3, 4. The use of clinical risk factors (CRFs) has been shown to enhance the performance of BMD in the prediction of hip and major osteoporotic fractures (5). In addition to a prior fragility fracture, other important CRFs include age, sex, body mass index, prolonged use of glucocorticoids, rheumatoid arthritis, parental history of hip fracture, current smoking, alcohol intake of 3 or more units/d, and secondary osteoporosis (6). These elements are integrated in the WHO fracture risk assessment tool (FRAX®) for estimation of individual 10-yr major osteoporotic and hip fracture probabilities (6).
Meta-analyses have confirmed that there is an improvement in the fracture prediction using BMD and CRFs together compared with using either BMD alone or CRFs alone 4, 5. This has led to broad endorsement of FRAX® and its integration into several clinical practice guidelines 7, 8, 9, 10, 11, 12, 13, 14, 15. Fracture and mortality rates are known to vary widely between countries (16). Therefore, population-specific FRAX® tools can be customized to the fracture and mortality epidemiology in that specific region (6). At present, more than 35 FRAX® models are available, and others are being developed.
Fundamental features of a predictive model’s performance are discrimination and calibration (17). Model discrimination addresses the question: “How well did the model perform in terms of risk stratification?” Of equal importance is model calibration, which asks the question: “Was the observed fracture risk consistent with the predicted fracture risk?” Although FRAX® has been independently shown in multiple cohorts to provide fracture discrimination comparable to the initial derivation cohorts (5), assessment of calibration is less frequently considered but incorrect calibration could affect performance of the model in clinical practice.
A 10-yr major osteoporotic fracture probability of greater than 20% is considered high risk and an indication for intervention according to the National Osteoporosis Foundation (NOF) of the United States and Osteoporosis Canada 7, 8, 18. The NOF also recommends that a 10-yr hip fracture probability of 3% or greater be considered for intervention, in addition to those with any BMD measurements in the osteoporotic range and those with prior spine or hip fractures. The current analysis was undertaken to explore how simulated miscalibration in a previously validated FRAX® tool would affect the numbers of individuals meeting these intervention criteria.
Section snippets
Subjects and Setting
In the Province of Manitoba, Canada, health services are provided to virtually all residents through a single public health care system. Bone density testing with dual-energy X-ray absorptiometry (DXA) has been managed as an integrated program since 1997; criteria and testing rates for this program have been published (19). The program maintains a database of all DXA results, which can be linked with other population-based computerized health databases through an anonymous personal identifier.
Results
The study cohort included 36,730 women and 2873 men. Baseline characteristics of the population are summarized in Table 1. Men were slightly older than women (mean age 68.2 ± 10.1 vs 65.7 ± 9.8, p < 0.001). Women had a lower mean femoral neck T-score (−1.5 ± 1.0 vs −1.2 ± 1.1, p < 0.001). FRAX® risk estimates for major osteoporotic fracture were greater for women than men (p < 0.001), but were similar for hip fracture probability. Using the reference calibration, 4137 (11.3%) women and 83 (2.9%) men exceeded
Discussion
This analysis found that relatively small changes in model calibration (even on the order of 10% relative miscalibration) can have large effects on the number of individuals qualifying for treatment. This effect is particularly evident with major osteoporotic fracture probability where a 50% relative miscalibration could more than double (overcalibration) or virtually eliminate (undercalibration) treatment of osteoporosis. Hip fracture probability is also affected by miscalibration, but the
Acknowledgments
The authors are indebted to Manitoba Health for the provision of data (HIPC File No. 2007/2008−49). The results and conclusions are those of the authors, and no official endorsement by Manitoba Health is intended or should be inferred. This article has been reviewed and approved by the members of the Manitoba Bone Density Program Committee. We would like to thank Ms. Helena Johansson and Dr. John Kanis for generating the FRAX® results for the Manitoba cohort.
Disclosures: William D. Leslie:
References (35)
- et al.
Fracture incidence and association with bone mineral density in elderly men and women: the Rotterdam Study
Bone
(2004) - et al.
FRAX and its applications to clinical practice
Bone
(2009) - et al.
Construction and validation of a population-based bone densitometry database
J Clin Densitom
(2005) - et al.
A reference standard for the description of osteoporosis
Bone
(2008) The importance of spectrum bias on bone density monitoring in clinical practice
Bone
(2006)- et al.
Rate of proximal humeral fractures in older Finnish women between 1970 and 2007
Bone
(2009) - et al.
A nationwide seven-year trend of hip fractures in the elderly population of Taiwan
Bone
(2009) - et al.
Bone mineral density thresholds for pharmacological intervention to prevent fractures
Arch Intern Med
(2004) - et al.
BMD at multiple sites and risk of fracture of multiple types: long-term results from the Study of Osteoporotic Fractures
J Bone Miner Res
(2003) - et al.
Low bone mineral density and fracture burden in postmenopausal women
CMAJ
(2007)
The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women
Osteoporos Int
A revised clinician’s guide to the prevention and treatment of osteoporosis
J Clin Endocrinol Metab
Implications of absolute fracture risk assessment for osteoporosis practice guidelines in the USA
Osteoporos Int
FRAX and the assessment of fracture probability in men and women from the UK
Osteoporos Int
Case finding for the management of osteoporosis with FRAX((R))-assessment and intervention thresholds for the UK
Osteoporos Int
FRAX assessment of osteoporotic fracture probability in Switzerland
Osteoporos Int
European guidance for the diagnosis and management of osteoporosis in postmenopausal women
Osteoporos Int
Cited by (16)
Circulating serum microRNAs including senescent miR-31-5p are associated with incident fragility fractures in older postmenopausal women with type 2 diabetes mellitus
2022, BoneCitation Excerpt :As DXA scans were not performed in the AGES-Reykjavik study, and as the FRAX website and the International Society of Clinical Densitometry (ISCD) and the International Osteoporosis Foundation (IOF) in their joint official positions do not recommend to calculate FRAX using a BMD measurement obtained with other technologies than DXA [62], we decided to not input our CT-derived DXA-equivalent areal femoral neck BMD into the FRAX calculator and therefore could not compute BMD containing FRAX major osteoporotic fracture ten year probabilities in the conventional way. However, in order to take advantage of our available CT-derived areal BMD information and as meta-analyses have shown improved fracture prediction using BMD and clinical risk factors together compared with using either BMD or CRFs alone [63,64], we created via Cox-Proportional hazard modelling (with clinical FRAX and DXA-equivalent femoral neck aBMD as covariates) a combined FRAXaBMD variable that was used as surrogate FRAXaBMD for all statistical analyses. In 17 subjects, baseline CT could not be performed, and one woman was too heavy for clinical FRAX calculations, leaving a total of 149 subjects in whom FRAXaBMD could be calculated.
Worldwide Fracture Prediction
2017, Journal of Clinical DensitometryCitation Excerpt :Such errors would, however, impact absolute FRAX-derived fracture probabilities, and thus daily practice at the individual and public health level, the latter because of a major impact on cost-effectiveness analyses (15). Indeed, Leslie et al showed that small changes in FRAX calibration, for example, as little as 10%, had a large effect on the number of individuals qualifying for treatment (162). The importance of high-quality data that best represents country-specific epidemiology was further underscored in a study directly comparing 8 national FRAX tools for fracture prediction and treatment qualification in 36,730 Canadian women, mean age 65.7 ± 9.8 yr, enrolled in the Manitoba BMD program.
Effect of Race/Ethnicity on United States FRAX Calculations and Treatment Qualification: A Registry-Based Study
2023, Journal of Bone and Mineral ResearchMajor osteoporosis fracture prediction in type 2 diabetes: a derivation and comparison study
2022, Osteoporosis InternationalCalibration of FRAX: A Journey, not a Destination
2021, Calcified Tissue InternationalComparison of treatment strategies and thresholds for optimizing fracture prevention in Canada: a simulation analysis
2020, Archives of Osteoporosis