“Hip protectors in the elderly: lack of effectiveness or just suboptimal implementation?” A reply to critics

Dear Editor,

Thank you for inviting me to respond to a paper published in your journal: “Hip protectors in the elderly: lack of effectiveness or just suboptimal implementation?” [1]. I recently came across this paper which contains an extensive critique of a paper published by myself and colleagues [2] reporting a cluster randomised trial of hip protectors.

Meyer and Mühlhauser write concerning our study that it has important methodological shortcomings which might have heavily affected the results. They go on to raise more than a dozen issues that they argue undermine the internal validity of the trial. They also state that “Access to the education programme is not possible neither by further references nor authors’ offer to contact them” (p. 90).

Of course critical debate is to be welcomed, but unfortunately no debate took place as I am not aware that any attempt was made to contact me, even though my full contact details, including email address, were on the paper in question, offered as the address for correspondence. Now that the paper has come to my attention, I will address the main issues raised by Meyer and Mühlhauser, first outlining the criticism then offering my reply.

Criticism: The population was mixed, including nursing home residents and non-nursing home (i.e. residential home) residents. Reply: In the UK, both nursing and residential homes offer residential care for sick, disabled or elderly infirm people, including the elderly mentally ill. Nursing homes in the UK are distinguished from residential homes only by the fact that they deliver “nursing care” as narrowly defined in legislation, as we indicate in Table 1 in our paper. Furthermore, the type of home was one factor used in the stratified randomisation employed in our study.

Criticism: Neither baseline data for participants nor the flow of participants are reported. Reply: As the journal in which our paper was published operated a strict limit on words and figures (2,500 words of text, two tables, one figure) and as this was a trial of a policy using clusters, we chose to report home rather than individual characteristics. Readers are thus able to judge the success of randomisation in terms of home level characteristics. On a more general note, I would add that it is important to take the purpose of a paper (in this case describing the evaluation of a policy) into account when offering a critique.

Criticism: Absolute numbers of participants are not reported. Reply: I accept this is a limitation of the trial as reported, in that we report bed occupancy within each home, obtained by taking a census on four occasions over the study period, from which the mean number of occupied beds was calculated for each home. Bed occupancy in the homes was high and stable, so we believe this figure is reliable for our purposes. Had Meyer and Mühlhauser contacted us, we could also have provided them with the results of our individual level analysis, which included the 1,179 residents in the intervention group and 2,952 in the control group for whom we had full individual data. Logistic regression, taking the cluster design into account, showed no significant difference between groups (odds ratio 1.08, 95%CI 0.77–1.53).

Criticism: No details are reported on the structure, theory or piloting of the information session, no references are given and the authors do not offer to be contacted. Reply: As noted above, we were writing to a strict word limit, so full details could not be given. However, the overall structure of the programme was described and two key references were given indicating the evidence-based approach to implementation. Papers on the piloting of our intervention and the educational approach taken in the main study have been published [3, 4] and further details are available on request. I am not sure what Meyer and Mühlhauser mean when they write that access is not possible through the authors’ offer to contact them. Publishing my contact details as the author for correspondence on the paper itself would seem to me to constitute such an offer. I am not aware that they took the offer up and they do not report such an attempt. By way of contrast, the authors of the Cochrane review that Meyer and Mühlhauser criticise did contact me for further information, which I provided.

Criticism: Outcome measurement is based on routine documentation of homes and chart review. Reply: We report that the main outcome (hip fractures) was cross-referenced from three sources, including quality-controlled information systems within the only two hospitals in Northern Ireland that treated patients with hip fractures. This information was cross-checked and I am confident it is accurate. Secondary outcomes were obtained from routine data collection sources.

Criticism: Numbers of falls and fallers are not reported. Reply: We report the number of injurious falls (as these are required to be reported to the Registration and Inspection Unit for the homes) as an indication of the relative risk of falling in intervention and control groups. No significant difference was found.

Criticism: It is unclear if intervention and control groups had similar risks of hip fracture. Reply: The groups were obtained through randomization and were found to be similar in terms of home level characteristics (as reported) and individual level characteristics (as would have been made clear on request).

Criticism: A minority of homes did not cooperate with implementation or outcome assessment. Reply: We report that all homes cooperated with outcome assessment in relation to hip fractures and injurious falls. The study was designed to investigate the effect of a policy of offering hip protectors. Therefore, an intention to treat analysis is appropriate based on data from all 127 homes entered into the study. This includes the “low-cooperation” homes that refused to give individual data, or to allow monitoring visits or (for those in the intervention group) to offer the hip protectors. Arguably, this is a strength of the trial as a pragmatic evaluation of policy in the real world. Your readers will note that we also reported a “per-protocol” analysis on the 114 homes on which we had full data, which produced a similar outcome.

Criticism: The results of the trial were not adjusted for the cluster effect. Reply: In the intention to treat analysis, the results were adjusted for clustering using a method similar to that described by Donner and Klar [5], as indicated in our section on statistical analysis. Here, we state that the appropriate unit of analysis is the home rather than the resident, and that the analysis was based on event rates per home. As noted above, had Meyer and Mühlhauser contacted us, we could also have provided them with the results of our individual level logistic regression analysis, which also took the cluster design into account.

Of course, no study or research paper is perfect and criticism is to be welcomed. However, contacting the authors of the paper to clarify the issues before publishing an extended critique would allow a more accurate picture to emerge for readers.