Measurement of patient reported disability using WHODAS 2.0 before and after surgical intervention in Madagascar

Patient reported outcomes (PRO) have been defined by the U.S. Food and Drug Administration as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else” [1]. In recent years PRO tools have become an increasingly important method of evaluating surgical outcomes. Since 2009, the National Health Service (NHS) in England has collected several PRO measures for four types of surgery: hip replacement, knee replacement, varicose vein and groin hernia surgery, and publishes quarterly reports [2]. Internationally, World Bank has advised countries to ensure that health services are client oriented [3]. Yet, despite this recommendation, PRO’s are still not widely reported in high or low income settings.

New or residual disability after surgery is of concern to both patients and clinicians. Definitions of disability distinguish between the impairment (physical or mental) and the impact the impairment has on the individual’s ability to work, interact socially and to care for themselves [4, 5]. Disability-free survival is said to be an ideal endpoint in surgical outcome studies because it reflects the primary goal for most patients [6], circumvents the problems of using more traditional composite endpoints [7], and can aid shared decision-making [8]. Yet despite this, there remains scant evidence on the impact of disability after surgery. Elderly patients are known to frequently not fully recover after surgery and suffer accelerated disability in the following months and years after surgery, but little is known about the impact of surgical intervention on younger age groups [9‐11]. The World Health Organisation Disability Assessment Schedule 2.0 (WHODAS 2.0) is a PRO tool validated in surgical patients [6]. WHODAS 2.0 is publicly available [12] and can be administered quickly by face-to-face or by telephone interview [13].

In low and middle income countries (LMICs), surgery is expensive. 81 million people are estimated to face catastrophic expenditure paying for surgical care [14, 15]. Because of the high cost to patients, it is imperative that surgical care be of high quality and has meaningful impact on patients. Since PRO tools measure quality of care from the patient perspective they have been used to calculate the health gains after surgical treatment in high income countries (HICs) [16] but there is little evidence of use in LMICs.

In LMICs surgical delivery does not only occur in government hospitals. Over 400 surgical non-governmental organizations (NGOs) operate globally [17], and, in some countries, more surgery is performed by NGOs than by the government sector [18]. These surgical NGOs should themselves be held to the same standard, so that the care provided does not merely ‘feel good’ as an act of heroism but provides quality care with meaningful patient impact. In this study, we aim to use a PRO tool to evaluate the patient-centred impact of NGO-delivered surgery. We hypothesized that surgical interventions would decrease patient reported pre-operative disability measured using WHODAS 2.0.

The Minister of Health of Madagascar gave permission for the surgical programs including the evaluation and Mercy Ships Institutional Review Board approved the study (MS-2016-003). The requirement for individual written consent was waived.

Pre-operative WHODAS 2.0 was recorded in 401 patients. The orthopaedic program only treated 82 patients instead of the 100 predicted, and the plastics program volume was reduced by 60% due to the surgeon suffering a fractured limb and therefore unable to operate. The male: female ratio was 0.42, median age 32 years (Range: 3–73; IQR: 19–46 years); median duration of surgery 115 min (Range: 16–523; IQR: 86–173 min); and median length of hospital stay 7 days (Range: 1–70; IQR: 3–13 days).

At 3 months, 176 out of 401 (44%) patients were contacted by telephone for postoperative WHODAS 2.0 evaluation. Three were excluded because more than one WHODAS 2.0 score was missing, leaving 173 for final analysis (Fig. 1).

There was a significant reduction in mean preoperative and postoperative disability scores (8.4% to 1.0%, p < 0.001, Table 1). When analysed per surgical speciality, each speciality showed statistically significant reductions in disability.

Sixty-eight patients out of 401 (17%) met criteria for disability (score > 25%) preoperatively (the number per speciality is shown in Table 2). No patient had residual disability after surgery.

Seventeen out of 401 (4%) patients experienced a post-operative complication and there were no deaths (Table 3). The preoperative disability scores of patients who developed a postoperative complication did not differ from those who did not (p = 0.21) although our study was under-powered to detect a difference this small.

Disability scores, demographic and hospital details for patients followed-up at 3 months versus those lost to follow-up are shown in Table 4. The group lost to follow-up were more likely to be female (65% versus 50%, p < 0.05), were younger (mean age 31 versus 35, p < 0.05), had longer hospital stays (10 versus 4 days, p < 0.001), and were more likely to have experienced post-operative complications (p < 0.05). No differences were seen between the two groups in preoperative disability (p = 0.25), ASA score (p = 0.46), or duration of surgery (p = 0.85).

Across a wide range of surgical specialties, surgical patients treated by an NGO in Madagascar had a significant reduction in self-reported disability as measured using WHODAS 2.0.

It is estimated that globally 143 million additional surgical procedures are needed annually in order to meet the global burden of surgical diseases [23]. In order to ensure quality of surgical scale-up, reliable and easy-to-use measures of surgical quality and patient impact are needed in LMICs. WHODAS 2.0 is a PRO that can measure the patient impact of surgical interventions. This study uses WHODAS 2.0 to demonstrate a reduction in patient-reported disability after NGO surgery in Madagascar.

PRO tools are designed to evaluate what matters to the patient rather than factors clinicians or other stakeholders consider important. Historically clinicians have focused on perioperative mortality rates and incidence of post-operative complications. The 2004 Dindo-Clavien classification [24], grades complications according to patient impact but is not widely used. At the population level, Disability Adjusted Life Years measure the burden of a particular disease and Quality Adjusted Life Years assess the cost-effectiveness of an intervention to treat the disease, but both are focussed from the stakeholder perspective. PRO tools assess what patients think they are able to do and how they feel. Thus PRO tools measure a patient’s health status or health-related quality of life at a single point in time, and are collected through short, self- completed questionnaires. They are widely used in high-income settings [2, 25], but many are difficult to administer and may be difficult for populations with high illiteracy rates to understand. WHODAS 2.0 is simple and quick to use taking approximately 5 min to administer by telephone [13]. Although a wide variety of PRO tools have been documented within National Institute for Health’s Patient Reported Outcomes Measurement Information System, they face two major challenges. Many are not in widespread clinical use, and more work is required regarding quantifying changes for performance and accountability purposes [16]. Our study adds to the evidence that WHODAS 2.0 can be used as a PRO tools to demonstrate patient impact in a LMIC.

Follow up in this study was 44%, which is comparable with other studies investigating surgical outcome in LMICs [26‐30]. Most studies evaluate immediate outcome (1–4 weeks), with very few evaluating longer term follow up [28]. Follow-up rates in LMICs are typically lower than those in high income countries (HICs) although even in HICs follow up rates can vary considerably. One HIC study looking at PROs of long-term conditions in primary care reported 38.4% follow up rate [31] while rates of 70% [25] have been reported after hip surgery. Challenges of follow up in LMICs after surgery include: nonfunctioning telephone numbers, transport costs, the requirement for the continued presence of a surgical team to administer post-operative questionnaires and to provide informed responses when difficulties are encountered. In this study, there was no difference between those followed up and those lost to follow up in preoperative disability scores, duration of surgery or length of hospital stay. However, the group lost to follow up was more likely to be female, younger, and to have experienced post-operative complications. Female and younger patients fit the profile of those least likely to have functioning telephone numbers. That they experienced significantly more postoperative complications is difficult to explain. One might consider that in HICs those with complications will likely return to complain or seek further help, but perhaps in a LMIC, they may be unwilling to return for fear of further complications. Since we did not aim to investigate the reasons for lack of follow up we are unable to comment further further.

Post-operative complications were not associated with post-operative WHODAS scores, further illustrating the value of PRO tools in evaluating surgical outcomes and impact. Reliance on more objective and traditional measures of surgical outcome, such as post-operative complications, may be insufficient. In one study in Bangladesh evaluating patient satisfaction in a Government hospital [3], it was noted that patient satisfaction was unrelated to surgical outcome, and that respect and politeness of providers, as opposed to their technical ability, was the greatest predictor of patient satisfaction. Similar findings have been seen in cleft lip and palate surgery in Nigeria [30]. Patients satisfied with the service and care they received are more likely to be committed to future treatments and would recommend the service to others [30, 32]. Also, since satisfaction is heavily influenced by things like the cost of surgery and the attitude of providers [30], traditional measurements of mortality and morbidity are insufficient to monitor patient satisfaction and impact. This study shows how WHODAS as a PRO can show a positive patient impact (significant reduction in disability) after surgery in a LMIC.

This study has several limitations. Working cross culturally as well as in different languages represents a challenge to data collection and whilst we ensured that we used experienced translators, information can be lost in translation. Baseline data was obtained through face-to-face interview whereas follow-up data by telephone and while WHODAS 2.0 is designed to be administered in both ways, we are unable to rule out any effect from the mode of interview. Our study presents paired data but did not adjust for confounders, and was underpowered to accurately compare the postoperative disability scores of those who experienced postoperative complications with those who did not. Since the group lost to follow up had more females, younger age, longer length of stay and more complications, the results may not be generalizable to the entire Mercy Ships surgical population. Bias may also be introduced by those lost to follow up in that they may have not wanted to share negative results. Mercy Ships provides surgery for free, and this may also affect patient’s willingness to report a negative outcome. Despite these limitations, our study has a number of strengths. The study demonstrates the use of WHODAS as a PRO tool in LMICs to show a reduction in disability across a wide range of specialties; the follow up rate of 44% is relatively high and the numbers are large compared to other follow up studies in LMICs.

Our study shows that there was significant reduction in patient reported disability after surgical intervention in a LMIC. We recommend that WHODAS 2.0 is used as a PRO tool by NGOs and other stakeholders involved in the delivery of surgical care to monitor patient-centered impact of surgical interventions. This will ensure that surgical health services remain patient oriented as recommended by World Bank [3].