Validation of the care transition measure in multi-ethnic South-East Asia in Singapore

Care transition, or the movement of patients from one healthcare setting to another, is a concern of many healthcare systems because it affects the overall quality of patients’ care. It has been well documented that inadequate discharge planning and/or patients’ poor self-management following discharge from hospitals increased avoidable rehospitalisation and the use of emergency room [1‐5]. It is not surprising that leading healthcare organisations for quality and safety such as the Joint Commission and The National Quality Forum (USA) are emphasising high-quality care transition [6‐12].

An important element in the measurement of quality of transition is patients’ experience [13, 14], as patients and caregivers are often the only common link across various care providing settings. In 2005, the American Institute of Medicine recommended research and development of measures to address patients’ experience during care transitions [15].

The Care Transition Measure (CTM-15) is, to the best of our knowledge, the only available measure of quality of care during transition, from the patients’ perspective [16‐19]. It was designed to measure the overall care transition experience and not merely the hospital discharge phase [20]. Studies have shown that the CTM-15 is a valuable tool for health system performance evaluation by measuring the quality of care delivered across settings [17, 20]. Recently, a shorter version of the CTM-15 comprising 3 of the original CTM items (CTM-3) [16, 21] was endorsed by the National Quality Forum [22]. The CTM-3 closely approximates the CTM-15 in measuring the quality of care transition and has an advantage of minimising cost and response burden for assessing quality of care [16]. Both the CTM-15 and CTM-3, however, have been validated in patient populations in the US and Israel only [23].

The purpose of our study was to test the psychometric properties of the CTM-15 and CTM-3 in Singapore, a multi-ethnic urban state in South-east Asia. The quality of Singapore’s healthcare services was regarded high by both the World Health Organization [24] and Singaporeans [25]. Although it is not in the form of national health insurance, Singapore’s healthcare system has successfully achieved the goal of providing good and affordable basic medical care to all Singaporeans. Nevertheless, as a fast aging country with high public hospitals bed occupancy, Singapore is also facing the challenge of growing concerns over quality of care transition [26, 27]. Therefore, there is a need to have a measure of care transition to assess the quality of care of patients in transition between care settings in Singapore. As the majority of the Singaporean population speaks either English or Chinese, we assessed both the English and Chinese versions of this instrument. We also compared the effects of three-point and five-point Likert response scales (currently used in the CTM) on the psychometric properties of the measures. This analysis was for maximising the usefulness of the instrument as fewer response choices pose less administrative and cognitive burden to respondents.

A total of 600 patients or their caregivers were interviewed. The overall participation rate was 58%. The main reason for refusal was lack of interest in the study. After excluding 21 respondents who were interviewed in Malay, 579 patients or their caregivers who were interviewed in English (n = 414) or Chinese (n = 165) with no missing data on the CTM-15 questions were included in data analysis. Table 1 summarises the basic demographic information of the participants of the study. It can be seen that the vast majority of respondents was proxies. English-speaking and Chinese-speaking respondents had similar characteristics except that higher proportion of respondents were patients themselves among those who were interviewed in Chinese (15.2%) than in English (3.9%) (p <0.01). Distribution of CTM item scores are displayed in Table 2. For both English and Chinese respondents, CTM-3 scores were lower than CTM-15 scores, and CTM scores based on the 5-point response scale were lower than those based on the 3-point response scale.

We found that the psychometric measurement properties of the CTM were generally satisfactory, although variations existed between different versions of this instrument. Generally, the long form is more reliable than the short form and English version is more valid than the Chinese version. The English version of the CTM-15 is the most valid and reliable measure of care transition. It exhibited adequate internal consistency and was able to discriminate between patients with and without ED visits or rehospitalisation and converged with reported experience with care post-discharge. These results were consistent with those in previous studies [16, 17, 23]. Also, three of the four factors underlying the CTM items in our study were similar to those observed in the US and Israel populations [16, 23].

Most of participants in our study were not patients but their proxies. This was because most patients designated a family member to answer the interview questions for them. Reasons for using a proxy included caregivers knowing better than patients about care-related matters, patients too weak to be interviewed, and patients having language barriers. Generally, using proxies to assess quality of care is a limitation because proxies and patients themselves may have very different perception of the care during transition. Nevertheless, it is necessary to use proxies when patients are not well enough to be the assessors. Moreover, it makes good sense to ask for the opinions of family members when it comes to assessment of quality of care from care users’ perspective in Singapore. Singapore is a small city country dominated by Asian culture and Singaporeans are very involved in the medical and long term care of their family members. Family members, particularly spouses and children not only provide informal care, but influence or even make medical and long term care decisions for the patient and also contribute to the healthcare costs. Singapore’s unique healthcare system allows contribution of healthcare costs from family members using individuals’ compulsory savings, which strengthens family bonds. Hence, it is not unexpected that the majority of our respondents were family members (Table 1). For this reason, a quality of care measure would be relevant and more valuable in Singapore with proxies’ assessment. The most important contribution of our study is therefore the provision of evidence for the usefulness of CTM when it is administered to proxies. To the best of our knowledge, this is the first study that explored the extended application of this instrument to assessor other than patients themselves.

The Chinese CTM-15 and CTM-3 were similar to their English counterparts in psychometric properties but exhibited poor discriminatory power. There are several possible reasons for this result. First, the poor discriminatory power could be partly be attributed to sampling error. This is supported by the observation that less than thirty patients reported ED visits or rehospitalisation in the Chinese-speaking subgroup (Table 1). It is possible that those patients visited ED or were rehospitalised for reasons not related to poor care transition. Therefore, the Chinese CTM should be reevaluated in a larger sample of patients in the future. Second, it could also be attributed to respondents’ poor understanding of the CTM items. Although we did not collect information on education profile of the patients, Chinese-speaking patients generally had fewer years of formal education than English Speaking patients in Singapore [32, 33] as English has been the primary language in education. Since the statements used in the CTM questionnaire are relatively long, a larger proportion of Chinese-speaking respondents might not comprehend the statements fully, which in turn affected the discriminatory power of the CTM. Another possible reason for poor understanding is suboptimal wording of the Chinese CTM which was the product of a single forward translation of the English CTM. Certain Chinese CTM items might not be accurate or clear. Nevertheless, this may not be a serious issue; factor analysis of English and Chinese CTM items resulted in similar latent constructs, suggesting no dramatic difference in response patterns between the two language groups. Third, Chinese-speaking respondents may have idiosyncratic response styles due to culture reasons. A previous study found that a visual analog scale for self-assessing overall health was not discriminative between Singaporean diabetics in differing health conditions [34]. Exact reason for this phenomenon is unknown. Hence, future studies using qualitative research methods such as in-depth interview or focus group discussion are needed to find out how respondents understand and respond to the CTM items, which will hopefully help us find ways to improve the performance of the instrument among Chinese-speaking persons.

It is interesting that the CTM scores based on the three-point response scale performed similarly well as those based on the originally five-point response scale. That means if the interviewer had offered only three response options (i.e. disagree, neutral, and agree) in the interviews, the resultant CTM scores would not be less reliable or discriminative. This result is not surprising as three-point response scales were found to be superior to five-point scales in some populations [35]. This is a useful finding as it suggests that the CTM can be administered in a more economical and easy-to-answer way.

It was not unexpected that the CTM-3 demonstrated worse internal consistency than the CTM-15. Internal consistency increases with increasing number of items. Although the reliability of the CTM-3 was suboptimal, it was highly correlated with the CTM-15 scores and exhibited good construct validity in English-speaking respondents. Therefore, the CTM-3 can be used as a substitute for the CTM-15 when administration burden is a concern or in large-scale studies where large sample sizes can compensate its suboptimal reliability.

There are some limitations in the study. First, our findings have limited generalisability. Factors that limit generalisability include relatively low response rate, all patients aged 50 years or older, half of the study sample enrolled in a care coordination program, most respondents being proxies, and some proxies possibly not experiencing the entire care transition process. However, our study for the first time demonstrated that the CTM was valid and reliable in proxies. Second, that we administered the CTM at the time when ED visits and rehospitalisation had occurred may have introduced response bias to the study. That is, those who had experienced ED visits or rehospitalisation might ascribe those undesirable events to care transition, which in turn intensified their memory of negative experience during discharge including those experienced during the discharge process. Arguably, it may be more appropriate to administer the CTM soon right after the hospital discharge (with the discretion of the problem we discussed next) and obtain information about ED visits and rehospitalisation from administrative records or inquire at a later time so that the respondents’ answers to the CTM questions are more clearly disassociated from ED visits or readmission events. Third, although genuine, the reports from some older patients may not be reliable. A previous study suggested delirium, including periods of altered perception, is highly prevalent among the elderly recently discharged to a post-acute care facility from hospital [30]. However, this should not be a big concern as the vast majority of our respondents were their caregivers.

The care transition measure is a valid and reliable measure for quality of care transition in Singapore. Moreover, the care transition measure can be administered to proxies using a simpler response scale. The discriminatory power of the Chinese version of this instrument needs to be further tested in future studies.