Assessing the performance of centralized waiting lists for patients without a regular family physician using clinical-administrative data

The objective of this study was to assess the performance of centralized waiting lists for unattached patients implemented in Quebec, Canada, in terms of both processes and outcomes.

Quebec’s tax-based system provides universal health insurance coverage for medical services to its eight million residents. Its healthcare system is structured on three pillars of governance. At the provincial level, the Ministry of Health and Social Services (MSSS) determines the priorities and orientations for the overall health system and ensures that the regional agencies apply the established policies. The 16 regional health and social services agencies (ASSS) are responsible for organizing and coordinating services in their respective regions. At the local level, there are 93 health and social services centres (CSSS) made up of local community services centres, long-term care facilities, and, in 85% of cases, hospitals [24]. Each CSSS is mandated to improve the health and well-being of a specific geographically-defined population [25]. In collaboration with local partners such as medical clinics, municipalities, and schools, these CSSSs ensure the delivery of services in their territory.

In Quebec, the majority of FPs work on a fee-for-service basis. Private practices are managed by FPs who are self-employed but paid by government through public health insurance. There are also local community services centres, which are government-managed and where physicians are paid on a salary basis; however, less than 20% of Quebec’s physicians work in these centres [26]. Formal attachment is a process by which patients are registered on an FP’s list of patients [27]. This facilitates accountability by defining the population for which the FP is responsible and fosters an ongoing relationship between the patient and FP [28]. Formal attachment of patients to FPs is relatively new in Quebec. It started in the early 2000s with the implementation of a new model of primary care, the Family Medicine Group model, but has since been extended to all practice settings [26, 29]. In Quebec, attachment is formalized through a written agreement which is signed by both the patient and FP. This formalization of attachment was part of a provincial government policy to add capitation-based financial payments for FPs to fee-for-service payments. Since the introduction of this new policy, most FPs have formally attached their regular patients. As of March 2014, 65% of the province’s population was formally attached to an FP [30].

To address the need to attach patients to FPs, in 2008 the MSSS and the Quebec Federation of General Practitioners (FMOQ) decided jointly to implement centralized waiting lists for unattached patients. The aim of these lists, called guichets d’accès pour la clientèle orpheline (GACO), is to facilitate access to an FP for the population of each local territory, based on a clinical priority scale and physicians’ availability to take on new patients. While the MSSS and FMOQ published a general framework to guide GACO implementation [31], in practice, there is considerable variation in GACO organization models and processes, as each of the 93 CSSSs were mandated to implement a GACO in their territory.

Figure 1 presents how GACOs generally work. Each CSSS has a local GACO. This local GACO serves the population, clinics, and FPs of the local territory. Each region has as many GACOs as it has CSSSs. In most cases, a local GACO is staffed by a secretary and a nurse working in collaboration with a medical coordinator, who is an FP from the local territory fulfilling this role in addition to his or her medical practice. Requests for an FP can be made to the GACO in several ways. Patients can make the request themselves either by phone, online, or by completing a form at a local community services centre and submitting it there or by mail. The request can also be made by a health or social services professional after seeing an unattached patient at a medical clinic or emergency department, after hospitalization, or in another care context. Finally, physicians formerly could make a request to the GACO on behalf of a patient they had agreed to attach, a process known as self-referral that was, however, later prohibited by new rules put in place by the government as of June 2013. The questions on the GACO enrolment form vary from one CSSS to another, but generally cover contact information, demographic characteristics, presence of certain medical conditions (e.g. diabetes, mental illness, hypertension), and information on health services use (e.g. emergency department visits, home care).

After the request for an FP is made to the GACO, a nurse assesses the patient’s health by phone. A priority code is then assigned to the patient by the nurse, sometimes in collaboration with the local medical coordinator, depending on the urgency and complexity of the case. Priority codes were implemented to ensure patients with the most pressing medical needs are attached to an FP first. The MSSS/FMOQ framework recommends maximum wait times for attachment to an FP for each priority level [31]. For priority 1 cases, which require immediate medical care (complex pathologies, high risk of decompensating), the recommended wait time is 30 days or less. Priority 2 patients should be attached within 3 months, and priority 3 within 6 months. Patients identified as priority 4 do not require urgent care, but should be attached to an FP within a year. For those identified as priority 5, considered in good health, there is no recommended wait time. Ultimately, patients are matched with an FP based on primary care workforce availability, scope of practice of FPs participating in the GACO, medical priority category, and date of request. Relevant information on the patient’s health is documented and sent to the receiving FP.

FPs’ participation in GACOs is voluntary. Those practising in the CSSS’ territory are not obligated to take unattached patients through the GACO. FPs who want to participate can contact the GACO intermittently to take on the desired number of new patients and can refuse any patient the GACO sends them. To increase the number of patients attached to an FP via GACOs and to encourage FPs to take on patients considered medically vulnerable, various financial incentives were implemented. Patients considered medically vulnerable are those who have at least one of the 20 vulnerability codes as defined by the Régie de l’Assurance Maladie du Quebec (RAMQ), Quebec’s health insurance board. These include 19 specific health problems (e.g. cancer, depressive disorders, intellectual disability) or being 70 years old or over [32]. Since the implementation of GACOs, financial incentives for the formal attachment of new patients have been modified on several occasions [26]. As of November 2011, FPs receive a bonus when they take on a patient via GACOs: 100$CAD for non-vulnerable patients and 200$CAD for vulnerable patients [33]. These bonuses are paid to the physician after the patient’s first visit.

In this article, we assess the performance of centralized waiting lists for unattached patients (GACOs) in Quebec over a 1-year period from April 1, 2013, to March 31, 2014. We focused on this period because: a) it represented the most recent data available; b) after 5 years of implementation, GACOs could be considered firmly established in Quebec; and c) changes made to the GACO framework in previous years had major impacts on pre-2013 data [26].

In total, the data from 86 GACOs located in 14 regions of Quebec were analyzed. The seven other GACOs were excluded because their data were either unavailable or incomplete in SIGACO. The data used therefore included all patients in the SIGACO database for 86 GACOs.

A committee of six decision-makers from the three levels of governance (provincial, regional and local), four healthcare professionals (two nurses and two physicians) involved in implementing and monitoring GACOs in Quebec, and four researchers from our team selected five performance indicators among those available in the SIGACO database (see Fig. 2). To identify the performance indicators presented in this article, the committee based its decision on the data from the previous year (2012–2013) and on the indicators’ usefulness in pinpointing areas for improvement of the performance of the centralized waiting lists. The committee aimed to select a limited set of indicators that comprehensively represented the GACOs’ different components (requests, prioritization, the waitlist, and attachment). Although factors external to the GACOs could have influenced the performance of several indicators, the committee selected indicators for which actions could be taken from within the GACOs to improve performance. We did not use data related to wait times for FP attachment, i.e., number of days elapsed between making a request to the GACO and becoming attached to an FP, because several irregularities were observed in the data and there were variations between regions in the way dates were recorded in the SIGACO database. Three of the selected indicators are related to the GACOs’ internal process (new requests for an FP, patients waiting for an FP, change in the number of patients waiting for an FP), while other two relate to GACO outcomes (patients attached to an FP through GACOs, vulnerable patients attached to an FP through GACOs).

To take into account the five indicators selected by the committee, which included both process and outcome indicators, we based our approach to assessing GACO performance on the Balanced Scorecard method [23]. The initial objective of the widely adopted Balanced Scorecard was to present managers with a concise yet comprehensive overview of performance [34]. Its underlying principles are that no single measure can provide enough information to gain an understanding of critical areas of performance and that processes driving performance in achieving key outcomes must also be measured. Use of a Balanced Scorecard has been shown to improve performance by: 1) better translating strategy into operational terms; 2) transforming strategizing into a continuous process; and 3) more closely aligning an organization’s processes, services, competencies, and units [35]. The more traditional Balanced Scorecard used in the private sector, with its emphasis on financial results, may not be well suited to evaluate the performance of healthcare interventions such as centralized waiting lists. However, many organizations have adapted the Balanced Scorecard to their contexts, integrating more appropriate process and outcome measures for use in the healthcare sector [36‐39]. This method allowed us, when describing GACOs’ performance, to take into account the various aspects of their processes as well as their results in terms of patient attachment to FPs and prioritization of vulnerable patients.

For each of the five indicators, we classified regions into tertiles of relative performance, where tertile 1 indicated relatively weak performance and tertile 3 indicated relatively strong performance. No region was classified in the strongest or weakest performance tertiles for all five indicators. Region B was the region most often classified in the strong performance tertile (4 times out of 5). Region I ranked twice in the strongest performance tertile and three times in the average tertile. At the opposite end, region G was the region that ranked most often in the weakest (twice) and average (three times) tertiles. Region D ranked twice in the weakest tertile, twice in the average tertile, and once in the highest tertile. Figure 6 presents an overview of these five indicators for these contrasted cases. No clear performance patterns emerged from classifying the regions into performance tertiles.

The results for the Pearson correlation coefficients were calculated at the local GACO level (i.e. the lowest level of data aggregation, n = 86). As such, indicator 1 (New requests for an FP made to the GACOs) was strongly and significantly correlated with indicator 2 (Patients waiting for an FP in GACOs) (r = 0.51 p <0.0001), meaning that as requests for an FP increased, more patients were waiting. Indicator 1 was also strongly correlated to indicator 4 (Patients attached to an FP through GACOs) (r = 0.61, p < 0.0001), meaning that as new requests increased, more patients became attached to an FP through GACOs. Indicator 3 (Change in number of patients waiting for an FP in GACOs) had a strong inverse correlation with indicator 4 (r = −0.53, p < 0.0001), meaning that as more patients became attached to an FP, the number of patients waiting for an FP decreased. There was a significant inverse correlation of average strength between indicator 5 (Vulnerable patients attached to an FP through GACOs) and indicator 4 (r = −0.31, p < 0.005), meaning that as more patients became attached to an FP through GACOs, the proportion of vulnerable patients became smaller. No significant correlation was found between the five performance indicators and the proportion of each territory’s population attached to an FP.

Like centralized waiting lists implemented in other Canadian provinces, the GACOs have the twofold objective of: 1) attaching patients to a family physician, and 2) giving priority to vulnerable patients [21]. The results presented in this article show that, in 2013–2014, the number of new requests for an FP made to GACOs throughout Quebec (260,046 requests) largely exceeded the number of patients who became attached to an FP through GACOs in that period (152,625 patients). This may mean the GACOs’ outreach was good, but that FPs’ availability or participation in GACOs (supply) was insufficient to meet the demand. Results also show that, in total, nearly 22% of patients waiting for an FP were vulnerable, whereas this proportion rose to almost 40% among patients who were attached to an FP through GACOs in 2013–2014. Vulnerable patients therefore seem to have been given priority in attachments to FPs. Overall, the GACOs seemed to have: 1) attached a large number of unattached patients to FPs even though the overall number of requests exceeded capacity, and 2) prioritized vulnerable patients in attachments to FPs.

The results showed very significant differences in performance between regions and large variations both among GACOs of different regions, and among those of a same region. GACOs’ performance varied with regard to indicators related to objectives (Patients attached to an FP through GACOs; Vulnerable patients attached to an FP through GACOs), as well as indicators reflecting processes (New requests for an FP made to GACOs; Patients waiting for an FP in GACOs; Change in number of patients waiting for an FP in GACOs). No region was classified in the strongest performance tertiles for all five indicators. Also, GACOs of a same region could perform relatively well for one indicator and not for others. The large variations in performance may be explained, in part, by the fact that the MSSS and FMOQ provided very few details on how the CSSS should implement the GACOs, “leaving each CSSS considerable latitude in the strategies they adopted” and leading to “large variability in what services GACOs offer” [21]. It is possible that implementing such a mechanism with few guidelines is one of the reasons we have observed such heterogeneity in the performance of GACOs.

Therefore, the results indicate there is room for improvement in GACOs’ performance for all the regions under study. Such improvement is necessary to reduce the large performance heterogeneity observed across the province—heterogeneity that can lead to important inequalities in access to an FP between regions and even between territories of the same region.

The correlations between indicators offer clues regarding paths to explore in identifying levers for improving GACOs. The number of new requests for an FP, i.e., the influx of patients into GACOs, appears to be an important performance indicator, as it was strongly correlated with the rate of patients waiting for an FP and with the rate of patients attached to an FP through GACOs. Moreover, the negative correlation of average strength between the rate of patients attached to an FP via GACOs and the proportion of these patients who are vulnerable indicates there may exist a certain tension between the two objectives of the GACOs. Indeed, it appeared that, as the number of patients being attached to an FP grew, the proportion of these patients who were vulnerable declined. Surprisingly, the GACO performance indicators did not seem to be correlated with the proportion of unattached patients on the territory, which we expected would have a significant influence on performance.

Clearly, the five indicators presented in this article paint an incomplete picture of GACO processes and outcomes, the choice of indicators being limited by the data available in the clinical-administrative database used. While these indicators enabled us to obtain a quick and comparable overview of the GACOs’ performance, developing performance indicators that reflect more accurately the processes and the outcomes of GACOs could help to better identify areas of improvement. For instance, since the indicators presented in this article come from aggregated data, we did not have information on the patients’ individual profiles, such as their specific health issues (mental health, hypertension, HIV/AIDS, etc.). We cannot know if the vulnerable patients who were attached to an FP were in fact the most vulnerable ones, i.e., those with multiple health problems or with the most complex health problems. Moreover, the indicators presented here are measures of processes and of immediate outcomes only, since the database that was used only focuses on the period extending from patients’ request for an FP to their attachment. Therefore, we are unable to present medium- or long-term outcome measures of the GACOs, such as use of healthcare services after attachment to an FP, or retention of the FP. A study comparing use of healthcare services before and after attachment to an FP through GACOs could assess whether GACOs actually improve access to primary care. Finally, the process and outcome indicators used can help identify performance variations and potential areas for improvement, but are insufficient to understand the source of the variation. Qualitative case studies comparing the processes and mechanisms of GACOs with relatively strong performance against those with relatively weak performance on certain indicators would be necessary to gain a better understanding of the sources of variations both from within the GACOs and from external local or regional influences.

The advantage of the Balanced Scorecard approach is that it allowed us to assess and compare the performance of the GACOs by using a combination of indicators that reflect multiple aspects of processes and outcomes. If, conversely, we had used only one indicator, such as the percentage of requesting patients who were subsequently attached to an FP through GACOs, we would lose relevant process information. Focusing on the large percentage might, for instance, obscure the fact that the actual number of requests processed by the GACO was small. Using multiple indicators also allowed us to take into account the GACOs’ twofold objective (attach patients and prioritize vulnerable patients) in evaluating their performance as well as various processes that provide details on the GACOs’ mechanisms.

The Balanced Scorecard is an interesting tool to use in the context of significant performance heterogeneity. No GACO in our study ranked in the strongest performance tertile for all indicators. It is therefore impossible to verify what the “best” GACO does and to develop general recommendations for all other GACOs based on what this “best” GACO is doing. However, it is relevant for a GACO with a relatively weaker performance for one indicator, such as prioritizing vulnerable patients in attachments to FPs, to know how another GACO that has a relatively strong performance for this indicator is doing. Comparing GACOs’ performance on various indicators can thereby stimulate a reflexive practice to improve processes and outcomes.

However, the tool itself is not sufficient. It has to be combined with communal reflection on the standards to be set for each indicator: How many new requests should be targeted? What proportion of attached patients should be vulnerable? An understanding of the Balanced Scorecard content and deliberation among the actors involved in the GACOs are necessary to improve the overall performance of GACOs [35‐37]. Moreover, using a combination of indicators, each with their particular strengths and weaknesses, raises the question of whether certain indicators are more important than others and whether some should be given greater weight in evaluating overall performance. Is a GACO with a large number of new requests and a large number of patients being attached to an FP, but with a small proportion of vulnerable patients in that number, performing as well as a GACO that has a small number of new requests, and hence fewer patients being attached to an FP, but of whom a large proportion are vulnerable patients? To adequately assess the performance of GACOs in Quebec, in all their complexity, it will therefore be necessary to clearly define strategic priorities and performance standards for each indicator and to adjust the Balanced Scorecard accordingly. In this way, the Balanced Scorecard can go above and beyond a simple assessment of performance and become a reflexive tool to improve GACOs’ processes and outcomes.