Creating performance intelligence for primary health care strengthening in Europe

As a starting point, we reviewed the literature for characteristics of primary care in existing frameworks, tools and surveys. We took as a basis a study by Kringos et al. (2010) being the most comprehensive review on the core dimensions of primary care to-date [26]. To update the review, we extracted priority areas and strategies of a contemporary European policy: the WHO European Framework for Action on Integrated Health Services Delivery [25]. The policy was developed and endorsed by countries and aligns with current European policies [18, 27] making it highly attuned to policy priorities in the region.

A literature review search strategy was developed to target recently published scientific and grey literature on frameworks and tools for health services delivery in general, and primary care in particular [28]. We conducted initial searches between October 2016 and May 2017 using PubMed to identify scientific literature published since 2010. We brought the existing systematic review up-to-date by searching new key terms including: avoidable hospitalization; chronic disease management; community-based care; drugs and medical devices; financial incentives; information systems; integrated care; job satisfaction; patient-centredness; patients with complex needs; population health management; responsible use of medicines; role of nurses; shared care plans; task-shifting; technology assisted care; unmet need; waste and appropriateness of care; and workload.

We hand-searched websites and databases of key international actors active in monitoring PHC, namely WHO using WHOLIS, the World Bank, Organisation for Economic Co-operation and Development (OECD) and European Commission and collated sources already known to the authors. Reference lists of relevant work identified were also reviewed and titles were searched in a snowballing approach. With the exception of global frameworks and a recent PHC initiative for low- and middle-income countries [13], work from the European context was prioritized.

Two authors (EB,DK) completed the initial document review. For each framework identified, the features of primary care, their respective classification and key definitions were extracted and logged in an Excel spreadsheet. The authors jointly carried out an analysis of the review findings to identify crosscutting themes.

To tailor the framework to the European context, we used the method of tracer conditions [29], like applied to monitoring UHC globally [16, 30]. Tracer conditions have been used in health services research on the premise that a carefully selected set of health problems makes it possible to profile the strengths and weakness of services delivery and health systems [31]. On this basis, we sought to construct a set of tracer conditions to inform the selection of indicators that–when analysed together– could serve to gauge the ability of primary care to respond to a range of health needs individually and concurrently as multimorbidities, while also measuring across population groups and life stages.

The selection of tracer conditions prioritized the following: relevance to the burden of disease in Europe; responsiveness to the strength of primary care; and representativeness of primary care’s functions. The final selection also gave consideration to the measurability of conditions and the parsimony of the set, weighing together the balanced representation of different target populations and life stages, gender importance and types of services. The overall manageability of the set was also prioritized for the selection of a core group of tracers that could serve the purpose of scoping the tool to high-priority health improvement areas.

Current global and European health policies were reviewed by two authors (EB,IK) as a proxy for priority health improvement areas. To prioritize conditions amendable to primary care, European lists of ambulatory care sensitive conditions were consulted [32], together with priority conditions included in an earlier study on the strength of primary care in Europe [12]. To achieve a comprehensive and holistic view to primary care, the type of condition (e.g. acute, chronic), relation to the life-course, gender importance and function of primary care (e.g. prevention, detection, treatment, management) were also considered. For each possible tracer condition, findings for the criteria considered were recorded.

To identify existing, internationally standardized indicators, we searched by features of the framework resulting from the first stage, in databases of international organizations (WHO, OECD, European Commission) and topic-specific databases of research consortiums e.g. cancer, medicines and tuberculosis. We reviewed existing global and European surveys related to health services delivery, patient-reported experience and outcome measures or conditions amendable to the strength of primary care. Standardized country reports by international organizations were also reviewed through searches on their respective websites. Indicators from the initial literature review and health strategies used to select tracer conditions were also extracted.

To select indicators for the framework, we prioritized the following:

We prioritized the first criterion on data availability in our selection to align to information systems and make use of the vast amount of available data, also minimizing the burden of new data collection. We undertook a detailed process of identifying sources by drawing on an existing study on health information systems [22]. We further expanded upon this to scan the availability of health services delivery data across Europe. The findings from this scan have been published elsewhere [21] and were used to ensure existing data sources are drawn from. Alternative sources where applicable, such as national documents (e.g. health policies, directives or prikaz, guidelines) or key informants, were also considered as part of asserting the measurability of indicators to merit their selection.

Indicators were mapped to the classification resulting from stage one. For the purposes of this mapping, indicator passports were developed to clearly define the scope and measurement of each individual indicator. The indicator passports draw from the literature reviewed and existing international standards where available to detail the following: alignment to the framework (domain, subdomain, feature), indicator/question title, indicator/question definition, numerator/denominator or answer choices, unit of measurement, rationale, relevant definitions, disaggregation, known limitations and possible data sources. A glossary of terms accompanies the indicator passports with emphasis put to capturing different terminology used in the European countries. The terms and definitions draw from existing international classifications including the International Classification for Health Accounts, International Standard Classification of Occupations and International Standard Classification of Education.

Country-nominated focal points representing ministries of health, health insurance funds, centres on health services or similar from 30 countries assessed face and content validity of the framework and selection of tracer conditions at a meeting in June 2017 [33]. The set of tracer conditions was also presented and validated with country representatives. They were asked to consider the relevance and comprehensiveness of the framework and tracer conditions in relation to their systems. In the same period, the framework was presented to members of WHO’s European Primary Health Care Advisory Group – a group of appointed experts to support the continued advancement of PHC [34]. All comments and discussion points were recorded and adjustments to features were made for a revised version of the framework that was then applied for the mapping and selection of indicators.

To review the indicators identified, we engaged more than 40 experts between November 2017 and June 2018. Reviewers spanned three profiles: (i) staff of WHO; (ii) experts in relevant fields from academia, think tanks and international organizations; and (iii) representatives of professional and patient associations, as well as practising clinicians.

Attention was put to ensure that reviewers were representative of countries across the region and included a range of language skills – with approximately one quarter (23%) being native Russian-speakers – and gender balance (49% females, 51% males). Nearly half of reviewers (42%) were trained medical doctors. Reviewers also included information specialists on European and central Asian countries.

Each reviewer received a written request for their feedback on a subset of indicators. Reviewers were provided the indicator passport, background on the framework and selection criteria. They were asked to score the indicator’s overall quality, provide comments or amendments and/or suggest an alternative indicator or source. One author (IK) consolidated the feedback from all reviewers. Indicators that were rated of poor quality, too vague or not meaningful for analysis were removed. For others, comments were used to revise the indicator passports. Comments included important feedback for updating indicators to current international standards as well as making explicit the limitations of indicators where known and for identifying alternative sources of data if available.

Further to technical reviews, we also conducted a preliminary test of the framework and indicators as an initial validation of its use in practice. The indicators were translated to Russian and applied in Kazakhstan on the basis of testing their applicability in the context of a Russian-speaking country and the characteristics and cultural nuances of health systems in central Asia and Caucasus countries. This process extended from December 2017 to June 2018 and included a series of workshops with national centres responsible for data collection. Further revisions were incorporated into the indicator passports and the glossary of terms.

We found a total of 50 different frameworks, assessment tools and survey instruments, recorded in Additional file 1 [35]. From the literature reviewed, we identified approximately 50 domains and 100 features of primary care. We further analysed these findings to identify crosscutting themes and to cluster related features. We noted several areas of clear consensus in the literature. For example, the contribution of primary care to first contact access, comprehensive services and continuity and coordination of care is recognized across frameworks [12, 14, 33, 36].

We developed a hierarchy for the classification of findings, adopting the classical framework of Donabedian’s structure-process-outcome logic model [37]. These components were classified and sequenced as the capacity, performance, and impact of primary care. We defined a taxonomy to describe these components from broad to specific as domains, subdomains, and features [35].

Under primary care capacity, we put attention to disaggregate the unique yet often over-looked resource needs of primary care, including the primary care workforce, medicines, diagnostics, facility infrastructure, and information system. We were also attentive to differentiate the system’s capacity, as the enabling system structures underpinning primary care, from the function of services delivery. We classified this as the model of care: the result of deliberate decisions taken that determine the contents, design, organization, management and quality improvement elements of services delivery.

This classification is a point of departure from existing frameworks. It captures the less prevalent ‘software’ of primary care’s capacity [13], such as managerial autonomy for determining staffing, budgeting and strategic planning in primary care facilities, prescribing authority of general practitioners to initiate or refill prescriptions and the existence and scale of quality improvement mechanisms, like practice audits, patient complaint systems or peer review teams. The taxonomy around the selection of services spans from identifying needs, to the selection of preventive care, diagnostic procedures, treatment and disease management services as well as services for supporting self-management. The model of care also bridges between the system on one side, and the provision of services and perspective of patients, on the other.

This domain, at the intersection with the performance of primary care, is captured as care contact. It is novel in distinguishing the ‘structural’ from its implications on performance. For example, we find skill-mix and multidisciplinary teams an agreed upon feature of primary care [12, 14, 33, 38, 39]. However, it has typically been classified as a feature of coordination. We argue the setup and structure of teams captures how providers are organized and their resulting level of interaction and joint-work rather as a measure of coordination.

In line with existing system frameworks, outputs via services delivery (access, responsiveness, safety, effectiveness) and health system outcomes (quality, equity, efficiency) were classified from a services delivery perspective, linking to the final component of impact on health outcomes (health status and well-being). Importantly, in distinguishing between outputs from outcomes, we recognize that suboptimal outcomes can be attributed to features of the health system beyond the scope of primary care services and have labelled it as such. Figure 2 illustrates the resulting framework. Like other frameworks, we acknowledge that primary care performance lies within larger socio-political contexts, though these factors are outside the scope of health systems [12, 19, 36].

Adopting the approach of people-centred systems, the framework begins with health outcomes. In this way, health outcomes are the lens through which capacity and performance are monitored. By cascading the framework’s domains, features and subdomains as shown, the taxonomy applies system thinking principles to associate connections between capacity, performance and impact. Importantly, as the field of systems thinking has revealed, changes to systems can have unpredictable and multi-direction results [40].

Fourteen disease-specific strategies actively being implemented in Europe and globally were identified and reviewed (Additional file 2). We screened 32 conditions, identified first and foremost as priority health improvement areas in Europe and selected a set of 12 grouped in 7 clusters based on our selection criteria. These conditions span: reproductive, maternal, neonatal and child health; communicable diseases; cardiovascular diseases; diabetes; respiratory diseases; cancer; and mental health (Table 1).

The selection process included a review within and across clustered to assess the conditions in combination in order to gauge the balance across the criteria applied and the manageability of the set. Guided by our primary aim to select a core set of tracers for profiling primary care, we prioritized one or two conditions per cluster, with one exception (cancer). In a last stage, we also considered the measurability of conditions in primary care [21], resulting in the exclusion of those specific to ageing, like dementia. Nonetheless, ensuring the representativeness of varied population groups and life stages was among the core selection criteria applied. For example, a range of primary care services for older adults are included such as, influenza vaccines (prevention), colorectal cancer (detection, management) and cardiovascular diseases (prevention, detection, treatment, management).

More than 20 international databases were reviewed to extract existing and reported on indicators. Fifteen global and European surveys were also reviewed on topics including medicines and e-health, patient experience, primary care doctors, and on tracer conditions like influenza and NCDs. An additional 8 standardized country reports were identified e.g. Health Systems in Transition series, OECD country reviews and WHO country assessments.

From this, we consolidated a total 243 indicators/questions. We plotted the indicators in our classification of subdomains/features. When mapped, we found the largest number of indicators for the structure domain (health workforce, financing) and health impact (morbidity, mortality), consistent with their wide use in health system performance frameworks [20]. The model of care domain was the least populated from this initial mapping and further targeted reviews were conducted to identify indicators for its features. In a final stage of development, we reassessed the initial classification with the indicators identified for refinements at the feature-level. Following this, three features remained without assigned indicators: acceptability, equity, and responsiveness.

We applied the tracer conditions to scope the selection and explicitly link indicators across the framework using theory of change principles as described [41]. In doing so, health outcomes are preceded by related performance and capacity indicators. For example, impact indicators on diabetes link to preceding diabetes-related performance indicators e.g. hospitalizations, managed insulin-levels, and capacity indicators e.g. prevention services for diabetes, existence of patient registries. Indicators that measured conditions outside the scope of the selected tracer conditions were excluded. A balance in the number of indicators for each tracer condition was also sought. We retained relevant disaggregations, such as public-private mix and rural-urban status for analysis purposes. The full set of indicator passports is available electronically in Additional file 3 and related glossary of terms in Additional file 4 [35].

We consolidated our findings in a tool we refer to as the Primary Health Care Impact, Performance and Capacity Tool (PHC-IMPACT). The tool includes a total of 139 indicators mapped to a detailed framework in a hierarchy of domains (6), subdomains (26), and features (63) as summarized in Table 2 and expanded in Table 3.

The measurability of each indicator has been safeguarded through an extensive process of identifying a possible source for each. More than half (61%) of the indicators can be sourced from more than one type of data source, increasing the potential measurability across countries. The range of possible data sources include: national and international databases (43 indicators); existing surveys (11 indicators); policies (27 indicators) and reports (10 indicators); and key informants (62 indicators) (Table 3).

This research set out with the aim to create robust performance intelligence for PHC strengthening in Europe. Through the three-staged research process described, the tool has been tailored for a classification, set of tracer conditions and selection of indicators that are sensitive to primary care, policy priorities and information systems in Europe.

The broad suite of indicators is intentional in order to allow the possible tailoring of the indicators, functioning as a menu of options to be selected on the basis of a country’s policy priorities. The customization of the tool is among its core advantages and an important feature to increase the tool’s responsiveness within and transferability across countries. Other unique features are found to include the following: diversified data sources accommodate a range of perspectives for a more holistic view to primary care; the translation and means testing of the tool in Russian attempts to redress context-specific classification challenges unique to the membership of the WHO European Region; the cascading of the taxonomy developed facilitates varied entry points to analysis and a uniquely detailed approach to capture the model of primary care; and, the indicator passports and glossary of key terms developed are a practical resources for data collection.

Prior to the use of the tool, a review of the indicators for further tailoring to the context of a specific country should be conducted. The selection of tracer conditions should also be reviewed, with the possibility to adjust this list given a country’s health priorities. This process of customizing the tool’s suite of indicators should rely on the meaningful engagement of key stakeholders for the full benefits of co-designing with the intended users.

The relevance of existing indicators for use in monitoring primary care was a significant limitation. For example, nurses in primary care are not defined in existing databases, surveys or reporting, ultimately limiting the extent to which the number, profile, role and performance of nurses in primary care can be assessed. We have attempted to redress these limitations with adjustments to standard indicators using existing definitions and classifications to improve their sensitivity to primary care. In particular, we have harmonized the varied terminology for the primary care workforce, types of facilities and levels of education that have previously limited the relevance of frameworks and tools across Europe. The glossary of terms developed supports the classification defined [35].

We also found metrics for equity and responsiveness to be limited. A similar finding was reported in a recent review on health system performance assessment frameworks [20]. We recognize the need to operationalize both in a more comprehensive manner, suggesting the use of composite indicators and highlight acceptability as a feature in need of further development. The disaggregation of indicators existing in international databases was retained where possible and includes gender, age, socioeconomic status and rural–urban classifications. These variables are found especially relevant for the analysis of equity. An approach that links the relevant indicators and disaggregations is suggested for a more holistic appraisal of equity considerations.

The tool is found novel in its attempt to capture the capacity of primary care beyond system structures, with indicators like prescribing authority to assess the ability of general medical practitioners to issue initial prescriptions and/or refills for treatment and the autonomy of managers on planning, staffing and budgeting. These features of the model of care, together with the domain on care contact, are found important specifications for depicting primary care that allow for further analysis and comparisons across countries.

To further improve the sensitivity of the tool, answer choices where applicable were disaggregated to capture responses as country-wide, in some regions or pilot status.

Collecting data should rely as much as possible on existing international databases, surveys and country reporting. The development of an electronic data processing system is already being explored to build linkages to existing databases and create a common living platform for interactive analysis. For other indicators, there is untapped potential to uptake data from national information systems, in particular on hospitalizations for ambulatory sensitive conditions where reporting for many non-OECD, non-EU countries of Europe is out-dated. For these countries, strengthening linkages with national systems should be prioritized.

An electronic questionnaire is being built to serve as a data collection tool for qualitative indicators. Where key informants are a data source, we suggest diversified informants to capture varied profiles (policy, managerial, clinical) for accurate responses on the tool’s range of topics. By soliciting a range of informants with complementary knowledge, this approach also facilitates multi-stakeholder engagement while ultimately yielding more objective and reliable results.

A modular approach could also be taken to data collection for a focus on topics like mental health, maternal and child health, and out-of-hours services based on a country’s priorities. While the tool has prioritized available data, the extent to which indicators are measurable across countries varies making some features more ambitious and future-oriented in some contexts. In instances where preferred databases or survey sources are not available, and the indicator cannot be answered by one informant, we suggest a pragmatic approach is taken using the method of expert consensus. This method is applicable for one quarter (27%) of the total indicator set. Drawing from established group-based methods, a Delphi technique followed by a consensus workshop could be used to generate estimates. The highly structured method preserves anonymity while capturing a range of perspectives. This data could support meaningful analytics as countries aspire to and work in parallel on advancing the use and alignment of data in their national information systems. Lessons on expert consensus methods in health services research could be explored for use here [42‐45].

PHC-IMPACT has been designed with view to the information needs of decision-makers in the WHO European Region both for international monitoring and country-specific priority-setting. The merit of a regional approach to monitoring has already been advocated for SDG 3 in the scope of financial protection [46]. We have excluded conditions or services considered a basic expectation or fundamental to primary care, such as family planning and childhood vaccination. The exclusion of these services is a prioritization of the tool’s sensitivity to disaggregate performance in high- and middle-income countries rather than a statement of their importance. Moreover, disease- or service-specific monitoring tools and instruments are in use for this purpose.

The comprehensiveness of the tool’s taxonomy brings depth to the analysis for country-specific use. It is seen as a vehicle for identifying priority areas for improvement, while also shedding light on the information landscape and overall availability of data and actors involved. For analysis purposes, the tool facilitates linkages along different continuums: a clinical continuum, linking prevention, diagnosis, treatment and disease management for tracer conditions; a performance continuum in the classical structure-process-outcome sequence; and a continuum of stakeholders cascading the delivery of services from the micro-level (health professionals, patients), to meso- (managers, regional health authorities) and macro-level (policy-makers, health insurers). These relations between indicators have strong analytical potential to improve the coherence of PHC strengthening and signalling of policy opportunities to accelerate performance gains.

For example, the availability and provision of services in primary care can be assessed across to the care continuum to gauge the extent to which the full continuum of services are available and if not, where gaps in services provision lie (i.e. lack of risk assessment services for cardiovascular diseases or mental health; limited role of primary care in follow-up for tuberculosis. This analysis draws on the indicators related to the selection of services. In a similar approach, the role of different primary care practitioners can be profiled, clustering the provision of services by general medical practitioners, nurses, or specialists working in primary care, among others. This analysis has the potential to shed light on the different roles and scope or practice of primary care practitioners, including important insights into the potential for general medical practitioners to confirm an initial diagnosis or role nurses in risk assessment or follow-up services in primary care. Other thematic clusters of indicators could aggregate indicators by policy-relevant themes such as patient engagement, out-of-hours primary care, integrated health and social care, prescribing practices in primary care, among others still.

The tool should be treated as a living resource to be adjusted and improved upon as new types and sources of data become available. This includes the continuous improvement and development of indicators that are defined according to global standards such as total primary health care expenditure and the UHC services coverage index. It requires piloting beyond the initial country-validation described to test its approach and robustness for country-specific, cross-country comparisons and use overtime. Ultimately, despite its attempt to be comprehensive, not all complexities can be accounted for.

Further to piloting PHC-IMPACT, areas for continued development include: developing composite indicators for features like equity and responsiveness and priority policy areas like scope of practice; selecting a core set of indicators for use in dashboards for the purposes of international benchmarking; improving metrics for hard-to-measure topics, like medicines, primary care workforce and acceptability; exploring methods of expert consensus; and intensifying data collection from existing sources and for newer indicators like patient reported experience measures in primary care. Patient experiences measures are found an important area for further research and development to allow cross-country comparisons with the necessary adjustments for their wider use in eastern European and central Asian countries. Tools developed in the scope of the Patient-Reported Indicators Survey (PaRIS) Initiative of the OECD are one platform that could be coordinated with and tools adapted from for use in the context of countries across the European Region [47].