Primary health Centres’ performance assessment measures in developing countries: review of the empirical literature

An initial search for articles from the following databases was carried out: Hospital/Health care (Pubmed, Embase), Psychology (JSTOR, PsycINFO) and Business (Emerald Insight). Endnote7 software was utilised to download and select articles. As the aim of the review was to include articles on PHC performance assessment in developing countries that are published in English, the search terms included ‘primary health centre’ or ‘health centre’ and ‘performance assessment’ or ‘performance’ and ‘developing country’ {(primary health centre or health centre) and (performance assessment or performance) and developing country}. The period of publication for the search was from the year 1979, a year after Alma Ata up to December 2014. Further, an additional search was done from January 2015 to October 2016 to include recent articles. The search terms were tested and Mesh terms were employed during search when available in the database. The search results from the databases identified were combined to eliminate duplicates. The details of the search strategy are given in Additional file 1: Appendices 1 and 2. A similar search was conducted in Google scholar and the articles fulfilling inclusion criteria were included for full review (Fig. 1).

The inclusion criteria employed were:

Articles that reported only on methodology of research, discussion on performance/performance assessment and review of literature were excluded. The authenticity of this review was ensured by including only published empirical articles, avoiding grey literature and reviews. On the basis of these inclusion and exclusion criteria, the articles were screened first by title, then by the abstract and full article, to select the relevant articles for full paper review.

Next, the selected full articles were assessed for quality. The complete evaluation of methodological characteristics as suggested by COSMIN and Noben and colleagues could not be applied, as methodologies of the articles selected were not advanced enough [27, 28]. Hence, a simpler instrument was developed, based on the quality of the description and the quality of the measurement property (the reliability and validity) itself. It is theoretically possible that the measurement property is weak although the description is good [28]. With our instrument for assessing the quality of description, the articles were scored as:0, if there was no description of the measurement property;1, if only a few points of the measurement property were described (little information); and 2, if there was a detailed description of the measure in the article (good information).

The quality of the measurement property was assessed based on four criteria:

The quality was scored as follows:

The results were relegated to a lower number if the higher criteria were not fully met or exclusively mentioned in the article. The qualitative article was scored between 0 and 2 as its measures would be unique, based on the theoretical framework used (scores 3 and 4 were irrelevant). The papers were analysed for quality by the first two authors independently, the differences were discussed and agreed upon.

The selected full articles were analysed and reviewed to understand and describe how the performance of PHCs had been assessed in developing countries. All selected articles were studied to understand how they had defined performance and the perspectives of the performance assessment (provider or patient). The topographical and content analysis of measures was done and these were compared with the measures of WHO performance assessment framework for comprehensiveness [29].

No ethical approval was needed for this narrative review.

After downloading the articles (4359 articles) using the search strategy described above, duplicates were removed, inclusion and exclusion criteria were applied systematically, which resulted in the selection of eight articles from the initial search and three articles from the additional search for the review. Further, searches from the reference tracking and in Google Scholar resulted in another four articles (Fig. 1). Thus, in total fifteen articles were included for quality assessment and content analysis (Table 1). Table 1 provides an overview of these fifteen articles.

Fourteen articles had a good description of the measurement properties. The quality of the measures varied. Measures in eleven articles scored 3, two articles scored 2 and two articles scored 1 (Table 1). None of the measures was tested for validity with good results.

Empirical research on PHC performance measures was published from Brazil, India, Papua New Guinea, Egypt, Bangladesh, Armenia, Malawi, Tanzania, Rwanda, Serbia and other developing countries. A descriptive analysis revealed that PHC performance was distinguished as individual staff/personnel performance and that of the centre/health system performance with a focus on consumer’s and/or provider’s perspectives. Figure 2 depicts the perspective and level of assessment.

Nine articles included in this review used quantitative methods, one article had exclusively employed qualitative methods and five had employed mixed methods. The methods of data collection included were interviews with clients, health workers, key informants and/or stakeholders [30‐38], focus group discussions [30], direct observation [35, 39], facility-based survey [34, 35, 40, 41], and secondary data analysis (records on costs, infrastructure, service provision, health indicators, number of patients served and other patient details) [30, 31, 34, 39, 42].

Five studies evaluated the personnel performance assessment (nurses, midwives, management team, community health workers and medical professionals). The assessment considered competency/clinical services and non-competency/non-clinical service-based components. The personnel performance was assessed using the Quick Investigation of Quality tool [34] (Fort and Voltero 2004). Adherence to the protocol was the most common method for competency based personnel performance assessment [30, 33, 35, 36, 38]. For the non-competency based personnel performance assessment, the measures used were punctuality, response time, absenteeism, time of reporting to work, time spent with patients, waiting time and cleanliness [33, 36]. Reproductive healthcare delivery was evaluated in all the papers [30, 33, 35, 36, 38].

It was found that autonomy of the team, availability of manpower, clarity in the job description, roles and responsibilities, working conditions, workload and environment, the level of motivation, their education and training along with good supervisory practices contributed to the performance of healthcare personnel [33, 38].

Of the 15 articles selected, 9 studies assessed the performance of the centre or the health system [31, 32, 34, 36, 37, 39‐44]. Satisfaction of patients, community, care providers and other stakeholders were the most common measures utilized for centre/health system performance assessment. The performance of the health system was assessed from the provider’s perspective (4 articles) [31, 34, 37, 39], the consumer’s perspective (4 articles) [32, 40, 42, 43] and both perspectives (2 article) [41, 44] at various levels.

The target groups in the studies were individual patients, local community, district or country level. Health care personnel such as nurses, doctors and health managers along with other key stakeholders like local governing bodies were included. From the respondent’s perspective, performance was defined as satisfaction, accessibility of the centre, case detection and success of the treatment provided in the country, care experiences, establishment of patient’s bonds for treatment success and organizational care. Centre performance assessment was assessed as provider and patient bonding leading to therapy compliance and treatment success. This was also assessed in terms of costs and effectiveness of the services provided, the satisfaction of the providers and other stakeholders on the availability of resources and functioning of the centre. The scope of the centre performance assessment was at the structural levels of centre, community, district or country.

The measures described in the articles were compared with the six aspects of the WHO health systems performance assessment framework (Table 2) [29]. In the articles included, measures for three aspects (overall level of responsiveness, distribution of responsiveness and distribution of resources) were considered in relation to specific diseases/services, though the WHO framework uses system-specific measures. Measures for disease/service specific WHO aspects (overall level of health, distribution of health in population and distribution of financial contribution) were less well represented in the fifteen articles.

Personnel performance was based on observation methods, assessed competency and non-competency based tasks. This correlated highly with patient satisfaction, an indicator of centre performance, as well as that of availability of resources, support and culture of the organization [45]. The investigators used the ‘Quick Investigation of Quality’ tool that had been validated earlier [45, 46]. Such validated tools for observation of professional performance provide a quick and effective method for assessment of PHC personnel performance in a developing country. As reproductive and child health is one of the main focus areas of primary healthcare and an important health indicator, it is hence appropriate to use this in the evaluation of PHC performance as well.

The performance of the centre was assessed using the Primary Care Assessment Tool [40, 41, 43]. This tool was validated in a developing country for family healthcare [41, 47]. It is interesting to note that this instrument could be used to assess the performance from the consumer’s as well as from the provider’s perspective. This tool provides an extensive list of surveys on different attributes of PHC. Based on the requirement of the evaluation, one could consider specific components. In other studies, measures such as costs, patients served and effectiveness along with the client satisfaction were identified. Cost effectiveness is an appropriate measure that can be used across all PHCs [48] and thus, helps in making right choices by identifying the most effective service or intervention or centre [48].

Satisfaction of various stakeholders including patients and providers is an accepted standard in performance assessment as they are interrelated. Community participation in the functioning of the PHC highly influences the performance of PHC and its personnel, so it could be included as a component in the PHC performance assessment [34].

The current practices of PHC performance assessment in developing countries were analysed with the WHO framework for performance assessment [17, 29]. The WHO framework is completely focused on output and outcome, with the structure and process considered intrinsic to the system. However, for ease and clarity, many authors have used the Donabedian model for assessing performance in primary health care [19, 23, 26, 49]. The current review highlights the limited representation of the performance measures in relation to aspects of the WHO framework (Table 2). In the articles, even the tools used to assess personnel performance (Quick Investigation of Quality tool) and centre performance (Primary Care Assessment Tool) did not represent measures from three aspects: overall level of health, distribution of health in the population and the distribution of the financial contribution [40, 41, 47].

Due to lack of resources and data [10, 18], covering the complete list of WHO aspects to the full extent will be difficult for developing countries. However, a concise list of measures with an appropriate representation of six WHO aspects and requiring minimum resources and data, needs to be developed for PHC performance assessment in developing countries. If the assessment uses standardised measures useful comparisons across regions/countries would be possible. Since the search results indicated that a very small number of articles were published, further research needs to be conducted in the developing countries on PHC performance assessment, enabling cross learning and knowledge base enhancement [21].

Although the number of articles in this review was limited, the studies covered diverse countries and continents. Assessing the quality of the papers was a challenge as articles used qualitative, quantitative and mixed methodology. The articles demonstrated reliability and validity methods such as correlating provider and user experiences, data/method triangulation, standardised tools/training of researchers. Hence, there is a need for research in developing countries to establish quality of measurement and standardisation of PHC performance assessment using rigorous statistical methods.

PHC performance assessment in developing countries is an emerging field but it is fragmented at present. Though human resource for health is a component of the WHO framework, the various sub-components to be included are not clear. Evaluation of the PHCs is an on-going exercise, but yet there is no established standard for assessing performance. Studies have shown that the performance of PHC depends on several things, one critical factor being personnel performance [21, 29, 50‐52].

What is measured, can be controlled. However, not all aspects of PHC performance were yet covered. Measures such as skill sets for execution of healthcare focusing on mother and child health, are already well available. They were emphasised in five studies of our review and this is in line with Millennium Development Goals and Sustainable Development Goals [1, 2]. However, other aspects like coverage/distribution of health, overall health of the population, distribution of financial contribution, including program specific achievements, various service components and process of primary care delivery are still not covered. The performance measures should be appropriate and adequate enough to enable accurate assessment on an ongoing basis to aid in monitoring and efficient management of the personnel/system. There is urgency to develop new and additional measures for PHC performance in developing countries. PHC performance is a matter of immense importance for policy agenda and political priorities [53].