Skip to main content

01.12.2012 | Research article | Ausgabe 1/2012 Open Access

BMC Health Services Research 1/2012

Attempt to increase the transparency of fourth hurdle implementation in Central-Eastern European middle income countries: publication of the critical appraisal methodology

BMC Health Services Research > Ausgabe 1/2012
András Inotai, Márta Pékli, Gabriella Jóna, Orsolya Nagy, Edit Remák, Zoltán Kaló
Wichtige Hinweise

Electronic supplementary material

The online version of this article (doi:10.​1186/​1472-6963-12-332) contains supplementary material, which is available to authorized users.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

AI. independent academic reviewer nr.1, draft checklist development, draft article, corresponding author MP. head of expert panel (HTA office), supervisor of the new checklist, article revision GJ. expert panel member (HTA office), supervisor of the new checklist, article revision ON. expert panel member (HTA office), supervisor of the new checklist, checklist and article revision ER. simulation model checklist questions, expert panel member, article revision ZK. independent academic reviewer nr.2, scientific guarantor, draft checklist development, article revision and approval. All authors read and approved the final manuscript.



In middle income countries the number of trained health technology assessment specialists is limited and the public budget for health technology assessment is considerably lower compared to developed countries. These countries therefore must develop their own solutions to improve the quality and efficiency of health technology assessment implementation in reimbursement decisions. Our study aimed to develop a scientifically rigorous and detailed appraisal checklist for economic evaluations of pharmaceuticals in the single health technology assessment process.


The research design entailed a review of economic evaluations, submitted for reimbursement of pharmaceuticals, by two independent academic reviewers to identify the most common methodological problems. Fifty economic evaluations submitted in 2007-2008, randomly selected by the Health Technology Assessment Office served as data sources. The new checklist was developed by an iterative working process: first by assessing ten economic evaluations, then improving the checklist by generating new question items, then employing the improved checklist to assess the next ten economic evaluations. After appraising 25 documents, the reviewers reconciled their opinions and improved the checklist with the researchers of the Health Technology Assessment Office during an expert panel discussion. The reviewers scrutinized the second 25 economic evaluations, after which the expert panel finalized the checklist with consensus.


The final checklist consists of 91 yes or no questions in 11 main topics concerning comparator selection, efficacy, effectiveness, costs, sensitivity analysis, methodological approach, transparency, and interpretation of results. The new checklist is based on current Hungarian evaluation practice. As the published checklist will be part of the official single health technology assessment process of pharmaceuticals, submitters will be able to assure the quality of their economic evaluation.


The transparent critical appraisal method should improve the consistency of pharmaceutical reimbursement decisions and facilitate the utilization of economic evaluations in other fields of health care decision-making in other Central-Eastern European countries.
Additional file 1: Hungarian critical appraisal checklist.(DOC 136 KB)
Authors’ original file for figure 1
Über diesen Artikel

Weitere Artikel der Ausgabe 1/2012

BMC Health Services Research 1/2012 Zur Ausgabe