Dietary protein intake and health-related outcomes: a methodological protocol for the evidence evaluation and the outline of an evidence to decision framework underlying the evidence-based guideline of the German Nutrition Society

Traditionally, nutrition research has primarily focused on the health impact of dietary carbohydrates and dietary fats. Accordingly, previous guidelines developed by the German Nutrition Society (Deutsche Gesellschaft für Ernährung, DGE) addressed health-related recommendations for the consumption of dietary fat [1, 2] and dietary carbohydrates [3]. Today, there is also an increasing number of studies investigating the impact of dietary protein intake on health-related outcomes, allowing the establishment of an evidence-based guideline on dietary protein intake. This upcoming guideline will focus on the role of dietary protein intake for the primary prevention and risk modification of various non-communicable diseases. Upon starting the development process for the evidence-based guideline on dietary protein intake, the existing methodology applied for the previous guidelines of the DGE had to be adapted to recent developments in the field. These novel aspects relate to the availability of topical evidence evaluation tools and the increased availability of systematic reviews (SRs) with or without meta-analyses (MA). A further novelty of the revised methodology includes new working principles, such as the four-eye principle and the outline of an Evidence to Decision (EtD) framework [4, 5]. Accordingly, the objective of the current protocol was to develop a revised methodological procedure for the systematic literature reviews and for the deduction of an evidence-based guideline for dietary protein intake. The key question to be addressed in this guideline to assess the overall certainty of evidence was defined as follows: Does dietary protein intake with regard to quantitative (higher vs. lower dietary protein intake) and qualitative considerations (total, plant-based, animal-based or supplemental protein intake) affect the development of selected health-related outcomes (i.e. blood pressure, body weight and other body weight-related outcomes, bone health, cancer, cardiovascular diseases, kidney health, muscle health, type 2 diabetes mellitus, inflammatory bowel disease) in the general adult population? For addressing this research question, this publication therefore describes in detail the novel methodological procedures of the literature reviews for the assessment of the overall certainty of evidence. The following steps for the derivation of summary conclusions and recommendations based on an EtD framework will only be outlined in general. Corresponding details will be published separately.

A guideline panel consisting of DGE staff members, DGE scientific board members and selected experts was constituted in 2016 to develop the evidence-based guideline on dietary protein intake. This panel selected a panel coordinator (BW) and reached a consensus-based decision on the outcomes to be addressed in the guideline, taking aspects, such as disease burden and expected relevance of protein intake, into account. Since the amount of protein reported in observational studies or used in intervention studies varied notably, the panel refrained from a strict unifying definition for high or low protein intake (such as above or below 0.8 g/kg body weight per day). Instead, within each chapter, the link of higher and lower protein intake as reported or used in the included studies to the specific endpoint will be evaluated. A methodology task force (AK, AEB, LS) was set up to delineate the procedures to be followed for establishing the guideline. A working group (AS, AMA, NK, AL, JH) was commissioned to perform the literature search and selection, data extraction and assessment of methodological quality, and outcome-specific certainty of evidence. Conflict of interest statements were obtained from all members of the guideline panel and published (Supplement 1).

In consideration of the increased availability of SRs with or without MA, it was primarily decided to develop an approach based on systematic searches for these study types and to perform umbrella reviews for each of the nine selected health-related outcomes. A continuous publication of each successively completed umbrella review, from now on referred to as guideline chapter, and a final overall recommendation based on these guideline chapter conclusions, were set out to be the backbone for the protein guideline. The evidence-based guideline on dietary protein intake was registered as an umbrella review in PROSPERO (CRD42018082395) [6].

Like the previous approaches to derive the guidelines of the DGE [7], a multi-step scheme consisting of nine steps constitutes the backbone for the methodological procedures. Figure 1 presents this nine-step scheme for the development of the DGE’s evidence-based guideline for dietary protein intake. Hereafter, each single step is outlined in detail.

Today, an overwhelming body of nutrition research study data is available which can hardly be critically evaluated and made applicable by a single user, be it a research scientist or a practitioner. The systematically developed and evidence-informed dietary guidelines of the DGE should provide a basis to assist nutrition experts as well as other experts to make appropriate professional decisions in their respective fields. Similar endeavours have been described by methodologists, pointing out the need to provide trustworthy nutrition guidelines by adhering to internationally accepted and transparently described literature review standards [17].

The upcoming guideline for dietary protein will reflect the current knowledge on quantity and quality of dietary protein intake and the risk association with several health-related outcomes (blood pressure, body weight and body weight-related outcomes, bone health, cancer, cardiovascular diseases, kidney health, muscle health, inflammatory bowel disease and type 2 diabetes mellitus). Importantly, in addition to the overall certainty of evidence, several additional aspects will have been considered in the overall recommendations on dietary protein intake, including considerations on problem priority, benefit and harms, dietary intake levels and preferences, ecological sustainability of a certain protein intake level and financial costs of recommendations, acceptability, feasibility, and potential impacts on health equity. Finally, the results of an open consultation will be handled before the final guideline on protein intake is published. To that end, this publication described in detail the overview on the procedures to assess the overall certainty of evidence, and outlined cornerstone aspects of an EtD framework that is to be adapted to the current GRADE EtD frameworks [14]. The GRADE EtD framework is a comprehensive and rigorous guideline approach that is considered to strengthen transparency, trust, and credibility of resulting guidelines [12]. Once finalised and applied, the EtD framework allows to inform about judgements made, including the evidence supporting those judgements [5]. By providing this detailed methodological description the necessity and demanded transparency of such processes [17] is assured.

The multi-step procedure followed in previous guidelines of the DGE (e.g., relating to carbohydrate [3] and fat intake [1, 2]) had to be adapted according to recent developments in the field of evidence synthesis and interpretation. In contrast to the previous guidelines of the DGE, the current approach is based on umbrella reviews while primary studies will not be included. We therefore had to adjust the method for the quality assessment of the included SRs. AMSTAR 2 [10] and NutriGrade [11] are applied to assess the methodological quality and outcome-specific certainty of evidence, respectively. Grading the outcome-specific certainty of evidence is considered of fundamental importance as it can improve the trustworthiness of findings [12]. For grading the overall certainty of evidence, a scheme was developed, accommodating nutrition research-specific aspects, including for example “biological plausibility”. These criteria extend the scheme used in previous guidelines of the DGE and are now adapted to the newly applied study quality and certainty assessments. By setting the criteria to be fulfilled for classifying the certainty of evidence into one of four categories, a standardised procedure for deriving conclusions on the relation between dietary protein intake and each of the addressed disease outcomes was strived for.

After finalising all umbrella reviews, the final steps of the guideline development will be entered by completing the EtD framework: all derived conclusions of the single, disease-specific umbrella reviews are to be summarised by weighting their relevance for the target population according to epidemiological, clinical, public health and/or ecological aspects, and to formulate overall recommendations. As this step will still take some time to be performed, only cornerstones of this process have been presented here and details are still to be developed and then published. This will allow to rely on the most recent developments in this field.

The strengths of this methodological protocol include a commitment to internationally accepted guideline development standards, such as performing the literature search in different databases, consideration of evidence from SRs with or without MA, evaluation of the certainty of evidence and use of an EtD framework (including aspects of sustainability). Adhering to these standards will ensure that the derived recommendations will be based on a high quality, novel overview of systematic literature reviews. Nevertheless, we are aware that including solely SRs with or without MA involves the potential risk of overlooking evidence from (recently published) primary studies. When setting up the methodology, we assumed that for the majority of guideline chapters the amount and nature of SRs will likely be sufficient to conduct umbrella reviews and that they will be sufficiently up to date.

The dietary recommendations derived in the guideline of the DGE on protein intake will be put forward by a group of experts in nutritional physiology, public health and clinical nutrition, nutritional epidemiology, and evidence synthesis methods. Respective details from each member of the guideline working group are made available (Supplement 1), thereby disclosing potential conflicts of interest.

The described methodological procedure has various limitations. The most relevant general aspect to be discussed refers to the predominantly available non-randomised studies, e.g. prospective observational studies, which have been repeatedly criticised to be prone to various biases and (residual) confounding [12]. To address this, vigorous application of respective assessment tools has been recommended [12]. Here, AMSTAR 2 is used for the methodological quality assessment of the retrieved SRs including risk of bias assessment, and NutriGrade to evaluate the outcome-specific certainty of evidence for each SR, also addressing risk of bias. It has, however, to be conceded that the guideline panel decided to evaluate the methodological SR quality using a modified version of AMSTAR 2 and to combine it with the outcome-specific certainty of evidence assessment/rating to obtain a complete picture of the quality of the included publications. We are aware that the approach to incorporate the methodological quality of SRs into the overall evidence has been criticised by the GRADE working group [18], as it was argued that it is problematic and misleading to include the quality of a SR as a factor to determine the certainty of a body of evidence. We thus follow the proposal of this working group to exclude low-quality SRs (better called a non-credible SR) from the overview of reviews [18]. In addition to the necessity of such assessments, it has been shown that a separation of the certainty of the evidence assessment on the one hand from confidence in recommendations on the other hand enhances clarity of proceedings and strengthens credibility of dietary recommendations although the certainty of evidence itself is not increased [12]. Such a separation is part of the here described procedure. A further limitation might be seen in the modification of answer options, e.g. of the original question 7. To abstain from demanding an explicit list of studies with exclusion reason while a flowchart is available is considered to be a pragmatic decision, given the available SRs and the rather conservative overall certainty rating.

Furthermore, both SRs and umbrella reviews as such hold limitations. Although SRs potentially provide stronger evidence than individual studies alone, methodological and statistical heterogeneity of the included studies represent, among others, difficult to solve challenges. In terms of fundamental approaches to umbrella reviews, instead of searching for the various SRs available, only the best or latest could be considered. Given the considerable overlap of SRs on a specific topic, this could be an option. However, since we aim for a comprehensive summary of the current body of SR evidence, it is preferred to include all SRs regardless of overlap. This decision is supported by the recently revised chapter of the Cochrane handbook “Overview of Reviews” [19].

A comprehensive protocol for systematic literature reviews relying on umbrella reviews and cornerstones of an EtD framework has been developed for the derivation of the upcoming evidence-based guideline for dietary protein intake of the DGE. Once the recommendations of the guideline are derived and published, nutrition and health experts as well as public health policymakers may profit from these guidelines; its implications include a better understanding of the desirable and undesirable associations/effects of protein consumption and provide consequently the option to promote a better-informed decision-making by the general population.