Development and validation of a brief general and sports nutrition knowledge questionnaire and assessment of athletes’ nutrition knowledge

The research was approved (S16/267) by the La Trobe University’s SHE College Human Ethics Sub-Committee (SHE CHESC). All participants were provided with the participant information statement and consent form (online) and ‘agreed’ to participate (electronically).

The data used to develop the abridged version of the NSKQ was the same data that was collected for the final stages of the full-length NSKQ validation. Participants (n = 181) were Australian athletes and university students, recruited from November 2016 to January 2017 [9]. The sample was predominately female (75%), aged 17–25 (52%), born in Australia (81%), and university educated (96%) (Additional file 1: Table S1).

A detailed description of the development and validation of the full-length NSKQ are described elsewhere [9]. The full-length NSKQ was developed in accordance with methods recommended by Trakman et al. [6], and the validation of the A-NSKQ was based on a modified version of these methods. In the validation of the A-NSKQ no new items were developed (steps 1–3) and face and content validity (steps 4 and 5) were not re-assessed (Table 1).

Item analysis (step 6) was performed using Rasch analysis only. Rasch analysis was conducted using RUMM2030 Professional software. The aim of this step was to remove items that were causing misfit to the Rasch model. Misfit was assessed based on overall item-interaction, overall person-interaction and overall item-trait interaction. A SD of 1 and mean of 0 for overall-item and overall-person interaction, and non-significant Chi-Square probability for overall item-trait interaction indicate compliance with the Rasch model (i.e. no misfit). In order to determine which items (or persons) were causing misfit, the following indicators were evaluated: individual item fit residuals, item characteristic curves (ICC), category probability curves (CPC) and differential item functioning (DIF). Detailed definitions and interpretation of the aforementioned statistics are beyond the scope of this report, but are described elsewhere [6, 12] and summarised in Additional file 1: Table S2. Stricter criteria were applied during item analysis for validation of the A-NSKQ. During the validation of the full-length NSKQ the researchers retained certain items that did not meet one or two of the aforementioned Rasch indicators because they were deemed relevant in terms of assessing gaps in NK and retaining content validity. However, in the validation of the A-NSKQ, all items that failed on a single indicator were excluded.

Assessment of internal reliability and dimensionality (step 7) were also performed using Rasch analysis only for the validation of the A-NSKQ. Reliability was assessed using the PerSepIndex, a summary statistic produced by RUMM 2030 that is analogous to Cronbach α. Both PerSepIndex and Cα are based on repeated split-half reliability assessment; a value of ≥0.7 is an acceptable measure of internal reliability [12]. Dimensionality was assessed using the perc5% statistic; values of 5% or less indicate unidimensionality, a requirement of the Rasch model [20].

Gathering of new data to re-examine the scale (step 8) was not conducted. Rather, a ‘proxy score’ for each respondent was calculated by subtracting the deleted NSKQ items from participants’ original score. Similar methods were used by past researchers [22] to compare scores on the original and revised General Nutrition Knowledge Questionnaire (GNKQ/R-GNKQ). Using the proxy scores, construct validity was assessed using the known-group comparisons method and temporal stability was assessed using the test-retest reliability method (Table 1). For the known-group comparisons assessment, NK (proxy) scores of individuals with and without a formal nutrition education (university course, university subject, diploma, and online course) were compared; statistically significant differences are indicative of construct validity [6]. For the test-retest reliability assessment the correlation of NK (proxy) scores on two attempts of the questionnaire, taken ten days to two weeks apart, were assessed; a correlation of ≥0.7 indicates temporal stability [6]. The correlation between NK proxy scores and original scores was also assessed.

The original NSKQ consisted of 89 items. Rasch analysis showed that three items had high fit residuals, 19 items had problematic CPCs, 38 items had problematic ICCs, and one item showed DIF (for country). Nine items failed on two indicators, therefore, in total, 52 items were problematic and were excluded from the A-NSKQ, leading to a 37-item questionnaire.

The A-NSKQ was not unidimensional (Perc5% = 7.39%). The two sub-sets of items that were most different from each other, based on principal components analysis (PCA), were not theoretically related. Therefore, the investigator (GT) divided the items into two sub-sections based on their content. The first sub-section (‘general nutrition knowledge’) included 17 items that assessed knowledge of energy density; the role and sources of macronutrients and micronutrients; and alcohol. The second section (‘sports nutrition knowledge’) included 20 items that assessed knowledge of athletes’ macronutrient and fluid requirements; weight loss and gain strategies for athletes; and supplementation for athletes. These two sub-sections fit the Rasch model and were unidimensional. Reliability of the whole scale (PerSepIndex = 0.8) and the SNK (PerSepIndex = 0.7) were acceptable; the value for the GNK (PerSepIndex =0.6) was below the requisite value of 0.7 (Table 2).

Individuals who had undertaken nutrition studies scored better in the GNK and SNK sub-sections (nutrition students = 77% versus non-nutrition students = 60%, P < 0.001 [GNK] and nutrition students = 60% versus non-nutrition students = 40%, P < 0.001 [SNK]), indicating construct validity. Test re-test reliability was demonstrated based on the proxy scores (r = 0.8, P < 0.001 [GNK] and r = 0.7, P < 0.001 [SNK]). There was a strong, positive correlation (r = 0.9, P < 0.001) between individuals’ score (%) on the NSKQ and their proxy score on the A-NSKQ. This indicates the removal of the items did not alter overall total (%) NK scores.

Two-hundred-and-seventy-three athletes followed the link to complete the A-NSKQ (response rate = 7%). The individual response rate of the NSKQ could not be calculated for non-elite athletes, because the questionnaire was distributed by a second party and the total number of invitations was not known; the response rate for team presidents who agreed to distribute the NSKQ was 8–13% (Additional file 1: Table S2).

After deleting responses of individuals who followed the link but did not agree to the participant information and consent form (n = 65; not included in completion rate calculations), incomplete responses (n = 22) and respondents who did not meet the eligibility criteria for age (n = 9), there were 177 usable responses. The completion rate for the A-NSKQ (85%) was higher than the average completion rate for the NSKQ (54%); completion rate calculations were undertaken in the same manner (Additional file 1: Table S2).

The median time taken to complete the A-NSKQ was 12 min, compared to 25 min taken to complete the NSKQ.

The characteristics of the participants who completed the A-NSKQ are outlined in Table 3. The sample was predominately female (61%) and aged 17–25 years (48%). Despite being distributed to the database of a recreational AF and netball league, 23% of athletes stated the main sport they played was not AF or netball, and 19% of athletes reported competing at the state, national or international level.

There were no differences in age, gender, country of birth, level of education and previous nutrition study between those who reported playing at the elite and non-elite level. However, individuals who reported playing a sport other than AF as their primary sport were more likely to play at higher levels of competition (X ²  = 104.526, df = 1, P < 0.001). Females were more likely to play a sport other than AF (X ²  = 4.637, df = 1, P = 0.031) and to be university educated (X ²  = 0.337, df = 1. P < 0.001).

The 37-item A-NSKQ can be completed in around half the time taken to complete the NSKQ (12 versus 25 min). The A-NSKQ covers the same key topics assessed in the NSKQ (weight management, macronutrients, micronutrients, supplementation, sport nutrition, and alcohol);however, it does not evaluate some aspects of these topics, such as the role of specific supplements, and the consequences of dehydration. Therefore, the content and face validity of the A-NSKQ should be reviewed by individuals planning on using the tool, to ensure that it serves their intended purpose.

The results of this study show that the A-NSKQ sub-sections (GNK and SNK) fit the Rasch model, indicating that the items meet the requirements that difficult questions are less likely to be answered correctly and individuals with higher levels of NK are more likely to perform well [20]. Likewise, the questionnaire demonstrated test-retest reliability and achieved or approached pre-determined cut-off values for internal reliability. However, a limitation is that these assessments were based on proxy scores. Construct validity was also determined based on the proxy scores, and was confirmed when the A-NSKQ was administered to non-elite athletes, as those with previous nutrition education scored significantly higher than those without any formal nutrition education (Additional file 1: Table S5). The criterion validity of the A-NKSQ could be assessed in future studies by administrating both questionnaires to participants and calculating the correlation between scores (%). However, it may be difficult to achieve adequate response rates for this step, as respondents are likely to find completing both tools burdensome. In addition, participants may research the correct answers to questions before the second completion, which would influence the validity of this method.

The invitation to complete the A-NSKQ was distributed to a large number of non-elite athletes via email, but response rates were low (7%). Although web-surveys are known to have lower response rates than mail-based surveys, these usually sit around 34% [23]. Response rates for NSKQ could not be calculated because it was distributed via Facebook groups and online forums, or via second parties where total exposure was not known. However, the percentage of non-elite teams who responded positively to be involved in data collection using the NSKQ (6–13%) was very similar to the response rates of individuals who participated in data collection using the A-NSKQ (7%). This is in line with some reports in the literature that questionnaire length has only a small effect on response rates [18]. Although the shorter length did not appear to have a positive effect on response rates, it did have a positive effect on completion rates. Completion rates in the present study were 85%, which is higher than completion rates (54%) achieved during the validation and use of NSKQ to collect data. As above, this finding is akin to what has previously been reported regarding questionnaire length and completion rates [17]. Future studies should consider distribution methods and survey design to ensure that response and completion rates are optimised. These include designing personalised invitations and sending reminders and notifications if responses have not been obtained [16]. However, personal interest in the topic is one of the key factors influencing individuals’ decision to participate in a survey, and this cannot be controlled by investigators [16].

The results indicate there is room for improvement in non-elite AF and netball players’ NK. Athletes’ mean score was 47%; similarly, non-elite, male AF players scored 51% on the NSKQ ¹. In the present study, athletes performed much better on questions of GNK than SNK, which is akin to results reported by Devlin et al. [24] and Kunkel et al. [25], but in contrast to results reported by Harrison et al. [26] and Barr [27]. Respondents answered items on hydration (q27) supplements (q35, q26, and q37) and micronutrients (q13, q 23, and q25) poorly. These items were also answered poorly when the NSKQ was administered to non-elite AF players¹. In the existing literature there are inconsistencies regarding gaps in knowledge with some studies reporting hydration [28], supplementation [29] and micronutrients [28] were better understood than other topics, and others finding the opposite to be true [26, 30].

A literature review undertaken by Trakman et al. [31] reported that 64% of studies on athletes’ and coaches’ NK found that prior nutrition knowledge, higher levels of education, previously undertaking a nutrition course or currently majoring in nutrition studies correlated with higher NK scores. Ten of fifteen studies that assessed the effects of gender on NK stated there were no significant differences between males and females. All studies that evaluated sporting calibre and type of sport reported these had no effect on NK scores. The present study reflected these findings; previous nutrition education and higher levels of education positively affected NK score, but a relationship between score and sport played, sporting level and gender were not detected. Older participants had better GNK, which is in contrast to findings in other cohorts [32]. Of note, however, the current sample had only a small proportion of athletes from sports other than AF, and playing at the sub-elite or elite level.