Reliability and relative validity of a food frequency questionnaire to assess food group intakes in New Zealand adolescents

This study was approved by the University of Otago Human Ethics Committee. A paper-based, three section adolescent-specific food questionnaire was developed. The food questionnaire is made up of three sections: The first section contains 12 multiple-choice questions on general eating habits, including intakes of food group servings, meal consumption patterns and frequency of takeaway consumption. These questions were adapted from previously published questionnaires [29, 30]. This study focusses on the validity of the FFQ (sections 2 and 3 of the food questionnaire), namely the New Zealand Adolescent FFQ (NZAFFQ).

The NZAFFQ was produced by combining and modifying the Health Behaviour in School-aged Children (HBSC) FFQ [24] and the Children’s Dietary Questionnaire (CDQ) [26]. These FFQs were developed to describe food patterns, but not nutrient intakes, of children and adolescents (4 to 16 years), and therefore contain only a limited list of food items. In particular, the HBSC FFQ included 15 items covering the most commonly consumed foods known to be important sources of fibre and calcium among European youth. The CDQ included 28 items described as ‘encouraged foods’ (fruits, vegetables, water, reduced fat products) and ‘discouraged foods’ (high fat or sugar foods, sweetened beverages and full fat dairy products) for adolescents in Australia. These two validated questionnaires formed the basic construct of the NZAFFQ as they covered different important aspects (i.e. variety and intake frequency) of an adolescent’s diet and have been used to derive index-based dietary patterns [26, 31, 32].

Section 2 of the food questionnaire assessed ‘usual consumption’ of 32 food items, covering 15 items from the HBSC FFQ [24] and included extra questions on food groups relevant to the New Zealand adolescent population. Changes to the original HBSC FFQ included the addition of questions on consumption of meats and different types of soft drinks. As in the original HBSC FFQ, frequency of intake was estimated by asking “On average, how many times a week do you usually eat or drink……” and participants could select one of the following response categories: ‘none’, ‘less than once a week’, ‘once per week’, ‘2 to 4 days a week’, ‘5 to 6 days a week’, ‘once a day’, and ‘more than once a day’. Although a specific time frame for ‘usual’ was not defined, we believed that this was likely to cover the period of the previous four weeks, based on results of our pretesting group interviews. For the last section of the NZAFFQ (Section 3), we assessed intakes of 13 fruits, 22 vegetables and 7 miscellaneous foods consumed in the past seven days, as in the original CDQ [26]. Modifications to the original format included renaming and regrouping of conceptually similar food items to ensure the suitability of the food items to New Zealand. For example, ‘sweet potato’ was renamed as the locally known Māori name ‘kumara’ while ‘orange’ and ‘mandarin’ were grouped together as one item. Section 3 assessed ‘most current intake’ (in the past week) to reduce the difficulty of recall and accommodate seasonality and availability of foods [33].

Further revisions were also made to ensure that the food lists in sections 2 and 3 covered the foods frequently consumed in New Zealand including those indicated in the 2002 National Children’s Nutrition Survey [9]. To improve face validity of the NZAFFQ, two registered dietitians and a nutritionist were consulted to review this FFQ before formal pretesting.

The NZAFFQ was pre-tested in a sample of 29 adolescents (13 males, 16 females) aged 14 to 15 years recruited from one secondary school in Dunedin, Otago. The day before the pretesting, participants were asked to complete the FFQs and answer feedback questionnaires about their comprehension of the NZAFFQ questions. Two group interviews (one for males, one for females) were conducted, each moderated by a trained research assistant, with additional observers present to identify further discussion questions based on feedback from the group. The main focus of the group interview was to obtain details on the participants’ understanding of the questions, the food items listed and examples. The group discussions were audio recorded and transcribed so that the feedback could be used to further refine the FFQ. The final NZAFFQ included 74 food items as either (i) single foods (e.g. yoghurt), or (ii) lists of similar foods (e.g. chicken, turkey or duck). For analysis, these foods were subsequently aggregated into 34 food groups of interest by grouping similar items as had been done in other studies [34, 35] (Table 1).

A convenience sample of adolescents aged 14 to 18 years was recruited to participate in the validation study via schools, sports clubs and youth groups in Dunedin based on the following inclusion criteria: aged between 14 and 18 years, absence of any disease that may influence nutritional status, ability to keep a food record, and those whose parents did not complete an opt-out consent form. All participants also provided written informed consent before participating in the study. Data collection was conducted in two periods from November to early December 2010 and March to May 2011.

Each participant was asked to keep a four-day estimated food record (4DFR) and complete the NZAFFQ twice within a fortnight. On the first visit day, each participant was asked to self-complete an FFQ in the presence of a research assistant. After the FFQs were checked for completeness and missing answers were obtained, participants were given verbal and written instructions on how to complete a structured Food and Drink diary for three weekdays and one weekend day. To facilitate accurate recording, the instructions incorporated standardized examples on the methods of recording and the food record was structured into six daily eating occasions. On each page, spaces were provided for recording of meal times, venue, whom the participants ate with, details of foods consumed including type, brand and amount of foods or beverages consumed. In addition, each participant was taught to use the portion aid kit provided for recording food quantities. The portion aid kit included a metric measuring cup, ruler, diameter circle and a coloured food photo booklet, which contained photographs of commonly consumed foods in different portion sizes [36]. At the end of the four-day record, participants answered supplementary questions on the type of sweetened drink, milk, bread and fat spread that they usually consumed. This qualitative information was collected to assist with coding of the food records. All food records received were checked for completeness. Participants were asked to clarify missing and unclear entries whenever possible. Within two weeks after the first completion of the NZAFFQ, all participants were asked to repeat the NZAFFQ for a second time to assess test-retest reliability. Based on results of previous research of FFQ repeatability conducted in New Zealand children [37], this two-week interval was chosen in order to minimize the variation in food intake responses due to true changes over time.

All data were entered into a Microsoft Excel spreadsheet and checked by a trained nutritionist. Recorded food items in the 4DFR were matched to the items as defined by the NZAFFQ. An example of this was ‘hash browns’ from the 4DFR assigned to ‘fried potatoes’ in the NZAFFQ. Food items within a composite meal were each allocated a proportion of the meal then assigned to their relevant food or food groups. For instance, a chicken burger was recorded to the three corresponding foods in the NZAFFQ: white bread/bun, poultry and lettuce/salad greens. All frequencies obtained from the 4DFR were adjusted to provide average weekly intakes. Food intakes were described as frequencies of intake (days per week) and used as the unit of comparison between the NZAFFQ and the 4DFR.

Spearman’s rank correlations were performed to evaluate the validity of the first FFQ administration (FFQtime1) relative to the 4DFR for ranking participants by frequency of consumption of the 34 food groups. Following grouping of participants into thirds, cross-classification analyses were undertaken to examine the proportion of participants correctly classified into the same thirds or grossly misclassified into extreme thirds of intake. For reliability (FFQtime1 vs. FFQtime2), intraclass correlation coefficients (ICCs) were calculated in addition to Spearman’s correlation analyses. Because ICCs take account of within- and between-subject variability in responses, it was deemed the most appropriate test to examine the agreement between the repeated FFQs in ranking individuals by food group intakes. All statistical analyses were performed using the statistical program STATA 11.1 (StataCorp, College Station, TX, USA). Significance levels for all tests were set at p < 0.05.

Of the 78 participants who volunteered to take part in this study, 38 (49%) participants completed both the 4DFR and two replications of the NZAFFQ. Fourteen participants (18%) completed two NZAFFQs only while three participants (4%) completed the 4DFR and the FFQtime1. More males than females did not complete all parts of this study (p = 0.006) but there were no differences in demographic or anthropometric data between those who completed all parts of the study and those who did not (data not shown). In total, 41 participants (16 males, 25 females) were included in the validity study while 52 participants (28 males, 24 females) were included in the reliability study. Participants were aged 15.0 ± 0.8 years (range 14.0 to 17.9 years). The mean time interval between test-retest administrations of the NZAFFQ was 12 days.

The median Spearman’s correlation coefficient (SCC) between the two administrations of the NZAFFQ was 0.71, and SCCs ranged from 0.46 for fruit juice or cordial to 0.87 for non-standard milk (Table 2). The median ICC was 0.69 (range 0.26-0.92) and 71% (24 of 34) of the food groups had ICCs above 0.6. At least 46% of participants were correctly classified into the same thirds for all food groups. No food groups had levels of gross misclassification above 10 percent, with the exception of meat alternatives (17%) and rice, pasta or noodles (25%).

As shown in Table 2, SCCs above 0.3 were seen for over two-thirds (23) of the 34 food groups in the FFQ. The median SCC was 0.40, and individual SCCs ranged from 0.04 for convenience foods to 0.70 for standard milk (whole-fat milk). High correlations (SCC ≥ 0.50) were observed for breakfast cereals, milk (standard and non-standard), eggs, sweet bakery products and sweet snack bars. Overall, the exact agreement between the methods in ranking participants into thirds was highest for meat alternatives (78%), but lowest for red or yellow vegetables and potatoes (27%). The mean percent misclassified into extreme thirds for all food groups was 12%.

Although validation studies of FFQs in adolescents have been previously reported, most studies validated their FFQs in terms of nutrients or absolute food intakes [47, 48]. Since this study focused on validating the actual responses of intake frequency, precise comparisons of this study with existing studies are not possible. One exception is a study by Vereecken and Maes [24], which validated a 15-item HBSC FFQ against a 7-day food diary using a similar approach to this study. These same 15 food items were included in the present NZAFFQ. A direct comparison of the two studies revealed that the NZAFFQ showed similar reliability (mean SCC = 0.71) compared to the original HBSC FFQ validation study (mean SCC = 0.67) for the 15 food items. Likewise, the validity of most food items was comparable to those of the original HBSC FFQ (mean SCC = 0.41), achieving an average SCC of 0.38. Vereecken and Maes found high correlations for milk (whole and semi-skimmed) and brown bread (SCC = 0.51-0.65) but low correlations for crisps, diet soft drinks, sweets and fruits (SCC = 0.10–0.34). Similarly, our study showed that validity was good for milk and breakfast cereals (SCC = 0.59-0.70), but less favorable for soft drinks (regular and diet), chips, crisps and sweets (SCC = 0.06-0.30). The most striking observation that emerged from the data comparison was the rather low validity for foods perceived as being ‘less healthy’. We speculate that these foods may be underreported in the NZAFFQ due to social undesirability [49]. This is evident for regular soft drinks where 50% of participants who reported usual consumption of ‘once or less per week’ in the NZAFFQ specified intake on two or more days during their 4-day recording periods.

This study has several strengths and limitations. The main limitation of this study was the small sample size (n = 41), which may have limited the observation of significant correlations in food group intakes. Previous authors have suggested that a sample size of at least 50 is desirable [27], and ideally a sample of between 100 and 200 should be used, particularly if the FFQ is designed to provide information on nutrient intakes [41]. Although the recruitment deadlines were shifted several times, it was difficult to recruit more participants. A high percent (47%) of those recruited failed to complete the study due to the demanding task of keeping a 4DFR, even though estimated rather than weighed records were used. Our low compliance rate fell within the response range of 48% to 60% typically observed in previous validation studies of adolescents [19, 50, 51]. The sample in our study may comprise participants who were highly motivated; hence generalizability of these findings to other adolescent populations in New Zealand may be limited. On the other hand, this reinforces the clear need to develop a simple FFQ to accurately assess diet among adolescents, including those who are unlikely to provide high quality food records.

In the absence of an absolute gold standard for dietary assessment, we chose an estimated food record as the reference method. This method is advantageous in its ability to capture all food intakes without the reliance of memory and hence has the fewest correlated errors with an FFQ [52]. Additional effort was taken to prepare the Food and Drink Diary as an easy-to-carry booklet to facilitate recording ‘in situ’. Although participants were instructed to conduct recording ‘at the time’ of food and beverage consumption, we acknowledge that this may not be entirely possible. Food underestimation may still occur due to forgetfulness and the limited food knowledge among adolescents [47, 53, 54]. In addition, as the present NZAFFQ also assesses food intakes in the past, the different time frame between the FFQs and the food records may have had an effect on the correlations. Nevertheless, we found similar correlations between this study and other studies with overlapped time frame [38, 39, 44].

The strength of this study lies in the design of a non-quantitative FFQ, which is relatively short and practical for use in time-limited surveys where detailed measures of food intakes are not feasible. We attempted to address the limited motivation and portion size estimation skills among adolescents by omitting the requirement to provide food quantities in the NZAFFQ. As a result, this FFQ was highly repeatable and could be self-completed within 15 minutes. The median SCC of 0.40 obtained from this study was comparable to other validation studies of quantitative FFQs in adolescent populations [38, 44, 50]. Encompassing a wide range of food items from different food groups, this FFQ may offer a viable approach to measure diet diversity and derive dietary patterns or diet quality indices in large studies of adolescents. Whilst the intended use of the NZAFFQ is to assess food group intakes of adolescents in New Zealand, there is a potential for the frequency data to be used alongside other more intensive dietary assessment methods such as the 24-hour diet recall to estimate usual intake [55]. As it was adapted from previously validated questionnaires and pretested rigorously, we believe that the food list sufficiently covers the common foods consumed by New Zealand adolescents and is hence suitable to assess food group intakes in this age group.