Design and Serendipity in Establishing a Large Cohort with Wide Dietary Intake Distributions : The National Institutes of Health–American Association of Retired Persons Diet and Health Study

Abstract

In this paper, we discuss the rationale for and design of a new prospective cohort study, the National Institutes of Health–American Association of Retired Persons (NIH-AARP) Diet and Health Study. We emphasize difficulties that hampered implementation of our original design and the (partly serendipitous) resolution of those difficulties. Finally, we present the baseline characteristics, especially the dietary intake distributions, for the cohort that ultimately emerged.

MATERIALS AND METHODS

Rationale

We designed this study to address three methodological problems impeding epidemiologic investigations of diet and cancer: 1) recall bias arising from postdiagnostic dietary assessment in case-control studies; 2) attenuation of true relative risks because of dietary measurement error; and 3) relative homogeneity of dietary intake among participants in studies conducted in one area or country.

To circumvent recall bias, we opted for a prospective cohort study, with dietary assessment taking place prior to cancer diagnosis. Although it is plausible that a diagnosis of cancer could systematically alter the retrospective reporting of typical diet prior to diagnosis, it is not proven that recall bias is a substantial problem in case-control (retrospective) studies of diet and malignancy. Within the context of prospective cohort studies, a few studies have compared prospectively and retrospectively reported fat and energy intake for cancer cases and noncases (1–4). Although the results of these studies are inconsistent and vary according to the nutrient assessed and the cancer being investigated, they have provided some empirical evidence for the existence of systematic recall bias in case-control studies of diet and cancer. However, even if there were a large series of cohort-based studies showing strong correlations between pre- and postdiagnosis food frequency questionnaire (FFQ) results for virtually every nutrient or food of interest in relation to a wide variety of cancers (which would certainly be a costly research program), investigators could still not be certain that such strong correlations would hold in case-control studies conducted in other demographic or geographic contexts. Thus, it is unlikely that even an extensive, rigorous (and, undoubtedly, expensive) research program would definitively exclude the possibility of recall bias in case-control studies of diet and cancer. Given this essentially intractable uncertainty about recall bias in case-control studies of diet and cancer, we opted for the prospective cohort design in which diet is assessed prior to diagnosis.

The FFQ is now commonly used to assess dietary intake in large epidemiologic studies. A large body of evidence suggests that the FFQ measures true diet with considerable error (5). Such error tends to attenuate the relative risks observed in studies of dietary factors and disease (6). Using a simple errors-in-measurement model and data from the 1987 National Health Interview Survey and the Women's Health Trial Vanguard Study, we calculated the cohort sample size required to take into account the attenuation resulting from the misreporting of dietary fat (7). To detect with 90 percent power, based on a 5 percent significance level for a two-sided statistical test of a “modest” relative risk gradient (one reflecting a relative risk of 1.64 for those who consumed more than 47.5 percent calories compared with those who consumed fewer than 25 percent of calories from fat) for dietary fat and colorectal cancer, we estimated that we would need a cohort of approximately 350,000. As with many sample size calculations, this was a crude estimate based on a number of assumptions about cancer incidence, the true distribution of dietary fat in our target population, the correlation between measured and true fat intake, and the accuracy of so-called reference instruments used in determining “true” intake. Moreover, this calculation was based on only one dietary factor when, in fact, we are interested in examining the relation to cancer of multiple dietary factors, with each being measured with error and having various degrees of correlation with the other factors. What was clear, though, was that our cohort would have to be very large—on the order of several hundred thousand—to offset the attenuation produced by measurement error.

Many populations are characterized by relatively narrow intake distributions for a variety of potentially cancer-related nutrients and foods. In countries such as the United States, for example, a relatively small proportion of the population consumes less than 20 percent of calories from fat or more than several servings of vegetables per day. A study conducted in this country or similar geographic regions is constrained in its capacity to examine the cancer risk associated with more “extreme” intake levels. To circumvent this problem of dietary homogeneity, we used a two-stage cohort construction strategy that allowed us explicitly to enrich our study population with persons at the extremes of intake.

Original design

We chose to establish our cohort within the membership of the AARP for several reasons: 1) the AARP is a very large organization, with a mailing list encompassing over 30 million men and women in the United States; 2) the demographics are favorable, since the AARP comprises both men and women age 50 years and over, an age when cancer occurrence is becoming more frequent; 3) the organization focuses on many health issues, which we thought would help in our recruitment; 4) AARP maintains regular communications with members through its magazine and bulletin; and 5) AARP leadership and research staff had participated earlier in a few smaller studies, were interested in collaborating with the NIH on a large-scale health study, and were willing to provide their mailing list and letterhead to help with recruitment.

On the basis of national survey data (8), we anticipated that only relatively small percentages of respondents to our baseline questionnaire would fall into extreme dietary intake categories (i.e., less than 25 percent and more than 47.5 percent of calories from fat). We reasoned that if we could mail the baseline questionnaire to a sufficiently large pool of people, we would be able to capture enough persons in the extreme intake categories to construct a cohort permitting stable estimates of cancer risk associated with the intake extremes. Assuming a response rate of approximately one in three, we calculated that we would need to send the baseline questionnaire to 3.5 million people. We planned to construct our cohort by including all respondents in the intake extremes and a random sample of persons who reported intake between these extremes.

In 1995–1996, we sent a 16-page-questionnaire to 3.5 million AARP members in six states (California, Florida, Pennsylvania, New Jersey, North Carolina, and Louisiana) and two metropolitan areas (Atlanta, Georgia, and Detroit, Michigan). In choosing our states and metropolitan areas, we considered only those areas with cancer registries certified as having at least 90 or 95 percent completeness of case ascertainment. To make the mailing process more efficient, we attempted to minimize the total number of states for our initial mailing and, therefore, picked states with large AARP membership. Because we also wanted to maximize the minority composition of the cohort, we targeted some smaller states and metropolitan regions with large minority populations.

The baseline questionnaire was a grid-based version of the new National Cancer Institute instrument, the Diet History Questionnaire. The Diet History Questionnaire was modeled initially on the Block questionnaire, underwent extensive cognitive testing during its development, and used a nutrient composition database newly derived from national survey data (9–11). The baseline questionnaire included 124 food items with portion size and 21 questions on intake of low-fat, high-fiber foods and food preparation. This questionnaire does allow for variation in eating patterns in different parts of the country to the extent that this variation is reflected in the line items and questions on cooking practices contained within the instrument. Although the questionnaire was designed for “general population” use, some regional and ethnic group-specific foods (flour and corn tortillas, for example) were included. The baseline instrument also included questions on demographic characteristics and other potential cancer risk factors (smoking, physical activity, family history of cancer, medical conditions, reproductive factors, and exogenous hormone use). We mailed this questionnaire in three waves: 250,000 in October 1995, 1 million in February 1996, and 2.25 million in May 1996.

We planned a calibration substudy among approximately 2,000 respondents to the baseline questionnaire. In addition to the baseline FFQ, each calibration study participant was to complete two 24-hour dietary recalls (administered by telephone an average of 25 days apart) and a second food frequency questionnaire.

We planned to use the cancer registries as our primary means of ascertaining cases. Although some registries have mechanisms for capturing data from persons who go to contiguous states or geographic areas for medical care, we expected that a small proportion of cohort members would move outside the purview of the registries in our target states and metropolitan areas. We therefore planned to mail a follow-up questionnaire to cohort members at the end of a 5-year period of observation. This short questionnaire would ask if cancer had occurred and whether the participant continued to reside in the registry area.

RESULTS

Implementing the original design

A total of 617,119 men and women returned the baseline questionnaire, a response rate of 17.6 percent. Data from the questionnaires were entered via Optical Mark Read (National Computer Systems, Minneapolis, Minnesota) scanners, capable of reading 12,000 pages per hour with 99.99 percent accuracy. The scanners used an “ink read” technique that permitted reading of questionnaires marked in pen rather than black lead pencil. All questionnaires were manually examined for damage prior to scanning. During scanning, we checked at least one record in every thousand by comparing the marks on the original document with the output of the scanner. Less than 0.1 percent of the questionnaires could not be scanned due to form damage; data from these damaged instruments were entered manually.

We excluded respondents for whom gender was unknown and those who skipped substantial portions of the questionnaire, were proxies for the intended respondent (this was indicated by a question on the front cover of the questionnaire), had more than 10 recording errors, reported fewer than 10 foods consumed, or subsequently requested to be removed from the study (table 1). A total of 567,169 persons (340,148 men and 227,021 women) respondents remained after these exclusions.

The 17.6 percent response rate to the baseline mailing was lower than we had (overoptimistically) expected from the AARP membership. Because we had calculated the initial mailing size (3.5 million) on the basis of a higher response rate, we faced the prospect of fewer respondents with extreme intakes available for the final cohort. (Due to fiscal and administrative constraints, we were compelled to move directly to the field in 1995 and could not conduct a pilot study to estimate the response rate prior to the first mailing wave.)

It turns out, though, that the intake distributions for respondents differed from those observed in national surveys. Our AARP respondents consumed less fat and red meat and more fiber and fruits and vegetables than comparably aged adults in the general US population. Moreover, the intake distributions for these dietary factors were wider than those in the national surveys. This combination of both a shifting and a widening of the intake distributions among respondents compensated for the less-than-anticipated response rate. (The lower response rate did have one fortuitous, if unintended, consequence: The return mailing costs were nearly a million dollars lower than the initial budget projection.)

In calculating our initial cohort sample size of 350,000, we focused on a single nutrient, dietary fat. After review of the baseline data, it was clear that we had satisfactory dietary information on an additional 200,000 men and women. Other dietary factors of interest were not strongly correlated with fat, so that persons in the extreme quintiles for fat are not necessarily those in the extreme quintiles for fiber, fruits and vegetables, and red meat. By including all 567,169 people in the final cohort, we were able to achieve a reasonably wide intake range for all four dietary factors.

Baseline cohort characteristics are summarized in table 2. The mean age was approximately 62 years. The cohort is predominantly White and more educated than the general population: Forty-five percent of the men and 31 percent of the women are college graduates. Baseline body size was comparable with national averages, with a mean body mass index of 27.2 in men and 26.9 in women. Although over half of the cohort reported being former smokers, only a little more than 10 percent of the men and 14 percent of the women reported current smoking. (These percentages of current smoking are somewhat lower than national numbers (12)). Physical activity (defined as lasting at least 20 minutes and causing either increases in breathing or heart rate or, alternatively, working up a sweat) showed considerable variation: 16 percent of the men and nearly 23 percent of the women never or rarely engaged in such activity, whereas 21 percent of the men and 16 percent of the women were physically active five or more times per week.

Table 3 shows, for each quintile of intake, the median values for four dietary factors: percent kilocalories from fat, dietary fiber (g/day), fruits and vegetables (servings/day), and red meat (g/day). To derive these data, we first excluded from the baseline cohort all men who reported consuming less than 800 kcal or 4,200 or more kcal and all women who reported an intake of less than 600 kcal or 3,500 or more kcal. (This is a typical approach to excluding calorie outliers, but it is not the only one that can be used.) The range of intake was similar in both men and women. The median values for fat as a percent of total energy intake ranged from approximately 20 percent in the lowest quintile to 40 percent in the highest quintile. Dietary fiber intake in the highest quintile was about three times that in the lowest. Fruit and vegetable intake was nearly four times greater in the fifth as opposed to first quintile. Median intake of red meat among study cohort members in the top quintile was more than seven times (for men) and nine times (for women) that of members in the bottom quintile.

To ensure that our calibration study paralleled that of the main study in comprising persons with extreme intakes of four dietary factors (fat as a percent of total calories, fiber, fruits and vegetables, and red meat), we randomly selected baseline questionnaire respondents within various combinations of intake strata for these four factors. Of the 2,795 respondents initially approached, 2,055 (74 percent) participated by completing the first 24-hour dietary recall interview, and 1,415 (51 percent) completed both 24-hour dietary recall interviews and a second food frequency questionnaire. The 24-hour dietary recalls were conducted by experienced interviewers who completed a rigorous training session, which included home study, demonstration interviews, interactive lectures, and role playing. Table 4 shows the correlations, adjusted for random within-person error in the 24-hour dietary recall, between food frequency questionnaire and recall values for the four dietary factors.

Table 5 shows the number of incident, site-specific cancers expected in the cohort after 5 and 10 years of follow-up. These estimates were based on 1994–1996 data for all persons covered in the Surveillance, Epidemiology, and End Results program of the National Cancer Institute (13).

In table 6, we present results of power calculations based on the distribution of dietary factors (table 3), the expected number of incident cancers (table 5), and the relative risk attenuation resulting from dietary measurement error (14). The calculations were designed to determine the power to detect “moderate” true relative risks of 1.6 and 1.8, reflecting a comparison of the cancer risks associated with the median intakes for the first and fifth quintiles for a given dietary factor. We used the calibration study data to determine λ, the attenuation factor; for estimation of λ, we used the standard model (5), which assumes that the reference instrument (our 24-hour recall) gives an unbiased estimate of true intake and has errors uncorrelated with those from the FFQ. As an example, for percentage of calories from fat in men, we calculated λ to be 0.54. Because the relation between the true and observed relative risk is given generally by [RR>_T]λ = RR_obs, the RR_obs for percentage of calories from fat is 1.6^0.54 = 1.29 for the true relative risk of 1.6 and 1.8^0.54 = 1.37 for a true relative risk of 1.8. We applied a newer, more robust model, one that does not make the assumption that the reference instrument provides an unbiased measure of true intake, to an empirical data set that contained information on dietary protein from a food frequency questionnaire and a “reference” instrument (4-day weighed food record) and had an unbiased biomarker (urinary nitrogen) (14, 15). Our analysis indicated that the attenuation was 34 percent greater than that predicted by the standard model. Therefore, the true relative risk of 1.6 would generate an observed relative risk of 1.18 (1.6^{[0.54 × 0.66]}); the true relative risk of 1.8 would yield an observed relative risk of 1.23. We found comparable degrees of relative risk attenuation in similar calculations carried out for women and for fiber, fruits and vegetables, and red meat.

We then calculated the power of the study to detect observed relative risks in the range of 1.1–1.3 for several cancers. These power calculations are presented in table 6.

DISCUSSION

Our two-stage stategy (wide screen, then inclusion of all persons at extremes and a subset of those in-between) was compromised by the lower-than-expected response rate. We ultimately did create a large cohort with wide intake distributions (table 3), but only after an unanticipated development that offset the low initial response: the shifting and widening in the intake distributions that became apparent with our first mailing wave. Even though we were not able to implement our two-stage strategy, this design may prove useful to other investigators interested in establishing cohorts with wide exposure distributions.

Our sample size calculations (table 6) show that, even allowing for attenuation due to dietary measurement error, we will have more than 90 percent power to detect moderate relative risk increases after 5 years of follow-up for four major dietary factors (fat, fiber, fruits and vegetables, and red meat) in analyses of colorectal, breast, and even pancreatic cancer outcomes. In the interest of space, we have chosen to present power calculations only for selected outcomes. Clearly, the power to detect moderate relative risk increases within 5 years of follow-up will be very high for prostate and lung cancer analyses. The power to detect moderate relative risk increases for non-Hodgkin's lymphoma after 5 years of follow-up will be even greater than that for pancreatic cancer. For ovarian cancer, we will require 10 years of follow-up to achieve 90 percent power to detect moderate relative risk increases. It should be noted, however, that should our AARP respondents turn out to be “healthier” than the Surveillance, Epidemiology, and End Results population, our expected number of cases (table 5) and power calculations (table 6) may be somewhat overoptimistic.

We are currently conducting within the cohort a pilot study of our endpoint assessment procedures. This study will determine the incidence rates for a few major cancers, the response rate to the follow-up questionnaire, the proportion of cohort members who no longer reside in the registry reporting areas (and, among these, the proportion of cancers reported on the follow-up questionnaire that can be confirmed through written requests to physicians and hospitals), and the proportion of cohort members who are lost to follow-up.

We developed a second questionnaire that was mailed in late 1996 to the baseline questionnaire respondents. This second questionnaire, derived from a series of instruments used in other National Cancer Institute studies, asked more-detailed questions on potential cancer risk than we could include in the baseline instrument because of the size of the FFQ and the constraints imposed by the mass mailing cost structure. This instrument also included questions on typical diet in adolescence and 10 years earlier. Although data on the accuracy of dietary information for adolescence obtained in later adult years are sparse, we concluded that the potential importance of adolescent diet-cancer hypotheses warranted collecting this exposure information. The response rate to this second questionnaire was approximately 63 percent. The administration of this questionnaire was a “one-shot” activity; we did not have resources available at that time for pretesting or for further mailings or telephone contacts to increase the response rate. Nevertheless, a subcohort of approximately 339,000 persons with information on past as well as current diet, in addition to more detailed information on family history, physical activity, body size, and other risk factors, may prove valuable.

Another ongoing pilot study will aid us in determining the optimum method for collecting buccal cells for DNA from the cohort. On the basis of preliminary findings from a recent buccal-cell collection pilot study within the cohort, we anticipate being able to collect DNA from about 200,000 of our cohort members. These specimens will add a molecular dimension to the study and contribute to the worldwide epidemiologic resources available for studying the interplay of genetic, dietary, and other environmental factors.

The NIH-AARP Diet and Health Study has the prospective design, large size, and wide intake range as well as a realistic possibility of acquiring a large number of biologic specimens to make a useful contribution to the nutritional epidemiology of cancer.

REFERENCES