Modifying the original NVS to develop the NVS-UK
The NVS nutrition label was adapted to conform to current UK food labeling practice and the questions were converted from US- to UK-style English. We did this with a web-based Delphi technique [
21,
22] that involved a panel of experts from clinical practice (medicine, nursing, pharmacy), public health, dietetics, research, adult education, and the food and drink industry. Recruitment of these experts was undertaken through the Health Literacy Group UK, a not-for profit organization that aims to raise the profile of health literacy as a remediable cause of health inequalities [
23]. All Health Literacy Group UK members were invited by email to participate and all who expressed an interest were recruited. We asked these experts to assess nutrition labels used in the UK, to compare their content and layout to the nutrition label used on the original (US) version of the NVS, and to suggest modifications of the original NVS nutrition label to make it concordant with UK nutrition labels. We also asked them to make suggestions for modifying the wording of the questions that accompany the nutrition label. The intent of these modifications was to make the style of English in the questions correspond to common usage in the UK.
Participants then used a web-based Delphi technique to score the layout of the modified nutrition label and questions, ranking them on a 5-point scale in which 1 indicated complete disagreement that the nutrition label and questions were suitable for use in the UK, and 5 indicating complete agreement. Further modifications of the nutrition label and questions were made in response to these scores and suggestions from participants, and rounds of web-based scoring were continued until consensus was reached (i.e. all participants scoring 4 or 5) that the label and questions were suitable for use in the UK and no more suggestions for improvement were being made.
Further Refinement of the NVS UK through Cognitive Testing in the Community
The nutrition label and questions were then tested for ease of understanding and acceptability by the public in a series of one-on-one interviews conducted by the market-research firm, Ipsos MORI. The individuals interviewed in this phase were residents of Lambeth borough in central London, an inner-city area with marked socio-demographic variation. Lambeth is the 14th most deprived of England’s 354 Boroughs, with a high proportion of residents from Black and Ethnic Minority (BEM) groups [
24]. Recruitment was in–street in Lambeth, with the time of day and recruitment site varied to ensure a wide cross-section of participants. A multi-stage sampling procedure was undertaken in 4 cycles over 6 weeks enabling the research team to assure that at least 30% of participants were from groups likely to have lower health literacy, such as members of BEM groups, those with education qualification levels below the standard English educational achievement expected at age 16 (5 grades A-C in the English matriculation examinations (GCSE)) [
25], and people from the lowest two social grades (grades D and E) on the National Readership Survey (NRS) social grading system. The NRS social grades are the standard for market research in the UK [
26]; and are shown in Table
1. Prospective participants with low levels of spoken English were screened out of the research. The interviews took place in participants’ homes.
Table 1
National readership survey social grades
A | Higher managerial, administrative and professional | 4 |
B | Intermediate managerial, administrative and professional | 22 |
C1 | Supervisory, clerical and junior managerial, administrative and professional | 29 |
C2 | Skilled manual workers | 21 |
D | Semi-skilled and unskilled manual workers | 15 |
E | State pensioners, casual and lowest grade workers, unemployed with state benefits only | 8 |
Each participant was asked to complete the NVS UK questions, comment on question wording and label content and layout, and explain the processes they used to answer each question. They were asked to give feedback on the length of the survey and the clarity and difficulty of the questions.
This was an iterative process in which successive rounds of 15–20 interviews were carried out. Each round was followed by a review of the responses by project investigators and further modification of the NVS label and questions as indicated by interview results. Interview rounds continued until no more modifications were suggested. Participants in the cognitive interviews all gave informed consent and were offered a £25 voucher as compensation for their time.
The socio-demographic characteristics of the participants in the cognitive interviews were compared with local and national population characteristics using Office of National Statistics (ONS) 2007 mid-year estimates [
27], ONS 2009 mid-year estimates [
28] and 2001 UK census data [
29].
Validation
Validation of the NVS-UK was assessed by comparing its performance to that of an accepted standard measure of health literacy, the TOFHLA [
16,
18], including the area under the Receiver Operating Characteristic (ROC) curve, and it’s correlation with education qualification attainment.
Data were collected on socio-demographic, lifestyle, and educational attainment in an interview that lasted 45–60 minutes. Age data were collected in 10-year age bands. The interview procedures were pilot tested with 20 Lambeth residents, following which the main validation survey was undertaken.
Instruments
The reference standard measure for HL used in this study, the TOFHLA, was developed from hospital materials and consists of a 50-item reading comprehension and 17-item numerical ability test, taking 22 minutes or more to administer. The reading items use a modified Cloze procedure, in which every 5th to 7th word in a passage is omitted and replaced with a blank space; the word to fit into each blank space is chosen from multiple-choice options. The numeracy items use prescription forms, clinic instructions, and medical insurance examples about which questions are asked requiring calculations. TOFHLA scores range from 0 to 100. A score of <60 represents inadequate health literacy; people with skills at this level are likely to experience the greatest barriers due to limited literacy and numeracy. A score of 60 to 74 represents marginal literacy; people scoring at this level may experience some difficulties understanding and using health information. Those scoring >75 have adequate literacy and are unlikely to experience problems arising from limited health literacy and numeracy skills.
Participants completed the NVS UK first followed by the UK-validated version of the TOFHLA.
Sample and recruitment
For validation against the TOFHLA, the sample size calculation was based on published reports on the validation of the original NVS, where correlation against the TOFHLA was 0.59 [
20]. An unacceptable correlation was considered to be 0.3 (i.e. accounting for 9% of variance), and (based on previous data) a plausible correlation for purposes of power calculation was defined as 0.5 (or more). All correlations that could be shown to be significantly higher than 0.3 were regarded as acceptable. At least 250 subjects were required to give 90% power to detect such a difference.
The recruitment area for the validation stage was widened to include the London Borough of Southwark. Southwark is a borough neighbouring Lambeth with similar socio-demographic characteristics i.e. high levels of socio-economic deprivation and a high proportion of people from BEM groups [
28]. Eligibility criteria were age 18 – 75 years, living at home, and ability to converse in English. We excluded potential participants if they were health care professionals (defined as people working in the National Health Service or private health care), did not live at home, had self-reported impaired vision (unable to read the test card), or were unable to hold a conversation with the interviewer due to cognitive impairment or inability to converse in English. Sampling aimed to recruit a sample reflecting the age, gender, NRS social grades and ethnic mix of Lambeth and Southwark. Recruitment was by postcode with interviewers assigned to clusters of postcodes with a high prevalence of residents fitting the desired recruitment profile. A total of 51 sample points were issued, with 7 interviews to achieve within each sample point. Interviewers knocked at the doors of potential recruits; if no-one eligible for the study was available or participation was declined, the interviewer went to the next address on their list. The interviews took place in participants’ homes with consent. Participants were given a £15 gift voucher in compensation for their time.
Data analysis
The principal analysis to determine the validity of the NVS-UK was to assess the correlation (Pearson r) between scores on the NVS and an accepted standard measure of health literacy, the TOFHLA [
16,
18] and by calculating the area under the receiver operating characteristic (ROC) curve. Validity was further assessed by the correlation (Pearson’s r) between the NVS and participants’ educational qualification attainment.
Optimal cut-off point(s) on the NVS UK for differentiating different levels of health literacy as identified by the TOFHLA were undertaken through calculation of the sensitivity and specificity for selected cut scores in the ROC analysis.
Statistical analyses were performed using STATA. V 11.2.
Ethics review
The cognitive testing and validation interviews were conducted by Ipsos MORI under the Market Research Society (MRS) Code of Conduct and Interviewer Quality Control Scheme (IQCS). Ethics approval for the study was granted by the London South Bank University Ethics Committee (ref UREC 1034). This project was exempt from NHS Research Ethics Approval as participants were not recruited from the NHS.