Validation of the NPAQ-short – a brief questionnaire to monitor physical activity and compliance with the WHO recommendations

Sample size was estimated using the method developed by Walter et al. [11]. The aim of the study was to estimate correlation coefficients between self-reported and objectively measured physical activity between 0.3 and 0.6 [3, 4], which resulted in a sufficient sample size of 44 people [11]. To account for loss to follow-up and to be able to stratify by sex, the sample size was increased to 120 people.

Participants consisted of a convenience sample, recruited through different sources: an announcement was distributed by using the LinkedIn page of the National Institute of Public Health, by contacting specific workplaces and organisations, and through the extended network of the research group. The recruitment process was adjusted regularly, aiming for a balanced distribution in the study population regarding sex, age, education and region. All interested individuals received written and oral information about the project and signed a written informed consent. No incentives were given for participation, but participants received a report with their personal results afterwards.

The content of the survey did not require registration at the Regional Committee on Health Research Ethics, the Capital Region of Denmark (H-16020894). Data managing procedures was approved by the Danish Data Protection Agency (2015–57-0008). The survey content and procedures were designed according to the Helsinki Declaration [12].

The interviewer-based questions in NPAQ were adapted into a two-question self-administered questionnaire, NPAQ-short. Both versions include one question on time spent on moderate and vigorous physical activity (MVPA) and one on time spent on vigorous physical activity (VPA). At first, questions from NPAQ were modified for self-administered use mainly by reducing the amount of text. This was done through discussions in the project group, existing literature and experience from other researchers.

Then, content validity was assessed using cognitive interviewing, which gave insight into the cognitive processes taking place when answering the questions, in order to improve the wording of the questions [13]. A total of 12 cognitive interviews were conducted with participants of different sex, age, and education. The interviews considered e.g. the different elements of the questions (help text, descriptions of intensity, and examples of activities). From the interviews, it became clear that the respondents preferred different elements in their response process depending on their activity patterns, and thus all elements were included in the final version of the questionnaire. However, the cognitive interviews led to smaller adjustments in the wording of the questions. The final version of NPAQ-short is displayed in Fig. 1.

To assess test-retest reliability and criterion validity NPAQ-short was included in a questionnaire, which was distributed to participants twice, two weeks apart. Two versions of NPAQ-short were included in the questionnaire; first participants answered the closed-ended version and then, separated by additional questions, they answered the open-ended version. The additional questions concerned sex, age, weight, height, education, occupation, health status, sedentary behaviour, and self-rated fitness. The NPAQ-short with both open and closed-ended questions are displayed in Fig. 1.

At inclusion, participants answered the questionnaire on a computer or in some cases on a printed version due to technical reasons. After two weeks they received an e-mail with a link to the same questionnaire. A single participant without e-mail address received the second questionnaire by mail. Two reminders about answering the second questionnaire were sent out after one and two weeks.

Physical activity was measured using Actiheart (CamnTech Ltd., Cambridge, UK), which measures heartbeat and acceleration and combines the two into a measure of energy expenditure [14, 15]. Actiheart was placed horizontally on the chest using two electrodes (SP-50 ECG Electrode) and set to record in 60 s. epochs. Before long-term monitoring a signal test was conducted and participants completed a steptest, which was used for individual calibration. The steptest was administered from the Actiheart Software and lasted 8 min during which participants stepped up and down on a 20 cm step bench following a given beat with increasing intensity [16]. The participants received the Actiheart at the first visit and returned it after one week either picked by a member of the research team or returned by mail in a prepaid envelope. Participants wore the monitor 24 h/day for seven consecutive days however, they were asked to take off the monitor in case of skin irritation, swimming or any water-related sports (the monitor could be worn during showering). In addition, participants were instructed to change the electrodes every second day in order to ensure the best signal and to prevent skin irritation.

During the monitoring time period, participants kept a log in which they recorded sleeping patterns, working hours and when they changed electrodes. Irregularities, e.g. problems with the monitor or non-wear time were also documented.

Test-retest reliability and criterion validity were analysed for each of the two questions in NPAQ-short separately. This was done in order to assess the ability of the NPAQ-short to monitor physical activity duration and intensity. Additionally, answers to both questions were combined into a joint measure that was able to stratify the population according to the WHO recommendations for which criterion validity was analysed.

Actiheart files were trimmed in accordance with the logbook in the commercial software (Actiheart 4.0.116, CamNtech), using standard procedures for identifying non-wear time periods and interpolating gaps with missing data. Non-wear time was identified as a continuous period of > 2 h with no activity or with information from the logbook. Gaps < 5 min were filled by automatic interpolation based on previous and subsequent data [16]. The activity measure was calibrated using self-reported height and weight, resting heart rate as an average over the measured nights and results from the step test. Based on this, energy expenditure and MET-values for each minute of wear time was calculated using the branched equation model [18].

Combining this data with information from the log on sleep, work and leisure each minute was categorised into one of these three domains. Leisure time minutes with MET> 3.0 were summed into minutes of MVPA/week and leisure time minutes with MET> 6.0 were summed into VPA/week [2]. Only MVPA minutes in bouts of 10 or more minutes were included due to WHO’s recommendations on physical activity (allowing for 2 min below threshold) [19].

In line with previous studies, a workday had to have valid data for at least 4 h of work and 3 h of leisure time and a leisure day had to have at least 7 h of valid data (excluding sleep) to be included in the analyses [20]. Participants were included in the analyses if they had valid data for a minimum of three days, constituting of at least two workdays and one leisure day, or three leisure days in case participants did not work.

One hundred twenty two participants were included in the study, filled out the first questionnaire and had the Actiheart fitted. Of the 122 participants 115 filled out the second questionnaire. During data processing Actiheart data from 26 participants (21%) were excluded, e.g. due to technical problems with the device or lack of logbook data, 7 participants (6%) had less than the three required valid days. Participants with complete Actiheart data did not differ significantly from the total group of participants regarding age, sex, and education (data not shown). A few participants were lacking or had invalid questionnaire data which left 117 with valid data from questionnaire 1 and 108 from questionnaire 2.

This resulted in 105 participants with valid data for the reliability analysis, 92 with valid data for analysis of criterion validity of the MVPA question and 89 for the VPA question and compliance with the WHO recommendations.

Spearman’s rank correlation coefficient (Spearman’s rho) was used to assess correlations between test-retest and between objective and self-report measures of MVPA and VPA. Based on results from similar studies Spearman’s rho ≥0.50 was considered acceptable reliability and Spearman’s rho ≥0.30 was considered acceptable validity [3, 4]. Validity was additionally considered using Bland-Altman plots.

For categorical variables correlations between objective and self-report measures of MVPA and VPA were assessed using Cohen’s Kappa. Taking existing literature into account, values ≥0.40 were considered acceptable reliability and values ≥0.20 were considered acceptable validity [21]. For variables with more than two categories a weighed Kappa was calculated. All statistical analysis were performed in Stata 14.1.

Participants (50% men) had an average age of 43 years and were equally distributed on educational level (Table 1). Median time wearing the Actiheart was 5 days (Interquartile Range 4;5).

Correlations between the individual measures of physical activity (assessed by the questionnaires two weeks apart) were Spearman’s rho = 0.82 for MVPA and rho = 0.80 for VPA, for questions with the open-ended answering modes. Spearman’s rho for men were 0.88 for MVPA and 0.82 for VPA, and for women 0.78 for both MVPA and VPA (data not shown). For the questions with closed-ended answering modes, the kappa-values were 0.66 for MVPA and 0.59 for VPA (Table 2). For men, the Kappa correlation value was 0.57 for MVPA and 0.48 for VPA and for women the correlation was 0.73 for MVPA and 0.68 for VPA (data not shown).

Spearman’s rho was 0.33 for the correlation between self-reported and objectively measured MVPA and for VPA the Spearman’s rho was 0.32. For MVPA there were no differences between men and women however, the correlation for VPA was higher for men (Spearman’s rho = 0.43) than for women (Spearman’s rho = 0.23).

The Bland-Altman plots showed a mean difference between self-reported and objectively measured MVPA of − 111 min (limits of agreement, LOA: − 680;457) and 26 min (LOA:-173;-224) for VPA (Additional file 3). Distribution of the data appeared cone-shaped, illustrating that the higher the level of both MVPA and VPA, the larger the difference between the self-reported and objectively measured values.

For questions with the closed-ended answering modes, the kappa correlation value between the self-reported and the objectively measured MVPA was 0.17 and 0.21 for VPA. The correlations of questions with open-ended answering modes were kappa = 0.20 for MVPA and 0.20 for VPA (Table 3). In general the kappa values were higher for men (kappa = 0.25 for MVPA and 0.31 for VPA) than women (kappa = 0.12 for MVPA and 0.13 for VPA) (data not shown).

Kappa for agreement between the question using a closed-ended answering mode and a categorised version of the open-ended question of NPAQ-short was 0.50 for MVPA and 0.48 for VPA. In general participants were categorised with higher levels of physical activity when using the open-ended compared to the closed-ended versions of the questions (data not shown). This is also seen in Table 4, which shows that mean values of MVPA from the open-ended questions generally are above the maximum of the respective closed-ended category. The same applies to mean objectively measured activity. For VPA, mean values from questions using open-ended answering modes and objective measured physical activity are in general close to the limits of the respective closed-ended categories.

For objectively measured physical activity 78% of participants were categorised with a physical activity level above the WHO’s recommendations, 81% for open-ended questions and 73% for closed-ended questions. The correlation between the objective and the open-ended questions was kappa = 0.42 and for the closed-ended kappa = 0.34 (Table 5).

The specificity, i.e. the proportion correctly classified as having physical activity levels below the recommendations, was 50% for open-ended questions, 55% for closed-ended questions. The sensitivity, i.e. the proportion correctly classified as having physical activity levels above the recommendations, was 90% for open-ended questions and 81% for closed-ended questions (Table 5). In general, analysis on the compliance with WHO’s extended recommendations showed lower kappa values (kappa = 0.33 for open-ended questions and 0.17 for closed-ended questions) (Additional file 4).

Sensitivity analyses were carried out using different requirements for valid data: 10 h/day of valid data, 4 days of valid data and 5 days of valid data, and separately with standardised values of MVPA and VPA. None of the analyses changed the results markedly.

Strengths of the present study include its size, as the number of participants allowed for the analysis to be divided on sex. Additionally, the Actiheart monitor was used to measure physical activity objectively and thus it included a combination of accelerometry and heart rate. The general wear time was high (median = 5 days), and the Actiheart measure was calibrated using a steptest, thus the objective measure was regarded of high quality. Furthermore, we tested both open-ended and closed-ended questions on the same participants making comparisons between the two answering modes possible.

Limitations include the sampling method (convenience sample). Even though the population represented a variety of people regarding age, sex and education, questionnaire measures showed that the participants in general had better self-rated health, were better educated, and had a higher level of physical activity compared to the general population [25]. This might affect generalizability of the results as participants might be more motivated and more concerned about their physical activity level than the general population and thus better at self-reporting physical activity.

Finally, it is important to remember that the NPAQ-short was tested with the aim of monitoring physical activity levels in large national surveys and it might therefore not be suitable for individual feedback or intervention purposes.

When choosing physical activity questionnaires for surveys they should be robust for changes in recommendations on physical activity and they should be validated in similar populations. Therefore, it might be relevant to assess the validity of the NPAQ-short in an even larger populations outside the Nordic countries – and to reassess validity if the international recommendations on physical activity change.

When monitoring levels of physical activity in the general population it is often desirable to use the most low-cost, accurate, and robust questionnaires. This paper contributes with a short-item questionnaire (NPAQ-short). When comparing results on validity from validations studies of bigger and more time-consuming questionnaires such as NPAQ, IPAQ and GPAQ, this study of NPAQ-short has found similar or even better validity.

The NPAQ-short has already been applied to The National Danish Health Survey 2017 which gathers information about different health and morbidity measures within the grown-up population in Denmark. In the future, it would be interesting to assess the validity of NPAQ-short in other countries and settings.