After the Focus Group transcription were analysed, the relevant variables to evaluate adolescents’ were: nutrition, physical activity, substance abuse, relaxation, hygiene, and the use of technology for communication or for leisure. No questions related to sexuality were considered because the data on the initiation of sexual intercourse in Catalonia indicate that adolescents initiate sexual activity at 15.8 years of age for girls and 15.7 years of age for boys [
76]. In secondary school, 65.3 % of boys and 72 % of girls had not engaged in sexual intercourse [
77], and thus, this component was not included because it would be irrelevant to a large group of adolescents.
In the past five years, there are not published validations of questionnaires that have scored using all of these variables. However, there are some who value certain variables separately. In 2011, Wright et al. [
78] validated the HABITS questionnaire to assess lifestyle in children 7–16 years of age. It consists of 19 items with closed-response questions related to diet (frequency of consumption of different types of food and drinks) and the time spent watching television and playing electronic games. In 2012, Muchotrigo validated the Healthy Lifestyle Questionnaire (Cuestionario de Estilo de Vida Saludable, in Spanish) [
79] for university students. In this case, 26 items with Likert-type responses are grouped into the following three components: sports activities, diet, and relaxation/sleep. Other studies have not validated instruments to assess lifestyle but may include this variable in some of analyses based on other questionnaires. In 2009, Vereecken et al. [
80] studied the relationship between breakfast habits and lifestyle in adolescents 11 to 15 years of age from 45 countries. To assess lifestyles, questions were based on issues relating to substance abuse (alcohol and tobacco), physical activity, hours watching television, and eating habits (consumption of vegetables, fruit, and soft drinks) that appear in the HBSC.
Furthermore, other questionnaires that assess lifestyle include other variables. In 2012, Taymoori et al. published the validation of the
Healthy Lifestyle Questionnaire (HLQ) for Iranian adolescents [
49]. The final version consisted of 36 items drawn from different questionnaires on lifestyle of adolescents that are grouped into the following six factors: life appreciation, health responsibility, nutrition, social support, physical activity, and stress management. In 2013, Dinzeo et al. validated an abridged version of the
Lifestyle and Habits Questionnaire (LHQ) for young (18–25 years of age) university students in the United States. The original version of LHQ consisted of 80 items, and its validation was published in 1998 [
81]. The new version contains 42 items grouped into the following eight factors: physical health and fitness, psychological health, substance abuse, nutrition, environmental awareness, social awareness, accident prevention, and the meaning of life. Despite their multidimensional approach, both of the questionnaires were noted to be missing questions related to topics that are important for teens, such as the use of the Internet to interact and communicate. Both questionnaires (HLQ and LHQ) use Likert-type responses.
In the four questionnaires mentioned above, questions always have closed responses. The VISA-TEEN also contains closed-response questions but includes open-response quantitative questions and one ordered-choice question as well. Open-response questions of a quantitative type permit more accurate information to be obtained about variables for which it is more important to know a number (hours, cups) rather than an approximate interval or a qualitative assessment of frequency of performance or consumption. Other questionnaires that also use open-response questions are of a quantitative type. Some examples include the
International Physical Activity Questionnaire (IPAQ), which poses questions concerning time in hours and minutes devoted to performing different types of physical activity during the past seven days [
73], and
Systematic Interrogation of Alcohol Consumption (
Interrogatorio Sistematizado de Consumos Alcohólicos, in Spanish) (SALGA), Department of Health of the Government of Catalonia [
82], which poses questions related to the number of standard drink units (SDUs) consumed in one week.
The analysis of the items in this study showed that all had good comprehensibility. The item that was rated as the most difficult was item number 4, concerning moderate and intense physical activity performed each day in reference to the previous week. The mean and median difficulty of this item were 3.11 and 3 points, respectively, based on a scale ranging from 0 to 10 (10 is the maximum difficulty). Qualitative inputs made by some of the subjects about this item suggested it should be divided into two parts. Additionally, examples of each type of activity were added to the final version of the questionnaire, thus facilitating the response process. All other items showed an average difficulty from 0.29 (question 9 concerning hand-washing and tooth-brushing) to 0.93 (question 1 related to food intake). In the latter case, it was considered especially important for this question to present a low difficulty, as ordered response questions tend to be difficult to answer [
83]. In this item, this type of question was considered most suitable for assessing the adequacy of consumption of different types of food reported by adolescents with respect to the food pyramid. The pyramid, in its latest version (2012), does not specify the portions of different types of foods to consume but rather establishes an order of consumption (daily, weekly, and occasional) [
84].
The diversity in the types of questions could complicate the comprehensibility and the response process by adolescents. To avoid these errors in the response process, and responding to suggestions made by some subjects who participated in the evaluation of comprehensibility, questions were ordered by type of response (ordered-choice followed by quantitative-response followed by closed-response) and instructions for answering each type were provided.
Reliability: internal consistency
Regarding reliability, the value of both α and stratified α is above 0.65, thus demonstrating acceptable internal consistency that permits the use of the questionnaire in descriptive population studies, the objective for which the questionnaire was developed. The analysis of the characteristics of each item helped to confirm that there were none that needed to be removed to significantly increase the value of α. The corrected item-total correlations ranged between 0.138 and 0.423. Because the correlation was significant in all cases, and after verifying that the removal of any item did not improve the value of α, it was decided to retain all items for the final version. Even items that had less than a 0.20 (but significant) correlation remained in the questionnaire because it was decided that they provided interesting and necessary lifestyle information about adolescents.
The reliability (temporal stability) studied using the ICC obtained for the lifestyle total score demonstrated a very good agreement between the two occasions. In the individual analysis of each item, values ranged from 0.54 (good agreement) to 0.95 (very good agreement). Therefore, the items studied were accepted. Content validity was validated by the theoretical analysis, the involvement of stakeholders, and the classification conducted by the experts in the development phase of the questionnaire.
In terms of construct validity (EFA and IFFS), from the assessment of exploratory factor analysis, five components were extracted. Four were expected, and each contained items that were conceptually related (physical activity, substance abuse, hygiene, and diet). A fifth component, which was designated as RUTL, contained items related to entertainment technology and sleep. The relationship between these two variables and confirmation that they can be studied within a single component is supported by studies demonstrating the relationship between these variables. In 2013, Spies, Shapiro and Margolin analysed the existing evidence on the relationship between the use of social networking and psychosocial development of adolescents. Among other consequences, the authors found a relationship between intensive computer use, including online communications, and hours and quality of sleep [
88]. The same year, Wolniczak et al. found a dependent relationship between Facebook and sleep quality [
89]. Also in 2013, Don et al. concluded that excessive Internet use negatively influences health because of its relationship with few hours of sleep [
90].
The IFFS values were acceptable for the RUTL and hygiene components and very good for physical activity, substance abuse, and diet. Therefore, the items assigned to each component are adequate, and the components are sufficiently independent from each other to allow for separate analysis.
For construct validity (hypothesis testing), first, the relationship between the total score for VISA-TEEN and that obtained for KIDSCREEN-10 was assessed. The correlation coefficient between the scores on the two questionnaires was r = 0.21 (
p < 0.001). This correlation, though weak, is significantly different from 0 and is positive. Additionally, it is superior to the one that presents the same KIDSCREEN with physical health measurement offered by the
Child Health Questionnaire (CHQ), which is r = 0.15 [
58].
Second, the association between the total score on the VISA-TEEN and the assessment of perceived health was checked using the SRH. Scores were found to be significantly different when they were ordered by group, observing that the best scores for lifestyle corresponded to adolescents who reported excellent health, and the worst scores were for those who reported having poor health. In the intermediate group, scores for VISA-TEEN diminished when perceived health worsened.
Post hoc tests showed that there were differences between all groups except between "Excellent" and "Very Good" and between "Fair" and "Poor". A relationship between some of the components of lifestyle and SRH was also found in the study conducted in Spain by Giron in 2012 [
91], which concluded that diet and substance abuse influence the perception of health in young people, and in the study conducted in Greece in 2011 by Darviri et al. [
92], which concluded that the factor most correlated with low SRH in adolescents is little physical activity.
Finally, the relationship of the various components of lifestyle with age, sex, and purchasing power was demonstrated. All components except hygiene were negatively associated with age (older, lower score). This tendency for declining health with age coincides with that found in other studies. Thus, in the case of physical activity, the HELENA study conducted throughout Europe shows how the average hours of daily physical activity diminishes with age (2 h at 13 years of age, 1.4 h at 17) [
93]. With respect to substance abuse, increased risk behaviour with age can be found in both the HBSC-2010 worldwide [
94] and in the local-level FRESC-2012 [
75]. As for diet, Diaz and Trave found that the KidMed rate of adherence to the Mediterranean diet diminishes with age [
95]. Additionally, both the HBSC-2010 and the FRESC-2012 show that consumption of fruit and the percentage of adolescents who eat lunch daily decreases with age. As for entertainment technology, results from FRESC-2012 show that the time spent chatting increases from 13 to 16 years of age, and then decreases slightly. The relationship with purchasing power was not significant for any component. This is inconsistent with other studies, such as HBSC-2010 [
2], which did find an association between some of the components studying lifestyle and purchasing power. This fact could be due to the statement previously mentioned above, namely, an update of the criteria used to establish the socio-economic levels may be necessary because such an update has not been performed since 2002.
According to the way that EMPRO specifies assessing the quality of questionnaires results perceived by patients [
57], criterion validity must be evaluated when there are shortened versions of existing validated questionnaires. This is not a prerequisite for new questionnaires, as it often happens that other validated measures assessing the same construct (gold standard) do not exist in order to make the comparison. In our case, we did not have a validated instrument that we could use as a criterion, and therefore, the criterion validity was assumed to be reaffirmed by the construct and content.
With respect to discriminatory power, the value of δ = 0.972 indicates that the questionnaire provides good discrimination. The KIDSCREEN questionnaire, which assesses HRQoL, provides discriminatory power between 0.94 and 0.98 in different versions of 52, 27, or 10-items [
58].