Background
The impact of neighborhood conditions on health and well-being outcomes has been gaining considerable attention over the past decade [
1]. Young people may be especially affected by the neighborhood they live in due to limited mobility restricting each individual’s school, family, and peers to a confined geographic area [
2,
3]. Many studies have explored the impact of neighborhood social conditions on adolescent health outcomes including self-rated health (e.g., [
2]), alcohol use (e.g., [
3]), and violence (e.g., [
4]). In line with this increased research interest, there is a need for measurement instruments that examine the features of the neighborhood in order to better understand the relationships between the neighborhood context and adolescent health and well-being [
5‐
7]. Despite this, there are few validated and reliable measures of adolescent neighborhood conditions [
6], particularly at the neighborhood level.
Most studies examining neighborhood level conditions make use of structural measures which are based on administrative data such as census information. Recently research has moved beyond examining the structural features of the neighborhood to better understand the societal conditions present at the neighborhood level. Survey data have proven to be a useful source in understanding the social conditions of the neighborhoods in which people reside [
5,
8]. However, many studies survey adults to understand the neighborhoods in which the adolescents live, leaving adolescents ignored as active agents within their own neighborhoods [
9,
10]. Schaefer-McDaniel [
11] argues that this represents a methodological flaw and that adults cannot fully represent with accuracy the experiences and perceptions of young people in their environment. Measures derived from adolescents’ perspectives are therefore considered more theoretically valid than adult measures of the adolescent environment, as young people may have different perceptions of their neighborhood than adults, are exposed to fewer neighborhoods to compare their own with, and access different areas of their neighborhood [
12].
Neighborhoods are experienced through an individual’s perceptions and as a collective attribute at an aggregate level (a shared characteristic). Where possible, examining both collective measures and individual perceptions is desirable to allow for the most complete picture of the role of neighborhoods in adolescents’ lives. Kawachi et al. [
13] argue that studies examining the relationship between neighborhood social conditions and health should consider both individual perceptions and collective conditions using multilevel frameworks and considering cross-level interactions. For instance, socially isolated individuals may still benefit from residing within a community with positive neighborhood conditions. The construction of valid and reliable measures that operate at both the individual and neighborhood level necessitates an assessment of both psychometric and ecometric properties. Psychometric properties refer to the extent to which items reliably capture a construct at the individual level, while ecometric properties refer to the reliability at the neighborhood level [
5]. Although some studies exist detailing the psychometric properties of adolescents’ neighborhood perceptions (e.g., [
6]) fewer studies examine the ecometric properties of these measures [
12].
An important consideration when deriving neighborhood scales that will be utilized in a variety of neighborhood settings is whether the scale items are operationalized similarly for different types of regions, and what adaptations might be needed to ensure scales are appropriate across neighborhood types. The same scales therefore may not be invariant between urban and rural areas [
14]. Neighborhood scales are considered invariant when items within the scale function similarly between different groups (see [
15,
16] for a more complete discussion). This makes comparisons between groups justifiable. Two types of invariance are most frequently considered 1) factor loading invariance (
metric invariance) and 2) intercept invariance (
structural invariance). Metric invariance indicates the factor loadings are equal across groups; if this condition is met, “weak” invariance is satisfied [
17]. Reasons metric invariance may not be met include: if respondents from different groups interpret the scale items differently or if certain groups have a higher propensity to extreme responses [
16]. Structural invariance indicates that a one-unit change in the item response results in the same change on the underlying factor for both groups. This meets the condition for “strong” invariance [
17]. Structural invariance may not be met if certain groups have a different reference point when making statements about themselves, there are differences in social norms, and/or certain groups are prone to respond strongly to an item despite having comparable factor values [
16,
18]. Structural invariance implies both the meaning of constructs and levels of the underlying items are the same between groups; thus allowing for group comparisons [
19].
This research seeks to construct multi-item scale(s) measuring adolescent’s social environment in the neighborhoods in which they live. Accordingly, both individual and neighborhood measures are derived from adolescent survey data. Psychometric methods are used to validate and measure reliability of individual level measures while ecometric methods are used to measure reliability at the neighborhood level [
20]. It is important to have both valid and reliable measurements prior to conducting statistical models using these constructs. Accordingly, the objectives of this research were to: a) establish valid and reliable measures of adolescent neighborhood conditions, b) assess the psychometric and ecometric properties of these measures, c) test for invariance between urban/rural classifications, and d) generate neighborhood level scores that can be used in further analysis.
Discussion
To our knowledge, this is the first attempt to construct neighborhood scales for adolescents at both the individual and neighborhood level that takes into account potential invariance across neighborhood type. Measures across two dimensions of adolescents’ neighborhood social environment were constructed with both yielding good reliability at the individual level and moderate reliability at the neighborhood level. However, it is important to note that the response system varied for the neighborhood questions and that the EFA results largely corresponded with this. Nevertheless, the two measures perform well in CFA.
The research findings of this study are consistent with past research of psychometric and ecometric properties of adolescent neighborhood scales. Studies of rural and urban United States adolescents found similar individual level reliabilities. For example, a measure of neighborhood attachment that used some similar indicators to this study reported a Cronbach’s alpha of .72 [
50] and a measure of neighborhood deterioration using comparable measures reported a Cronbach’s alpha of .75 [
4]. Additionally, findings are consistent with a study of neighborhood level social capital in Dutch adolescents which found what the authors deem acceptable levels of neighborhood social capital at .57 [
39].
We found that adjustments to the originally specified model improved model fit and measures of invariance. The results of invariance testing indicate “weak” (
metric) invariance between different urban/rural locations for the six-item model was certainly met. There is also evidence of “strong” (
structural) invariance, however, these results are more sensitive to estimation procedure and invariance test used and therefore should be interpreted with caution. Issues with
X
2
difference test have been widely noted as it is sensitive to sample size [
15,
34,
36]. Therefore the other approaches used to test for invariance may be more appropriate and we can be reasonably confident that strong invariance is met.
Regarding the ecometric analysis, we were able to construct measures that reflect collective attributes that showed moderate reliability. Trade-offs between neighborhood sample size and reliability had to be considered as reliability decreases as a function of within neighborhood sample size. There is no established cut-offs for reliability in ecometric analysis and so the researcher must consider the trade-off between sample size and reliability as well as the purpose of the scales prior to use in future analysis. Estimates of convergent validity were as expected indicating that valid measures of the neighborhood level social environment can be constructed using survey data from adolescents. This is similar to findings based on surveys of adults [
5].
A potential limitation of the current study is that an administrative definition was used as a proxy of neighborhoods. The IDZs were constructed with consultation from those with local knowledge (by consultation with Community Planning Partnerships who coordinated the views of local people and regional officials); however, these partnerships are administratively based and therefore do not necessarily include adolescents. Additionally, the questions in the HBSC survey asked adolescents about their “local area” in which they lived but did not specify how local area should be defined and we were unable to determine how the administrative boundaries relate to the adolescents’ perceptions of their local area boundaries. This may contribute to within neighborhood variability [
5]. Despite these limitations, IDZs reflect a neighborhood definition for which other data from government sources can be linked.
Another consideration when interpreting the results is the potential for bias due to the presence of missing cases; particularly the proportion who were missing due to non-reporting of postcode data and missing data due to a low number of respondents within neighborhoods. This is a common issue in studies that collect neighborhood data but are not able to target at the neighborhood level, such as in school-based surveys (e.g., [
47]).
Although the measures established in this research are suitable for individuals experiencing urban and rural conditions in Scotland they may not be invariant cross-culturally. Further studies are needed to better understand how perceptions of neighborhoods may vary between countries. This represents an important avenue for future research of neighborhood characteristics. Additionally, the compromise between reliability, sample size, and having an appropriate number of respondents per neighborhood is an important area for future research.
Acknowledgments
The Health Behaviour in School-aged Children (HBSC) study is an international survey conducted in collaboration with the WHO Regional Office for Europe. The authors would like to acknowledge the HBSC international research network that developed the study’s research protocol. For details, see
http://www.hbsc.org. Funding for the Scottish Survey was provided by NHS Scotland. This work was also supported by the 600th Anniversary PhD Scholarship which was awarded to Gina Martin by the University of St Andrews.
Additionally, we thank the Regional and Island Authorities for granting permission for their schools to participate in the survey; and all the young people who generously completed questionnaires; and the schools and teachers who kindly agreed to administer the survey.