Sampling challenges in a study examining refugee resettlement

To assess the various sampling techniques adopted by other researchers working with similar refugee groups, we conducted a literature search in 2007 using Google Scholar, ProQuest and PubMed databases. Because of our ethnic focus and desire for cultural sensitivity, we only included studies in western settings involving refugee or asylum seeker participants from our target regions, namely Afghanistan, Iraq, Iran or the Kurdish areas of the Middle East. A number of published articles were identified, reporting a range of different approaches to participant selection and strategies for sampling. Many of these had a qualitative focus and as we were principally interested in quantitative sampling strategies, we carefully reviewed nine relevant articles which undertook quantitative assessment and analysis (Table 1). Of these, a range of different techniques were used: one study did not specify the sampling method [3], two obtained random samples (using national survey data, or based on country of birth) [4, 5], one approximated a random sample by using a systematic technique where every 4^th person was included, and another attempted a random selection in collaboration with clan leaders [6, 7]. However, methodological issues with these approaches can be identified. For example, a systematic technique relies on access to a database of refugees that is of sufficient size to allow an adequate sample size to be drawn, and in practice, this is often not feasible. Some concern can also be expressed about a study where collaboration with clan leaders for participant recruitment may introduce gatekeeper bias, or raise concerns about coercion [8]. Of the remaining studies, in two the investigators approached all members of the study group e.g. all asylum seekers registered with an NGO [9, 10], while the other two used convenience sampling methods [11, 12]. Our eventual conclusion was that there are multiple challenges in obtaining information from these vulnerable refugee population minorities and that no apparent consensus exists on the most appropriate methodologies for use with these populations.

The exploratory study adopted a mixed methods approach, combining qualitative interview responses to open ended questions about health and resettlement experiences with quantitative assessment of psychological distress, subjective wellbeing and general perceived self efficacy. The initial phase of the study involved extensive profiling of each host community through interviews with refugee community leaders, interpreters and cross cultural workers, refugee agencies, health workers, refugee legal advocates and immigration personnel [13‐15]. Data obtained guided language considerations and awareness of specific cultural requirements, as well as helping to define the social structure of each community [15]. Pre-translated and culturally validated Farsi, English and Arabic versions of the Kessler-10 Psychological Distress Scale (K-10) [16, 17], General Perceived Self Efficacy Scale (GPSE) [18‐21], and Australian Unity Well Being Index (Personal Well Being Index PWI) [22] were provided for self completion, or with interpreter assistance if necessary, to quantitatively measure the variables of interest. Nearly 90% of questionnaires were self completed during the first part of the interview, although this varied amongst the groups with almost no Australian Kurds or New Zealand Afghans requiring extra assistance. The second part of the 1-2 hour interview involved more in-depth discussion around open ended questions; this was normally conducted in English. Translations of all additional study materials (information sheets and consent forms), prepared using standard back translation methods [23] were provided in Farsi and Sorani (Kurdish dialect), as well as English, to ensure ethical standards were maintained and to help overcome reticence to participate due to host language limitations [14]. A full description of the reasons for selection and characteristics of the instruments is available elsewhere [24].

Ethical approval for the study was obtained from the Human Research Ethics Committee, Curtin University in Perth, WA.

Study participants were of Kurdish or Afghan ethnicity, 18 years or older at the time of the study, who arrived in Australia or New Zealand as refugees or asylum seekers between 1988 and 2008, and who were resident in either Christchurch or Perth at the time of data collection in 2008. The inclusion of participants settled for up to 20 years allowed a longer term focus, and also meant that the Kurdish group resident in Perth, who mostly arrived during the 1990 s, could be included. Although this meant a small number of eligible participants were children at the time of arrival, in practice all except two were of school age and had clear recollections of the resettlement experience. We are unaware of any other studies looking at former refugees up to 20 years post-resettlement, so our findings should be of wider significance.

As one aim of our study was to compare groups, quantitative data was needed, so we wanted to obtain as representative a sample as possible, which reflected the composition of the groups, given these constraints. We acknowledge that truly representative samples can only be obtained through random sampling techniques, but within the context of our exploratory study of previously unstudied groups, our goal was primarily to provide as wide a cross section of participants as possible. Because former refugee communities may be socially invisible and wary of outsiders [13, 25], a snowball sampling method, where identification of potential participants, introductions and approval of study objectives and researchers is assured through personal endorsement, was the preferred approach. It has been acknowledged that when 'attempting to study hidden populations for whom adequate lists and consequently sampling frames are not readily available, snowball sampling methodologies may be the only feasible methods available.'[26] However, to reduce selection bias inherent in this method, multiple entry points into the communities were used, choosing as wide a range of people as possible to provide further contacts [13, 25, 27]. It has been suggested that greater heterogeneity in snowball samples, and improved representativeness can be achieved by increasing sample sizes, using quotas for key demographic variables, use of multiple starting points for snowball initiation and using a small number of links within each chain [13, 27]. Similarly, less reliance on community based or gatekeeper organisations for locating potential participants may ensure more marginalised or less socially active individuals are located [13]. With the exception of the Christchurch Kurdish community, between six and eight discrete snowball initiation points were used with each of the other three groups, with a variety of people radiating out from each 'entry' point. Although a quota was not formally applied, care was taken to ensure that these key people represented a cross section of each community by including a range of different Afghan ethnic groups (Hazara, Tajik, Uzbek etc) and religious affiliations (Sunni, Shi'a etc) who move in different social circles [8]. Contacts were not generally sourced through refugee resettlement agencies to reduce possible gatekeeper bias [13, 27] in selection of individuals who could provide 'appropriate' responses and also to minimise identification of the study with agency agendas. For the New Zealand Kurdish group however, the small population size and cohesive, well structured community meant all eligible adults could be approached. As our study was exploratory, rather than hypothesis testing, sample size was based upon logistical considerations, including the small size of the communities from which respondents could be drawn, so our aim was to recruit between 40-50 individuals from each group to provide sufficient data for some basic descriptive statistical comparison. The specific strategies used for community access in each location are described in Table 2.

Quantitative data was analysed using SPSS 12.0 (SPSS Inc.). Frequency distributions for demographic variables were calculated, with results split by resettlement location and refugee community. Non-parametric tests were performed to assess differences between groups.

Lack of a clear sampling frame [13, 25, 26], a widely acknowledged problem with refugee research, and limitations with Census data [8, 13, 28] hinder attempts to accurately gauge representativeness. However, to assess the likely representativeness of our sample, aggregated Afghan and Kurdish data was compared with 2006 Census data for Western Australian (WA) residents born in Afghanistan, Iran and Iraq (Table 4) [29‐31]. Appropriate Census data from New Zealand was not available for comparison, although anecdotal reports from within the communities suggest that our sample reflected local demographics. With such hidden populations, it has been suggested that construction of a 'tentative map' of community demographics developed in consultation with professionals working with the target groups or community members may be helpful in order to judge representativeness [26]. In our case, this was done informally through consultation and discussion with community members. Due to the invisibility of Kurdish ethnicity in immigration statistics, Iraqi and Iranian census data was used as a compromise substitute, although significant limitations exist with this. In particular, the Kurdish geographic region spreads across Iran, Iraq, Turkey, and Syria, with small populations elsewhere in the Middle East, and it is an individual's country of birth that is recorded in government statistics. As the Kurds comprise minority groups in these countries, a direct comparison with natal population demographics is likely to be misleading. However, as shown in Table 4 our sample did achieve a good gender and age balance when compared with the ethnic populations in WA. Religion was predominantly Muslim in our sample, which is consistent for people of Afghan and Kurdish background. The Iran and Iraq-born groups in WA also include a significant number of Baha'is, Armenian apostolic adherents and Catholics who sought refuge from religious persecution as minorities in the 1980 s [31], which explains the disparity with this variable. Overall, our Afghan sample was slightly better educated (59% with some form of tertiary qualification, compared with 34% of Afghans in WA), and both groups were settled longer (almost half arrived before 1996, compared with about one third of the respective populations). Our sample also appears to speak better English; however, this result is distorted by the definition of speaking English. In the Census, participants are asked to rate their English ability as speaking English very well, well, not well or not at all; only those who speak it well or very well were included in the table data. In comparison, we dichotomised responses into functional/no functional English to assess levels of English literacy, so our results will appear positively skewed for this variable.