Background
Methods
Main search strategy – peer-reviewed literature
Initial inclusion criteria
Final inclusion criteria – and study quality assessment
High | ‘High’ level of evidence for a representative sample (or ‘low risk’ of selection bias).
| 1.The investigators describe a clear, defined and reliable sample frame for the target group •Sample frame already known e.g. hidden group reliably detected in census data or registry •Sample frame created by the researchers and includes the vast majority of the hidden target group 2.The investigators describe a random component in the process of drawing from the sample frame such as: •Referring to a random number table; •Using a computer random number generator *Note: Must fulfil both 1 and 2 criterion for a judgement to be made of ‘high’ level evidence for representativeness. |
Medium | ‘Medium’ level of evidence for a representative sample (or ‘medium risk’ of selection bias) | Sampling frame and sampling processes are applied from both the high and low below criteria. Non-random sampling with •Multiple efforts and techniques used with the a priori aim to approximate a representative sample including two or more of: o Respondent driven sampling (RDS) o Quota sampling o Maximum variation o Multiple starting points for snowballing o Sample representativeness ascertained e.g. sample compared with census demographics of the hidden group. |
Low | ‘Low’ level of evidence for a representative sample (or ‘high risk’ of selection bias) | 1.The investigators do not use a comprehensive sample frame for the hidden target group •Sample frame likely to exclude a significant proportion of the target group 2.The investigators describe a non-random component in the sampling process, for example: •Snowballing •Convenience sampling 3.Low sample numbers 4.Low response rates |
Unclear | ‘Unclear’ evidence for a representative sample (or unclear selection bias). | Indicates a lack of information about the sample frame and sample drawn. |
Grey literature search strategy
Data collection
Results
Author(s) | Country | Study design & focus | Sample frame: Type of ‘hidden’ population residing in the community | N and response rates |
---|---|---|---|---|
Fenta et al. 2006 [32] | Canada | Cross-sectional, mental health | Ethiopian immigrants/refugees |
n = 340, response rate 85%. |
Silove et al. 2007 [39] | Australia | Cross-sectional, mental health | Vietnamese refugees who have been in Australia for 10+ years |
n = 1,161, response rate 82% |
Australia
|
Longitudinal, mental health
|
Refugees granted residency within previous 3–6 months
|
n
= 2,400, response rate approximately 60%
| |
Australia
|
Cross-sectional, health (or applicable to health)
|
Irregular maritime arrivals to Australia issued with a protection visa within specified timeframe
|
n
= 1,008, response rate 47%
| |
Serbia
|
Cross-sectional, mental health
|
Refugees, predominantly from former Yugoslavia and Croatia
|
n
= 3,684, response rate not reported
| |
Canada
|
Cross-sectional, mental health
|
Refugees (Afghan 22%)
|
n
= 501, response rate 41%.
| |
Cochran et al. 2013G/Ao (2016) [23]* | USA | Cross-sectional, mental health | Bhutanese refugees |
n = 579, response rate 73% |
Maximova & Krahn 2010 [36] G
| Canada | Cross-sectional, mental health | Refugees (63% Yugoslavian) |
n = 525, overall response rate (in parent study) 95% |
Gerritsen et al. 2006 [33] | The Netherlands | Cross-sectional, mental health | Refugees & Asylum seekers |
n = 178, response rate (for refugees) 59% |
Spring et al. 2003 [40] | USA | Multiphase epidemiologic study, torture prevalence | Somalian and Oromo refugees |
n = 1,165, response rate 97.1%. |
Bhui et al. 2006 [24] | UK | Mix-method, mental health | Somalian refugees |
n = 143, response rates 76%–83% |
Bilsborrow et al. 2011 [25] | USA | Cross-sectional, wellbeing | Colombian migrants (including asylum seekers) in Ecuador |
n = 234 households, response rate 76% |
Blight et al. 2006 [26] | Sweden | Cross-sectional, mental health | Refugees from Bosnia-Herzegovina |
n = 413, response rate 63.5%. |
Heeren et al. 2012 [34] | Switzerland | Cross-sectional, mental health | Asylum seekers who arrived less than 2 years ago in Zurich |
n = 126, response rate 68.3% |
Khavarpour & Rissel 1997 [35] | Australia | Cross-sectional, mental health | Iranian migrants and refugees |
n = 413, response rate 99% (phone) n = 161 (follow-on postal survey) |
Qiu et al. 2012 [38] | China | Cross-sectional, investigating sampling & applicable to health research | Migrants in China |
n = 1,270, response rate not reported |
Vial et al. 2014 [18] | USA | Cross-sectional, health | Men who have sex with men |
n = 3,640, response rate not reported |
Wylie & Jolly 2013 [19] | Canada | Cross-sectional, health & investigating sampling | Men who have sex with men and sex workers |
n = 578, response rate not reported |
Bogic et al. 2012 [27] G
| Germany | Cross-sectional, mental health | Refugees from former Yugoslavia |
n = 854, response rate 52.9% |
Dunlavy 2001 [31] G-Thesis
| Sweden | Cross-sectional, mental health | African refugees and immigrants |
n = 420, response rate not reported |
Main search and separate grey search
Author(s) | Community representative sample* | RAS | Sampling method(s) | Sampling techniques | Sampling considerations | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
a priori aim | Used registry/census data | ||||||||||
Multiple non-probability methods | Probability (random) component | Network-based | In sampling/recruitment | In assessing representativeness | |||||||
Snowballing | RDS | Online | |||||||||
Fenta et al. 2006 [32] | high | Y | Random sampling from created frame | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ Lists from Ethiopian organizations, telephone directory | ✗ |
Silove et al. 2007 [39] | high | Y | Probabilistic sampling from created frame (house-to-house screening) | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ |
high
|
Y
|
All eligible refugees listed in government settlement database were invited
| ✗ |
All in sample frame invited to participate
| ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | |
high
|
Y
|
Sample frame of eligible refugees listed in government settlement database. Quota sampling also used
| ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | |
med/high
|
Y
|
Multistage stratified sampling
| ✗ |
Unclear
| ✗ | ✗ | ✗ | ✗ | ✓Municipality registries
| ✓ | |
med/high
|
Y
|
Random sampling from created frame
| ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓
Government database
| ✓ | |
Cochran et al. 2013G/Ao (2016) [23]* | med/high | Y | State-based stratification with random sampling from created frame. Supplementary purposive and probability proportional to size sampling | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ |
Maximova & Krahn 2010 [36] G
| med/high | Y | Systematic sampling (every nth name) from sampling frame | ✗ | Systematic every nth name in database | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ Government settlement database |
Gerritsen et al. 2006 [33] | med/high | Y | Random samples of refugees were obtained from population registries, plus asylum seekers living in randomly selected reception centres | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ |
Spring et al. 2003 [40] | med/high | Y | Multiple purposive sampling methods: Targeted, convenience, snowball sampling | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✓ School enrolments, birth statistics, state resettlement records |
Bhui et al. 2006 [24] | medium | Y | Community based sampling (convenience) & primary care registry lists (random) | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ |
Bilsborrow et al. 2011 [25] | medium | Y | Oversampling (probability sampling) first with supplementary snowball sampling | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ |
Blight et al. 2006 [26] | medium | Y | Random sample drawn from a large registry of community living target group | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ |
Heeren et al. 2012 [34] | medium | Y | National register of adult asylum seekers (sampled consecutively) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ |
Khavarpour & Rissel 1997 [35] | medium | Y | Snowball sampling with strategies to access diverse social networks | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ |
Qiu et al. 2012 [38] | medium | N | Respondent driven sampling | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✓ Gender ratio |
Vial et al. 2014 [18] | medium | N | Field (convenience) and online sampling | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
Wylie & Jolly 2013 [19] | medium | N | Respondent driven sampling | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ |
Bogic et al. 2012 [27] G
| medium | Y | Multiple random and non-random sampling: resident registers, snowballing, community-based sampling | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ |
Dunlavy 2001 [31] G-Thesis
| medium | Y | Non-probability stratified quota sampling, community-based snowballing with multiple starting points | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ |
Evidence level of a community representative sample
Sampling techniques
Other design issues
Author(s) | Long recruitment(>12 months) | Engagement with ‘hidden’ group | Barriers noted |
---|---|---|---|
Fenta et al. 2006 [32] | ✓ 12 months | Field-workers spoke the target language | Difficult to identify Ethiopian Muslim names in the telephone directory Potential candidates may have been excluded if they had no telephone, stable address or membership in the Ethiopian organizations used to develop the sample frame |
Silove et al. 2007 [39] | ✗ | Field-workers spoke the target language | Sampling strategy favoured Vietnamese refugees living in ethnically dense areas |
✗ |
Community consultation during development of design and methodology Community Engagement officers (members of local migrant communities) recruited to advocate for study, assist with recruitment etc.
Field-workers spoke the target language
Interviews conducted in respondent’s homes
|
The high mobility of the target sample made obtaining accurate contact information challenging
| |
✗ |
Bilingual assistants available to assist with survey administration
|
Participants in initial sample excluded if lived in non-metro areas of target cities, lacked a valid phone number or encountered significant language barriers
| |
✗ |
Contact details of refugees living in private accommodation not all available/correct in municipality records – highly mobile refugees may have been excluded. Substitutions identified by “trustees” – no explanation of how these selections were made
| ||
Citizenship and Immigration Canada 2011 [28] G-Report
| ✗ |
Promotional materials (posters, FAQ brochures) distributed to service provider organizations to encourage eligible participants to respond
|
Consent given through the returning of a postal questionnaire. Possible self-selection bias (e.g. higher proportion of university education). Poor health or mental health could have been associated with non-response
|
Cochran et al. 2013G/Ao (2016) [23]* | Not reported | Field-workers spoke the target language Interviews conducted in respondent’s homes | Lack of contact information for eligible participants |
Maximova & Krahn 2010 [36] G
| ✗ | Refugees without available addresses in the government database were excluded, as were those who had relocated from study site | |
Gerritsen et al. 2006 [33] | ✗ | Field-workers spoke the target language Potential respondents contacted by letter and in person | Recruitment only conducted in municipalities that agreed to provide researchers with contact details of potential participants One third of potential participants had incorrect contact details or were absent when interviewers visited |
Spring et al. 2003 [40] | ✓ 25 months | Field-workers spoke the target language. Interviews conducted in respondent’s homes. Field staff maintained a presence in the communities, including after hours and weekends. Created marketing materials (e.g. posters) and recruited at many varied community events and locations | Limited to one person per household. Analyses indicated some significant differences on outcome variables depending on recruitment strategy |
Bhui et al. 2006 [24] | ✓ 12 months | Researchers of same ethnicity as target population networking with local stakeholders to gain trust. Data collection also at weekends and evenings | Census data in the UK does not include country of origin. Authors note that this makes establishing a reliable sampling frame difficult. It was also noted that research fatigue and a failure to see immediate benefits to health and social status were additional barriers to participating in research |
Bilsborrow et al. 2011 [25] | ✗ | Use of archival census data could not identify recent or highly mobile refugees/migrants, or those living in the country illegally | |
Blight et al. 2006 [26] | ✗ | Attempts made to reduce focus on ethnicity in the questionnaire & cover letter to account for refugees who no longer identify as refugees | Consent given through the returning of the postal questionnaire. Poor health or mental health (such as concentration difficulties) could have resulted in non-completion. |
Heeren et al. 2012 [34] | ✗ | Reasons for non-participation included lack of time, indifference, distrust of researchers. Authors noted that RAS may feel intimidated or fearful of the interview situation, which may remind them of interviews or interrogations with officials in their home country | |
Khavarpour & Rissel 1997 [35] | ✗ | Field-workers spoke the target language | The mailed survey component of the study required participants to supply a postal address. This loss of anonymity was a noted barrier to participation |
Qiu et al. 2012 [38] | ✗ | Recruitment from multiple locations to promote respondent convenience | Identified barrier was that participants generally did not travel far to participate Difficult to obtain the trust of potential seeds in a short time |
Vial et al. 2014 [18] | ✗ | Staff partnered with community organizations and local stores frequented by target population | 21.9% of participants who completed the survey were excluded: approximately half of these did not meet inclusion criteria and others had missing data |
Wylie & Jolly 2013 [19] | ✗ | Multiple methods for seed selection improved access to target group | Seed selection significantly influenced which subgroups within a population were accessed |
Bogic et al. 2012 [27] G
| ✓ 22 months | Interviews conducted at multiple sites | Authors suggest that the difficulty in recruiting a representative sample of refugees was linked to the absence of detailed population data in the target countries. The lack of registry data in the UK (compared to Italy and Germany) resulted in variation in recruitment methods across countries, which may have led to non-representative samples |
Dunlavy 2001 [31] G-Thesis
| Not reported | Local cultural, community and political organizations assisted with recruitment Interviews conducted in locations convenient to participants Translators available to assist with survey administration | Snowballing methodology naturally excluded those not connected with the social networks targeted in the study |