Background
Methods
Identification of publications
Screening and eligibility
Classification
Extent of community participation
Seven levels of community participation | Four phases of project development | |||
---|---|---|---|---|
Diagnosis | Development | Implementation | Evaluation | |
1. No participation | Completely top-down, community is not informed about or asked about issues in their community. | Top-down, community is not informed about the development of the project. | Top-down, community is not informed about the implementation of the project, only about activities they’re involved in | Top-down, community receives no information about evaluation. |
2. Passive participation | Outsiders decide on the issues that need to be addressed, community is informed. | Outsiders control development, community is informed, but has no input. | Outsiders control the implementation, community is informed, but has no input. | Outsiders control the evaluation, community is informed, but has no input. |
3. Participation by information | Outsiders have control, community participates by providing information about their community. No feedback to the community and no checking for agreements. | Outsiders have control over development, community potentially provides information about what they want, but outsiders don’t necessarily respond to this. | Outsiders control implementation, community might provide information useful for implementation, but outsiders don’t necessarily listed to this. | Outsiders control evaluation, community provides information through surveys and/or interviews, focus groups. Findings are not shared or checked for accuracy. |
4. Participation by consultation | Outsiders define problems and consult with community about their agreement, using outsider defined processes. | Outsiders consult with community about potential projects to develop, but outsiders make final decision. | Community participates in activities decided upon by the outsiders | Outsiders define evaluation process, community provides information and might make suggestions for improvement and feedback provided |
5. Functional participation | Outsiders have predetermined goals and community assists in defining issues within those goals, outsiders make final decisions. | Community works together with outsiders to develop projects decided upon by the outsiders. | Community and outsiders work towards implementation of projects, based on outsiders’ goals and processes. | Community and outsiders work together in evaluation, based on goals as set by the outsiders. |
6. Interactive participation | Outsiders and community work together to identify the issues in the community and set goals for the project. | Outsiders and community work together to develop suitable projects to address the agreed upon goals. | Community and outsiders implement the developed projects together, community has control and uses local resources. | Evaluation methods are decided upon together and conducted in partnership. |
7. Self-mobilisation | Completely bottom-up, community identifies their own issues and sets their own goals, might contact outsiders to assist them where needed. | Bottom-up, community makes decisions about project development, apply for funding and potentially contact outsiders where needed | Community implements projects, contacts outsiders for resources where needed, but remains in control over resources. | Community conducts evaluations, potentially contacts outsiders for assistance, but stays in control over evaluation. |
Critical appraisal of methodology
Qualitative methods
Quantitative methods
Scoring for critical appraisal of methodology
Outcomes of the studies
Results
Extent of community participation
First author (year) | Four phases of project development | |||
---|---|---|---|---|
Diagnosis | Development | Implementation | Evaluation | |
Gauld et al. (2011) [40] | 1a | 2–5b | 2–5 | 5 |
Green et al. (2009) [67] | 4 | 4 | 4 | UNK |
McMurray (2012) [49] | 7 | 6 | 6 | 3 |
Parker et al. (2006) [48] | 6 | UNK | 5 | 4 |
Murphy et al. (2004) [41] | 3 | 7 | 5 | 4 |
Hunt (2010a) [59] | 7 | 7 | 7 | 6 |
CLC (2012a) [60] | UNK | UNK | 4 | 3 |
CLC (2012b) [51] | 7 | 7 | 7 | 3/4c |
CLC (2012c) [61] | UNK | UNK | UNK | 3/4 |
CLC (2012d) [62] | UNK | 3 | 3 | 4 |
CLC (2012e) [63] | UNK | 4 | UNK | 3 |
Taylor (2005a) [56] | 1 | UNK | 5 | 4 |
Taylor (2005b) [64] | UNK | UNK | 4 | 4 |
Ramsay (2005a) [57] | UNK | UNK | 5 | 4 |
Ramsay (2005b) [68] | 1 | 3 | 4 | 4 |
Burchill (2005) [65] | UNK | 3 | 4 | 4 |
Higgins (2005) [52] | 7 | 7 | 7 | 4 |
Bromfield (2005) [55] | 1 | 5 | 4 | 4 |
Ramsay (2005c) [66] | 2 | UNK | UNK | 4 |
Tsey (2003) [69] | 1 | 3 | 4 | 3 |
Tsey et al. (2004); [39] | 5 | 6 | 6 | 5 |
Smith (2004) [53] | 7 | 3 | 6 | 6 |
Lee et al. (2008) [42] | UNK | 2–5 | 4–5 | 4 |
Tyrrell et al. (2003) [43] | 6 | 6 | 6 | 5 |
Guenther (2011) [58] | UNK | 2 | 5 | 2 |
Salisbury (1998) [44] | 5 | 5/6 | 5/6 | 6 |
Hunt (2010b) [59] | 1 | 2 | 4 | 5 |
Moran (2003/2004) [45] | UNK | 4 | 4–1 | 2 |
McCalman (2005) [54] | 7 | 7 | 7 | 4 |
Jarvie (2008) [46] | 1 | 5 | 5 | UNK |
Shannon et al. (2001) [47] | 3 | UNK | UNK | 3 |
Seven levels of community participation | Four phases of project development | |||
---|---|---|---|---|
Diagnosis | Development | Implementation | Evaluation | |
1. No participation | 7 | |||
2. Passive participation | 1 | 3 | 2 | |
3. Participation by information | 2 | 3 | 6 | |
4. Participation by consultation | 1 | 5 | 10 | 14 |
Least active involvement sub-total (levels 1–4)
|
11
|
11
|
10
|
22
|
5. Functional participation | 1 | 4 | 8 | 4 |
6. Interactive participation | 3 | 4 | 5 | 3 |
7. Self-mobilisation | 6 | 5 | 4 | 0 |
Most active involvement sub-total (levels 5–7)
|
10
|
13
|
17
|
7
|
Unknown | 10 | 7 | 4 | 2 |
Total | 31 | 31 | 31 | 31 |
Methods used in studies
Methodological quality of studies with a qualitative component
First author (year) | Data collection | Data analysis | Potential bias | Implications |
---|---|---|---|---|
Qualitative only studies (n = 21) | ||||
Gauld et al. (2011) [40] | Little detail | Not described | Not described | Generalised to far northern Queensland communities |
Green et al. (2009) [67] | Detailed description, including interview questions | Detailed description and linked to literature | Not described | Generalised to organisations working with Indigenous communities in Australia and policy |
McMurray (2012) [49] | Detailed description of field work | Not described | Position of researcher described | Implications for the funding agency |
Parker et al. (2006) [48] | Not described | Not described | Not described | Described for health promotion work in Indigenous communities |
Murphy et al. (2004) [41] | Detailed description | Not described | Not described | Appreciative inquiry methods and culture projects |
CLC (2012a) [60] | Detailed description | Description of who did analysis and triangulation | Not described | Described for organisation |
CLC (2012b) [51] | Detailed description | Description of who did analysis and triangulation | Not described | Described for organisation |
CLC (2012c) [61] | Detailed description | Description of who did analysis and triangulation | Not described | Described for organisation |
CLC (2012d) [62] | Detailed description | Description of who did analysis and triangulation | Not described | Described for organisation |
CLC (2012e) [63] | Detailed description | Description of who did analysis and triangulation | Not described | Described for organisation |
Taylor (2005a) [56] | Little detail | Not described | Not described | Generalised to comparable projects |
Taylor (2005b) [64] | Little detail, mention of development of evaluation tool | Not described | Not described | Described for future communities wanting to implement project |
Ramsay (2005a) [57] | Little detail | Not described | Not described | Generalised to Indigenous communities with comparable issues |
Ramsay (2005b) [68] | Little detail | Not described | Not described | Discussed for working with Indigenous communities |
Burchill (2005) [65] | Very little detail | Not described | Not described | Not described |
Higgins (2005) [52] | Very little detail | Not described | Not described | Not described |
Bromfield (2005) [55] | Very little detail | Not described | Not described | Discussed for practice |
Ramsay (2005c) [66] | Very little detail | Not described | Not described | Discussed for practice |
Tsey (2003) [69] | Detailed description | Detailed description | Position of researcher is discussed | Generalised to community development projects and practice |
Tsey et al. (2004); [39] | Very detailed description | Detailed description | Position of researcher discussed | Discussed for practice, policy and researcher |
Hunt (2010b) [59] | Not described | Not described | Not described | Not described |
Mixed Methods Studies (n = 8) | ||||
Hunt (2010a) [59] | Detailed description of fieldwork | Described | Researcher position and bias described | Described for organisation’s community development work |
Smith (2004) [53] | Very detailed description | Description of who analysed data, but not methods | Position of researcher discussed | Generalised to other communities, implications for project described |
Lee et al. (2008) [42] | Detailed description | Described | Not described | Generalised to communities with similar problems. |
Tyrrell et al. (2003) [43] | Not described | Not described | Not described | Discussed for practice and results |
Guenther (2011) [58] | Detailed description | Detailed description | Position of researcher discussed | Discussed for policy and practice |
Salisbury (1998) [44] | Little detail | Not described | Researcher position discussed | Generalised to health services in Indigenous communities |
Moran (2003/2004) [45] | Detailed description | Description of who analysed data, but not methods | Researcher position and bias discussed | Discussed for practice |
McCalman (2005) [54] | Very little detail | Very little detail | Not described | Discussed for practice |
Methodological quality of studies with a quantitative component
1st author, year | Selection bias (A) | Study design (B) | Confounds (C) | Blinding (D) | Data collection methods (E) | Withdrawal & drop-outs (F) | Intervention integrity (G) | Analysis (H) | Summary rating |
---|---|---|---|---|---|---|---|---|---|
Mixed method studies (n = 8) | |||||||||
Smith (2004) [53] | Moderate | Moderate | NA | Weak | Weak | Moderate | Collection of quantitative data stopped before real community action started. | Community-level allocation, individual-level analysis. No appropriate analysis of change in child growth over time. | Weak |
Lee (2008) [42]a | Weak | Weak | NA | Moderate | Weak | Moderate | Many youth involved in the interventions, no information on consistency, other community initiatives were running simultaneously (including stricter supply controls and rewards linked to school attendance). | Community-level allocation and analysis. Statistical methods described in other publication. Dates of data collection (2001–2004) do not line up with dates of intervention (2003–2005), no post-test data. | Weak |
Tyrell (2003) [43]a | Weak | Weak | NA | Moderate | Weak | Moderate | No description of who was exposed to the project and who weren’t, nor of possible external influences on outcomes. | Allocation on community and individual level. Evaluation on community, organisational and individual level. No statistical analysis (outcomes as percentages only). | Weak |
Guenther (2011) [58] | Strong | Weak | NA | Weak | Weak | N/A | All participants were part of the project; not all participants attended every session; it is likely that the results were influenced by other interventions put on the families. | Individual-level allocation and analysis; statistical analyses (frequencies and t-test) were appropriate; analysis performed on actual intervention status. | Weak |
Salisbury (1998) [44]a | Weak | Moderate | NA | Weak | Weak | Moderate | No description of exposure or consistency; no mention of other interventions influencing outcomes; tested for population growth (which didn’t grow) | Unit of allocation and analysis are on organizational level. No statistical analysis. | Weak |
Hunt (2010b) [59] | Weak | Weak | Weak | Weak | Weak | Weak | No description of exposure or consistency, potential influence of other interventions running in the communities at the same time. | Unit of allocation community and organisational level, unit of analysis individual level, no statistical analysis. | Weak |
Moran (2003) [45]a | Moderate | Moderate | NA | Weak | Moderate | Weak | 92 % of participants reported awareness of town plan. Outcomes may be influenced by the cycle of optimism and pessimism. | Unit of allocation is community level, analysis is done on community and individual level. No statistical analysis. | Weak |
McCalman (2005) [54] | Moderate | Moderate | NA | NA | Weak | Weak | All evaluation participants were exposed to intervention; consistency was not measured; outcomes likely influenced by other factors. | Project was allocated at organizational level, data were collected on community level, cannot be sure whether changes at community level are caused by changes at the organizational level. No statistical analysis | Weak |
Quantitative only studies (n = 2) | |||||||||
Jarvie (2008) [46]a | Weak | Weak | Weak | Weak | Weak | Strong | One community withdraw temporarily, others stayed. There’s a chance that outcomes are influenced by other developments going on at the same time | No statistical analysis, intervention offered at community level, data gathered at population level. | Weak |
Shannon (2001) [47]a | Weak | Moderate | N/A | Weak | Weak | Moderate | No description of exposure to intervention or consistency in delivery. Outcomes may be influenced by other factors | Community-level allocation and analysis. Appropriate statistical analysis. | Weak |
Outcomes
First author (year) | Project aim | Outcomes of study |
---|---|---|
Gauld et al. (2011) [40] | Developing culturally relevant rehabilitation service for adults with acquired brain injury. | Experienced increase in knowledge about and access to services for people with acquired brain injuries. |
Green et al. (2009) [67] | Community empowerment through arts and cultural practice | Social issues addressed; Non-indigenous staff reported improved understanding Indigenous issues; community members acquiring new skills; experienced increase in supportive relationships and friendships |
McMurray (2012) [49] | Increase self-determination of women of the community | Increased networking; improved community governance; increased livelihood opportunities. |
Parker et al. (2006) [48] | Introduce Indigenous games in schools to increase physical activity | Process evaluation: most people were satisfied with forum, workshops and activities; project is transferred to other communities. |
Murphy et al. (2004) [41] | Enable indigenous people to identify positively with their culture | Youth experienced acquiring wide range of skills; development of pride and connectedness to community and culture; improved self-esteem. |
Hunt (2010a) [59] | Encourage healing and harmony for and between (non) Aboriginal people. (Partnership Oxfam and Yorgum) | Family issues were addressed; improved parenting skills; less stress; behavioural change; empowerment. |
CLC (2012a) [60] | Increase community understanding and control of usage of mine royalties to facilitate community development | Increased community understanding and perceived and actual community control; increase perceived benefits of project and increase in projects that support the whole community. |
CLC (2012b) [51] | Use aboriginal royalties to support education and training initiatives | Perceived improvements in school performance and increase youth employment; growing capacity and ability of committee. Observed increase in school attendance because of school excursions. |
CLC (2012c) [61] | Improve quality of dialysis service | Service now strongly contributes to health and wellbeing of family, patients and community |
CLC (2012d) [62] | Develop community initiatives and plans for commercial enterprises | Good relationships are built with stakeholders; increased perceived community control; creating activities for community. |
CLC (2012e) [63] | Establishing community development to achieve benefits from income from national parks | Commitment to projects that increase community benefits; planning of projects emerged. |
Taylor (2005a) [56] | Build capacity of Indigenous Health Worker(IHW) to address childhood asthma and educate community about asthma. | Increased skill transfer and development of IHW; increased confidence in administering own asthma medication; improved relationships doctors and IHWs |
Taylor (2005b) [64] | Revitalizing cultural knowledge through traditional games to improve health and build capacity. | Youth experienced increased confidence. Revitalized cultural pride. Indigenous and non-Indigenous people drawn together; empowering. |
Ramsay (2005a) [57] | Increase awareness of nutritional need of children and improve early childhood health. | Observed increased awareness nutritional needs, decrease failure to thrive kids and increase in fruits and vegetables in store. Increase confidence in buying healthy food; increase of healthier kids in community. Establishment of community garden. |
Ramsay (2005b) [68] | Developing and publishing literacy resources to improve literacy | Publishing and increased use of picture dictionary as effective tool to teach English as a second language. |
Burchill (2005) [65] | Revitalizing cultural knowledge through multimedia databases and developing computer skills. | Observed improvement of computer and literacy skills; increase in self-pride and pleasure; generations are drawn together. |
Higgins (2005) [52] | Empower Indigenous youth and strengthen links with their culture | Experienced increase in job offers, improved wellbeing of youth. Reported increase in youth entering higher education; decreased expulsions. |
Bromfield (2005) [55] | Develop confidence, self-esteem and pride in Indigenous history, | Emerging of real career pathways; observed change in children’s confidence and behaviour. |
Ramsay (2005c) [66] | Identify and assist emerging youth community leaders | Youth getting more active in community; youth staying in school longer; observed increase in youth taking employment opportunities |
Tsey (2003) [69] | Improve physical, mental, emotional and spiritual wellbeing of individuals and families. | Experienced improvements in parenting skills and confidence; improved student behaviour. |
Tsey et al. (2004); [39] | Restore men’s rightful place in the community | Progress towards goal; increase in self-awareness and confidence; taking more responsibility in family life; no improvement in addiction problems. |
Smith (2004) [53] | Improving child growth and increasing community involvement. | Increased understanding between community and staff of health service; increased community action; no improvements of child growth. |
Lee et al. (2008) [42] | Address youth substance misuse and crime and develop youth activities | No changes in school attendance (2003: 55.9 %; 2005: 51.3 %), or youth apprehension (2003: 68; 2005: 75); decline in cannabis use (2001: 80 %; 2004: 74 %, p = .003), statistically significant for females (13–36 years, p = .008) and older males (>16 years, p = .007). |
Tyrrell et al. (2003) [43] | Improve knowledge about and management of diabetes | Increase in visits to health professionals; improved adherence to diabetes management protocol; 65 % decrease in sugar purchases; increase in fruit (81 %) and vegetable (11 %) purchases; no change in biochemical control. |
Guenther (2011) [58] | Strengthen and empower families to help children succeed in life | Non-significant increase in school attendance (48.4 to 53 %; p > .1); improvements in family environment; no improvements in parental involvement in education; children show more respect towards teachers and other children. |
Salisbury (1998) [44] | Improve development and delivery of Aboriginal and Torres Strait Island Mental Health service | Increase in utilization of the service (1994: 73 people; 1997: 770 people using service). |
Hunt (2010b) [59] | Build financial capacity in Aboriginal communities | Increased knowledge, confidence and understanding of financial and money management; increase in employment and re-engagement with education. |
Moran (2003/2004) [45] | Establishing healthy and sustainable living environment | New healthy settlement was creating; satisfaction of tenants with new buildings; dissatisfaction with level of involvement |
McCalman (2005) [54] | Restore men’s rightful place in the community | Reduced injury and suicide rate in community; increase in self-esteem and confidence; increase in seeking help instead of going to drugs. |
Jarvie (2008) [46] | Improve relationship between communities and government and build community capacity. | Reduction Indigenous students in lowest literacy bands (2005: 16 %; 2006: 6 %); increase in TAFE enrolments (2001: 1480; 2006: 1718); 32 % increase year 11 and 12 completions; 71 % increase of students finishing certificates and 50 % increase in diplomas; 45 % drop alcohol related hospitalisations, 13 % drop in diabetes-related hospitalisations; decrease in thefts (21.6 %) and breaks (15.8 %) from dwellings. |
Shannon et al. (2001) [47] | Reduce injuries in the community | Significant reduction in frequency of injuries before (96; SE = 4.8) and after (65; SE = 3.08) start of the project (Student’s t = 5.07, df = 21, p < 0.001). |