An intergenerational program based on psycho-motor activity promotes well-being and interaction between preschool children and older adults: results of a process and outcome evaluation study in Austria

We conducted a multicenter, mixed-methods study using an exploratory sequential design. We started with a needs assessment, targeted at all stakeholders, followed by the development of the intervention program and a pilot phase that included a qualitative process evaluation. Based on the results of the pilot phase, the intervention program was adapted and implemented in a rollout phase including an outcome evaluation as well as a final qualitative process evaluation (Fig. 1). A participatory approach was followed [31] and older adults and kindergarten children were defined as equally important target groups, participating in all steps of the evaluation study. Furthermore, the perspectives of professionals (caregivers, kindergarten teachers) and parents were explored in the course of the needs assessment and process evaluation. Moreover, professionals were actively involved in data collection and were trained to conduct the assessments for the outcome evaluation. In addition, non-participatory observations through members of the evaluation team took place during the second, as well as the fifth or sixth sessions of the pilot, to develop an understanding of the setting of interest. Qualitative and quantitative data were collected during different time-points of the study, analyzed separately and then discussed and integrated in a final report.

A convenience sample of eight geriatric institutions and eight kindergartens in Vienna was selected to participate in this study. We considered a range of different institutions essential for ensuring rich qualitative data and an outcome evaluation that is applicable for real life settings. Subsequently, older adults and kindergarten children, as well as professionals (caregivers and kindergarten teachers) and parents were recruited by the participating institutions. Older adults aged 50+ had to be either residents in a long term care institution or visitors of a senior day-care centre. Children, between 2 and 7 years of age, were eligible to participate if they attended the kindergarten on a regular basis. Participants of both groups had to be able to join an intergenerational, psycho-motor activity group session as assessed by their caregivers. Multimorbidity, restrictions in mobility, e.g. use of a wheelchair, mild cognitive impairment and limited German language skills were no reasons for exclusion. However, older adults with severe depression, severe dementia, as well as those who were permanently in need of care were excluded from participation in the study. Participant recruitment started in October 2014 and ended in March 2016. Health professionals and kindergarten teachers were trained in recruiting eligible participants as part of a half-day training course by the evaluation team. Based on their expertise, another person was recruited as a surrogate to keep the group size constant, in case a child or older adult could not, or did not want to continue the intervention program. A balanced number of older adults and young children taking part in different psycho-motor activities was regarded as a basic requirement in the intergenerational program. Recruitment of the participants was left to the decision of the professionals working in the institutions to apply as accurately as possible the real life situation. Participants for the qualitative study were selected using a maximum variation strategy depending on the different stakeholder groups and considering the following criteria: coming from different institutions, both sexes, different functional mobility and cognitive status, different culture and ethnicity, different (former) profession and levels of training.

The study was submitted to the responsible ethics committee and approved without further detailed review (EK 14–274-VK_NZ). Eligible participants were first asked whether they would like to participate in the study and were informed in detail about the study and the right to withdraw at any time. All participants gave oral and written informed consent. Participating children assented to the intervention program in addition to their parents’ written consent. All data were handled with strict confidentiality and were only used for research purposes.

The structured intervention program was newly developed and further adapted based on the results of the needs assessment and the pilot evaluation in the course of the study [32]. A short overview about the content of the intervention program and the activity-inducing material used is given in Additional file 1. Each session focused on intergenerational contact by means of psycho-motor activities and was facilitated by two trainers. The trainers had experience in working with children and older adults in group settings. The intervention sessions included activities that facilitated sensory, perceptive, motor and social skills by using different types of material, which should induce and be an incentive for creative psycho-motor activities, including tubes, ropes, marbles, newspaper and plastic cups (Additional file 1). The trainers suggested intergenerational activities which had to include a motor component and should facilitate the creative ideas of the participants. Besides ritualized welcome and closing procedures, participants were supported in occupying themselves with different appealing material and working together as a group. While interacting with the material, the participants were in motion and in contact with each other. The pilot phase included ten sessions which were conducted in two intergenerational groups. Each group consisted of a kindergarten (with up to seven participating children) and a senior citizen institution (with up to eight participating older adults). During the rollout phase, 20 sessions took place in eight intergenerational groups. The sessions were implemented on a weekly basis with a duration of one hour.

An extensive qualitative process evaluation took place throughout the project. The results of the needs assessment and the formative process evaluation in the pilot phase were used to develop and to refine and adapt the intervention after the pilot phase. Furthermore, the final qualitative focus group session was scheduled at the end of the rollout phase to ensure that the perspectives and needs of all stakeholders were comprehensively included until the very end of the project. Identifying points of strength and barriers concerning feasibility and sustainability of the intervention program at different time points, and carrying out appropriate adjustments are important steps for ensuring best results.

Qualitative methods including group interviews with children [33] and focus groups with older adults, professionals and parents were used to assess the needs for an intergenerational group activity. Furthermore, the implementation and feasibility of the intervention program were evaluated [31, 34‐37]. Data collection and analyses were iterative processes and carried out in parallel [38]. During the preliminary stages of exploring the perspectives of different stakeholders on the intervention program, initial structures of interview schedules were developed. They followed a review of qualitative literature about intergenerational contact, and experiences with other intergenerational group programs. The interview questions for the needs assessment on an intergenerational group activity focused on values and preferences, as well as expectations of children and older adults, professionals and parents (Additional file 2). The interview questions for the pilot and the rollout phases focused mainly on how participants experienced the impact and sustainability of the intervention program (Additional file 3). The questions were pretested in older adults and children before conducting the focus groups and the group interviews, and were adapted according to their feedback [39].

All focus groups and group interviews were conducted in a familiar environment (either geriatric facility or kindergarten) and were chaired by trained and experienced moderators (EM, KW, TS) together with one assistant. Field notes on observations of the group interactions and the primary topics of discussion were taken during the focus groups. At the end of each focus group, a summary of what was discussed in the session was presented to the group to enable the participants to verify and amend emergent issues. All group interviews and focus groups were conducted in German, digitally recorded and transcribed verbatim. The quotations were subsequently translated by a native speaker for publication. After each data collection, a debriefing between moderator and other members of the evaluation team took place to review the course of the focus group and group interview [40].

Several strategies were used to improve and verify the trustworthiness of the qualitative data: data triangulation was achieved by interviewing different stakeholders at various sites, comparing findings to literature throughout the project duration and by including qualitative and quantitative methods of data collection [34, 41, 42]. After analyzing the qualitative data, all results were discussed with an experienced researcher (TS) who had not been involved in the analysis of the transcripts.

A modified form of the framework method was applied [43]. Data analysis was facilitated by using QSR International’s NVivo 10 qualitative data analysis software [44]. Firstly, the transcripts were thoroughly read to gain an overview of the collected data. In a second step, data were divided into meaning units (defined as specific units of text, either a few words or a few sentences with a common meaning) and the concepts contained in the meaning units were identified. Concepts could refer to the main topic of a meaning unit, but a meaning unit could also contain more than one concept. A concept is a separate theoretic entity with some attributes different from other concepts. Thirdly, all concepts contained in the meaning units (lower-level concepts) were grouped under higher-level concepts and sorted into a table depicting an analytical framework. Higher-level concepts are more general and cover the attributes of the lower-level concepts. In a final step we formulated requirements (needs assessment) and feedback statements (pilot- and rollout phases) out of the higher-level concepts. An example of a meaning unit, as mentioned by a child is: “I’m sorry that it [the group sessions] is over now! They [trainers] always brought such nice material […]. The hair ties were so cool. And once, Leopoldine [older adult] made a very cool paper airplane for me.” Three concepts were identified within this meaning unit: emotion, interesting material, and intergenerational contact. Finally, two of them (emotion, intergenerational contact) were grouped under the higher-level concept intergenerational contact takes place.

Basic socio-demographic information was obtained from all participants. Well-being and active engagement assessed through facial expressions was the primary outcome and engagement/behavior, intergenerational interaction and self-efficacy were the secondary outcomes. As we also included older adults with cognitive impairment, a modified form of the time sampling method [16, 45] was used to assess facial expressions and engagement/behavior. According to the standard protocols a designated amount of time, a so-called observational unit (120 s) was defined and facial expression as well as engagement/behavior were observed and recorded every ten seconds within a two minutes period of time during the most active part of the intervention. For example facial expression was assessed within 60 s according to the categories happy/smiling, neutral, lethargic or grumpy. Spontaneous intergenerational interaction (yes/no) and self-efficacy were assessed based on observation if it ever occurred at least once in a session. Self-efficacy was assessed on a 5-point grading scale ranging from 1 (very good) to 5 (not sufficient) by rating the following criteria: trying out something new, showing confidence in her/his own abilities, coping with demands and responding adequately to unexpected situations. The baseline assessment took place during the second session, which was the first unit in which children and seniors came together. The follow-up assessment took place at the final session. There were at least two assessors of the participating institutions (a health professional and a kindergarten teacher) responsible for assessing the participants. Each assessor scored only one participant’s facial expressions in a given time frame (one minute). Assessors were trained in the observation of participants and subsequent scoring during a half-day training course delivered by the evaluation team. Video sequences of similar group interventions were shown, scoring was practiced and individual feedback was given. All assessments were carried out according to the standard protocols. Furthermore, frequency of participation in the intervention program, reasons for not attending, as well as adverse events (AEs) were documented throughout the pilot and the rollout phases.

All quantitative data were checked for accuracy and completeness and data clarification forms were sent to the study sites. Descriptive statistics were used to summarize the characteristics of the participants. Metric variables were tested for normal distribution. In case of non-normally distributed data, we depicted medians and interquartile ranges (IQR) in addition to mean values and standard deviations (SD). Nominal data were shown as absolute and relative frequencies. The outcome of the intergenerational intervention program on the older adults and the children was calculated by comparing the data at the beginning and at the end of the rollout phase and applying appropriate inferential methods for paired samples by using the “last observation carried forward” method (the missing follow-up visit value was replaced by the subject’s previously observed value at baseline). The older adults and children who were enrolled as surrogates in the rollout were assessed in the respective first session when they started to take part in the program. To determine potential significant differences for non-normally distributed variables, we used Wilcoxon Signed Rank Tests for paired data and McNemar tests for nominal data. The effect size was calculated by using the formula r = \( \frac{\mathrm{Z}}{\sqrt{\mathrm{N}}} \) as suggested by Rosenthal [46], where Z is the z-value of the Wilcoxon Signed Rank Test and N is the number of observations over the two time points. As multiple tests were performed, Bonferroni correction was applied. The Bonferroni-corrected level of statistical significance was set at α = 0.001 (0.05/36 pairwise tests). The Statistical Package for Social Sciences (SPSS 24) was used for all calculations [47].

In total, 196 participants, including 93 older adults, 78 kindergarten children, 13 professionals and 12 parents, from 16 institutions (eight kindergartens and eight geriatric facilities) took part in the present study, from November 2014 to June 2016 in Austria, Vienna. Table 1 presents the characteristics of the participants.

Eleven focus groups with older adults, professionals and parents and eight group interviews with children were conducted over the whole study period (total number of participants involved n = 73, Table 1). The results of the qualitative process evaluation (summarized in Table 2) revealed nine requirements for the development and implementation of an intergenerational intervention program for older adults and young children. Six main themes emerged from the data of the formative process evaluation of the pilot. They helped to adapt the program for the rollout according to the feedback of participants, professionals and parents. Finally, the results of the process evaluation at the end of the rollout phase were organized in five overarching themes supporting the results of the outcomes evaluation and pointing out the value of the developed intervention program from the perspectives of all stakeholders.

The rollout phase started in spring 2016 with 120 participants (Fig. 2). In total, 21 persons had to stop their participation in the group sessions. To keep the group sizes constant, 20 new persons were recruited as surrogates during the period of the rollout and assessed according to the protocol. There were no significant differences at baseline in participants who took part throughout the rollout, compared to the surrogate participants (Additional file 4). Children and older adults who took part from the beginning of the rollout until the end participated on average in 16.4 sessions (SD ± 2.9) out of 20. Those who were included later on as surrogates participated on average in 11 sessions (SD ± 5.7; Mann-Whitney U-Test: p < 0.001).

All metric variables were non-normally distributed. The outcome evaluation of the rollout revealed a statistically significantly higher number of individuals being actively engaged with happy facial expressions (primary outcome) after the intervention when compared to baseline (Table 3; Wilcoxon Signed Rank Tests for paired data: p < 0.001, mean ± SD baseline 2.1 ± 1.8, mean ± SD follow-up 3.1 ± 1.7; effect size r = 0.34). When children and older adults were analyzed separately, the significant difference was confirmed in the children (mean ± SD baseline 1.6 ± 1.6; follow-up 3.3 ± 1.8; r = 0.47; p < 0.001). However, although a numerical difference was found with the same trend in older adults, no significant difference was observed (Table 3). Accordingly, a statistically significantly lower number of neutral facial expressions in all participants (total) and in the group of children was reported.

Moreover, a significantly higher number of children (53 out of 62 children; 88.5%; Mc Nemar test: p < 0.001) initiated intergenerational contacts at the end of the intervention when compared to baseline (32 children out of 62; 51.6%). Older adults, also showed more self-initiated intergenerational contacts at follow-up compared to baseline (70.5% of older adults initiated intergenerational contact at follow-up, compared to 60.3% at baseline; p = 0.109). Some of the secondary outcomes in the areas of engagement/behavior and self-efficacy showed signs of improvement (changes in mean ± SD and median) in children and older adults (Table 3 and Additional file 5).

In total, 49 adverse events were reported. Of these, 21 were minor, such as restless behavior in children. The major adverse events include one older person and two children falling down without hurting themselves (three falls in total), nine older adults feeling dizzy and having to sit down and another six of them complaining of feeling unwell during a session (solved by making a small break; all older adults participated again in the same session just after the break). The trainers documented six times that one of the older adults had shortness of breath during an activity and four times that one of them was agitated. All adverse events were solved without sequelae.

A main aim of our study was to support the development and adaptation of the intervention program. Therefore, the focus was placed on a continuous process evaluation with a feedback in different phases of the project to be able to incorporate this into building up a perfectly targeted intervention to the needs of the participants. A limitation, however, was the lack of a control group. Future studies with a randomized controlled design could be planned based on the results of our study and could show potential effects in comparison of intervention to the control group. Sample sizes for future studies can be based on our effect sizes.

During the rollout some participants dropped out and were replaced by others to keep the group sizes constant. Although this replacement was essential for the intergenerational activities to take place, this is a limitation regarding the data analysis of our study as surrogates participated in a numerically lower number of intervention sessions when compared to the other participants. In the present study, professionals working at the participating institutions were responsible for the recruitment of the participants. In case of any problems, they were offered a support service to facilitate participant recruitment according to the study protocol. As the professionals who performed the assessments observed every group session, the attendance of these people was nothing special for the children and older adults. For this reason, we assume that it was unlikely that participants were just acting because they were being observed.

Furthermore, another assessment might have brought out additional effects for older adults. However, we used assessments that were already established in the literature in similar studies [16, 45]. An important argument for choosing these assessments and looking at happy facial expression was that older adults with and without cognitive impairments, as well as kindergarten children would be able to express their joy and satisfaction in that way. A limitation of this assessment, however, might be that facial expression might differ between individuals. Another limiting factor might have been, that self-efficacy was assessed by professionals only. However, due to the age of the participants and the potentially impaired cognitive abilities some of them might have had, we decided not to use self-reported assessments after an initial pilot test (e.g. self-reported self-efficacy).

Concerning sustainability and transfer aspects of the project, most institutions decided to carry on a similar intergenerational intervention program in the future. In addition, other institutions decided to send some of their professionals and caregivers to be trained in the principles of the intervention program. On the basis of findings from this study further research is recommended to investigate the effectiveness of intergenerational intervention programs based on psycho-motor activity using a randomized controlled trial design including long time follow-up. Furthermore, additional outcomes could be of interest, like possible changes in physical functioning in both target groups.