Social Insurance Literacy of Dutch Workers Receiving Disability Benefits and its Associations with Socio-Economic Characteristics

In this cross-sectional study, members of an online panel of claimants of the Dutch SSI, called the UWV, were approached to fill in a shortened and culturally adapted version of the SILQ (SILQ-NL37) and additional questions on socio-economic characteristics. The Central Medical Ethical committee of the University Medical Center Groningen (UMCG) reviewed and approved this study (Number: 201900597).

1753 members of an online panel of the Dutch SSI were invited to participate in this study. The SSI frequently uses the online panel to ask for feedback on quality of service, policy changes and for scientific research. Prior to our study, these members pro-actively registered for the panel and provided informed consent to be approached for these purposes regularly. Data on age, gender, education, postal code and living situation were collected in an online questionnaire system.

For our study, members were eligible for inclusion when they were 1) client of the SSI, 2) received a work disability benefit after the two-year waiting period of sick leave. This benefit falls under the regulations of the WIA [15]. The WIA distinguishes between people who are fully unable to work (WIA-IVA), and people who have residual working capacity (WIA-WGA). We specifically selected the last group in our study sample, which has the largest total population and is the most diverse in background characteristics (i.e. age, education) [16].

The questionnaire was programmed in the questionnaire system, and for all items it was obliged to select an answer, preventing missing answers. In November 2019, the SSI send an e-mail explaining the study and with a link to the questionnaire to eligible panel members. Based on the explanatory information, panel members chose if they wanted to participate or not. After one week a reminder was sent. Two weeks after the first e-mail, data were exported in an SPSS-file.

Social Insurance Literacy Questionnaire—to explore the SIL of individuals and the comprehensibility of the system, we administered the SILQ-NL37. This measure is derived from the original version of the SILQ, developed in 2019[6], which contained 45 items on individual abilities and 24 items on system comprehensibility. For the abilities, the SILQ focuses on different abilities (i.e. obtaining, understanding and acting upon information) within different domains (i.e. contacts and communication, navigating the system, and decisions and appeals). Items regarding system comprehensibility are organized within the same domains.

At an early stage, the original SILQ was evaluated with regards to content validity. This evaluation was performed through a Delphi-study. An international expert panel reviewed the items of the SILQ in relation to the definition and domains of SIL. Revisions of the SILQ were made until consensus was reached upon the relevance and comprehensibility of the items, and thus the content validity considered strong. While further evaluations on the validity and reliability of the SILQ were performed in Sweden, such as through Rasch analysis, the development of the Dutch SILQ-NL37 began parallelly in collaboration between the research groups. Hence, these instruments share the same theoretical foundation but have been developed in slightly different directions, such as branches of the same tree, for instance due to differences in the Dutch and Swedish systems and culture. Results on the evaluations of the original SILQ are forthcoming.

The development process of the SILQ-NL37 included multiple steps. The aims of the adaptations were to ensure the measure represented the daily practice of the Dutch SSI, simplify language, and to shorten administration time. To reach the last aim, in step 1, one researcher selected the 3–5 items from the original SILQ that represented the Dutch SSI best. For example, a question regarding the understanding of leaflets was left out, because these were not used anymore as information was digitalized. In step 2, two other researchers and four employees of the SSI checked all items. They supported to simplify language, but also how items could be improved to meet daily practice within the Dutch SSI. In a last step, the adaptations were checked a second time by the same persons, and small changes in language were made until the SILQ-NL37 was considered final for our study.

The final SILQ-NL37 consisted of 37 items with 5-point disagreement-agreement Likert scales, where ‘strongly disagree’ gave a score of 1 on a specific item and ‘strongly agree’ a score of 5. Participants could also answer ‘not applicable/don’t know’. To explore the individual abilities and system comprehensibility, the total SILQ-NL37 scores were calculated for each participant by summing up the answers on 25 items on individual abilities and 12 items on system comprehensibility separately. The total scores were expressed as a percentage between 0 and 100 by dividing the calculated score by the total possible score (125 for individual abilities, and 62 for system comprehensibility), and multiplying by 100. Table 1 defines the measured abilities and domains within our study, and also shows how many items were asked within those abilities and domains.

All statistical analyses were performed in SPSS, version 26.0. From the respondents, we excluded participants with ≥ 15% ‘not applicable/don’t know’ answers from the analysis on total scores for both individual abilities and system comprehensibility. While for specific ability and domain analyses, only complete cases were included. To check the effects of this exclusion, chi-square tests were performed to compare the excluded and included participants, and to learn if group distributions among those participants were different. For age, education, Dutch language skills, years of work disability benefits and postal codes, we regrouped variables into three groups. From multiple answers on living situation, we extracted if a participant was with or without a partner. This was done to optimize group sizes for statistical analyses. Details on the original groups are in Online Appendix 1.

In addition to the evaluations of the SILQ in Sweden, we measured internal consistency–reliability to explore the measurement properties of the SILQ-NL37. We calculated the Cronbach α for the full set of items measuring individual SIL and system comprehensibility, but also for separate domains and abilities. Additionally, we used Spearman correlation tests (rho) to examine correlations between and within the different abilities and domains of the SIL. Cronbach α indicated high internal consistency-reliability for all items measuring individual SIL (0.93) and system comprehensibility (0.94). For the independent abilities and domains consistency-reliability was moderate to high (0.77–0.91). Spearman rho’s indicated a moderate to strong correlation between different abilities and domains of the SIL (rho = 0.54–0.80). We found weak to strong item-correlations within the independent abilities and domains of the SILQ-NL37. Online Appendix 2 and 3 give the results in detail.

Descriptives and frequencies were used to describe the means and standard deviations (SD) of the SILQ-NL37 scores for all participants and within subgroups, organized according to the collected socio-economic background characteristics.

To determine the prevalence of limited SIL, we used pairwise k-means clustering to create two groups of limited and adequate SIL, similar to HL studies [8, 17]. K-means clustering is a data driven method in which SPSS creates a predefined number of groups with similar answer patterns, determined by having as low variance as possible among responses on the different variables in each group. In our clustering procedure, we preset the number of groups on two. SPSS then ran 50 iterations with all 25 variables for individual SIL abilities. During an iteration and for each independent variable, SPSS created two groups with respondents as closely centered around an emerging mean. The grouping results of the independent variables were combined to divide people into the groups of limited and adequate SIL. After 50 iterations, this procedure led to two groups of limited and adequate SIL, with low within group variance. We used frequencies to describe group distributions among these groups.

To examine associations between SIL and socio-economic characteristics, we performed three analyses. Among participants with limited and adequate SIL, we performed Chi Square tests to detect significant differences in the two groups created with k-means clustering. Second, to examine associations between socio-economic factors and total SILQ-NL37 score, we calculated within group means for both the total individual abilities and system comprehensibility score. We performed independent t-tests for binary variables and linear regression analyses for categorical variables to find associations with the socio-economic characteristics. Third, to gain additional insight in associations between socio-economic characteristics and the SILQ-NL37 score, analyses were performed for the three separate subdomains; Contacts and Communication, System Navigation and Decisions and Appeals and for the three abilities; obtaining, understanding and acting upon information. We calculated the within group means and checked for significant group differences with independent t-tests and linear regression analyses for categorical variables. We performed Bonferroni post-hoc tests to correct for the multiple group comparisons in our analyses. Only results that remained significant after Bonferroni post-hoc tests were reported.

Table 2 shows the socio-economic characteristics of the 567 (32%) study participants. About 55% was male and 74% was aged ≥ 50 years. Study participants closely represented the members of the online panel, but were more often male, older and higher educated compared to the total population receiving disability compensation (see Online Appendix 1). From the 567 respondents, 57 respondents were excluded from the analyses on total scores for individual abilities, and 91 from the analyses on total scores for system comprehensibility due to ≥ 15% ‘not applicable/don’t know’ answers. These respondents were lower educated and more often had bad to moderate language skills differed compared to the included sample. The mean SILQ-NL37 score for the sample was 71.2 (SD = 12.9) for individual abilities to obtain, understand and act on information of the social insurance system. The mean SILQ-NL37 score for system comprehensibility was 66.3 (SD = 15.8).

Of all participants, 34.9% had limited SIL, which is characterized by a total individual SILQ-NL37 mean score of 57.4 (SD = 8.8). 65.9% of the participants had adequate SIL, which is characterized by a mean score of 78.6 (SD = 7.4). Participants in the group with limited SIL, significantly more often were without a partner (p = 0.018). Participants with limited SIL generally disagreed or neither agreed or disagreed with the items measuring obtaining, understanding and acting upon information within the different domains, meaning that they had doubts about possessing the needed abilities (mean item scores: 2.34–3.46). Participants with adequate SIL, generally, agreed or strongly agreed (mean item scores: 3.45–4.29) they possessed the needed abilities to obtain, understand and act on information within the different domains. Table 3 gives more information on group distributions.

Several findings revealed an association between individual SIL scores and socio-economic characteristics. A higher individual SILQ-NL37 total score was significantly associated with living in the northeastern region of the Netherlands (p = 0.031) compared to the southern region. Additionally, though non-significant, the analysis indicated a lower individual SILQ-NL37 total score was associated with being between 50 and 59 years compared to being over 60 years (p = 0.081) and being without a partner, compared to being with a partner (p = 0.064). Table 3 gives results on the relationship between individual SIL scores and socio-economic characteristics.

Additionally, there were several associations between SIL abilities (obtaining, understanding and acting upon information) and domains (contacts and communication, navigating the system and decisions and appeal), and socio-economic characteristics. A lower mean score for the ability to obtain (p = 0.041) and understand information (p = 0.049) was significantly associated with being of male sex, in comparison with being of female sex. A lower mean score for the ability to act on information was significantly associated with being between 50–59 years, compared to the younger age group (p = 0.020). A lower mean score for the ability to act upon information was associated with having bad to sufficient Dutch language skills, compared to having good (p = 0.043) and very good Dutch skills (p =  < 0.001). A higher mean score on the domain contacts and communication was significantly associated with living in the northeastern region, compared to living in the southern region (p = 0.002). A lower mean score on the domain decisions and appeal compared was significantly associated with being without a partner, in comparison to being with a partner (p = 0.008). Details are in Table 4.

No significant associations between total system comprehensibility SILQ-NL37 total score and socio-economic characteristics were found (see Table 3). However, the analysis indicated, though non-significant, a higher reported score for system comprehensibility was associated with having bad to sufficient Dutch language skills (p = 0.063), and being of lower education (p = 0.146) or with a partner (p = 0.085), compared to people with better skills, higher education or without a partner.

There were a few associations between specific domains of system comprehensibility and socio-economic characteristics. Lower mean scores for the domains navigating the system (p = 0.009) and decisions and appeal (p = 0.018) were associated with being between 18–49 years, compared to being of older age. A lower mean score for the domain navigating the system also was associated with having high education (p = 0.026), very good ability to speak and understand Dutch (p = 0.008), and being without a partner (p = 0.016), compared to having low education and bad to sufficient Dutch ability or being with a partner. Details are in Table 5.

Our study has strengths and limitations. A first strength is that we approached SIL in various ways by using a clustering method, total individual and system SILQ-NL37 scores and scores for specific abilities and within domains. We thereby provide the first comprehensive overview of the role of limited SIL in social insurance systems and its association with socio-economic characteristics. A second strength is that we, with help of employees of the SSI, adapted the preliminary SILQ to the Dutch context. With this approach, we ensured the questionnaire closely reflected the Dutch social insurance system, which prevents information bias as a result of administering unsuitable questions.

The most important limitation refers to a high risk of selection bias on several levels. First, the recruitment via the online panel leads to bias, as members have pro-actively signed up for an online panel and need skills to do that. Compared to the general population, participants were higher educated (53% vs. 25%). Most had good to very good Dutch skills (91%), which is expectedly better than in general, as we know around 25% of the people with disability benefits are migrants. It is possible the included participants with low education in our study are not representative for clients of the SSI with low education in general, for example because they have better skills in reading or using digital technology. This offers a potential explanation why we don’t find an association between individual SIL and educational level. Second, in all analyses, we excluded participants with ‘not applicable / don’t know’ answers. Our group comparisons indicated the excluded participants were lower educated and more often had bad to moderate language skills. This limits the possibility to generalize our results. Based on the above, we expect to underestimate the prevalence and role of limited SIL. Third, our participation rate of 32% is too low according to study quality assessment instruments [27]. However, since the participants closely represented all online panel members, we expect that participation bias did not influence the results. Another limitation is that our spearman correlation tests did not always show strong correlations, and this might result in measurement bias. Although we cannot be sure. If we assume, for example, obtaining, understanding and acting upon information are similar abilities, strong correlations are expected. But, if we treat them as different individual non-related abilities, it is possible correlations are moderate or even low. This uncertain measurement validity might also have influenced the results of our study. So in this case, the upcoming validation study of the original SILQ might add to the evidence and provide answers.

Our study has implications for research and practice. More research is essential to further unravel the role of SIL in social insurance systems. Since SIL is a new concept, it is important to validate the original SILQ, which is currently undertaken by Swedish researchers. Larger cross-country cohort-studies can help to gain insight in the prevalence of limited SIL, associations of SIL with socio-economic characteristics, but also if these characteristics act as confounders. Based on our results, we suggest to further unravel the role of social support, as a potential proxy measure of SIL. Additionally, these studies should unravel the influence of SIL on different social insurance related outcomes, such as received work disability benefits and perceived system justice. This is important because we know limited HL has a negative impact on multiple outcomes, such as active participation in the treatment [28] and mobility during rehabilitation from physical injuries [29, 30]. Additionally, it is important to learn more about the role of professionals in their contact with people with limited SIL, for example if they are able to tailor their communication, and to develop and test interventions targeting clients and professionals in social insurance systems to optimize support of people with limited SIL.

This study also has implications for practice, especially since the number of people receiving work disability benefits has risen in multiple countries [31, 32]. Our study shows a group of 35% with limited SIL expresses to have lacking abilities to obtain, understand and act upon information. We therefore suggest social security institutes to take action to identify people with limited SIL and support these people better. Based on HL research [33], we advise to simplify written information and use more visual strategies. Another approach would be to make employees more aware of limited SIL and train them in strategies that were effective for people with limited HL, such as providing easy information, checking understanding and including the social network [34].