Background
The past couple of decades have seen growing concern over sick leave in working populations in Western societies as a public health problem. Sick leave has a multi-factorial aetiology [
1‐
3]. Several scientific approaches have therefore been used which may explain the variety of approaches used to measure sick leave within an epidemiological framework: frequency of sick leave spells per individual, the total length of absence during a specified period, incidence rate, cumulative incidence and duration of absence spells [
4]. However, in light of the large number of studies on sick leave, it is remarkably that only few validation studies have been performed.
Among the four traditional sources (employer’s personnel files, insurance-based data, national social security registers and self-reported data) from which sick leave data are traditionally retrieved, register-based sick leave data is an option available only in few countries. Even where registers are available, self-reported sick leave data are usually more easily acquired than data from other sources. Company-based data retrieved from employers’ personnel files is considered a golden standard, mainly because these data are also used for calculating earnings [
5‐
8].
To identify sick leave measure validation studies, a systematic literature search was performed in PubMed. The search terms “Sick leave”, “absenteeism”, “presenteeism”, “work”, “registries”, “self report”, “questionnaires”, “reproducibility of results”, “validity/validation”, “sensitivity and specificity”, “predictive value of tests”, and “accuracy” were combined in the search. Twelve validation studies were selected. The validity of sick leave data reported in questionnaires or in interviews was studied and analysed against data retrieved from employers’ personnel files [
5,
6,
8‐
13], insurance companies [
9,
14‐
16] and a national social security register [
17]. None of the studies validated insurance-based data either from companies or from national social security registers against employers’ personnel files.
The validation studies found discrepancies between self-reported length of absence and insurer-reported compensation payments. The inconsistencies spoke against self-report and were associated with work status, cause of absence as well as personal characteristics [
14,
15].
A study from the Netherlands found poor agreement between workplace-registered sick leave data and data collected in a questionnaire [
11]. The ability of this questionnaire to detect frequency of sick leave spells was reported to have a sensitivity of 55% and specificity of 83%. The remaining nine studies concluded that self-reports yielded acceptable validity [
5,
6,
8‐
10,
12,
13,
16,
17]. The total length of absence was the most widely used measure of sick leave [
5,
6,
8‐
10,
12,
13,
16], but also prevalence [
5,
12,
17], frequency of spells [
5], incidence and duration of sick leave spells [
16] were used.
Recall periods ranged from 2 weeks to 4 years and were discussed in several papers [
6,
9,
10,
17]. These studies were largely unanimous that shorter recall periods could increase the precision of self-reported sick leave. In two papers, the optimum recall period was recommended to be no longer than 2–3 months to obtain valid measures of absence lengths [
6,
10].
The Whitehall II study and a Swedish replication of Whitehall II found a worse recollection the longer the absence length [
8,
13] which indicates that valid self-reporting may be limited to absence of short duration. Finally, relatively high sensitivities were found in studies where data on absence length, frequency and prevalence of absence were provided as pre-specified categorical questionnaire options [
5,
17].
In Denmark opportunities for register-based research are unique [
18]. Data on social public transfer payments, like sickness benefits are registered on a weekly basis in a national register called DREAM [
19]. Employees’ sickness benefit paid in excess of two weeks is refundable from the municipality according to the Danish Sickness Benefit Act [
20]. DREAM data on sickness benefit has been used for follow-up studies, where return to work (RTW) [
21‐
27], sick leave defined as absence > 2–3 weeks [
28‐
32] and long-term sick leave defined as absence >8 weeks [
29,
31‐
40] have been used as endpoints. Study populations have also been defined from the DREAM register [
24‐
26,
29,
41,
42]. One attempt has been made to validate the DREAM register [
43]. In this study a random sample of 5,221 Danish citizens were asked about which kind of income they received in a particular week in 2001. According to the DREAM register 82 persons were receiving sickness benefit and of those 38 responded this kind of income; yielding a positive predictive value of 31.7%. The article concluded the DREAM register to be a feasible tool for social and economic research in Denmark. DREAM data on sickness benefit has so far not been validated against workplace-registered sick leave.
Aim
1) To validate registered sickness benefit data from DREAM against workplace-registered sick leave spells of at least 15 days. 2) To validate self-reported sick leave days during one year against workplace-registered sick leave.
Methods
Study population
The study is a cross-sectional study of municipal elder care workers in Aarhus (the second largest city in Denmark). To validate DREAM data a total of 3,554 individuals employed throughout 2006 were identified in the employer’s computerized personnel files. Details on the study population are given in Table
1. Their median age was 47.5 years. The majority were women (95%) who were primarily working as home care workers, assistant nurses and nurses. The male employees worked mainly as home care workers, assistant nurses, maintenance workers, and in the administration. Nineteen percent worked full-time, i.e. 37 hours per week, 70% worked between 36 and 30 hours per week, 7% worked between 29 and 20 hours per week and the remaining 1% worked less than 20 hours per week.
Table 1
Description of elder care workers employed throughout 2006 in the municipality of Aarhus (N = 3,554)
Age in years, median (interquartile range) | 47.5 (39–54) |
Gender (% female) | 95.3 | 3,387 |
Workplace, local centres:
| | |
Frederiksbjerg | 9.2 | 326 |
Hasle-Gellerup-Toftegården | 7.5 | 267 |
Holme og Skåde | 7.4 | 264 |
Hørgården og Vejlby | 8.1 | 289 |
Skelager/Bjørnshøj | 4.2 | 148 |
Trøjborg og Abildgården | 5.6 | 199 |
Viby og Rosenvang | 6.6 | 235 |
North | 7.1 | 251 |
Northwest | 7.5 | 267 |
South | 8.2 | 291 |
Southwest | 5.5 | 197 |
West | 9.1 | 324 |
City | 7.2 | 255 |
Others | 6.8 | 241 |
Profession:
| | |
administration | 5.4 | 190 |
activity | 6.3 | 222 |
kitchen and café staff | 1.1 | 39 |
cleaning | 5.5 | 195 |
social and health care assistant level I and II | 68,0 | 2419 |
nurse | 10.4 | 369 |
maintenance | 0.6 | 21 |
not defined | 1.9 | 66 |
remaining staff | 0.9 | 33 |
Hours worked per week:
| | |
37 | 20.8 | 738 |
31-36 | 40.4 | 1,434 |
21-30 | 34.6 | 1,229 |
−20 | 1.7 | 59 |
missing | 2.7 | 94 |
All subjects in this study were covered by the Danish national health insurance, which provides sickness benefit to those who are unable to work due to disease or injury.
Agreement between self-reported sick leave and workplace-registered data was studied in 2,311 responders to the “Working in eldercare” survey [
44] in 2005. The response rate was 73% and 2,139 of these responders were also included in the validation of DREAM data from 2006.
Beside questions about the working environment and health the employees were also asked: “How many sick leave days have you had within the last 12 months?”. The two study populations did not differ regarding gender, age, profession or hours worked per week.
Measures of sick leave
When an employee needed sick leave, he/she reported absence to his or her immediate superior. An absence form was printed out by the superior, in which the first date of the absence spell was written. This form was handed to the employee, who wrote the last date of the absence spell when he/she returned to work and signed it together with the immediate superior. The dates on the form were entered to a computerised duty roster (from which earnings are calculated) by the immediate superior or a secretary. In each of the 14 municipal eldercare workplaces in Aarhus a number of immediate superiors were responsible for entering absence dates. Simultaneously these absence dates were transferred to the company’s absence records along with a categorization of the absence spell (sick leave, care leave, child’s first sick day etc.). These raw data on absence were retrieved from the workplace records, but only spells related to sick leave were included in the present study. The dates of the first and last day of each sick leave spell were available for each individual. A sick leave spell was counted in calendar days regardless of whether all of these days were work days. Overlapping, consecutive, or duplicate sick leave spells were merged into a single spell.
Sick leave spells entitled to municipal refunding according to the Danish Sickness Benefit Act include the following four categories: 1) spells lasting more than 14 days; 2) spells related to pregnancy; 3) spells due to sick leave in a “flexi job”, i.e. modified job due to permanently reduced workability; or 4) recurrent or anticipated spells due to chronic disease. Spells in categories 2) - 4) shorter than or equal to 14 days were exempted from the employer period and refundable from the first day of absence. For every refundable sick leave spell, we identified the week number in which the spells were seen.
From workplace absence to sickness benefit compensation
When a workplace-registered absence spell was assumed by the employer to be entitled to tax-financed compensation according to the Danish Sickness Benefit Act, a notification was sent from the human resource department in the municipality of Aarhus to an external private IT-company. This private partner provided data-handling services for the municipality and was responsible for notifying the social services in the sick-listed employee’s municipality of domicile, which grants sick leave compensation benefits. From the social services a form was sent to the sick-listed employee. Within a week detailed information about the particular absence spell, treatment, workplace, current work tasks, education and prospects about the return to work process had to be returned to the social services. This information was exchanged with a social worker from the municipal job centre, engaged in the process of resumption of work. If/when the absence spell was discontinued the social services was notified by the social worker. Data on tax-paid sickness benefit compensation granted in the municipality of domicile are pooled in a central database. This information is further processed and collected in a national register of sickness benefits and maternity payments, which becomes part of the DREAM register.
Measures of sickness benefit
The DREAM register is administered by The Ministry of Employment [
19]. The name is a Danish acronym (Den Registerbaserede Evaluering Af Marginaliseringsomfanget) which translates into “The evaluation of marginalized groups of individuals based on registered social public transfer payments”. The DREAM register includes all Danish citizens with a CPR number who have received social public transfer payments at some point since mid-July 1991. Each person is registered once a week with a code representing the type of reimbursement received that particular week (currently 109 codes are available). Codes are ordered hierarchically; low-ranked codes are overwritten by high-ranked codes, e.g. sickness benefit codes (Table
2). The weekly recordings cover reimbursements of 1 to 5 days of actual workdays lost. By January 2006, data on more than 3.5 million citizens of a total Danish population of 5.4 million had been entered into the DREAM register.
Table 2
DREAM codes related to sickness benefit reimbursement
891 | Sickness benefit |
892 | Sickness benefit while being on part time sick leave |
893
|
Sickness benefit while being unemployed (this code is of no relevance in this study)
|
894 | Sickness benefit while being re-trained |
895 | Sickness benefit while being employed under special condition “flexi job” |
The weeks of 2006 were isolated in the DREAM register and the 3,554 employees were identified by their CPR number. The weeks in 2006 coded 891, 892, 894 or 895 were identified.
Statistical analyses
Agreement about prevalence of sick leave, aim 1
We used workplace registered sick leave as the reference standard in the comparison of employers’ personnel files and DREAM data on sickness benefit. Whether refundable sick leave and sickness benefit was registered in the employer’s personnel files and DREAM respectively, was addressed in each week of 2006 and cross tabulated. Hence 52 2×2 tables were constructed and sensitivity, specificity and positive predictive value (PPV) of DREAM data were calculated. Logistic regression was applied to obtain single sensitivity probabilities, i.e. the probability of having received sickness benefit registered in DREAM when the workplace data said so. For this analysis, individuals were included as cluster variables to adjust for 48 repeated measurements from each study member. Week 1, 2, 51 and 52 were excluded to avoid possible imprecision due to sick leave carried forward from the previous year and prolonged into the forthcoming year. Gender, dichotomised age (< 41 years), workplace (14 geographically defined categories) and profession (administration, activity, kitchen and café staff, cleaning, social and health care personnel level I and II, nurse and maintenance) were included as explanatory variables.
Concordance with sick leave spell durations, aim 1
A receiver-operating characteristic curve (ROC-curve) was used to study the performance of DREAM data on sickness benefit. Weekly cut-off points in DREAM were used to find the optimum discrimination of sick leave spell durations of >8 weeks defined in the workplace register.
The mean duration of the first workplace-registered sick leave spell was calculated as well as for the corresponding number of weeks of sickness benefit reimbursement registered in DREAM. Differences between and averages of these durations formed a Bland-Altman plot and was used to illustrate the relationship between the two durations. The assumptions behind a paired t-test were also appraised from this plot (Wilcoxon´s signed rank test is the non-parametric test equivalent). Included in the analyses were the first spells of sick leave retrieved from the employers’ personnel files and the first registration of sickness benefit in DREAM, whenever agreement about starting week had been established.
Employees who had pregnancy-related sick leave according to the employers’ personnel files at some point in 2006 (95 women) constituted a special group. All analyses were performed both with and without this group and separately for the group as such.
Concordance between self-reported and workplace-registered sick leave length, aim 2
Raw data on absence were retrieved from the workplace records, but only spells related to sick leave were included. All sick leave spells 12 months prior to the response date were identified. If a spell encompassed the response date or the date 365 days prior to that, the spell was “shortened” to ensure the duration did not exceed these dates. All sick leave spells were summarised into total length in calendar days. These were compared to the self-reported sick leave days from the questionnaire.
Mean annual length and differences in days were stratified on gender, age, profession and working hours per week. To enhance the comparability to other studies we used some of the same sick leave measures used in a Swedish study [
13]. Both measures of sick leave were categorised into: 1) 0 days, 2) 0>days=<7, 3) 7>days=<14, 4) 14 > days < 28, 5) 28>=days<56 and 56 days or more. An expanded 2x2 table was constructed and sensitivity and specificity were calculated.
All analyses were performed in Stata version 11.2.
Discussion
Our study showed an excellent agreement between workplace registered sick leave and DREAM registered sickness benefit compensation. Except for pregnancy-related sick leave, the DREAM register identified workplace-registered spells exceeding 14 days with very high sensitivity and excellent specificity. To identify sick leave spells beyond eight weeks, the optimal cut-off point in the DREAM register was nine weeks. On average, DREAM data overestimated the workplace-specified duration of sickness spells by 1.4 weeks. The ability to recall accurately declined the higher number of workplace-registered absence days. This was apparent even with short lengths; only 47.5 (95% CI: 42.3 – 52.7) % and 37.1 (95% CI: 31.4 – 43.1) % recalled accurately that they had had 7–14 days or 15–28 days of sick leave, respectively.
Other validity studies
To our knowledge, no previous study has attempted to validate a national social security register against workplace-registered sick leave so comparison with other studies is not possible.
Studies providing self-reported data on total absence length, frequency and prevalence as pre-specified categorical options reported relatively high sensitivities (range 79-91% with workplace registers as reference standard) [
5,
13,
17]. The sensitivity of 91% was found when responses were categorised in having had at least one day of absence within one year, but the specificity was low (74%) [
13]. Regarding absence of more than 28 days within one year the sensitivity was 67% and the specificity was 98%. We were able to reproduce the results reported in the study by Voss et al. in our study. In the validation of DREAM data on sickness benefit, both sensitivity and specificity were high although we adopted a much stricter requirement of sick leave registration within the same week than the criterion of sick leave within the same year used in studies using self-reported sick leave. Other studies requesting more information on sick leave reported lower sensitivities. Thus, a decline in sensitivity from 79% to 64% [
17] and from 79% to 13% [
5] were seen when additional information about diagnosis was required.
In this study, we analysed the validity of DREAM registered sick leave spell duration. A frequent topic addressed in validation studies is the respondent’s ability to precisely recall sick leave duration within a specified timeframe. Recall periods of down to two months have been shown to produce discrepancies between workplace data and self-reported duration of absence in approximately 13% of the cases. This percentage increased to approximately 50% when the recall period was extended to 12 months [
6]. In our study population the ability to recall absence lengths accurately was found in 662 cases (28.7%). Approximately 30% perfect agreement between self-report and workplace registered data among female responders was also found in the Whitehall II study [
8]. For comparison, DREAM data was in perfect agreement with workplace data in 69.7% of the cases on which weeks were reimbursed and on the respondent being on sick leave. However, DREAM data overestimated the spell duration by a mean of 1.4 weeks. Overestimation was less pronounced at shorter spells. Due to the registration procedure in the DREAM register, an expected and systematic overestimation can explain some of the variation seen in our study because one week in the DREAM register covers reimbursement of 1 to 5 days of actual workdays lost.
The agreement between DREAM data on sickness benefit and workplace data did not depend on the women’s age. This changed when pregnancy-related sick leave was included in the analyses because of the combined effect of being young and therefore more prone to be pregnant. Furthermore, the age dependency could not be found among the male employees. Because pregnancy-related sick leave is recoded from sickness benefit to maternity payment, sick leave among young women is underestimated and caution is advised when analysing sick leave in this age group of females.
Male employees seemed to give more valid self-reported data than women [
8,
9,
13]. The massive female employee domination in the public sector and in our study population hampers firm conclusions about potential gender differences in terms of diagnostic accuracy of sick leave in our data.
Prior validation studies have established that the ability to correctly recollect the duration of absence decreases with the duration of the absence [
6,
8,
13]. This was also the case regarding our self-reported data. A longer absence length was associated with increased discrepancies between workplace-registered and self-reported sick leave. Under- or over reporting was evident for 1,063 and 586 individuals, respectively. When using categorisation of sick leave duration the number of responders who underestimated their sick leave declined to 518. Still the ability to recall sick leave length accurately decreased with increased lengths of absence.
Methodological considerations
The present study included an entire population of eldercare workers in Aarhus, employed throughout 2004 and 2005 or 2006. In the validation of DREAM data nobody was lost to follow-up, thus selection bias was not an issue. In the validation of the self-reported sick leave, a total of 3,147 eldercare workers were employed throughout 2004 and 2005, but only 2,311 (73%) responded. Responders and non-responders had a mean workplace registered annual absence length of 2.5 (SD: 5.0) weeks and 4.2 (SD: 8.0) weeks, respectively (results not shown). We expect those with increased absence lengths to recall with less precision compared to those with shorter lengths. Therefore, some selection bias may have been present and caused overestimated results regarding agreement - in line with findings of Burdorf et al. [
5]. This is a major disadvantage of self-reports as compared with register data.
The large number of participants yielded a high power reflected in precise measurements and tests. Yet, the population was strongly dominated by women and the power to reveal gender difference was therefore questionable.
There was a lower agreement between DREAM data and workplace data in the measures obtained at the beginning and towards the end of 2006 than during the remaining part of the year. This can be explained by different registration procedures. We do not expect excluding these weeks to have caused information bias because having sick leave in these particular weeks would appear at random.
The registrations in both the DREAM register and the employers’ personnel files are based on manual entry and human error might explain the disagreement between the two registers on 1.55% of the employees. This validation study was based on data from one municipality. Even if manual entry of sick leave data was done by a number of different administrative employees working on different geographical settings and workplaces within the same municipality, the procedure was most likely systematised in a way that differs from that used in other companies. Future research projects should repeat the validation study using data from workplaces from the private sector and state institutions. The municipal health care workers account for approximately 20% of the total municipal workforce of 500,000 persons in Denmark [
45].
Possible implication of the DREAM data validation
Until 2011, we found 22 studies that used data retrieved from DREAM for sick leave research. Studies defined the study population from the DREAM register as those who had received sickness benefit for at least two to three weeks before baseline [
24‐
26,
29,
41,
42]. The DREAM register correctly registered sickness benefit reimbursements in 96.7% of those weeks that were also verifiable in the employers’ personnel files in our study. In general some bias should be expected because the DREAM register does not register sick leave as such but sickness benefit and other disability benefits. However, we believe that the agreement between sickness benefit and sick leave in our study is reasonable and that it supports the use of DREAM data as a selection tool to identify sick leave periods with little or no selection bias.
DREAM data used for follow-up studies [
29,
31‐
40] defined long-term sick leave as absence >8 weeks. When a cut-off point of nine DREAM-registered weeks of sickness benefit was chosen, the ability to discriminate workplace-registered spells of >8 weeks was optimal. However, from our point of view, this misclassification was non-differentiated which would tend to bias the results toward the null hypothesis.
In comparative register studies on sickness absence where the DREAM register is considered for use; it will be of great importance to recognise the recoding of pregnancy related sickness absence into maternity benefits, because it is reducing the validity in fertile-aged women.
Reliability of self-reported sickness absence declines with increasing length of absence and for studying absence beyond the period paid by the employer, national registers seems preferable to self-reported data if reliable personnel files are not available.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
CMS conceived the study, carried out statistical analyses and drafted the manuscript. NTA and NF carried out and / or supervised the statistical analyses. All authors participated in the design of the study, helped to draft the manuscript and interpreted the results. All authors have read and approved the final manuscript.