Can self-reported disability assessment behaviour of insurance physicians be explained? Applying the ASE model

The research group of the organizations participating in this study - TNO Quality of Life, the EMGO Institute of the VU Medical Centre and the Employee Benefits Insurance Agency (UWV) - drafted the questionnaire for insurance physicians who perform disability assessments of long-term sick-listed employees ("clients"). At the start of 2008 UWV drew up a list of addresses of all insurance physicians working for the agency. In March 2008 UWV sent the questionnaire, together with a covering letter containing an invitation to participate in the research, to the home addresses of insurance physicians. A reminder was sent two weeks later. Not all the physicians belonged to our target group, but it was not possible to make a selection in the mailing. In total we wrote to 750 insurance physicians. Our estimate was that the target group consisted of 450 insurance physicians: insurance physicians actively employed by UWV in May 2008 who had performed work disability assessments of long-term sick-listed employees in 2007 or in preceding years. The criteria for inclusion were mentioned in the accompanying letter. The participants sent the completed questionnaire to TNO (Netherlands Organization for Applied Scientific Research). The response consisted of 231 questionnaires (estimated response approximately 51% of the target group).

As we lacked the necessary data of the target population to do a full non-response analysis, we checked whether the group of participants (N = 231) was representative of the total population of insurance physicians working for UWV (N = approximately 900) in terms of age, gender, and working hours per week.

As this study was based on a survey under (insurance) physicians only, approval by a Medical Ethical Commission was not necessary under Dutch law.

In drawing up the questionnaire we used existing and newly developed concepts. These concepts were chosen on the basis of literature studies and four interviews with insurance physicians. In a pilot study two insurance physicians completed the questionnaire while thinking aloud in order to enable us to test whether the questions were correctly understood. Finally, two other insurance physicians were timed while they completed the questionnaire. For a detailed description of the questionnaire, the operationalization of the constructs of the ASE-model, the construction of measurements, and the imputation of missing values we refer to Steenbeek et al. [27]. Below we give a summary of the construction and content of the measures.

The operationalization of the theoretical concepts of the ASE-model and the validation of the subsequent measurements are described in detail by Steenbeek et al. [27]. We operationalized the theoretical concepts of the ASE model with 28 additive scales and 20 dimensions. An additive scale is constructed of numerical categories of items that can be meaningfully added. Some additive scales were formed on basis of already validated scales; other scales were formed after extracting factors and performing reliability analysis. For some items it was not possible to construct an additive scale. We grouped these items on a theoretical basis and used homogeneity analysis to analyse the dimensions behind these grouped items. The object scores of the dimensions that were meaningful and gave additional information were selected as variables. We call these variables 'dimensions' (indicated by 'D' in Figure 1), contrary to the variables which we constructed as additive scales, which we call 'scales' (indicated by 'S' in Figure 1). We did not reverse individual scales and dimensions so that higher scores meant a more positive outcome.

The meaning of all measurements (28 scales and 20 dimensions) is presented separately (see Additional file 1). The measures of Attitude reflect the attitude towards the insurance physician's own job [39], development of skills, support by management, the WIA and the social security system. The measures of Social norm reflect how much the insurance physician is influenced by the staff, the employer (UWV), employee representative bodies, colleagues and society in general. The Self-efficacy scale of Scholtz et al. [40] was adjusted for this study and measures the Self-efficacy specific for behaviour performed during the assessment interview, e.g. the ability to resolve difficult issues involving clients, the ability to stick to the proper procedure when a client is being difficult, the ability to stick to the planned course of the assessment interview, the ability to have a number of different solutions available when faced with a problem during the assessment interview, the ability to find a way of dealing with the situation no matter what happens during the assessment interview, etcetera. Various aspects of Barriers were measured: cooperation and co-determination [39], work pressure [41], emotional workload [42], decision-making authority [43, 44], emotional exhaustion [45], experiencing difficult clients and divers stimuli for the quality of the assessment work of insurance physicians. The measures of Knowledge reflect whether insurance physicians feel they possess sufficient medical information and knowledge. The measures of Intention reflect to what extent insurance physicians intend to perform disability assessments according to the professional standards of their professional organization, the disability legislation, and of their employer, UWV. These standards regard the importance of three aspects of objectives, task-setting and criteria of insurance physicians in the Netherlands, which concern their core business: a) stimulate recovery, return to work, self-perception and reintegration, b) estimate residual capacity, sickness, disorders, limitations and handicaps, and c) collect a consistent and verified account of daily activities of the client. The measures of Behaviour with respect to the assessment process reflect conflict handling [46], dedication [47], and various aspects of the insurance physician's assessment interview technique. Finally, measures of Behaviour with respect to the assessment itself reflects compliance with permanent full disability rules, and whether or not expressing a formalistic approach towards a clients' possibilities and limitations.

In the questionnaire, we measured the following background variables: gender, registered as insurance physician, additional medical specialization (GP, OP, medical specialist), working hours (week), years of experience as an insurance physician, assessments mainly under WIA or WAO or Wajong (Invalidity Insurance for Young Disabled Persons Act) legislation, client's industry/sector (agriculture, fishing and food, construction and timber, manufacturing, retail and wholesale, transport, financial services, temporary work, health, education, rest of public sector, other).

Our original purpose was to estimate a structural equation model with latent variables for the theoretical constructs of the ASE model, using Lisrel [48]. This proved impossible, however, as the operationalized model was fairly complex with 21 background variables and 48 additive scales and dimension scores [27] (see Figure 1). Furthermore, the number of cases (n = 231) did not allow us to reduce the number of variables with a structural equation model that included latent variables. In addition, the individual scales and dimensions of an ASE-construct are not suited to form one consistent composite scale, i.e. the individual scales and dimensions cannot be regarded as items that can be meaningfully added to each other. We therefore decided to reduce the number of variables (scales and dimensions) by separately estimating each theoretical ASE construct as latent variables in measurement models, which have to be considered a part of the results of the estimated model. Self-efficacy formed an exception because it consisted of only one scale. This procedure reduced the 48 scales and dimensions to eight ASE variables: Attitude, Social norm, Self-efficacy, Barriers, Knowledge, Intention, Behaviour: process and Behaviour: assessment.

The factor loadings were calculated as standardized coefficients. Factor scores are individual scores on basis of the factor loadings and the individual scores of the observed scales and dimensions on which a latent variable loads. The factor scores were treated as observed variables when we estimated a path model: a model with only observed variables.

Before assessing the various measurement models, we normalized the scales and the dimensions by applying the normal score procedure [49], using Prelis 2.72 [50]. The scales and dimensions were assigned to an ASE construct according to Figure 1. Maximum Likelihood was used as estimation procedure. Because we did not reverse individual scales and dimensions, factor loadings of the latent ASE-variables could have positive or negative coefficients. Non-significant loadings of a latent ASE-variable on the observed variables were deleted. We chose p < 0.10 as the level of significance because we did not want to miss any interesting relationships. Associations between measurement errors of the observed variables were allowed if suggested by the Lisrel program (Modification Indices > 3.84). The model fit was good [51] if the Chi-Square of the model was less than twice the number of degrees of freedom, if the Root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean Square Residual (SRMR) were both smaller than 0.05, and if the Comparative Fit Index (CFI), was equal to or greater than 0.90. We verified that the standardized residuals were normally distributed.

The direction of the factor score of Social norm was reversed so that all factor scores of the ASE-constructs indicated a higher score on the meaning of the ASE-construct. Because it proved impossible to estimate a measurement model for the latent variable Knowledge alone (i.e. a not positive definite correlation matrix of measurement errors), we estimated the latent variable Knowledge and the latent variable Barriers in the same measurement model, because theoretically both play a role in the relationship between Intention and Behaviour in the ASE model. Furthermore, because the measurement model of Intention proved to be a Heywood case [52], i.e. a (saturated) model with a standardized factor loading of 1.05 for one of the three scales and a negative disturbance term (-0.10), we fixed the standardized measurement error of this scale at 0.05.

For the path model the background variables were defined as exogenous variables. The scale for Self-efficacy and the factor scores for the other seven ASE constructs were defined as endogenous variables. Because all variables, including the factor scores for the ASE-constructs, were treated as observed variables, we presented them as rectangles instead of ovals in the figures, according to the convention in path analysis [48]. In the start model all the exogenous variables could have a direct effect on the endogenous variables. The direct effects between the endogenous variables were specified in accordance with the theoretical ASE model in Figure 1. We began by selecting the significant (p ≤ 0.10) estimated parameters of the exogenous on the endogenous variables. Then we fitted the model by adjusting the parameters for the endogenous variables on other endogenous variables one by one, i.e. by closing non-significant parameters (p > 0.10) and opening parameters with the largest Modification Index (at least > 3.84) if this was theoretically justified. These adaptations did not include direct effects between the determinants Attitude, Social norm and Self-efficacy, because direct effects between these determinants are not justified on theoretical grounds. Instead, relationships between these three variables could be opened as associations between the disturbance terms of these endogenous variables. For the fit of the estimated path model we used the same kind of model fit parameters as for the measurement models. In addition we ensured that the correlations of the parameter estimates were < 0.7: large correlations may indicate that the model is nearly non-identified and that some of the parameters cannot be determined from the data [48]. We analysed the product moment correlation matrix, resulting in the estimation of standardized parameters. We used the Lisrel program [50] to estimate both the measurement models and the structural model.

The mean age of the participants (N = 231) was about 51 years (sd = 7), about 41% were female, and they worked on average about 33 hours a week. The participants did not substantially differ from the population of insurance physicians (N = approximately 750) in this respect. About 86% was registered as insurance physician and about 15% was also registered in another medical speciality. The average number of performed assessments per week was about 9 (sd = 4). Insurance physicians had on average 16 years (sd = 8) of experience. We refer to Steenbeek et al. [27] for further details.

The seven factor score variables were normally distributed (with mean = 0 and variance = 1) as was the scale for Self-efficacy (median = 32,0; mean = 32.81; sd = 4.21). The factor loadings for all latent variables and the model fit parameters are listed in table 1. All measurement models for the latent variables showed a good fit.

We estimated latent ASE constructs that represented most of the assigned scales and dimensions. All of them could be described and interpreted. They were then used to build a path model that showed a good fit.

One noteworthy aspect of the final model is the positive connection between Behaviour: assessment and Self-efficacy. An insurance physician who is stringent, guideline-based and disorder-oriented strengthens his sense of Self-efficacy. This in turn strengthens the professional Intention and leads to greater eagerness, control and willingness to compromise, after which the formalistic behaviour tends to decline (see Figure 2). This seems to indicate a striking feedback mechanism in the model.

If we look at the triangle of Attitude, Behaviour: process and Behaviour: assessment, we see that Attitude has two positive direct effects on Behaviour: process and Behaviour: assessment and we see a negative direct effect between Behaviour: process and Behaviour: assessment. This may appear contradictory but one should bear in mind that Behaviour: process and Behaviour: assessment are not opposites (the direct effect between the two is also weak) so it is perfectly feasible for the same - and other - aspects of Attitude to exert a positive effect on both.

The positive connection between a positive attitude to the profession and system (Attitude) and being more influenced by others (Social norm) seems to concur with the positive direct effect of being more influenced by others (Social norm) on greater professional intention (Intention). This indicates that insurance physicians that perform more according to the norms of others have a stronger professional Intention and, at the same time, feel more positive about the profession and the social benefits system.

Another notable finding is the direct negative effect of experience of more barriers (Barriers) on eagerness, taking control and willingness to compromise (Behaviour: process) as opposed to the direct positive effect of receiving knowledge and information more frequently (Knowledge) and sticking to the rules (Behaviour: assessment). It is conceivable that physicians who stick more to the rules have a greater need for knowledge and information (about guidelines, third parties etc.). The latter profile goes hand in hand with a positive attitude to the profession and the social benefits system (Attitude). On the other hand, insurance physicians who experience more barriers (Barriers) - expressed as heavier work pressure and more frequent tiredness - may be less eager and feel more often that they have no control over the course of the assessment interview (Behaviour: process). The experience of more barriers also goes hand in hand with a less positive attitude to the profession and the social benefits system.

In contrast to what was hypothetically expected of the ASE model, we found no direct effect for Attitude on Intention. This may be related to the fact that only three aspects were incorporated in Intention: promote recovery; importance attached to capability in the assessment; and importance attached to the account of the daily activities and home situation of the client. We also found no direct effect from Intention to Behaviour: assessment. This could mean that the questions to measure intentions (Intention) in the questionnaire were insufficiently attuned to behaviour during the assessment interview (Behaviour: assessment). In addition, the absence of direct effects between Attitude and Intention on the one hand and Intention and Behaviour: assessment on the other, may be caused by the narrow distribution of the variable Intention; the majority of insurance physicians scored high on the three additive scales that formed the latent variable Intention [27]. Finally, it is important that the scales and dimensions that are loaded from a specific latent variable (a measured ASE construct) can show positive and negative relationships with the scales and dimensions of another latent variable. As a consequence, relationships between measurements of the ASE constructs may not be found.

The results of the exogenous variables show that insurance physicians who have trained in a medical specialization (besides training as an insurance physician) differ from insurance physicians who have not. Training in another medical specialization turned out to be connected with being more influenced by others (Social norm), higher Self-efficacy and a less stringent, guideline-based, disorder-oriented approach (Behaviour: assessment). These doctors listen to the opinions of other people but their confidence in their own professional ability makes adherence to the rules less important to them.

Insurance physicians who, relatively speaking, carry out more assessments per week seem to pay less heed to the opinions of others. They also have confidence in their professional ability and a less strong professional intention whereby they do assume a stringent, guideline-based, disorder-oriented approach. Steenbeek et al. [27] showed that insurance physicians experience serious barriers amongst which work pressure was most frequently cited. If more working hours per week is indeed related to heavier work pressure, this paper shows that this does not result in lower self-efficacy but rather in a more stringent, guideline-based, disorder-oriented approach.

Steenbeek et al. [27] found that half of the insurance physicians in the study did not have a positive opinion of the Work and Income (labour capacity) Act. This could be related to the years of experience. Our study showed that insurance physicians who have been longer on the job have a less positive attitude to the profession and the system. As noted earlier, Barriers is negatively connected and Knowledge is positively connected with Attitude. This fits in with the result that insurance physicians with more years of experience pay little heed to the opinions of others and demonstrate high self-efficacy, but they also say that there is a shortage of medical data and third-party information in the assessments.

A large quota of clients from the industrial sector is connected with a more stringent, guideline-based, disorder-oriented approach whereas a large quota of clients from the retail sector is connected with a less stringent, guideline-based, disorder-oriented approach. These findings may have something to do with the culture of the organizations where the insurance physicians worked before the Employee Insurance Benefits Agency was founded in 2002. At that time the tasks were performed by Industrial Insurance Associations, which were autonomous organizations with their own sector and culture [53].

Several studies have investigated the relationship between elements of the Theory of Reasoned Action and the Theory of Planned Behaviour (i.e. elements of the ASE model [27, 54]) for physicians and healthcare workers in general. These studies were conducted in specific curative settings such as clinical or primary care [55‐57] or, in the case of a systematic review study, in a curative healthcare setting in general [58‐60]. Most of them found a relationship between Attitude and Intention, between Social norm and Intention, and - if self-reported behaviour was included in the study - between Intention and Behaviour. They conclude that there is a predictable relationship between the intentions of a health care professional and their subsequent behaviour. However, special care should be given to methodological issues, especially to better define the context of behaviour performance of physicians and health care workers.

Our measurements for Intention and Behaviour: assessment reflect to what extend insurance physicians follow the rules. Therefore, studies that show specific relevance to the present study are TPB-based studies on guideline adherence by health care professionals [61‐68]. These studies show that social norm (or subjective norm) is often related to intention and seldom to behaviour. In our study we also found a relation between Social norm and Intention, and not between Social norm and Behaviour. In addition, these other studies showed that attitude, perceived behavioural control and self-efficacy are related to both intention and behaviour. In our study, we found that Self-efficacy was related to both Intention and Behaviour: process. However, Attitude was only related to Behaviour: assessment, and not to Intention. Bonetti et al [64] found that higher perceived control and higher self-efficacy in diagnosing and treatment was related to less adherence to the guideline. We found similar results for insurance physicians who have trained in a second specialization. In contrast to our findings Bonetti et al. [64] also found a relation with intention.

We know of only two studies that investigated self-reported behaviour in a non-curative, occupational health setting on the basis of elements of the ASE model. In a study among occupational physicians Rebergen [34] found no relation between self-reported guideline adherence and performance rates. Self-reported guideline adherence correlated significantly with perceived behavioural control, Social norm and positive job stress. In an earlier study of the process of sick-leave assessment among insurance physicians Croon and Langius [33] found relationships between 'Attitudes' and 'Belief in the importance' of certain issues (i.e. Intention) on the one hand and between 'Attitudes' and 'Belief in importance' with 'working styles' (i.e. self-reported preference in behaviour) on the other.

Armitage and Conner [69] found in a meta-analysis of TPB studies that around 27% of the variance in behaviour and 39% of that in intention were explained. In our path model, only 7% of the variance of intention and about 14% to 15% of that of both behaviour variables were explained. This is poor considering the explained variances in the mentioned meta-analysis. Despite the many variables included in our model, most of the variance of intention and behaviour is unaccounted for, i.e. is due to unmeasured influences or/and measurement errors. This raises the question about the validity of the ASE-constructs in the investigated path model. The original 48 additive scales and dimension scores were investigated by Steenbeek et al. [27] and were considered as valid, because of good properties. We used these scales and dimensions scores for the measurements of more general latent ASE-constructs in our model. It is possible that the measurement of the latent ASE-constructs in our model is on such general, high level, that this produced relatively weak relations between these ASE-constructs in our estimated path model.

One of the strengths of this study is that the response to the survey was considerably higher than for most other surveys among insurance physicians. Another is that we managed to measure the latent ASE constructs, which were interpretable, and in relating them in a good fitting path model.

One of the weaknesses is the cross-sectional research design, which does not allow for the analysis of causal relationships. Another is that we may not have sufficiently synchronized the questions for the various ASE constructs. For example, the theory on intention and behaviour says that intention is an important predictor of behaviour. We probably did not sufficiently define Intention for our goal: explain the assessment behaviour of insurance physicians (Behaviour: assessment). If we had, Intention might have played a more pronounced role in the model. On the other hand, we did find a relationship between Intention and behaviour related to the assessment process (Behaviour: process). A final weakness relates to the fact that our final model was established on the basis of certain specification choices. It is theoretically possible that other choices would result in another fitted model. We did, however, explore some of the alternative models to arrive at our final model. However, since a lot of modification went on in the model-building process, the findings would need to be replicated, most preferably by confirmatory path analysis in an independent sample. Especially, it is necessary to replicate the scales and dimensions, which we used in the measurement models in this study for the ASE-constructs. Perhaps, items could be developed that distinguish between specific goals and their saliency of intention of social insurance physicians, In addition, items should be developed that describe behaviour more specifically in terms of the Target, Action, Context and Time (TACT) principle [70].

In 2004 the Dutch Council for Health Care Research (CHR), a government body, stressed the need to research the variation in disability assessments performed by insurance physicians in the Netherlands [71]. Relevant information could possibly be acquired if the assessments of the Dutch insurance physicians in this study as well as the effects of their work environment and client characteristics could be linked to the outcomes of compensation claims for loss of income due to diminished work capacity. Disability assessment has been the subject of various studies in the Netherlands [24, 25, 72‐74], but these have taken place in a laboratorial rather than a 'real life' setting. If, for example, Attitude (towards both the profession and the social security system) turns out to be strongly related to the outcome of disability assessments, it would be important for social insurance physicians to receive awareness training so that they can handle the potential influence of their opinions when assessing the functional disabilities of a client.