Introduction
While the assessment of analytical precision within one medical laboratory has received much attention in scientific enquiry, extent and sources of variability that occur between laboratories quantifying the same parameter remains incompletely understood. In view of the fact that the clinical value of a laboratory test depends directly on its reproducibility and comparability[
1], the scientific community has made great efforts to promote analytical precision within laboratories lately[
2‐
4]. For example, external quality assessment programs (EQA) were introduced to improve comparability between laboratories and are seen as an essential part of quality management systems today[
3‐
5]. Nevertheless, data on variability between laboratories remain limited and analysis of variance components are scarce. Available investigations indicate a large variability, even in the context of coagulation parameter measurements[
5‐
8] thus jeopardizing the comparability of results between different institutions.
On the other hand, knowledge is also scarce for causal sources of variability between laboratories. Possible influences may be the type of the parameter, reagents and calibrators used, the level of standardisation within each laboratory and the adherence to guidelines[
9]. If investigations could identify factors that contribute to the variability between laboratories, efforts could be made to improve comparability efficiently. In this study, using the example of haemostasis, we quantified the variance components when performing coagulation tests with identical analytical platforms in different laboratories and computed intraclass correlations coefficients (ICC) for each coagulation test. Thus, we aimed to quantify the extend of variation between state-of-the-art laboratories in relation to the variation of the subjects and to determine the sources of this variation.
Results
A total of 360 fibrinogen measurements, 180 PT measurements, 160 F II, F V, F VII, F VIII, F IX, F X and F XI measurements, and 140 F XIII measurements were available for analysis because only fibrinogen was determined two times and one laboratory did not provided all tests. Raw data for all analysis are available at Additional file
1: Table S1.
The intraobserver variability for fibrinogen measurements, indicating the variability within a laboratory, ranged from 0.02 to 0.04. This variability occurs from the variability within a laboratory (including the technician handling the samples). The variability between laboratories ranged from 0.006 to 0.097. This indicates that laboratories used the platform in different ways or that they interpreted the manual differently. This detailed analysis was only possible for the fibrinogen measurement because fibrinogen was determined in plasma samples of both points in time.
The results allow discussing how prone a platform – the specifications, the complexity in the handling and other reasons – might be to create higher variability when used in different laboratories and contexts.
The ICC for fibrinogen ranged from 0.37 to 0.66 and from 0.19 to 0.80 for PT between the laboratories. Though, the amount to which non-subject factors is accountable for the variance of the parameters is very varying. Low ICC indicates that non-subject factors are major components of variance.
The ICC of FII when assessed with platform 3 was as low as 0.04 as compared to 0.44 when measured with platform 2. For the remaining factors the ICC’s ranged from 0.22 (F V) assessed with platform 3, to 0.93 (F VIII) assessed with platform 2.
For the two platforms providing enough data, the average ICC across all parameters was 0.64 (standard deviation 0.25) for platform 2 and 0.56 (standard deviation 0.28) for platform 3. Overall, the ICC’s were at 0.60 on average. Details are available on Table
2.
Table 2
Intraclass correlations of various coagulation tests and intraobserver variability of fibrinogen tests with various platforms
# laboratories
| 1 | 2 | 4 | 2 |
Fibrinogen
| 0.85 | 0.37 | 0.65 | 0.66 |
Within- laboratory variability ¶ | | 0.02 | 0.03 | 0.04 |
Between- laboratory variability + | | 0.097 | 0.03 | 0.006 |
Prothrombin time
| | 0.19 | 0.24 | 0.80 |
F II
| | 0.44 | 0.04 | |
F V
| | 0.63 | 0.22 | |
F VII
| | 0.71 | 0.82 | |
F VIII
| | 0.93 | 0.61 | |
F IX
| | 0.86 | 0.83 | |
F X
| | 0.82 | 0.70 | |
F XI
| | 0.90 | 0.65 | |
F XIII
| | 0.57 | 0.81 | |
Discussions
Variance components that could be attributed to technicians or laboratory procedures were substantial, led to disappointingly low intraclass correlation coefficients for several factors and were pronounced for some of the platforms, probably for those allowing more instrument adjustments. Our results emphasise that variability of parameters between state-of-the-art laboratories remains considerable. This may have a relevant impact on the reproducibility and comparability of the results.
Our results confirm and extend previous reports on a wide variability of coagulation parameters. A wide variability of PT measurements was already noticed with the introduction of early automation coagulometers[
13]. An investigation of the World Federation of Haemophilia (WFH) EQA programme and the United Kingdom National External Quality Assessment Scheme (UK NEQAS) conducted in the UK and in emerging countries revealed a good to acceptable variation with regard to PT and aPTT (coefficient of variation [CV]: 10.1-20.4%) but an extensive variation with regard to FVIII, FIX and von Willebrand factor determination (CV 6-154%)[
14]. Considerable variations in determination of von Willebrand factor antigen (vWF:Ag) and ristocetin cofactor activity (vWF:RCo) were recognised in investigations of the U.S. College of American Pathologists proficiency testing (CV 3.2-30.9%; n = 171 laboratories)[
14], the European Concerted Action on Thrombosis and Disabilities Foundation (CV 10-40%; n = 181 laboratories)[
15] and UK NEQAS (CV 15-50%; n = 200)[
16].
Agreement with regard to subtherapeutic, therapeutic or supratherapeutic levels in the monitoring of unfractionated heparin using activated partial thromboplastin time or anti-Xa level was very limited in a cross-validation study[
17] as well as a study using the results of the annual Ontario Quality Management Program[
6]. An interlaboratory agreement of 16-52% and a coefficient of variation of 10.5 – 65% were reported respectively. Another investigation used the results of the Italian External Quality Assessment Scheme and found a wide variability in estimation of quantitative D-Dimers (coefficient of variation up to 47%)[
18]. A high degree of variation between laboratories in the identification of coagulation factor inhibitors was shown in several investigations, which used data of external quality assessment programs[
7,
19‐
21]. Other investigations identified the type of the reagent[
22], the method of the assay[
23] and the calibrator[
24,
25] as factors that bias the measurements between different laboratories and the coagulometer influencing the precision of the measurements[
22].
In contrast to these previous investigations, our study illustrates the degree of variation between laboratories using state-of-the-art coagulometers and reagents in relation to the variation of the subjects. We showed the degree of variation for PT, fibrinogen and coagulation factors II, V, VII, VIII, IX, XI and XIII. Furthermore, we characterised factors associated with technicians and laboratory procedures as a relevant source of this variation. These factors include also the choice of the reagents, which may have influenced the higher between-laboratory variability of platform 2 in contrast to platform 3.
Our investigation has several limitations. First, it is an exploratory study with a relative low number of determinations. However, it facilitates future investigations with more determinations and a proper power analysis. Second, “factors associated with technicians and laboratory procedures” are determined as a global factor. The design of the study did not allow discriminating additional potential factors such as type of the reagents, which are also inadequately addressed by the literature.
Future investigations have to separate these factors to discriminate between organisational factors and single technicians. Third, only fibrinogen determination was analysed for two points in time, because of the study design. If future investigations would determine more parameters two times, better information on differences between parameters would be possible. Forth, only state-of-the-art laboratories specialised in determination of coagulation factors measured the samples in our investigation. The results could possibly be different if routine laboratories would have been investigated.
The strength of our study is that was done it in a nation-wide design including only state-of-the-art coagulation laboratories of all tertiary hospitals. Therefore, a selection bias appears unlikely. Furthermore, a wide range of coagulation parameters was determined. Though, the observed effects were internally confirmed by other parameters. Moreover, it is one of the first investigations that observed the sources of this variation and – to our knowledge – the first investigation, which observed factors associated with technicians and laboratory procedures as possible source for variation of results between laboratories.
The present results suggest that variability of coagulation parameters between specialised laboratories is substantial despite state-of-the-art coagulometers and reagents, great efforts to guarantee precision of laboratory tests and external quality assessment programs. The results indicate that factors associated with technicians and laboratory procedures are a relevant source of this variation. Though, organisational factors are a promising field of work for future investigations on the sources of this variability. Furthermore, efforts to enhance laboratory organisation may be an attractive area of activity for reducing variability of the measurements of coagulation parameters as well as improving quality of laboratory results within laboratories. This hypothesis is supported by the fact that platforms, which allow more adjustments, were associated with a greater variance than others with a more rigid application. Furthermore, parameters, which necessitate a greater effort in structuring operation procedure, have had a lower variance than more simple parameters.
In conclusion, variability between laboratories in determination of coagulation parameters was considerable and variance components that could be attributable to technicians and laboratory procedures were substantial. Once confirmed in larger studies, our findings call for sustained efforts to raise the level of standardization of structures and procedures involved in the quantification of coagulation factors.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
LA, AA-S, LMA, WK, AM, GR, HS, DAT and WAW designed the RIVAMOS study, performed the research and collected the data. MN, LMB and WAW designed the present part of the investigation, analysed the data and drafted the manuscript. All authors revised the manuscript, added important intellectual content and approved the final version of the manuscript.
Part of the investigation was presented at the 56th meeting of the “Gesellschaft für Thrombose und Hämostaseforschung (GTH)” in St. Gallen, Switzerland, February 1–4, 2012.