The SAIL databank: linking multiple health and social care datasets

Methods were devised to address two questions. The first question assessed the accuracy of accepting the NHS number supplied in routine NHS data as the basis of a unique identifier. The second question assessed the effect on numbers of records matched of varying the techniques applied in matching each of three different datasets to the NHSAR. As this study involved work with potentially person-identifiable variables it was conducted in Health Solutions Wales (HSW) who act as the Trusted Third Party (TTP) in providing HIRU with a data pseudonymisation service [6].

Three test datasets of person-based records from the health economy of Swansea were used in this study. These were: a primary care dataset from across the general practices (GP) in the area; a secondary care dataset of hospital in-patient data from the Patient Episode Database for Wales (PEDW); and a local authority social services dataset called the PARIS system. The PARIS system is an electronic record of individuals receiving various social services including, mental health, learning disabilities and elderly care under the auspices of the local authority. These will be referred to as the GP dataset, the PEDW dataset and the PARIS dataset, respectively. As part of NHS primary and secondary care services, the GP and PEDW datasets are structured to include an NHS number. The PARIS database, as part of social services, does not contain NHS numbers. The criteria used to assess matching efficacy were: forename, surname, gender, postcode of residence and date of birth. These will be referred to as the set of matching variables. The NHSAR was used as the reference dataset and records in the test datasets would be expected to have a match on the NHSAR.

The MACRAL (Matching Algorithm for Consistent Results in Anonymised Linkage) algorithm was developed for the work of HIRU. MACRAL is an SQL-based algorithm that is used to apply DRL and PRL methods to the set of matching variables. DRL looks for an exact match on all five variables. The probability-based linkages make use of a variety of techniques, including some which allow similar but not identical query strings to be accepted as possible matches [14]. These include Lexicon matching and Soundex matching. The Lexicon used in this study is a Welsh-specific list of alternative forenames, based on variants in the registered name given by persons listed on the NHSAR, such as Betty, Elsie, Liz, etc. for Elizabeth. Soundex matching is a standard technique that uses codes for variant phonetic spellings of the forename or surname. Probabilities are assigned to the match success, and these are based on likelihood ratios calculated using a Bayesian approach of prior and posterior odds, by taking into account the distributions of the set of variables on the NHSAR for the Welsh population. For example, it takes into account the occurrence of common surnames, such as Jones, in deriving the likelihood ratio to create the weighting assigned to the match. It also recognises the non-independence of certain pieces of information, such as the male gender and recognised male first names, in generating the likelihood ratio.

The posterior odds are calculated as:

Posterior odds = prior odds * likelihood ratio

The likelihood ratios are calculated as follows:

Firstly, where the demographic variables match (e.g. on surname) -

And where the demographic variables do not match -

In this way, pairs of variables found to match increase the odds of a match and pairs of variables that don't match decrease the odds. This is applied to each of the set of five variables for each record to produce the final cumulative probability of a match. Acceptable matching thresholds for a given dataset can be specified as required. A range of matching probabilities with cut off points of 99%, 95%, 90% and 50% were assessed for each of the three datasets in this study.

This was addressed by matching each of two NHS datasets against the NHSAR on the set of matching variables, and using the results to allocate an NHS number to the records in those datasets. These were a GP dataset and a PEDW dataset. A GP dataset of registered patients (n = 229,127) was extracted for this study, and of this, the sub-set of 229,117 records with a valid NHS number was used. A sub-set of the PEDW data was used to ensure manageable computations, and it was arbitrarily set as records with admission dates on the 15th of every month from 1998–2007 (n = 290,650). Of this sub-set, records with a valid NHS number were used to develop a test dataset (n = 264,868). The resulting GP and PEDW datasets included the set of matching variables. Supplied NHS numbers were validated by using the NHS check digit algorithm [15]. DRL and PRL methods were applied to the set of variables to allocate an NHS number to the records in the GP and PEDW datasets. The degree of agreement between the allocated NHS number and the NHS number supplied in the dataset was checked and used to calculate specificity and sensitivity values. These are defined respectively in this context as: number of matches found to correspond to the same NHS number in the GP or PEDW dataset, and total number of matches made. Where this process resulted in disagreement this was taken as an error in the matching process or in the GP or PEDW dataset, as this work uses the assumption that the reference dataset (the NHSAR) is 100% accurate. There are four possible outcomes in the record matching process: true positive (correct match), false positive (mis-match), true negative (no link present) and false negative (link missed) [16]. However, as all analyses were conducted on anonymised data it was not possible for us to check the actual source of any error, which could be done by reviewing individual clinical notes. This is a limitation of the study and we aim to address this issue in the future so that we can differentiate between the sources of error.

The second question measured the impact, on the numbers of records that could be matched, of adopting different probability thresholds and techniques. Cut-off points of 99%, 95%, 90% and 50% were used for each of the three datasets. The GP dataset of registered patients (n = 229,127) extracted for this study was used in full (i.e. including the records without a valid NHS number). Registered patients were chosen to ensure that they were resident in the area and therefore expected to be included on the NHSAR. The PEDW dataset described earlier, including the records without a valid NHS number (n = 290,650), was used. Finally, the assessment was conducted on the PARIS dataset. The numbers on this system are much smaller than on many NHS systems and the entire database (n = 18,540) was anonymised and matched with the NHSAR to assess what proportion of records could be linked to a unique individual within the NHSAR. In each case, the numbers of records in agreement with the NHSAR were taken as successful matches and those resulting in disagreement as error (as previously). However, error rates are not quoted in this case, as the datasets included records without a valid NHS number, and the NHS number is used as the cross-check to calculate the error rate.

The initial question assessed the level of accuracy that could be obtained by using a valid NHS number as the basis of an anonymous identifier in routine data. Table 1 shows the results of comparing the NHS number supplied in the GP and PEDW datasets with the NHS number allocated via PRL & DRL methods. The level of agreement between supplied and allocated NHS number was high with disagreement (error) levels of < 0.2%. DRL produced the lower disagreement level, as would be expected with higher specificity, but PRL enabled the greater proportion of records to be linked.

The effect of varying the matching probability threshold and technique on the numbers of records that could be matched was assessed for each of the three test datasets and the results of these analyses are summarised in Table 2. The percentage of records in the GP dataset that could be matched to the NHSAR was > 99.99%. Varying the acceptable PRL threshold for record matching had negligible effect on the high proportions matched. For the PEDW data, 91.1% of the sample records contained a valid NHS number and by combining these with DRL, the matching rate was increased to 96.6%. The highest match rate was achieved using the combination of valid NHS numbers, DRL and PRL at the 50% threshold.

Of the 18,540 records in the PARIS database, 14,158 (76.4%) were successfully matched to the NHSAR with DRL, with further records being matched using various PRL thresholds. Again the combination of DRL and PRL (50%) yielded the greatest value with 95.2% records matched, leaving a remainder of only 4.8% that could not be matched.

The results obtained from these analyses informed the decision to operate the algorithm in the sequence shown in Figure 1. Firstly, having assessed the accuracy of NHS numbers in routine data and achieving a high degree of agreement with the NHSAR, records with valid NHS numbers are accepted. Next, DRL is carried out on the set of matching variables. Following from this, the remaining unmatched records are subjected to PRL methods down to the 50% threshold. Datasets from non-NHS organisations enter the process at DRL. As a result, an ALF can be allocated to the matched records and this is used as the linking field for each individual in the dataset.

This assessment confirmed the suitability of accepting a valid NHS number as the basis of allocating the unique identifier: the ALF. The error levels were extremely low, and as would be expected from its greater specificity, were lower for DRL than for PRL. However, PRL with its higher sensitivity resulted in a greater proportion of records being linked than DRL with only slightly higher error rates.

A comparison of probability thresholds and techniques using the GP data resulted in consistently high levels of records matched. As the NHSAR is essentially a list of all patients registered with general practices, whilst some anomalies may occur due to delays in registering new patients, very high levels of NHS number completeness and agreement with the NHSAR were to be expected. This was found to be true for DRL and any variant of PRL with negligible effects on the high proportions matched or on error levels.

High rates of matching were also achieved with the PEDW data demonstrating the notable efficacy of the methods. Although the PARIS database of social services data does not contain NHS numbers, it does contain names, genders, postcodes of residence and dates of birth. It would be expected, therefore, that the matching rates would be considerably lower than were obtained for the NHS datasets. However, using a combination of methods, over 95% of the records were matched. The success of matching on these criteria is a particularly significant result. It means that for datasets, such as these, that originate outside the healthcare sector, the ALF derived from the individual's NHS number (recorded on the NHSAR) can still be consistently applied to their anonymised records. This enables a broad scope for record-linkage studies.

In table 2, the slightly greater numbers of records matched by the sequential process compared to PRL at the 50% threshold are most likely due to rare occurrences of duplicate records on the NHSAR. In those cases, records with an exact match on all five variables would be matched by DRL, but would not be matched by PRL as the highest score/second highest score would be < 2 (as set out in Figure 1).

It is recognised that the increase in sensitivity of lower threshold PRL is accompanied by a decrease in specificity. This increases the risk of acceptance of false positive matches which could have important implications for the analysis of health-related data, particularly if it is to be used to inform clinical practice. As we were unable to distinguish between types of error, we cannot quantify our false positive and false negative rates at this stage. Because of this, the record matching rate is taken into account when extracting data for analysis. The analysis can be repeated including and excluding the records matched at lower PRL thresholds to check for consistency in the results, and to inform the sample that should, therefore, be used for each particular application of the data.

Record linkage is widely recognised as having far-reaching consequences for the development of innovative approaches to research [3, 8]. This study has demonstrated high levels of matching efficacy across three disparate datasets in health and social care that compare favourably with the published literature. For example, specificity values of 100% [17], 99.4% [18], 98% [19] and 89.7% [20], with corresponding sensitivity rates of 92% [17], 99.2% [18], 94% [19] and 99.9% [20] have been reported across various types of record linkage study. The results also compare well with record linkage software packages, such as Link Plus and The Link King [21]. It is worth noting that variations obtained in matching efficacies may be due to the quality and levels of completeness of the datasets as well as to the technical aspects of the linkage systems.