Primary and secondary data in emergency medicine health services research – a comparative analysis in a regional research network on multimorbid patients

For this analysis, we compared three primary data study populations with respective secondary data HIS populations regarding socio-structural (age, gender) and health- and care-related characteristics (triage category, transportation to ED, case and discharge type, multi-morbidity). Our research question was: What are potential implications of using primary and secondary data for analyzing care within EDs? In addition, our analysis aims to show potential insights from the comparison of both data types and methodological-practical suggestions derived from this investigation’s experiences.

In this analysis, we investigated data from the regional research network EMANET (Emergency and Acute Medicine Network for Health Care Research Berlin). The overall scientific goal of EMANET is the identification, development, and implementation of measures for optimized health care of multi-morbid patients in emergency and acute care [21]. In the first funding period from 2016 to 2020, three mixed-methods studies were conducted in all of the eight interdisciplinary EDs in Berlin’s central administrative district Mitte. The participating hospitals differed with regard to ownership (state and federal authorities, denominational organizations, non-profit organizations), total numbers of hospital beds (from approximately 150 to 1,700), and yearly ED patient numbers (from 10,000 to 110,000). However, since all EDs of the respective city district were included in the EMANET network, the study ensured access to a group of ED patients representative for this specific urban area. The network’s aim was to investigate the specific characteristics of and healthcare provision for acute patients with one of three model diseases: (a) EMAAGE aimed at the emergency and follow-up health care of patients with hip fractures; (b) EMACROSS focused on cross-sectoral care provision for patients with respiratory diseases (with an initial focus on outpatients, which was expanded throughout the recruitment process to also include inpatients) [21, 22] and (c) EMASPOT targeted comorbid mental health conditions, such as depression and anxiety, in elderly patients with cardiac symptoms and diseases [23].

Inclusion into the primary data sample and extraction of secondary data was performed for patients in participating study EDs with at least one of the respective leading symptoms and diagnoses (see Additional Table 2) [24]. Model diseases and respective diagnoses were initially chosen from a publication on ambulatory care sensitive conditions (ACSC) in the German healthcare setting and adapted to the need of patient recruitment in the ED so that symptom diagnose codes were included [25]. ACSC are a group of common chronic and acute illnesses considered not to require inpatient treatment if appropriate ambulatory care is received [26]. The basis of our analysis consisted of three data types: (a) primary data from the three sub-studies with baseline surveys, as well as data from an electronic case report form (eCRF) for the period from 2017 to 2019, (b) a screening log that monitored and documented the recruitment process and reasons for non-participation for the period of the primary data collection, and (c) secondary data from the HIS of participating EDs for the year 2016. The HIS dataset included all ED visits of patients with respective study-related diagnoses and posed a complete representation of the relevant ED population in the specified time frame of one year.

For primary data collection, trained study personnel recruited patients between June 1, 2017 and June 28, 2019 during fixed time periods, generally on weekdays between 8 am and 5 pm, in all of the eight participating EDs. Occasionally, recruitment was extended to weekends or weekday evenings. Patients were interviewed by study nurses in the acute ED situation (EMASPOT, EMACROSS) or postoperatively on hospital wards (EMAAGE). Potential study participants were identified by study nurses via patient screening in participating EDs using data from the HIS. Inclusion criteria were project-specific leading symptoms and age (50+ years for EMASPOT and 18+ years for EMAAGE and EMACROSS). If necessary, medical, nursing, or administrative ED staff was consulted for the clarification of inclusion- and interview-relevant questions (e.g., regarding suspected diagnoses and leading symptoms or patients’ ability to be interviewed). Since the inclusion of patients in the primary data sample was based on leading symptoms and not on final and confirmed diagnoses, there was a possibility that a patient no longer possessed one of the study-relevant diagnoses at the end of his or her ED treatment. In these cases, respective patients were subsequently excluded from the study population. The screening process was documented in printed structured questionnaires, i.e., screening logs [4]. After inclusion and informed consent, study nurses interviewed patients for approximately 30 to 60 minutes with handheld tablets containing electronic versions of the study-specific questionnaires. Printed study materials were additionally available in German, English, Arabic, and Turkish language. All participants gave written permission for review of their individual electronic health records for study-specific aims.

For secondary data collection, HIS data of the eight EDs were retrieved for all patients that were treated in one of these EDs during the year 2016 and met the inclusion criteria of at least one of the EMANET sub-studies with respect to age and coded diagnoses according to the International Classification of Diseases (ICD), 10^th Revision [27] (see Additional Table 2). Since data were anonymized before extraction, patients’ consent was waived. All EDs received a list of predefined variables for extraction including patients’ sociodemographic information, diagnosis ICD-codes of ED and inpatient treatment, and parameters of ED care.

Primary data collection in patient interviews was tablet-based and data was automatically transferred after entry to a Research Electronic Data Capture (REDCap) tool hosted at Charité – Universitätsmedizin Berlin. Each study participant received a unique pseudonym. Screening log data and administrative participant data was saved separately in a Microsoft Access database. Further information from patients’ electronic hospital files was manually entered by study nurses into a study-specific eCRF in another REDCap database using the respective participant pseudonym. The central data management of EMANET collated data sets by using participant pseudonyms.

Anonymized secondary data were prepared and delivered to the central data management of the coordinating unit of EMANET by the participating EDs, the information technology (IT) departments of the respective hospitals or the vendors of the respective HIS in CSV (comma-separated values) or Microsoft Excel files. Data delivery followed established data protection procedures on password-protected devices as described in the project’s data protection concept. The central data management checked all data for completeness and plausibility and linked all files of the participating EDs to yield one final data set with secondary data from all EDs. Due to varying documentation standards between the participating EDs, it was necessary to harmonize the data to establish comparability. HIS data were adapted according to data harmonization rules consented by a working group of EMANET researchers. These rules basically followed the data harmonization recommendations of the INDEED project [28]. All primary and secondary data was stored on servers of the Charité – Universitätsmedizin Berlin.

For comparative analyses between primary and secondary data it was necessary to create two differently tailored datasets of secondary data. One secondary data set was adapted to the recruitment conditions and inclusion criteria of the primary data sample (see Fig. 1 and Additional Figs. 1, 2, 3 for each sub-study), i.e., secondary data for EMACROSS included only patients registered in EDs between 8 am and 5 pm analogous to patient recruitment times of the primary data sample and secondary data in EMASPOT only included patients with an age of 50 years or older and registered in EDs between 8 am and 5 pm. The second data set included all patients presenting with relevant diagnoses (for EMASPOT: age ≥ 50 years) independently from the time of ED admission.

In this analysis, the following variables were selected as suitable for comparison between data types: age, gender, triage category, transportation type, case type, discharge type, and multi-morbidity. The choice of variables is justified by their general availability within all eight HIS in the participating hospitals, the comparatively small number of missing data within variables, and their relevance to clinical routine in EDs. Except for multi-morbidity, all variables were generated by directly interviewing patients or by data extraction from each ED’s documentation system. Triage categories were structured according to the Manchester Triage System (MTS). Transportation to the ED was coded in walk-in (including patients accompanied by relatives), by ambulance services (transportation of non-urgent and mobility-impaired patients), by emergency medical services (EMS), and by EMS accompanied by an emergency physician. Case type was coded into in- or outpatient. Discharge type for inpatients was differentiated in discharge to home or an existing care arrangement, transfer to another hospital or other health care facility, death, or other. Multi-morbidity was defined according to van den Bussche et al. as three or more chronic diseases of a predefined list of ICD-codes [29]. For our comparative analyses, ICD-10 diagnoses documented in HIS and diagnoses reported by participants were used to determine multi-morbidity.

The aim of the statistical analysis was to compare primary data and secondary data regarding their similarity in the distribution of various ED-related parameters. For descriptive statistics, the continuous variable (age) was characterized by its mean and standard deviation (SD). Categorical variables (gender, triage category, type of transportation to the ED, multi-morbidity, discharge type, case type) were summarized as numbers (percentages) of subjects. We computed 95% Wald confidence intervals for the distribution of percentage estimates of categorical variables for population proportions of n≥5. In order to estimate the distribution of variables in primary data in comparison to the ED population with respective diagnoses in secondary data, we calculated binomial tests, chi-square goodness-of-fit tests, or one-sample t-tests according to scale level. Available case analysis (pairwise deletion) in SPSS version 27 (IBM Inc.) was used for all analyses. In order to discuss differences between variable distributions in the study-specific primary and secondary data samples, primarily differences in point estimates and confidence intervals of each variable were consulted. Graphical distributions in the form of bar charts for multi-categorical variables (MTS category, transportation to ED, discharge type) are available in Additional Figs. 4, 5 and 6. We included all relevant items of the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) Statement in this manuscript [30].

In the population consisting of patients with hip fractures (EMAAGE; n=326 in primary data sample; n=439 in secondary data sample), the distributions of age and gender were similar between both samples (see Table 1). Although statistically significant, only minor differences were found in the triage category, i.e., slightly more patients with triage category 2 (“very urgent treatment”) were included in the primary cohort compared to the secondary data population (16.4% vs. 12.6%; see Additional Fig. 4). Concerning the other triage categories, we observed only small deviations between 1 percentage point (pp) and 2pp, rendering the distribution of patients in both populations between triage categories similar. Concerning case type, while all patients in the primary sample were hospitalized after ED treatment, 3.2% of patients from the secondary sample were coded as released after their ED stay in HIS data. The variables transport type, discharge type and multi-morbidity showed more profound differences. In transport categories, differences between 3pp and 5pp were found (see Additional Fig. 5). However, the proportion of patients transported by EMS was similar, i.e., the confidence interval included the estimate of the secondary data sample with respective diagnoses. The discharge type altered between cohorts, e.g., 16pp more patients in the primary data sample were transferred to another health care facility, while 13pp less patients were discharged home or to a pre-existing care arrangement (see Additional Fig. 6). Finally, the primary data sample included 8pp more multi-morbid patients compared to HIS data (65.3% vs. 57.4%).

In patients with respiratory diseases (EMACROSS; n=472 in primary data sample; n=1,563 in secondary data sample (presentation between 8am and 5pm), n=3,410 in secondary data sample (without time restriction)), characteristics between primary and secondary data samples showed pronounced differences in the comparison of deviations regarding pp and confidence intervals (see Table 2). However, the distribution of gender and multi-morbidity did not show any relevant differences in the three samples. Study participants (53.6 years) were on average six years younger than patients in the secondary data samples (59.9 years and 59.1 years, respectively). More patients were discharged home or to existing care arrangements (87.0%) than in the secondary data samples (80.3% and 79.7%, respectively; see Additional Fig. 6). Mortality was lower in the primary data sample (0.2%) than in the secondary data samples (8.3% and 7.7%, respectively). Accordingly, less patients in the triage categories 1-3, and correspondingly more patients in categories 4 and 5 were recruited (see Additional Fig. 4). The type of transport to the ED differed: primary data contained fewer patients who were transported by ambulance and EMS, but slightly more patients who were accompanied by an emergency physician, and far more walk-in patients (see Additional Fig. 5). Finally, EMACROSS recruited substantially more outpatient cases compared to the proportion of outpatients in the secondary data samples.

In the third project EMASPOT (n=644 in primary data sample; n=2,777 in secondary data sample (presentation between 8am and 5pm), and n=5,480 in secondary data sample (without time restriction) patients with cardiac symptoms and diseases of comorbid mental health conditions (MHCs) were screened and recruited. Differences in percentage points in the distribution of patient characteristics in primary and secondary data samples were rather small, although statistically significant (see Table 3). Data types did not differ concerning case type. Study participants were slightly younger (68.4 years) than the secondary data samples (69.8 years and 69.5 years, respectively). Slightly less patients from triage category 1 were recruited into the primary sample (0.5%) in comparison to the secondary data sample (1.4% and 2.4%, respectively; see Additional Fig. 4). Small differences of 3pp to 4pp were found for discharge types; however, no specific direction or recognizable structure was detectable (see Additional Fig. 6). The EMASPOT sample showed a clear difference in gender distribution, where 7pp more men were included in the primary data sample. As in the other sub-studies, moderate differences between primary and secondary data were found in the distribution of transport type (see Additional Fig. 5). The largest discrepancy of 14pp between populations was found in the prevalence of multi-morbidity with higher frequency for study participants (68.3%) in comparison to secondary data samples (54.1% and 52.6%, respectively).

Of all eligible patients, 56.3% in EMAAGE, 45.7% in EMACROSS and 45.4% in EMASPOT gave consent to participate in the respective studies. Reasons for non-participation of eligible patients were described elsewhere as a summary for all three study populations [24]. Gender and age – the only available categories for comparison in non-responders and participants – showed slight differences. In EMAAGE, proportionally more women participated. In EMASPOT and EMACROSS, participants were younger than non-responders (see Additional Table 1).

Missing values in primary data samples and in the complete secondary data sample (with presentations throughout the day) applied to MTS category, transportation to ED, and discharge type (see Additional Table 3). Furthermore, data on case type was missing in EMACROSS and EMASPOT. Analyses of missing data in the secondary data sample revealed that information was partly not available (in the required format) from all eight ED HIS for the above listed variables: data on case type was not available in one ED; transportation to the ED and discharge type in two EDs; and MTS category in four EDs. Concerning the amount of missing values at the ED level, all of the above listed variables were supplied by three EDs, one missing variable each was observed in two ED HIS datasets, two missing variables each in a further two ED HIS datasets and three missing variables were observed in one ED HIS dataset.

Our research combines comprehensive data of two types on three ED patient populations with common model diseases. Nevertheless, our analyses are subject to limitations. Firstly, samples of primary and secondary data were drawn from two different time periods due to research-practical reasons and availability of secondary data. This might have influenced absolute numbers in sample composition. However, no changes in the relative distribution of patient- and care-related characteristics in participating EDs should have occurred in the rather short period between 2016 and 2019, as there were no major changes in the prevalence of studied (mostly chronic) diseases, medical guidelines for ED treatment of these diseases, and in the structure or processes of ED care in Germany. Secondly, some variable categories in primary and secondary data sets originally differed and were thus harmonized retrospectively for comparative analyses, which might have introduced minor distortions of results. Thirdly, three variables (triage category, transportation to ED, and discharge type) showed high proportions of missing values, especially in secondary data, which might have influenced results. Reasons for high missing values in secondary data samples were mainly due to the fact that some EDs did not transmit data on certain variables, e.g., because this information was not collected in the respective ED at all (no mandatory field in documentation forms) or information was not collected systematically for every ED patient. If the amount of missing secondary data from specific study EDs would be systematically associated with the above-mentioned ED factors, the distribution of the respective variables in our analysis of secondary data might be biased. However, from descriptive analysis of ED features and the pattern of the amount of missing values per ED, no systematic bias in this direction became evident so that we can reasonably assume that missing values in our datasets occurred completely at random. Lower missing values in primary data in the same variables might point to the advantage of primary data collection by trained study nurses, who closely observed the care process of study participants during recruitment and manually retrieved not readily available information from all electronic documentation in the patient file. Fourthly, identification of cases in primary and secondary data was different (leading symptoms in primary recruitment and diagnoses in secondary data), which might have affected the comparability of populations. Lastly, the secondary data sample consists of cases from eight EDs. However, the number of patients treated in each ED differs vastly between EDs. As documentation of variables was not harmonized prior to data extraction, systematic differences may occur.