Annual influenza epidemics cause substantial morbidity and mortality, and the majority of patients with influenza-like illness present to primary care. Point-of-care influenza tests could support treatment decisions. It is critical to establish analytic performance of these platforms in real-life patient samples before uptake can be considered. We aimed to assess the analytical performance and ease of use of the cobas® Liat® PCR POCT in detecting influenza A/B and RSV in samples collected from patients with influenza-like illness in primary care. Sensitivity and specificity of the cobas® Liat® POCT are calculated in comparison with a commercial laboratory-based PCR test (Fast-Track Respiratory Pathogens 21 Plus kit (Fast-Track Diagnostics)). Samples with discordant results were analysed additionally by the RespiFinder 2Smart (PathoFinder) using an Extended Gold Standard (EGS). Acceptability was scored on a five-point Likert scale as well as a failure mode analysis of the cobas® Liat® POCT was performed. Nasal and oropharyngeal swabs were obtained from 140 children and nasopharyngeal swabs from 604 adults (744 patients). The cobas® Liat® POCT had a sensitivity and specificity of 100% (95% CI 99–100%) and 98.1% (95%CI 96.3–99%) for influenza A, 100% (95% CI 97.7–100%) and 99.7% (95%CI 98.7–99.9%) for influenza B and 100% (95% CI 87.1–100%) and 99.4% (95%CI 98.6–99.8%) for RSV, respectively. According to trained lab technicians, the cobas® Liat® POCT was considered easy-to-use, with a fast turn-around-time. Cobas® Liat® POCT is a promising decentralised test platform for influenza A/B and RSV in primary care as it provides fairly rapid results with excellent analytic performance. Point-of-care influenza tests could support treatment decisions in primary care. Cobas® Liat® POCT is a promising decentralised test platform for influenza A/B and RSV in primary care as it provides fairly rapid results with excellent analytic performance.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Background
Annual influenza epidemics cause substantial morbidity and mortality and the 2009 H1N1 influenza outbreak was defined as a pandemic by the World Health Organisation (WHO). [1, 2]
About 50% of patients presenting to primary care with influenza-like illness (ILI) during a period of high influenza incidence, with some variation by country, usually have laboratory-confirmed influenza infection. [3] Unlike in Japan for example, [4] in European primary care, point-of-care testing for influenza is not routine practice, making it difficult for clinicians to distinguish influenza from other viral infections that cause ILI at the point of care with a high degree of certainty. [5] It is common practice throughout Europe to base treatment or advice to patients with ILI on signs and symptoms alone, given that point-of-care tests (POCTs) may neither be feasible nor perform adequately, and that antiviral treatment is hardly ever prescribed in primary care. [6] However, guidelines recommend use of antiviral drugs during the influenza season for high-risk groups. [7, 8] POCTs for routine use in primary care and (cost) effective treatment and willingness to prescribe would have major implications for care delivery. The performance of novel POCTs intended for use in primary care is seldom determined using samples obtained from patients consulting in primary care. [9] A valid, rapid test to detect influenza in primary care patients could help target antiviral treatment and informing patients about avoidance of risk for transmission and likely clinical course. Several new POCTS for common respiratory infections in primary care are merging onto the market. [10, 11] We aimed to use the opportunity of an existing trial, the “Antivirals for influenza-Like Illness? An rCt of Clinical and Cost effectiveness in primary CarE (ALIC4E)” trial, [12, 13] to evaluate the cobas® Liat® POCT, selected as it is easy to use and has excellent proof-of-principle supporting data. [14‐18] However, before uptake into routine care, it is critical that analytic performance be established using samples obtained from real patients in the setting in which the test might be marketed.
Anzeige
Our primary objective was to assess the analytical performance of the cobas® Liat® POCT in detecting influenza A/B and RSV in samples collected from patients consulting with ILI in primary care. In addition, we aimed to explore ease of use.
Methods
Study endpoints
The primary endpoint was the sensitivity and specificity, and positive and negative predictive values of the cobas® Liat® POCT in comparison with laboratory-based PCR testing. The secondary endpoint was the acceptability and failure mode analysis of the cobas® Liat® POCT.
Study population
We included a selection of swabs taken from patients presenting to primary care with symptoms of ILI, willing to participate in the ALIC4E trial, who met the inclusion criteria and gave informed consent. [12, 13] There were no additional exclusion criteria. For the purposes of the trial, ILI was defined as a sudden onset of self-reported fever, with at least one respiratory symptom (cough, sore throat, running or congested nose) and one systemic symptom (headache, muscle ache, sweats, chills or tiredness), with symptom duration of 72 h or less. [12, 13]
Recruitment for the ALIC4E trial was during periods of heightened influenza incidence which was determined by reviewing the data from the European Centre of Disease Prevention and Control (ECDC) in combination with local and regional alerts during the winters of 2015–2016, 2016–2017 and 2017–2018. The ALIC4E Trial team confirmed to each recruiting network when their sites could begin recruitment. Recruitment took place through recruiting sites (GP Practices, Out of Hours surgeries or Paediatric Centres); recruitment was paused when influenza activity dropped again below the epidemic threshold.
Anzeige
Design
This was a method comparison study where a selection of swabs taken at baseline during the ALIC4E trial from participants presenting to primary care are analysed both on the cobas® Liat® POCT with the Influenza A/B & RSV Assay, as well as the laboratory-based Fast-Track Respiratory Pathogens 21 Plus kit (FTD, Fast-Track Diagnostics). The selection of swabs was aimed to detect a representative range of test positive and test negative samples for the viral pathogens relevant to the assay under investigation, including: all RSV positive samples, and, in function of the epidemiology of the circulating strains, a more or less equal proportion of samples testing positive for influenza A H1N1, influenza A H3N2 and influenza B.
For the laboratory-based PCR test, extraction was performed on NucliSENS easyMag (bioMérieux) and amplification on Lightcycler 480 (Roche).
Sample analysis on the cobas® Liat® POCT started 12th February 2018 and ended 30th June 2018.
For the POCT, all patients aged up to 16 years recruited during the ALIC4E trial had an oropharyngeal and nasal flocked swab taken at baseline by the responsible clinician or recruiter. These two samples were placed in one 3-ml Universal Transport Medium (UTM) (Copan). Those aged 16 years or older had a nasopharyngeal swab taken which was also placed in a 3-ml UTM. Once at the local laboratory, the samples were frozen and stored at − 70 °C (if a deep-freezer was not available on site at the local laboratory, storage at − 20 °C was deemed acceptable) and then transported to a central laboratory in Antwerp, Belgium. Samples went through a freeze/thaw cycle prior to aliquoting at the central laboratory in Antwerp.
A selection of the samples collected during the three influenza seasons (2015–2016, 2016–2017, 2017–2018) of the ALIC4E trial, frozen and stored at − 70 °C, were used, to ensure a sufficient large number of positive samples of the various viral pathogens. We enriched the sample for flu positivity.
The fresh samples collected in the third influenza season in Belgium were sent to the central laboratory for analysis both before and after a freeze-thaw cycle.
All laboratory analyses were done in the central laboratory at the University of Antwerp by trained lab technicians.
Prior to testing patient samples, the 2016 Quality Control for Molecular Diagnostics (QCMD) influenza panel (proficiency testing) was tested on the cobas® Liat® POCT, containing 10 samples with dilutions of different subtypes of influenza A and influenza B, to assess the performance of the POC device.
Anzeige
Samples with discordant results were analysed additionally by the RespiFinder 2Smart (PathoFinder).
The participant’s date of recruitment, gender and participant trial ID number were used as identifiers for these samples. Only the laboratory and trial team had access to this information for the purposes of sample identification and tracking.
Acceptability and failure mode analysis
To assess acceptability and certain aspects of failure mode analysis, [19] the device operators were asked to complete a survey (see electronic supplementary material S1).
Acceptability was scored on a five-point Likert scale (1 = completely disagree to 5 = completely agree), including questions regarding device start-up, cartridge handling, operating the device, duration of the test, the provided manual and presentation of error codes. The risk of misinterpretation of failure modes was scored as: low risk–medium risk–high risk–difficult to assess, considering the following errors: expired cartridge, insufficient sample volume, incorrect cartridge insertion, unexpected test result and mismatch between the error code and actual error. The detection of the failure modes was scored as: always perceivable–probably perceivable–not perceivable for the same set of potential errors.
Anzeige
Operators were asked to list the strengths and weaknesses of the device.
Sample size calculation and statistical analysis
We aimed to analyse a selection of swab results obtained using both platforms from a minimum of 730 participants recruited during three consecutive winters (samples selected from all stored frozen samples from season 1 and 2 and all new samples from season 3 until required sample size achieved), giving 90% power at 95% confidence to detect a maximum difference in results obtained from the two methods of 10% at a presumed sensitivity of 95%.
An expanded gold standard (EGS) approach was used to calculate sensitivities and specificities (positive by at least two tests). Results of this analysis were considered the reference standard. Sensitivity, specificity and positive and negative predictive value were determined. Wilson 95% confidence intervals were calculated with the Binom Package. All statistical analyses were performed using R 3.4.3, (https://www.r-project.org/).
Results
Quality control for molecular diagnostics panels
The 2016 influenza QCMD panel, containing 10 samples with dilutions of different subtypes of influenza A and B demonstrated excellent performance of the POC device (see electronic supplementary material S2.a).
Anzeige
Description of population
Nasal and oropharyngeal swabs were obtained from 140 children and nasopharyngeal swabs from 604 adults (744 patients); median age was 33 years (range 1 to 88 years); 55.6% of patients were women, 91.8% had not been vaccinated against flu in the preceding 6 months and 93.3% had not received pneumococcal vaccination in the preceding 5 years (Table 1).
Table 1
Baseline characteristics of patients providing samples for this analysis.
N = 744 (100%)
Age
Median (range), years
33 (1–88)
12 months to < 5 years
37 (5.0%)
5 years to < 18 years
103 (13.8%)
18 years to < 65 years
560 (75.3%)
+ 65 years
44 (5.9%)
Country of origin
Belgium
208 (28%)
Poland
117 (15.7%)
Spain
1 (0.1%)
UK
99 (13.3%)
Czech Republic
54 (7.3%)
Denmark
17 (2.3%)
France
13 (1.7%)
Greece
21 (2.8%)
Hungary
58 (7.8%)
Ireland
9 (1.2%)
Lithuania
72 (9.7%)
The Netherlands
24 (3.2%)
Norway
26 (3.5%)
Sweden
25 (3.4%)
Sex
Male
330 (44.4%)
Female
414 (55.6%)
Flu season
1 (2015–2016)
423 (56.9%)
2 (2016–2017)
255 (34.3%)
3 (2017–2018)
66 (8.9%)
Ethnicity
White
553 (74.3%)
Black
3 (0.4%)
Hispanic
4 (0.5%)
Asian
6 (0.8%)
Arabic
6 (0.8%)
Other
23 (3.1%)
Unknown
149 (20.0%)
Flu vaccination in the last 6 months
Yes
58 (7.8%)
No
683 (91.8%)
Unknown
3 (0.4%)
Pneumococcal vaccination in the last 5 years
Yes
37 (5.0%)
No
694 (93.3%)
Unknown
13 (1.7%)
Smoking status
Yes
121 (16.3%)
No
591 (79.4%)
Occasionally
32 (4.3%)
Sample type
Nasopharyngeal swab UTM
619 (83.2%)
Oropharyngeal + nasal swab
125 (16.8%)
POC-PCR results
Negative rapid test
227 (30.5%)
Positive rapid test for:
517 (69.5%)
- influenza A
318 (42.7%)
- influenza B
165 (22.2%)
- RSV
30 (4.0%)
Analytical performance
The analytical performance of the cobas® Liat® (Roche Diagnostics) POCT for diagnosing influenza A/B and RSV, and a laboratory-based platform, the Fast-Track Respiratory Pathogens 21 Plus kit (FTD, Fast-Track Diagnostics), was evaluated on 744 samples. Eleven samples with discordant results were analysed additionally by the RespiFinder 2Smart (PathoFinder). We encountered 8 run errors in total on the cobas® Liat®, resulting in 736 successfully run POCT tests.
Time to result for cobas® Liat® was within 22 min (2 min hands on time and 20 min analysis time).
The cobas® Liat® POCT had a sensitivity and specificity of 100% (95% CI 99–100%) and 98.1% (95%CI 96.3–99%) for influenza A, 100% (95% CI 97.7–100%) and 99.7% (95%CI 98.7–99.9%) for influenza B and 100% (95% CI 87.1–100%) and 99.4% (95%CI 98.6–99.8%) for RSV, respectively, with the EGS as reference standard (Table 2).
Table 2
Analytical performance of cobas® Liat® POCT on frozen samples using the Expanded Gold Standard (EGS) as reference
The results for the fresh samples (n = 19) are shown in the electronic supplementary material S2.b, with similar findings and no samples with discordant results.
Acceptability and failure mode analysis
Five independent trained device operators (KJ, KL, KVDV, MK, VM) assessed the user-friendliness of the cobas® Liat® POCT device with a median acceptability of four out of five on the five-point Likert scale. Potential concerns identified were the touch screen readability and proficiency of the included manual (median scores of 3 out of 5) and interpretation of error codes and how to respond to them (median scores of 2 out of 5). (Fig. 1) The risk of misinterpretation of failure modes was generally considered low, apart from the risk of misinterpreting collection of an insufficient sample volume (deemed as high risk). Although most items regarding detection of failure modes were scored as ‘always perceivable’, agreement of an error code to the actual error was scored as ‘not perceivable’ by all raters.
Fig. 1
Acceptability of the POCT device
×
The device’s size, analysis speed, ease of use and limited sample preparation were considered important strengths. The lack of a sample holder during pipetting, the size of the touch screen and difficulties reading the cycle threshold (Ct) values were generally considered potential weaknesses.
Discussion
Principal findings
The cobas® Liat® POCT demonstrated excellent analytical performance to detect influenza A, influenza B and RSV, and was considered easy to use, with a fast turn-around-time.
Comparison with previous studies
The high sensitivity and specificity for influenza A, influenza B and RSV found in our study was comparable to previous findings, with better diagnostic accuracy than other routinely available point-of-care devices. [14‐18, 20‐30] The diagnostic accuracy of POCTs for influenza has been examined in individual studies and several systematic reviews. [31‐33] Although the impact of the cobas® Liat® POCT on clinical decision making during visits to the emergency department has been established, [34] the effect on patient-relevant clinical outcomes still has to be determined.
Strengths and limitations
We were able to collect paired samples in a real-life clinical setting of patients presenting to primary care with ILI in this adequately powered study, from 14 different countries across three influenza seasons. Samples were collected, transported and analysed in a standardised and rigorous way and compared to routinely available standard central laboratory multiplex tests. Sampling method varied by age which needs to be taken into account when considering the applicability of our results. Nasopharyngeal swabs were taken from adults and nasal and pharyngeal swabs were taken from children. Our results may therefore not be replicated fully through self-swabbing or nasal swabbing. While nasopharyngeal swabbing may provide the most accurate results, the choice of sample type for children was in accordance with routine care constraints, where more invasive methods are considered less appropriate.
Combining the samples from three different influenza seasons provided us with a representative range of influenza A and B samples, with very few cases of samples testing positive for RSV, which reflects clinical reality.
The vast majority of our samples were stored and frozen samples. However, fresh samples collected during season 3 indicated similar patterns, further strengthening our findings.
The cobas® Liat® POCT in our trial, was operated and maintained by trained lab technicians, familiar with standard operating procedures of POCT devices. This limits the generalisability of our findings to routine primary care both in terms of clinical performance and usability, even when the cobas® Liat® POCT would be performed by trained general practice staff. Although similar results were found in fresh samples, using frozen samples does not necessarily reflect routine primary care where samples would normally be analysed at the point of care, immediately following sampling.
Implications for research
Studies of POCTs for influenza are needed, both in adults and children, with a particular evidence gap for primary care settings. Now that analytical performance has been established in primary care, trials of the impact on clinical course with cost-effectiveness analysis are required to establish the impact on patient-related outcomes, such as antibiotic prescribing rate, antiviral treatment and use of additional tests (chest radiography, blood cultures and routine blood tests). Combining influenza tests with other POCTs, such as C-reactive protein, may help rule out bacterial co-infection to reduce antibiotic prescribing rate in primary care settings.
Implications for clinical practice
We have shown that the cobas® Liat® POCT is accurate, providing timely results in the hands of trained operators. However, the evidence to implement the test in primary care is still scarce. Considering the patient pathway is primordial to ensure adoption of a novel POCT. Time to result should ideally be less than 15 min to accommodate the already pressured consultation length in primary care. Furthermore, these tests should not substitute clinical appraisal. Clinicians should consider local initiatives before widespread implementation to evaluate costs, appropriateness and impact on further testing. The use of antiviral treatment and the impact of POCT to decide whether or not to prescribe still needs to be established and will guide future research on the role of POCT for influenza in primary care.
Conclusions
Cobas® Liat® POCT is a promising decentralised test platform for detection of influenza A/B and RSV in primary care settings, provide fairly rapid results with excellent analytic performance.
Acknowledgments
This article presents independent research part funded by the NIHR former Diagnostic Evidence Co-operative (DEC) Oxford and ongoing Community Healthcare MIC. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. JV had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. We would like to thank Kevin Jacobs and Kailey Van De Velde for their help in analysing the samples and completing the user-friendliness questionnaire.
Compliance with ethical standards
Conflict of interest
Dr. Butler reports receiving advisory board fees from Roche Molecular Systems and Dr. Butler and Dr. Verbakel report receiving grant support from Roche Molecular Diagnostics. No other potential conflict of interest relevant to this article was reported.
Ethical approval
All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional research committee (NRES Committee South Central (Oxford B) approval and Clinical Trial Authority (CTA) approval by The Medicines and Healthcare products Regulatory Agency was obtained and all participating countries gained national Research Ethics Committee and CTA approval as required) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
We included a selection of swabs taken from patients presenting to primary care with symptoms of ILI, willing to participate in the ALIC4E trial, who met the inclusion criteria and gave written informed consent. [13]
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Mit e.Med Innere Medizin erhalten Sie Zugang zu CME-Fortbildungen des Fachgebietes Innere Medizin, den Premium-Inhalten der internistischen Fachzeitschriften, inklusive einer gedruckten internistischen Zeitschrift Ihrer Wahl.
