An evaluation index system for regional mobile SARS-CoV-2 virus nucleic acid testing capacity in China: a modified Delphi consensus study

Coronavirus disease 2019 (COVID-19) [1] is an infectious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [2]. Since the disease was first reported, the virus has rapidly spread globally [3], leading to the coronavirus pandemic [4] and a severe global recession [5]. The global spread of the virus has been drastic; in particular, more than 243 million cases have been confirmed in more than 200 countries, including 4.9 million deaths, as of 26 October 2021 [6]. The infection can be spread by asymptomatic, presymptomatic, and symptomatic infectors [7]. The Chinese government implemented unprecedented nonpharmaceutical public health measures in the early stages of the COVID-19 outbreak to control the local spread of COVID-19 and stabilise the epidemic. The SARS-Cov-2 virus has undergone several mutations, whilst the epidemic has been expanding worldwide, resulting in virulence alterations that impact illness severity around the world [8]. The introduction of vaccines against COVID-19 globally and in China has undoubtedly improved epidemiological situations. Nevertheless, extensive data suggests that the immune protection conferred by vaccines declines over time, allowing for the emergence of new diseases [9, 10].

Given the above background, the possibility that an epidemic will occur in vaccinated populations remains high because of the emergence of new SARS-CoV-2 variants and postvaccination infection [11]. Imported SARS-CoV-2 infectors and/or contaminated commodities from abroad would bring the potential risk of the local COVID-19 epidemic in China. Analysing respiratory discharges using real-time quantitative polymerase chain reaction is the most reliable method for virus detection [12]. The goal of ‘early discovery, early reporting, early isolation, and early treatment’ necessitates rapid and precise testing to respond effectively to the epidemic. Early data from the China epidemic supported the idea that effective and prompt testing would reduce the time between the start of symptoms and diagnosis, thereby lowering the number of severe and critical cases [13]. Besides, data from Brazil also emphasized that identifying interval time could favour efficiently carrying out prevention actions to contain the COVID-19 pandemic [14]. In summary, the case for mass, community-wide polymerase chain reaction (PCR) testing for COVID-19 of the right individuals at the right time remains strong.

Since the containment strategy against COVID-19 has been adopted in China, the measure of large-scale PCR-based testing of SARS-CoV-2 in throat swab samples is critical for sustaining containment in mainland China [15], particularly for controlling numerous local outbreaks caused by imported viruses. For example, between 11 June and 14 July, 2020, 11.9 million persons were tested at Beijing’s Xinfadi market during the outbreak [16], contributing to an optimal balance between epidemic containment and economic protection in Beijing [17]. In another two cases of COVID-19 epidemic control, 4.5 million and 10.9 million people received nucleic acid testing for SARS-CoV-2 virus within 5 days in Dalian [18] and Qingdao [19], respectively. Guangzhou performed a large-scale nucleic acid testing amongst 18.7 million people within 3 days to control the spread of COVID-19 [20]. Between 8 and 21 August 2020, a comprehensive community testing approach was performed in Vietnam, together with an innovative sample pooling mechanism, contributing to the ultimate success of COVID-19 control in Da Nang City [21]. New Zealand’s efficient mass testing contributed to the government’s success in curbing the spread of the coronavirus [22]. The significant number of asymptomatic individuals and evidence of substantial presymptomatic transmission highlighted the efficacy of mass testing in controlling the illness [22]. Timely and effective large-scale detection has great potential to bring benefits for containing the epidemic and supporting the government in reopening economic activities [23, 24].

Comprehensive, active and innovative PCR testing strategies need heightened requirements for nucleic acid detection capability to respond effectively to the epidemic that may be caused by new SARS-Cov-2 virus variants. On 2 September 2020, the State Council’s interagency task force released the Plan on Advancing Coronavirus Nucleic Acid Testing Capacity Building (the Plan) [25]. According to the Plan, regional mobile nucleic acid testing capacity is needed to ensure a highly responsive mobilisation mechanism; thus, all individuals residing in corresponding locations can receive nucleic acid testing within a short period in cases when local COVID-19 patients (either symptomatic or asymptomatic ones) are discovered in routine screening measures under the government’s regular epidemic prevention strategies. Finding the weak links and loopholes is critical in building nucleic acid testing capacity against COVID-19. However, no comprehensive and practical evaluation tools are available for relevant departments to find and strengthen the weak links of testing capacity in different regions.

Previous studies reported several challenges encountered in a massive nucleic acid testing [26‐28]. Firstly, the time-consuming detection period and expensive testing reagents are the main limitations of a massive testing. In particular, the high cost of the detection reagent needed in a massive testing causes stress to a region’s economic budget [26]. The equipment for nucleic acid testing is in short because of the concurrent increase in global requirements. Secondly, a vast workforce is needed to maintain the program. The effectiveness and scope of this program to contain the outbreak also depend on a comprehensive national strategy [27]. In addition, the close contact between medical staff and COVID-19 infectors in throat swab collection increases the risk of virus infection for the staff [28]. Compared with first-tier cities, such as Beijing and Guangzhou, third- and fourth-tier cities have a relatively poor response and reaction ability in handling public health emergencies. Cross-infection of SARS-CoV-2 occurred in the several rounds of massive testing against the COVID-19 outbreak in Yangzhou, China, because of the lack of experience, poor planning and management [29]. Given the evidence mentioned above, the primary goal of this study was to provide a scientific and comprehensive evaluation index system of regional mobile nucleic acid testing capacity. This index system will help optimise relevant issues in the whole process of massive testing, ensure the safety of working staff and subjects, and avoid cross-infection. Additionally, our study provides evidence for large-scale nucleic acid detection management in other countries.

The testing capability evaluation index system was built using a modified Delphi approach in this study, including 5 first-level indexes, i.e. ‘Personnel team construction’, ‘Laboratory performance building and maintenance’, ‘Emergency response guarantee’, ‘Information management system for nucleic acid testing resources’ and ‘Regional mobile nucleic acid testing emergency response system construction’, 17 second-level indexes and 90 third-level indexes. The AHP approach was used to calculate the weight coefficient for the three-level evaluation indexes. Moreover, the weight coefficient was reliable with consistency ratios of all less than 0.10. As policymakers worldwide seek to improve COVID-19 prevention, detection and response amid a flare-up of cases driven by the highly contagious Delta variant, the introduction of new nucleic acid testing programmers appears likely to continue. However, comprehensive and practical evaluation tools have not been available to monitor and evaluate nucleic acid testing capacity within and across jurisdictions. For the first time, the outcomes of this research provided robust references to benchmark mass testing capacity in China, which may then be refined for comparison with the outcomes from other countries developing and offering mass testing.

The evaluation index system for assessing regional mobile nucleic acid testing capacity was scientific, comprehensive and diversified with the following characteristics. Firstly, the theoretical basis was efficient and reliable. The initial draft of the evaluation index system was constructed based on the technical specifications and laws and regulations promulgated by the state and literature reviews, with reference to the Plan [25], the Protocol [34] and the Guidelines. In this study, relevant panelists were selected based on specific features (e.g. age, profession title and working year). These panellists were influential and active in COVID-19 epidemic prevention and control. Our team implemented revisions provided by specialists from many departments and perspectives, thereby ensuring that the system was suited for usage in various places with varying epidemic risk levels. Secondly, the authority coefficient (0.71) and questionnaire response rate (100 and 96.7%) were within the acceptable limits (more than or equal to 0.70) [35, 39]. Thirdly, this evaluation index system was systematic and comprehensive because we considered not only the whole process of nucleic acid detection but also the nucleic acid testing preparation, nucleic acid detection implementation and testing result presentation. Moreover, timely financial and material support and the construction of the emergency response system were considered. Fourthly, the diversified subject could use the evaluation index system to adjust and perfect the regional mobile novel coronavirus nucleic acid test activities. Each provincial or city-level government can conduct a self-assessment according to its situation. Besides, the government can evaluate the capacity of different areas according to the index. Diversified evaluation might assure fairness and impartiality in the overall nucleic acid testing capacity assessment.

The evaluation system followed the principles of integrity, emphasis, hierarchy, comparability and operability to achieve the purpose of ‘early detection, early reporting, early isolation, and early treatment’ of COVID-19 patients [40], and reduced the risk of large-scale spread of epidemics, by making arrangements in five aspects: regional mobile nucleic acid testing emergency response system construction, personnel team construction, emergency response guarantee, laboratory performance building and maintenance, information management system for nucleic acid testing resources. ‘Personnel team construction’ and ‘laboratory performance building and maintenance’ had the highest weights, indicating that they were important in strengthening the nucleic acid sampling and testing capacity in the personnel and laboratory techniques. Cross-contamination and other diagnostic mistakes, including those caused by an increase in specimens, a shortage of laboratory personnel, and a lack of quality control, may occur in a laboratory setting [41]. Stratigraphically speaking, our evaluation index system was efficient and reliable to some extent. COVID-19 testing on a large scale necessitates many people, many of whom may be untrained and working in a stressful environment. The regional mobile nucleic acid testing task necessitates a dynamic deployment of staff on duty in each part and a standby shift. For the emergency circumstance of COVID-19 mass testing, an emergency human resource allocation plan and a reserve echelon rotation mechanism must be developed, and proper rotation and collocation must be carried out [42]. As a result, the study group argued that long-term and standard human resource management procedures should be devised to ensure the smooth progression of the mass testing activity.

As for testing capacity, ‘Laboratory performance building and maintenance’ integrated the specimen reception, testing and biosafety management into a network that can respond effectively and efficiently to emergencies. Sufficient qualified personnel and efficient laboratory performance support provinces, cities and districts to provide increased sample analyses in epidemic outbreaks or other large-scale emergency events requiring surge capacity testing of samples and products. As reported, two air-inflated COVID-19 test laboratories, namely Huo-Yan Laboratory [43] and Falcon laboratory [44], were built in 1 day to meet the surge of testing demands in Guangzhou. Saudi Arabia, Brunei, and Kazakhstan are amongst the countries and territories that have started or built Huo-Yan labs to combat the COVID-19 virus [43]. New sampling and testing approaches have been described in recent studies [45‐47], indicating that further training with drills and assessments is required to assure standard and uniform sample collection and testing by all employees.

The weight of ‘Emergency response guarantee’ in the first-level indexes was the third largest. The guarantee of emergency nucleic acid testing supplies was essential for medical personnel to implement emergency testing work and safety protection [48]. Therefore, the procurement and use of emergency supplies and other management work were directly related to the speed of response to the outbreak and the final effect of emergency testing work [49]. Perfecting the procurement, reserve and use management system of emergency nucleic acid testing supplies is the key to improving emergency management and handling capacity. The weight of ‘Information management system for nucleic acid testing resources’ in the first-level indexes followed ‘Emergency response guarantee’. Establishing an information management system for nucleic acid testing resources could comprehensively, accurately and dynamically grasp the information of nucleic acid testing institutions, sampling and testing personnel, thereby achieving efficient and accurate management of their deployment. The first-level index ‘Nucleic acid mobile detection emergency system construction’ had the lowest weight; however, this finding did not mean it was unimportant. Studies have found that mass gatherings organized in places with high population density can easily lead to widespread and cluster outbreaks of epidemic diseases, such as the cluster epidemic of Buddhist gatherings in Ningbo [50] and the outbreak epidemic in the health training centre of Jilin Province [51]. Once an outbreak occurs, the difficulty of epidemic prevention and control increases if all cases cannot be identified quickly. Therefore, when COVID-19 patients were reported, the nucleic acid mobile detection emergency system should be activated as soon as possible to prevent the epidemic from spreading.

For the second-level indexes, we adopted expert opinions to merge the ‘Daily testing personnel’, ‘Reserve testing personnel’, and ‘Mobile response testing personnel’ to ‘testing personnel’. We also added ‘Sampling personnel’ because proper specimen collection is the most important step in the laboratory diagnosis of infectious diseases. A specimen not collected correctly may lead to false or inconclusive test results. The novel coronavirus testing specimens shall be collected by qualified technicians who have received biosafety training (who have passed the training) and are equipped with the corresponding laboratory skills.

The time spent in a large-scale nucleic acid program is an important factor for taking further interventions to control infection spread. We considered this issue along two lines (both authorities and residents). For authorities, the main aim was to find the virus infectors and block transmission at the fastest speed. Thus, several periods (e.g. the detection time of nucleic acid) are critical for authorities to take decisive action, which is needed as soon as possible. For residents, the major concern is the time from throat swab collection to the release of the test result. Thus, we defined indexes for the ‘Testing results presentation and feedback’ (Table 4). The results’ reporting time for residents/patients was defined as the time from throat swab collection to the release of the test result. It can represent the total turnaround time (time from throat swab collection or arrival of the sample at the laboratory until the result is communicated to the patient/person or authorities). Consequently, the total turnaround time and phased time can be used to assess the regional nucleic acid testing capacity.

Some experts believed that the reward for emergency work should be quantified according to the workload of different areas, and the epidemic’s severity should be considered. The study group argued that the flexible index should be set with the goal of protecting the health of all employees in mind, and that it should be updated dynamically as the epidemic progresses.

In the setting of the global COVID-19 pandemic, our findings supported the application of evaluation index system in China. Moreover, generalising the evaluation index system to other countries was prudent. China is a special country in terms of health care because of its large population size and unbalanced health care development in different areas [52]. The Chinese government has been spending a lot of manpower and resources to overcome this public health issue. The large-scale nucleic acid testing strategies support China’s sustained containment of COVID-19, regardless of backward and developed regions in China. Consequently, according to China’s experience, this evaluation index system may be prudently extrapolated to many of other low- and middle-income countries, each one with its particularities.

Several strengths characterized our study. Firstly, this study was the first to construct the evaluation index system of regional mobile nucleic acid testing capacity. Thus, this study provided a reference for nucleic acid mobile detection capability evaluation. It also has guiding relevance for ensuring nucleic acid detection capability. Secondly, Delphi method allows the development of defensible, valid, and reasonable solutions based on expert opinion [31]. The Delphi method is regarded as a structured approach for evaluating and combining human judgment. Rowe et al stated that the Delphi method can be used when the researcher is convinced that the technique can generate more accurate assessments and judgments than that provided by individuals [53]. Our objective was to develop a comprehensive tool for evaluating the regional mobile SARS-CoV-2 virus nucleic acid testing capacity. Thus, we resorted to the Delphi technique. Considering the issues of seniority, interfering or inhibiting personality traits that easily occur in a face-to-face meeting, the Delphi method is reliable in obtaining an objective and unbiased point of view. The experts from regions with strict epidemic control requirements in different provinces and cities were selected to avoid bias in the judgment of approaches, practices, and backgrounds. Thirdly, we ensured anonymity between participants, avoiding the interplay effects. Fourthly, the items’ importance, operability and sensitivity were comprehensively considered in this study, indicating that the representativeness of weights was multidimensional and practically meaningful. However, this study also had limitations. Firstly, we realised that the decision-making process inevitably involved subjectivity and judgmental inputs in terms of panel selection, item selection and dispute resolution. Secondly, the initial outbreak of COVID-19 was in Wuhan; hence, a limitation may exist because approximately one-third of the experts were from Hubei. Furthermore, more than a third of the specialists invited were from research institutes. Their perspectives could not be more comprehensive than frontline healthcare staff.

We established an evaluation index system for regional mobile nucleic acid testing capacity based on a scientifically designed Delphi process. Several important points obtained from this study. Firstly, the evaluation index system proposed specific, objective, and quantitative evaluation criteria that aid the government in containing the pandemic and resuming economic operations. In particular, the evaluation index system may help relevant departments to find and strengthen weak links of testing capacity in different regions. Secondly, the index weights informed the functional departments about the priority in managing a massive nucleic acid test. This evaluation index system may be generalised to other countries with prudence considering the shared, different, and complicated conditions in different countries. International consideration and feedback about the suitability of the evaluation index system are encouraged to develop an international consensus for virus nucleic acid testing against COVID-19.