Comparison of the analytical sensitivity of three real time PCR kits used in the detection of the SARS-CoV-2

Background: COVID-19 infection is diagnosed by a real-time Reverse transcriptase Polymerase chain reaction (RT-PCR), which can detect the presence of the SARS-CoV-2 virus as early as day one of symptoms. However, the performance of these kits can vary widely. We assessed the performance of three different PCR kits from different manufacturers used for diagnosis of COVID-19 in Sri Lanka. Methods: The performance of two PCR kits were compared to the Da An gene PCR kit in a total of 68 samples. The known positive samples were from the patients admitted to the National Hospital of Infectious Diseases. The performance was evaluated in samples with a ct value >35 (n=30), samples which were positive with ct values of the samples between 30 to 35 (n=10), those that gave an inconclusive result (n=8) and those that were negative (n=20). Results: The three PCR kits had highly varied sensitivity for the positive with high ct values (>35) and lower ct values (ct values 30-35). For instance, the 3rd kit (RealStar® SARS-CoV-2 RT-PCR Kit 1.0), only detected 6/10 (60%) of samples with ct values between 30 to 35. The GeneFinder™ and the RealStar® SARS-CoV-2 RT-PCR Kit 1.0, both showed low sensitivity for positive samples with high ct values (>35, suggesting low viral loads), with the GeneFinder™ giving positives for only 1/30 (3.3%) samples and the RealStar® SARS-CoV-2 RT-PCR Kit 1.0 giving positives for only 8/20 (26.7%) of samples. Conclusions: PCR kits vary in sensitivity and it is crucial that PCR kits are thoroughly evaluated prior to using them and kits with higher sensitivity should be used in the compatible machine for optimum results.


Introduction
Infection due to SARS-CoV-2 originated in Wuhan, the capital city of the Hubei province in early December 2019 1 . It caused a massive outbreak in the Hubei province within a very short period, with the number of cases rising to 72,314 by mid-February 2 . Since then it has spread to the rest of the world and is now found in 212 countries, reporting a total of 4,254,778 cases and 287,293 deaths as of 12 th May 2020 3 . Due to its spread in all continents the WHO declared it as a pandemic on the 11 th of March 2020 4 . Many countries have taken extreme measures such as lockdown and imposing curfew to curb the spread of the pandemic, causing significant impacts to the economies, education and lives of individuals. All countries rely on early detection of infected individuals to isolate and mitigate transmission.
Currently COVID-19 infection is diagnosed by a real-time Reverse transcriptase Polymerase chain reaction (RT-PCR), which can detect the presence of the SARS-CoV-2 virus as early as day one of Conclusions: PCR kits vary in sensitivity and it is crucial that PCR kits are thoroughly evaluated prior to using them and kits with higher sensitivity should be used in the compatible machine for optimum results. symptoms 5 . Most PCR protocols use several primers targeting one gene (CDC protocol) or a different gene targets such as the envelop, nucleocapsid, spike, ORF1 and RNA-dependent RNA polymerase (RdRp) 5, 6 . All the above primer sets were shown to be equally sensitive, except the primers targeting the RdRp gene7. In addition, the primers targeting the 2019-nCoV_N1 primer-probe set was found to be more sensitive in the detection of lower amount of virus than the 2019-nCoV_N2 primer-probe set 7 . The duration of illness at the time of collecting of the sample, the type of sample collected and the technique of collection all influence the likelihood of detection of the virus 5 . While some studies have claimed that the sensitivity of the realtime RT-PCR is around 45% to 60% 8 , other studies have shown that the sensitivity was around 63% to 72% depending on the sample 9 . Repeated testing of consecutive samples from the same patient, increased the positive rates from 70% from the first sample to an additional 24% when testing a second sample 10 . Therefore, the sensitivity of the real-time RT-PCR test assay appears to be affected by multiple factors including the type of primer-probe set used and the frequency of repeat samples. Carrying out large number of tests for several months poses a huge burden to resource poor developing economies and therefore, repeat testing of samples is sometimes challenging. However, in order to curb the spread of the virus, early detection and isolation of individuals is crucial.
There are several commercially available real-time RT-PCR kits, which are widely used by different countries. In order to improve diagnostic sensitivity, it is essential that commercially available kits with the highest sensitivity are used. In this study, we evaluated the performance of three commercially available PCR kits for the detection of SARS-CoV-2 that are widely used in Sri Lanka. We evaluated their performance in identifying COVID-19 confirmed patients and in those who were found to be negative or who gave an inconclusive result.

Methods
In this study the performance of Da An gene PCR kit (Detection kit for 2019 Novel Coronavirus (2019-nCoV) RNA (PCR-Fluorescence probing), DAAN Gene Co., Ltd. Of Sun Yat-sen University, China), the RealStar ® SARS-CoV-2 RT-PCR Kit 1.0 (Altona Diagnostics GmbH, Germany) and the GeneFinder TM COVID-19 Plus RealAmP kit (OSANG Healthcare Co., Ltd, Korea) were evaluated. The performance of the Da An gene kit was used for comparison with the other two kits because it has been approved by the regulatory authority of the country of manufacture for in vitro diagnostic use and this has been specified by the WHO. In addition, during the initial evaluation of the Da An gene kit along with several other kits that were evaluated in our laboratory (n=7), we found that this kit gave the best results even at high ct values (reflecting low viral loads). As the Da An gene has been recommended to be used in the ABI Biosystems 7500, we used this PCR kits performance on this specified machine as the reference standard.
The primer-probe sets included in each kit, the fluorescence of the dyes and the detection limit is mentioned in table 1. The sensitivity of these three kits were compared as they are widely available in Sri Lanka and used by many laboratories. Although the Da An gene PCR kit does not specify CFX96 ™ Real-Time PCR Detection system (Bio-Rad), as a compatible machine, as this kit is widely used in this machine in Sri Lanka, we determined the performance of this kit on this machine compared to performance of this kit using the ABI Prism ® 7500 SDS (Applied Biosystems).

Patient samples
We evaluated the performance of these two PCR kits in comparison to the Da An gene PCR kit in a total of 68 samples. The comparison was carried out for the PCRs run on the Da An gene PCR kit, as recommended by the manufacturer. These included samples that were positive with a ct value >35 (n=30), samples which were positive with ct values of the samples between 30 to 35 (n=10), those that gave an inconclusive result (n=8) and those that were negative (n=20). A ct value of <40 was considered as a positive result as recommended 5 . These samples were obtained from COVID-19 patients who were admitted to the National Institute of Infectious Diseases. These samples consisted of RNA extracted from sputum or a nasopharyngeal swab using the Qiagen kit and the RNA was stored at -80C, until the assays were carried out. The comparison of the Da An gene PCR kit on the ABI Biosystems 7500 PCR machine (recommended usage) compared to the performance of this kit on the BioRad CFX96 PCR machine (not listed under the compatible PCR machines) was also compared. This comparison was carried out as the majority of the PCR machines in many laboratories in Sri Lanka is the BioRad CFX96.

Results
The comparison between the three different kits is listed in

Discussion
There have been many concerns regarding the sensitivity of the SARS-CoV-2 PCRs and the likelihood of the false negative rates. It was recently shown that the false-negative rates at the time of onset of symptoms was 38%, which decreased to approximately 20% by the end of the first week since onset of symptoms and increased to 66% by day 21 of illness 11 . However, apart from the sensitivity been affected by sample type, sampling techniques, day since onset of illness we have shown how the sensitivity is affected by the type of PCR kits been used. There are many PCR kits marketed by many companies, which have varied performance.
The performance of some PCR kits was less than satisfactory compared to the one used for comparison (Da An gene PCR kit). In addition, it is evident that when certain PCR kits are used on PCR machines which are not listed as compatible ones, there could  be issues with interpretation of results. For instance, the same PCR kit when run on incompatible PCR machine gave suboptimal results (86.7% positivity). Therefore, it is crucial that PCR kits are thoroughly evaluated prior to using them, especially when the sensitivity of the PCR kit is not declared by the manufacturer. In addition, the use of PCR kits on the compatible machine for optimum results is crucial.