Comparison of Rapid Antigen Test with RT-PCR for COVID-19 Diagnosis: Performance and Limitation

Introduction

Tiny yet formidable microbes have unleashed calamities upon humanity, from Ebola to malaria and dengue. Among them now reigns SARS-CoV-2, the coronavirus. Its global toll is 757 million infections and 7 million lives lost as of 28 February 2023 [1]. Emerging from Wuhan's seafood market on 31 December 2019, this relentless adversary swiftly increased, prompting the World Health Organization to declare it a pandemic on 11 May 2020 [2]. In Pakistan, the dawn of 26 February 2022 brought news of two significant cases-harbingers of a widespread outbreak. These cases, originating in Karachi and Islamabad, marked the genesis of SARS-CoV-2's impact on the Pakistani population [3]. However, a new menace emerged: B.1.1.529 (Omicron). First witnessed in South Africa on 24 November 2022, Omicron's 55 mutations, including 30 in the spike protein, posed alarming concerns of heightened transmission and detection evasion [4, 5]. Therefore, early and accurate detection of COVID-19 is crucial to curb transmission. Real-time reverse transcription polymerase chain reaction (RT-PCR) is the gold standard, but it is expensive and time-consuming. Nucleic acid amplification-based tests (NAATs) and serological assays are also available for diagnosis [6-8]. Advances in molecular diagnostics have reduced assay time to less than 10 minutes, with some assays detecting viral RNA in 45 minutes [6, 7]. The single-tube in vitro extraction-free amplification assay provides accurate viral ribonucleic acid (RNA) detection in 36 minutes [9]. COVID-19 rapid antigen diagnostic kits (RAKs) are used for quick diagnosis of suspected SARS-CoV-2 patients, but their accuracy needs evaluation [10]. RAK performance varies based on sample type, sampling time, and the specific kit. Some studies suggest RAKs are inferior to RT-PCR, while others highlight their quick detection and cost-effectiveness without specialized instruments [11, 12]. Certain studies report high sensitivity and specificity of RAKs, particularly in emergencies or when handling multiple samples [13, 14]. The current evaluation focuses on a specific RAK kit chosen for its relevance in resource-constrained settings and its ability to improve patient convenience. Utilizing saliva as a specimen eliminates the need for uncomfortable nasal or oropharyngeal swabs, enhancing the accessibility and acceptability of COVID-19 testing. This feature is particularly valuable in settings with limited access to sophisticated laboratory facilities. The chosen RAK kit offers an efficient and accessible testing solution, improving the overall testing experience for individuals [15]. RAKs' sensitivity is influenced by the cycle threshold (CT) value and sample type. A high level of sensitivity is reported when the CT value is low and vice versa [16, 17]. In Pakistan, two studies compared RAKs with RT-PCR using nasopharyngeal swabs (NSP) and saliva, revealing higher sensitivity with saliva-based RAKs [18]. The World Health Organization (WHO) recommends the Panbio™ rapid antigen kit by Abbott Laboratories for COVID-19 detection and point-of-care testing. However, conflicting results arise from previous studies, as lower sensitivity values were reported by Krüger et al. and Bulilete et al. These findings highlighted the importance of evaluating RAK performance across different CT values and sample types to ensure accurate and reliable SARS-CoV-2 detection [19, 20].

This study utilized logistic regression to calculate the odds ratio and examine the relationship between Rapid Antigen Kits (RAKs) and PCR testing for COVID-19 diagnosis. Accurate data on RAK performance and diagnostic utility is urgently needed.

The study aims to compare the sensitivity and specificity of commercially available COVID-19 RAKs with real-time RT-PCR, providing a reliable, cost-effective, and precise method for SARS-CoV-2 detection.

Material and Methods

Study Type, Settings, and Population

A cross-sectional, single-blinded, one-centered study was undertaken at a private sector lab setting in Lahore, Punjab, Pakistan. Within the Lahore region, 7,580 samples were meticulously collected between May 2021 and June 2022 from individuals aged 8 to 80 years. The IDC Laboratories, endorsed by the Punjab Healthcare Commission (PHC) under the auspicious Letter IDC/L&A/2021/COVID-157, boast multiple nationwide collection centers and state-of-the-art testing facilities.

SARS-CoV-2 Real-time RT-PCR

Nasopharyngeal swabs were obtained using transport swab kits and placed in 2mL universal viral transport medium vials. RNA extraction was performed with the GF-1 Viral Nucleic Acid Extraction Kit. PCR testing utilized the Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v4 on the Anatolia Montania 4896 Real-time thermal cycler. The amplification protocol involved a 50℃ temperature for 20 minutes to activate the reverse transcriptase enzyme and a 95℃ temperature for 10 minutes to activate the Taq enzyme for one cycle each. The PCR assay consisted of 40 cycles with denaturation at 95℃ for 15 seconds and annealing and extension of the primers at 60℃ for 30 seconds. Sigmoid curves analyzed the results, with FAM, VIC, and ROX detecting target genes. The terminal fluorescence was measured at a step of 60℃ for 30 seconds. CT values below 36 were deemed positive, excluding samples with higher values.

SARS-CoV-2 Rapid Antigen Test

The PCL Spit Rapid Antigen Test Kit (Germany), a membrane-based immunochromatographic assay, was utilized to detect SARS-CoV-2 nucleocapsid protein in saliva samples. This kit was selected due to its widespread regional usage, preliminary efficacy data, rapid turnaround time, cost-effectiveness, and suitability for resource-limited settings. Saliva samples with varying ranges of CT values were tested. Samples were mixed with buffer, shaken, and three drops were added to the strip. The dark-colored test and control lines indicated positive results. Among 320 samples tested, including 224 PCR-positive and 96 negative samples, 56 negative samples had CT values above 36, confirming negativity.

Statistical Analysis

The study employed logistic regression and odds ratio calculations to evaluate the diagnostic performance and association between the Rapid Antigen Kit and PCR testing for COVID-19 diagnosis. A 2x2 contingency table was constructed from the test results to compute the odds ratio, considering true positives, false positives, true negatives, and false negatives. Logistic regression analysis incorporated the Rapid Antigen Kit as an independent variable and PCR testing as the dependent variable. Covariates like age, gender, and relevant demographics controlled for potential confounding variables. Regression coefficients, odds ratios, and statistical significance gauged the relationship between the testing methods. Specificity, sensitivity, accuracy, and positive and negative predictive values were computed using the proposed formula [21].

Results

Demographic Characteristics and Gender Distribution

A total of 7,580 samples were received at the diagnostic center during the specified period. Most samples were from males, accounting for 95.6% (n=7,249), while females constituted 4.3% (n=331) of the total samples. The distribution of patients across different age groups was as follows: <20 years (164; 2.16%), 21-40 years (5,276; 69.60%), 41-60 years (1,973; 26.02%), and 61-80 years (167; 2.20%). It is important to note that all age groups had a predominance of male samples, with more than 95% belonging to males (Table 1).

Table 1. Frequency distribution of patients

Rapid Antigen Kit (RAK) Results

The sample size of 320 was chosen to ensure statistical robustness and representativeness in evaluating the kit performance. It allows for a sufficiently large and diverse sample population, increasing the reliability and generalizability of the study findings. Additionally, a larger sample size enables a more accurate estimation of sensitivity, specificity, and other diagnostic parameters, providing a comprehensive assessment of the kit effectiveness. The samples were processed using the Rapid Antigen Kit for detecting SARS-CoV-2 nucleocapsid protein; 152 tested positive on both the Rapid Antigen Kit and PCR (true positive). In comparison, 72 samples tested negative on both tests (true negative). However, 24 samples (7.5%) were falsely positive, and 72 samples (22.5%) were falsely negative (Figure 1)

Figure 1. (a) The kit sensitivity represents the total proportion of actual positive, actual negative, false positive and false negative results of the Rapid antigen kit against the patient viral load (CT) value (b) Sensitivity of the kit in comparison to the cycle threshold values. The values of <20, 21-25, and 26-30 represent high, intermediate, and comparatively low patient viral loads, respectively.

Association between Rapid Antigen Test (RAK) and Reverse Transcription Polymerase Chain Reaction (RT-PCR)

Our study revealed a significant association between the Rapid Antigen Test (RAK) and gold standard RT-PCR for COVID-19 diagnosis. Individuals testing positive on RT-PCR had 6.333 times higher odds of testing positive on RAK (odds ratio = 6.333, 95% CI: 2.918 to 13.835). These findings support the reliability and usefulness of RAK as a diagnostic tool for COVID-19, particularly in resource-constrained settings with limited access to RT-PCR testing.

Kit performance metrics

The RAK exhibited a sensitivity of 67.86% (95% CI: 61.31%-73.92%) and a specificity of 75.00% (95% CI: 65.12%-83.28%). The positive likelihood ratio was 2.71 (95% CI: 1.90-3.88), while the negative likelihood ratio was 0.43 (95% CI: 0.34-0.54). With a disease prevalence of 70.00% (95% CI: 64.65%-74.97%), the positive predictive value was 86.36% (95% CI: 81.57%-90.06%), and the negative predictive value was 50.00% (95% CI: 44.46%-55.54%). Overall accuracy was 70.00% (95% CI: 64.65%-74.97%). Table 2 summarizes these findings comprehensively.

Table 2. Performance metrics of the rapid antigen test

Analysis of Sensitivity of the Rapid Antigen Kit (RAK) based on CT Values of RT-PCR

The sensitivity of the Rapid Antigen Kit (RAK) was analyzed in comparison with the threshold cycle (CT) value intervals of Reverse Transcriptase PCR (RT-PCR). The results revealed varying levels of performance across different CT intervals. For RT-PCR CT intervals of ≤20, the Rapid Antigen Kit demonstrated a sensitivity of 100% (95% CI: 88.8-100), correctly identifying all positive cases. In the CT interval range of 21-25, the sensitivity of the Rapid Antigen Kit was 63.15% (95% CI: 38.63-82.77), with 16 false negatives and 64 true positives. For CT intervals of 26-30, the sensitivity decreased to 22.22% (95% CI: 7.96-45.88), with 56 false negatives and only 16 true positives. Overall, the Rapid Antigen Kit exhibited a sensitivity of 67.86% (95% CI: 61.31-73.92) across all CT intervals, with 72 false negatives out of 224 positive cases detected by RT-PCR (Table 3).

Table 3. Sensitivity of rapid antigen kit in comparison with threshold cycle of reverse transcriptase PCR

Logistic Regression Analysis and Odds Ratio

A logistic regression analysis assessed the association between the Rapid Antigen Test results and PCR CT values. The analysis revealed a statistically significant association (p<0.05) between the Rapid Antigen Test results and the PCR CT values, indicating that the CT values can predict the likelihood of a positive result on the Rapid Antigen Test. The obtained odds ratio was 6.333, with 95% confidence interval of 2.918 to 13.835, suggesting that for every one-unit increase in the PCR CT value, the odds of a positive result on the Rapid Antigen Test decrease by nearly six times. These findings emphasize the importance of considering PCR CT values when interpreting the results of the Rapid Antigen Test. Individuals with lower CT values were more likely to test negative on the Rapid Antigen Test, while those with higher CT values were more likely to test positive (Table 4).

Table 4. Bivariate logistic regression analysis of factors associated with COVID-19 test results

Discussion

The global COVID-19 pandemic has profoundly impacted various aspects of life, affecting industries, agriculture, education, and healthcare [22]. Healthcare professionals have expressed concerns about diagnostic and treatment approaches for SARS-CoV-2. Scientists have focused on developing efficient, cost-effective, and rapid diagnostic methods [23-25]. Nucleic acid-based detection of specific viral genes has been recommended, along with development of quick and reliable in-vitro diagnostic kits, including magnetic nanoparticle-based extraction kits [26-28]. Our study sought to evaluate the diagnostic performance of home-based rapid antigen testing kits, pre-coated with antibodies against the N genes of SARS-CoV-2, compared to reverse transcriptase-real-time polymerase chain reaction (RT-PCR) testing. The kits yielded visual results within 15 minutes based on the appearance of a colored line in nasopharyngeal samples. Our objective was to assess the Rapid Antigen Test efficacy, accuracy, sensitivity, and specificity in diagnosing COVID-19 relative to PCR testing. The Rapid Antigen Test demonstrated a sensitivity of 67.86% (95% CI: 61.31%-73.92%) and specificity of 75.00% (95% CI: 65.12%-83.28%), implying moderate accuracy in detecting COVID-19 cases. Logistic regression analysis revealed a significant association (p<0.05) between the test results and PCR cycle threshold (CT) values, with higher CT values predicting a lower likelihood of a positive Rapid Antigen Test [29-31]. The odds ratio of 6.333(95% confidence interval: 2.918-13.835) further supported this relationship, suggesting that a one-unit increase in PCR CT value yielded the sixfold reduction of the odds of a positive Rapid Antigen Test result. This suggested that higher CT values, indicating lower viral load, were associated with a decreased likelihood of a positive result on the Rapid Antigen Test. The implications of these findings are significant for the clinical use of the Rapid Antigen Test. While the test is useful for quickly detecting individuals with high viral loads, it may have limitations in identifying cases with lower viral loads. Considering the PCR cycle threshold (CT) values when interpreting the test results is crucial. Confirming the diagnosis with PCR testing is necessary to avoid false negatives if the Rapid Antigen Test is negative, but the individual has a low CT value. Despite its advantages of speed and cost-effectiveness, cautious interpretation of the test performance is advised, especially in cases with a high clinical suspicion but negative Rapid Antigen Test results.

Performance of the Rapid Antigen Test

The positive likelihood ratio of 2.71 suggested that individuals who test positive with the Rapid Antigen Test were 2.71 times more likely to have the virus than those who test negative. The negative likelihood ratio of 0.43 indicated that individuals who tested negative with the Rapid Antigen Test were 0.43 times as likely to have the virus as those who tested positive. These likelihood ratios provided additional insights into the diagnostic accuracy of the Rapid Antigen Test.

Comparison of Rapid Antigen Test and PCR

The crosstabulation of the Rapid Antigen Test and PCR results demonstrates the agreement between the two testing methods. The Rapid Antigen Test showed a sensitivity of 67.86%, indicating its ability to correctly identify individuals with the virus. The specificity of the Rapid Antigen Test was 75.00%, indicating its ability to identify individuals without the virus correctly. These findings highlighted the moderate accuracy of the Rapid Antigen Test compared to PCR, which is considered the gold standard.

Disease Prevalence and Predictive Values

The disease prevalence of 70.00% reflected the proportion of infected individuals in the study population. The Rapid Antigen Test demonstrated a positive predictive value of 86.36%, accurately identifying true virus carriers. However, the 50.00% negative predictive value suggests that individuals testing negative may still harbor the virus. Categorizing patients by threshold cycle (CT), values revealed varying sensitivity and specificity patterns. High sensitivity (100%) and no false negatives were observed with CT values <20. Sensitivity decreased to 63.15% with CT values of 21-25, and sensitivity further reduced to 22.22% with CT values of 26-30, accompanied by increased false negatives: higher viral loads correlated with easier detection by the rapid antigen test. The sensitivity declined as soon as the viral load declined with an increase in the threshold cycle. The results are comparable to the study of Porte et al. (2020) [32]. Several studies reported varying kit sensitivity results from 93.9% to 70% and 100 to 92.2% specificity [11, 32-35]. The results are comparable with the current study, which has reported 67% sensitivity and 75% specificity. The study by Pandey et al. (2021) for the comparison of RT-PCR and rapid antigen detection kits showed an overall 53.6% sensitivity and 97.3% specificity [36]. The kit sensitivity rate reduced from 88.2% to 61.5%, with an increase in CT value from 25 to 30. The current study's false positive rate of samples was 7.5%, and the false negative rate was 22.5%. The results are comparable to the study of Li et al., 2020, where a high level of false negativity was reported in hospitalized patients in China [37]. The stability concern of the rapid antigen kit could be attributed to hospitalization, while dilution of the antigen with VTM poses a limitation.

Consequently, the reliability of antigen kits diminishes in cases of low viral load, making them less significant for asymptomatic patients. Sensitivity could improve by directly immersing samples in a lysis solution rather than using a transport medium. Nasopharyngeal swabs, not saliva samples, are recommended due to negative results observed with some saliva samples. This study's strengths include a robust sample size of 320 participants, meticulous real-world clinical assessment, and comprehensive evaluation methods. Limitations include the cross-sectional design and failure to distinguish between symptomatic and asymptomatic patients, potentially impacting kit sensitivity. Additional studies are needed to develop strategies for optimal use. The rapid antigen assay is unsuitable for large sample volumes due to its low turnaround time and manual nature. Sensitivity analysis of the Rapid Antigen Kit, considering the CT value intervals of RT-PCR, provides valuable performance insights. Results indicate that the kit sensitivity is influenced by the CT interval, showing varying performance levels. The Rapid Antigen Kit demonstrated exceptional sensitivity for RT-PCR CT intervals of ≤20, correctly identifying all positive cases. This suggests that the Rapid Antigen Kit effectively detects COVID-19 cases with lower CT values, indicating higher viral loads. However, as the CT intervals increased, indicating lower viral loads, the sensitivity of the Rapid Antigen Kit decreased. In the CT interval range of 21-25, the sensitivity dropped to 63.15%, indicating that the Rapid Antigen Kit may miss many positive cases in this range.

Further, for CT intervals of 26-30, the sensitivity decreased even further to 22.22%, suggesting that the kit performance significantly declined in cases with lower viral loads. These results highlight the importance of considering the viral load and CT values when interpreting Rapid Antigen Kit results. The sensitivity of the kit is influenced by the level of viral replication, with higher sensitivity observed in individuals with higher viral loads. Therefore, caution should be exercised when using the Rapid Antigen Kit as a sole diagnostic tool, especially in cases with higher CT values. More studies on diverse populations must be conducted to generalize the kit performance.

Conclusion

The study found moderate sensitivity (67.86%) and specificity (75.00%) of the Rapid Antigen Test compared to PCR for COVID-19 diagnosis in a private healthcare setting in Lahore, Punjab, Pakistan. Test performance correlated with viral load, showing higher sensitivity in cases with lower CT values indicating higher viral loads. The test had limitations in detecting lower viral load cases, particularly in asymptomatic or mildly symptomatic patients. Rapid antigen kits can be screening tools for symptomatic patients or those with higher sickness levels. However, confirmatory RT-PCR testing is recommended in cases with adverse test outcomes or high clinical suspicion. Further research is needed to validate findings and explore additional factors impacting test accuracy.

Acknowledgements

This study stands out as a testament to the unwavering dedication and self-reliance of our research team. Despite the absence of financial support, grants, fellowships, or scholarships, we embarked on this scientific journey driven by our passion for knowledge and our commitment to advancing the understanding of COVID-19 diagnostics. We are immensely grateful to the exceptional laboratory personnel whose invaluable contributions ensured the smooth execution of this study. Furthermore, we extend our heartfelt appreciation to all those whose unwavering inspiration and support fueled our pursuit of scientific excellence. Their influence will forever be etched in the fabric of our research endeavors.

Conflict of Interest

The authors declare no conflict of interest or financial support received for this study.

Ethical Approval

This study was approved by the IDC Research Ethics Review Committee on April 25, 2021 (approval letter number: IDC/HR/2023/04-12). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.