The requirement for products to have the same cost/benefit profile really hampers innovation in the marketplace. A less sensitive test (literally, as a % of PCR) over the cumulative test-testing window (e.g., −2 to +5 days from symptom onset) may be desirable when used in a specific part of that window where it doesn’t actually have as severe of sensitivity disadvantage (e.g., −1 to +1 days). Depending on the disease, we may not want to compromise on specificity at all. These are just the “cost” profiles (haha I left out price) - the personal benefit is diagnosis (it’s the same whether it’s PCR or antigen).
But there is also a public health benefit to early diagnosis, which does differ across the tests (depending on test provider and logistical situation, this could be a 1 day vs. 15 minutes difference, or several(!) days vs. 15 minutes difference). Particularly in a situation where we’re trying to stop the spread of a disease that doesn’t need immediate serious treatment, a higher false positive rate may be acceptable (note: that means lower specificity, not lower sensitivity). This is why it makes sense to let a variety of products on the marketplace that meet a minimum threshold so the world can tradeoff along these various attributes (the FDA is charged with finding an appropriate quality floor, but that’s just a matter of dialing in the floor cutoff). Whether these tests will be prioritized by public health authorities (and the public) depends on the actual public health policy smorgasbord adopted and tastes among test properties.
So the FDA, inspecting these cost and benefit profiles, is not approving a test for diagnostic use based on its public health properties (which depend on the whole public health approach) and is instead looking at its personal diagnostic properties. Is the test not actually for diagnosis? That should certainly change the criteria. And I think, clearly, antigen tests are for diagnostic use. The goalposts of antigen vs. PCR testing are not different—“they test for different things” is a mischaracterization of what they do. They use different indicators but are testing for the same thing (COVID) to inform the same behaviors (which is also evident in how we’re talking about them as a substitute for PCR!). Notice that antibody testing uses yet a different indicator to test for a different thing (adaptive immune response) with different behavioral goals.
You may think the quality floor was set too high [I think so too, if that would have meant 80% sensitive tests in 2020 giving way to the 90% ones we see now, given the imperative of testing capacity] (or that quality floors shouldn’t exist at all, which is very delenda est) and generally the FDA has been too strict and slow in this emergency, but that’s not the same complaint as saying the FDA has been unfair or stupid on antigen tests in comparing to PCR.
And I think, clearly, antigen tests are for diagnostic use. The goalposts of antigen vs. PCR testing are not different—“they test for different things” is a mischaracterization of what they do.
The way antigen tests are used, they are frequently used to check whether or not someone is infectious when attending a given event on the day after the test was taken. If a person is infected but there’s only were little virus in their upper respiratory system, they aren’t infectious so it’s less problematic when the test doesn’t detect that.
I somewhat like the distinction between “testing for infectiousness” and “testing for whether I have it” (especially from a public health, rather than personal healthcare, standpoint). “People want to go to parties so they want fast, even if slightly less sensitive, tests because sometimes they don’t really even care about their own health status, just whether they can reasonably party” is also a great reason to try to market the product (let party organizers or other organization police what tests they will accept or whether they will expect pre-testing), but the FDA needs to assess that sensitivity somehow. What is the standard for infectiousness that the FDA is going to use to have a second evaluation pathway if there really is a distinction with a difference here? Probably some indicator of the amount of infectious virus present. A plaque assay (yields plaque-forming units, PFU) would be great, counting infectious particles, but the test itself takes days, needs BSL-3 conditions, and is resource intensive in other ways as well. Don’t compare to PCR, compare to PFU...that’s even more demanding against would-be antigen test manufacturers. And PFU has a pretty strong relationship with Ct. Compare to PCR then!
People are rightly dissatisfied with the status quo and looking for someone to blame for the lack of tests, but the problem is not that the FDA compares to PCR, it’s the threshold they wanted. I know “evidence-based decision-making” is abused and gets a rap, but I don’t think effectively advocating for evidence-less decision-making will make us less wrong (in case that sounds too harsh, what was the alternative to PCR that was supposed to validate antigen testing for us?).
I know “evidence-based decision-making” is abused and gets a rap, but I don’t think effectively advocating for evidence-less decision-making will make us less wrong
The EU and the UK also have regulatory agencies that require evidence for accepting tests.
Assays should have a sensitivity of 90% or greater for subjects with a Ct < 25.
That means that the test has to has a sensisitivty of 90% for patients with “Very high viral load”
The world is bigger then the US. In a case like this it makes sense to look at other countries who have enough tests for everyone who wants instead of just declaring the task impossible.
This is again a threshold, not comparator, complaint. Ct values are generated by PCR. Instead of using a crosstab for all samples, this approach is to use a crosstab for a subset of samples with higher viral load. It’s reasonable! IIRC from a previous paper, this (90% of Ct<25) has a similar effect as just reducing the overall cutoff to (80% of all). It’s also reasonable to use studies from other countries or to follow other agencies, in either case the ones we think are credible, which is again about the evidence threshold. What I’ve been hammering on is that the idea these tests are so different that they’re noncomparable is not sensible.
The requirement for products to have the same cost/benefit profile really hampers innovation in the marketplace. A less sensitive test (literally, as a % of PCR) over the cumulative test-testing window (e.g., −2 to +5 days from symptom onset) may be desirable when used in a specific part of that window where it doesn’t actually have as severe of sensitivity disadvantage (e.g., −1 to +1 days). Depending on the disease, we may not want to compromise on specificity at all. These are just the “cost” profiles (haha I left out price) - the personal benefit is diagnosis (it’s the same whether it’s PCR or antigen).
But there is also a public health benefit to early diagnosis, which does differ across the tests (depending on test provider and logistical situation, this could be a 1 day vs. 15 minutes difference, or several(!) days vs. 15 minutes difference). Particularly in a situation where we’re trying to stop the spread of a disease that doesn’t need immediate serious treatment, a higher false positive rate may be acceptable (note: that means lower specificity, not lower sensitivity). This is why it makes sense to let a variety of products on the marketplace that meet a minimum threshold so the world can tradeoff along these various attributes (the FDA is charged with finding an appropriate quality floor, but that’s just a matter of dialing in the floor cutoff). Whether these tests will be prioritized by public health authorities (and the public) depends on the actual public health policy smorgasbord adopted and tastes among test properties.
So the FDA, inspecting these cost and benefit profiles, is not approving a test for diagnostic use based on its public health properties (which depend on the whole public health approach) and is instead looking at its personal diagnostic properties. Is the test not actually for diagnosis? That should certainly change the criteria. And I think, clearly, antigen tests are for diagnostic use. The goalposts of antigen vs. PCR testing are not different—“they test for different things” is a mischaracterization of what they do. They use different indicators but are testing for the same thing (COVID) to inform the same behaviors (which is also evident in how we’re talking about them as a substitute for PCR!). Notice that antibody testing uses yet a different indicator to test for a different thing (adaptive immune response) with different behavioral goals.
You may think the quality floor was set too high [I think so too, if that would have meant 80% sensitive tests in 2020 giving way to the 90% ones we see now, given the imperative of testing capacity] (or that quality floors shouldn’t exist at all, which is very delenda est) and generally the FDA has been too strict and slow in this emergency, but that’s not the same complaint as saying the FDA has been unfair or stupid on antigen tests in comparing to PCR.
The way antigen tests are used, they are frequently used to check whether or not someone is infectious when attending a given event on the day after the test was taken. If a person is infected but there’s only were little virus in their upper respiratory system, they aren’t infectious so it’s less problematic when the test doesn’t detect that.
I somewhat like the distinction between “testing for infectiousness” and “testing for whether I have it” (especially from a public health, rather than personal healthcare, standpoint). “People want to go to parties so they want fast, even if slightly less sensitive, tests because sometimes they don’t really even care about their own health status, just whether they can reasonably party” is also a great reason to try to market the product (let party organizers or other organization police what tests they will accept or whether they will expect pre-testing), but the FDA needs to assess that sensitivity somehow. What is the standard for infectiousness that the FDA is going to use to have a second evaluation pathway if there really is a distinction with a difference here? Probably some indicator of the amount of infectious virus present. A plaque assay (yields plaque-forming units, PFU) would be great, counting infectious particles, but the test itself takes days, needs BSL-3 conditions, and is resource intensive in other ways as well. Don’t compare to PCR, compare to PFU...that’s even more demanding against would-be antigen test manufacturers. And PFU has a pretty strong relationship with Ct. Compare to PCR then!
People are rightly dissatisfied with the status quo and looking for someone to blame for the lack of tests, but the problem is not that the FDA compares to PCR, it’s the threshold they wanted. I know “evidence-based decision-making” is abused and gets a rap, but I don’t think effectively advocating for evidence-less decision-making will make us less wrong (in case that sounds too harsh, what was the alternative to PCR that was supposed to validate antigen testing for us?).
The EU and the UK also have regulatory agencies that require evidence for accepting tests.
From https://ec.europa.eu/health/sites/default/files/preparedness_response/docs/covid-19_rat_common-list_en.pdf it seems that the alternative for using the straight comparison to PCR is:
That means that the test has to has a sensisitivty of 90% for patients with “Very high viral load”
The world is bigger then the US. In a case like this it makes sense to look at other countries who have enough tests for everyone who wants instead of just declaring the task impossible.
This is again a threshold, not comparator, complaint. Ct values are generated by PCR. Instead of using a crosstab for all samples, this approach is to use a crosstab for a subset of samples with higher viral load. It’s reasonable! IIRC from a previous paper, this (90% of Ct<25) has a similar effect as just reducing the overall cutoff to (80% of all). It’s also reasonable to use studies from other countries or to follow other agencies, in either case the ones we think are credible, which is again about the evidence threshold. What I’ve been hammering on is that the idea these tests are so different that they’re noncomparable is not sensible.