I don’t buy any inference drawn from that study that is in the realm of “20% of the population of NYC was exposed to SARS-CoV-2 and developed immunity”
I think that will be the primary message that people will get from that study.
It's the "for starters" that concerns me. A lot of us were always saying that official case counts are much too low, and antibody surveys were supposed to be the definitive proof that we were responsibly waiting for. Now they're starting to come out, and still nobody believes it. I worry it's a moving goalpost, and no evidence will ever be enough to make people start reconsidering their beliefs.
> antibody surveys were supposed to be the definitive proof that we were responsibly waiting for.
Simple question: Why?
For most coronaviruses, antibodies reflect only a temporary immunity, that is usual gone in 6-24 months, due to the nature of these viruses.
All an antibody survey shows is that antibodies can be created, not that they are effective long term. Showing a longer term immunity takes statistical analysis, usually after that temporary window has ended.
In fact, antibody surveys may not even show an effective temporary immunity, if the wrong kinds of antibodies are being screened for.
Knowing this, why was the antibody surveys supposed to be some golden bullet? The advice from the medical community was "this is being actively studied, wait and see."
The surveys provide the medical community with important data, but they don't really provide us with policy making data, and they certainly don't predict the future for the general population when exposed to the virus.
I think the argument being made is less about lasting immunity but reevaluating the actual risks for the general population. 8 million people live in NYC and there has been 0.155 million confirmed cases so far. If the actual infection rate is 20% then that represents a 10 fold overestimation of morbidity and mortality.
A good place to keep an eye on for the short term would be Sweden. Despite the lack of lockdown their disease penetrance is still on par with the UK.
> If the actual infection rate is 20% then that represents a 10 fold overestimation of morbidity and mortality.
10 fold over what? A problem since the beginning is that many people are confusing CFR and IFR. Worse is when people compare the IFR of COVID-19 to the CFR of the flu. Regardless, the IFR for COVID-19 has been thought to be .5-1% since the beginning. If we assume the NY antibody study is mostly correct (even with the sampling errors), I believe it puts the IFR in the .5-1% range [1]. If that IFR holds it still means 1.6-3.3M deaths in the US assuming the healthcare does not get overwhelmed.
> If that IFR holds it still means 1.6-3.3M deaths in the US assuming the healthcare does not get overwhelmed.
You cannot assume that 100% of people will be infected. Looking at case studies like USS Roosevelt (840 of 5000) and Diamond Princess (712 of 3,711) as the worst case prevalence because they are much higher-R environments.
So basically your IFR based fatality numbers could be divided by roughly 5.
In both case quarantines were put in place and/or people were eventually evacuated. Yes, there is a limit where not 100% of the population will be infected. Given the R0 of COVID-19, currently herd immunity is thought to be reached between 60%-80% of the population getting infected. So even if we are generous and take the low end of the IFR we get 960k - 1.28M deaths to reach herd immunity.
There is some news out that is putting the IFR closer to .3% on the low end. That is great news if it holds up. The problem is that the numbers out of NY, if flawed would bring the IFR lower than reality, and they are ~.75% IFR.
However, biased antibody studies (no self-selection criteria) that may have high rates of both false negatives and false positives do not represent anything about the current level of estimation whatsoever.
Which is why when these studies happen, the public is told to wait for it to be assessed, rather than pretending all of us are remotely qualified to judge the content and draw conclusions from it about what actual risks the general population might be facing.
> Antibodies are definitive proof of a previous infection, which is what I was talking about.
When the studies in question have a high rate of false positives, that is absolutely not the case. It may simply be a statistical anomaly, from taking the incorrect confidence interval.
Currently, from the studies taken, it looks like we have high rates of both false negatives, and false positives. Which means that the testing does not give you an accurate picture of whether a population group has previous infections or not.
I don't think anyone is moving goalposts. Most of the antibody studies that have come out have had serious flaws either with the tests themselves or the sampling. The recent NY one was ok, but still had sampling issues because it only sampled people who were out and about during a lockdown. I would expect those people to have a higher prevalence of exposure.
With that said, the extrapolated numbers for NY do fall in line with the original IFR of .5-1% The downside is that if that is the IFR then the US is looking at 1.6-3.3M deaths assuming hospital systems can keep up as the infection spreads.
Edit. It's also important to talk about infection counts (what antibody tests are looking for) and case counts (people who show symptoms and end up seeking medical care). In the past when people were saying it's just the flu they were comparing COVID-19 IFR to the flus CFR.
I totally agree that the confirmed case counts are way too low, because even most people who were symptomatic weren't able to get tested (e.g. me), let alone random people who were asymptomatic.
But the 21% study is seriously flawed because it didn't do a random sampling of the population. We need that at a minimum to know with any certainty what the actual exposure rate is. The figures that are coming back from studies using random samples in other places have been much lower.
It did a random sampling. We should do followups to screen off possible biases people have proposed, but stopping random people in the grocery store is by any reasonable standard randomization.
No. It’s a random sampling of people who are out during lockdown. It can’t being extrapolated to the whole population when large parts are not leaving their homes.
You do both. This is why study design matters. And it's one of the reasons all of the early antibody studies have issues (the other being test accuracy).
There is no such list. No US state has a master list of all residents. The DMV has a fairly high percentage but even that tends to miss children, older people, undocumented immigrants, etc.
I would be willing to bet that if you combine all the different lists that New York State and its various agencies have (DMV, DOE, Department of Taxation and Finance, NYC ID, jury duty, voter registration, social services, etc.), that you would easily get >99% coverage of all people who've resided here for at least one year.
This would be a much better list to sample randomly from than "go to a grocery store and test everyone who walks in".
I should point out on /r/nyc, some local redditors saw the testing going on all week in the same location and posted about it, informing others. I suspect this led people who wanted a free test to actively seek them out, especially because it's so hard to get tested otherwise. I'm pretty sure I had it over a month ago and I still haven't gotten tested, so if I'd seen those posts in time I'd have headed over there to get tested myself. Point is, the sample is even further biased because word spread around and some number of people getting tested there were actively seeking it out for reasons.
Now you're talking about a huge legal issue just to get access to the data, followed by a huge record linkage issue to remove duplicates. So with the time pressure involved, your proposal is so completely impractical as to be ridiculous.
These are all state agencies. They're already sharing data with each other anyway (e.g. the jury selection tool is getting feeds from many of these other sources).
What huge legal issues? This is all the government. Of course it has lists of all of its citizens, and can and does use said lists.
You say "almost like", but scientific studies rarely sample the population this way because researchers generally don't have access to a list of all residents.
The state is running these studies. I guarantee you New York State has many good lists of people living in the state. Start with the jury duty list, for example. It pulls data from the DMV, voter registration files, state tax filers, non-driver's IDs such as NYC ID, and more. That covers all the adults. You can get a good list of adults residing in the state to pull your random sample from, and to include the children go get data from the school system and/or just test whatever children live with any given adult that you pick randomly.
Well, the type of information you're trying to gather is rather unique. Usually we just wait for a virus to run course then test lots of people to see the resulting case counts. But we can't do that here. Normally you just use a control and test group, but that doesn't work for figuring out underlying infection rates.
There are some tests trying to sample everyone in a geographic area (SF Mission census block) but the data isn't out yet because they're conducting tests as we speak.
I guess we'll see. I don't share your certainty. Although I'd love to be able to go outside sooner.
Also elsewhere in this thread it's mentioned that the Florida and Santa Clara results could be entirely explained by high type II error in the test. The Florida test appeared to have a false positive rate of ~15% when independently validated, which is basically the infection rate they found. In other words, this is a specific form of base rate fallacy where the test accuracy is really low.
> It will also show an undercount of at least an order of magnitude.
It seems pretty likely that the data will come out showing at least 10%, so it's literally impossible for it to undercount by an order of magnitude.
How do you think a random sample of inhabitants would be off by a whole order of magnitude, anyway? Can you explain the mechanism whereby that might happen? The only thing that comes to mind would be using a worthless test with a 90+% false negative rate.
That's definitely relevant to the question of why it's difficult to do such a study. However, it's not relevant to the question of whether such a study is necessary to make strong inferences about the population as a whole. The difficulty of making the right study does not change our ability to draw inferences from the wrong study. (We can't.)
I think that will be the primary message that people will get from that study.