This is the world where the US federal government’s response to COVID was to ban private COVID testing,
Private COVID testing, in early 2020, ran the risk of diverting millions of people away from public testing sites, which, in early 2020, may have been the only way for the military to get data about the pandemic (which was poorly understood at the time, and at the time it was right for biodefense officials to be paranoid about it).
The military had no reason to trust state governments to use all the PPE they ordered in a manner that was optimal for national security, especially if the military was always the first to get solid info about COVID.
They wanted citizens to choose not to buy PPE, which again makes sense with the limited stockpiles of early 2020.
I don’t mean to give anyone any hope where hope isn’t due. Nate’s arguments about incompetence and non-coordination generally hold pretty strongly, including with COVID, even if these aren’t the best examples (resource/information hoarding).
However, I think this highlights a common problem among alignment thinkers where Government is assumed to be totally incompetent, when in reality there is an extremely complicated mix of competence disguised as incompetence and incompetence disguised as competence (and, of course, plenty of undisguised incompetence). This especially differs depending on the area; for example, in China during the Cultural Revolution, the nuclear weapons program was relatively untouched by the mass purges that devastated almost everything else in the government and severely disrupted the military.
Goodhart’s Law is not to be taken lightly, and there’s tons of people in the National Security Establishment who take Goodhart’s Law extremely seriously. Maybe not plenty, certainly not enough, but definitely a lot.
Nope, I don’t buy it. Having read Zvi’s Covid posts and having a sense of how much better policy was possible (and was already being advocated at the time), I just don’t buy a framing where government Covid policy can be partly considered as competent. I’m also dubious of treating “the military” as a monolithic black box entity with coherent behavior and goals, rather than consisting of individual decision-makers who follow the same dysfunctional incentives as everyone else.
If you have sources that e.g. corroborate sane military policy during the early Covid months, feel free to provide it, but for now I’m unconvinced.
Zvi depends on Twitter for a lot of analysis, which is a really serious Goodhart’s law situation, and it resulted in him being really wrong about Goodhart-intensive areas like predicting that the Shanghai lockdown would fail. Again, his analysis is still generally top-notch but it’s far from perfect. It’s more about outperforming large popular news outlets by as much as possible, which he does plenty of. But international affairs is a different beast entirely when it comes to systematic error.
Economic priorities can be considered more important than public health, especially with Omicron and variants, and even more so during a major war, and yet even more so when both sides deliberately use the threat of unpredictably spiralling economic crisis to erode the opposite side’s consensus to maintain force. Persuading people to go to work, instead of not going to work (e.g. due to fear of brain damage), seems to be a good idea that powerful people can rally behind right now.
I never stated that the military is a monolithic entity, it is consensus-driven depending on whatever consensus is possible in the context of compartmentalization e.g. what actions seem obvious to a lot of different people. Governments too corrupt to occasionally rally behind a goal have not been a dominant force in global politics for centuries. Any sort of effective central command structure is anti-inductive to study because it’s built to fool foreign intelligence agencies and domestic business interests. And, obviously, partisan gridlock is usually not the dominant factor influencing core national security priorities.
It is also really difficult to disentangle ordinary bureaucratic actions with military operations carried out through bureaucracies. This is actually extremely common. Among other things, it helps protect military agencies from blame since it’s really important that they remain as popular as possible, for several reasons.
The “pure incompetence” model appears extremely naive compared to the “mixed competence-incompetence” model, and the CDC’s lack of Disease Control and Prevention is only a single data point in a large-n system; even if that was entirely a convulsion, which I never ruled out in the CDC’s case.
I don’t trust Zvi’s COVID posts after sharing one of them with people who were much more clueful than me and getting embarrassed by the resulting critique.
However I also don’t trust the government response to have been even close to optimal.
During COVID, Denmark and a number of other European countries suspended dispensing the AstraZeneca vaccine over worries that it would lead to blood clots. Zvi yelled that all of them were “ludicrously stupid several times over” for doing that, because he did not believe the blood clots were real. However, according to Denmark’s official calculations, given the amount of alternative vaccines they had access to, the relative laxness of their COVID restrictions, etc., it was actually worthwhile to use the other vaccines instead of AstraZeneca. The calculations didn’t look obviously wrong to me, though I got busy before I could check them properly. As far as I understand, it’s currently the general scientific consensus that it can indeed cause blood clots once in a while?
For now, I don’t think basic notions like “all governments were incompetent on covid” are particularly easy to dispute?
To provide two examples:
The only country I know of to a do human challenge trial for Covid was the UK, and as I understand it, that trial only began in February 2021.
I’m not aware of any attempts to ban gain-of-function research, let alone of any bans that were actually implemented.
These seem like reasonably plausible examples of cases where countries could do better, but given the blood clot mistake I wouldn’t want to assume that there aren’t problems I’ve missed.
because he did not believe the blood clots were real
I’d need a source on that. From what I recall, the numbers were small and preliminary but plausibly real, but orders of magnitude below the danger of Covid (which IIRC incidentally also causes blood clots). So one could call the suspension penny-wise but pound-foolish, or some such. Not to mention that IIRC the suspension resulted in a dip in Covid vaccinations, so it’s not clear that it was even the right call in retrospect. I also recall hearing the suspension justified as necessary to preserve trust in the vaccines, when the result seemed to be the opposite.
And remember, even if the calculus was correct for Denmark, didn’t this ultimately result in a temporary suspension of this vaccine throughout Europe?
Based on just what we’ve discussed here, I don’t at all think it warrants being considered an error on Zvi’s part.
And with that being said, I don’t particularly care about policy details like this, when governments failed on the big stuff like human challenge trials or accelerating vaccine discovery or production. We’re discussing all this in the context of AI Safety, after all, and if governments don’t get the big details right, it doesn’t matter if they’re decent or terrible at comparatively minor issues. The latter is ultimately just bike-shedding. (E.g. I can see governments convening panels on algorithmic bias, but at best they’ll be irrelevant to the core issue at hand, and at worst they’ll distract from the important stuff.)
I’d need a source on that. From what I recall, the numbers were small and preliminary but plausibly real, but orders of magnitude below the danger of Covid (which IIRC incidentally also causes blood clots).
And remember, even if the calculus was correct for Denmark, didn’t this ultimately result in a temporary suspension of this vaccine throughout Europe?
I’m not sure about the specifics and I’m about to go to bed so I can’t check. But I think the other countries suspended them after Norway did out of a precautionary principle (i.e. if they are not good for Norway then we should probably re-evaluate whether they are good for us), and then often reinstated them after reevaluating.
It seems like a good policy to me for a country to pay attention to findings that a medicine may be surprisingly dangerous.
I also recall hearing the suspension justified as necessary to preserve trust in the vaccines, when the result seemed to be the opposite.
I feel like it would be exactly the kind of mustache-twirling consequentialism that Eliezer calls out as un-genre-savvy to continue using a vaccination that your calculations say aren’t worth it in terms of naive cost/benefit because switching out the vaccine policy might harm trust. Like I’d rather not have the government decide that I’m too panicky to allow showing doubt.
A dimilar point applies to the suspension in much of Europe. While it might’ve been consequentially better on some level of analysis for Norway/whoever to take into account what effects their suspension would have on other countries, it feels too manipulative to me, and I’d rather they focus on their job.
I wasn’t disputing that Zvi mentioned the blood clot story, I was disputing your characterisation of it. Quoting from literally the first two paragraphs from your link:
And even if all the observed clots were extra, all were caused by the vaccine, all were fatal, and that represented the overall base rate, and we ignore all population-level benefits and economic issues, the vaccine would still be worth using purely for personal health and safety by multiple orders of magnitude.
The WHO and EMA said there was no evidence there was an issue.
This is not consistent with your characterisation that Zvi made an error because “he did not believe the blood clots were real”.
I feel like it would be exactly the kind of mustache-twirling consequentialism that Eliezer calls out as un-genre-savvy to continue using a vaccination that your calculations say aren’t worth it in terms of naive cost/benefit because switching out the vaccine policy might harm trust. Like I’d rather not have the government decide that I’m too panicky to allow showing doubt.
I’m not disputing that. The assumption and argument here is that the calculations come out positive: we have world A where people get a few more blood clots from a very rare side effect of a vaccine, but a lot fewer blood clots as a Covid symptom, versus world B where people get no blood clots from the vaccine but a lot more people get blood clots as a Covid symptom. That’s what I meant by calling the policy of choosing world B penny-wise but pound-foolish.
(EDIT: And in a yet more competent civilization, you wouldn’t even have to make the tradeoff in the first place, because you could just tell your patients “by the way, we’ve discovered another rare side-effect” instead of suspending the vaccine outright.)
I wasn’t disputing that Zvi mentioned the blood clot story, I was disputing your characterisation of it. Quoting from literally the first two paragraphs from your link:
You are right also said that it was not worth it to deal with even if they were real (which contradicts what the Danish calculations showed, as I mentioned), but the argument he lead with was that they were not real:
There was always going to be something that happened to correlate with vaccination days to some extent, somewhere, over some time period. The number of blood clots experienced after vaccination wasn’t even higher than the base rate you would otherwise expect.
(I had actually initially misremembered him as leading with the argument that they were not worth dealing with, but when I wrote my original post I decided to go back and double-check and found that he was focusing on them not being real instead. So I rewrote it before posting it to say that he didn’t believe them.)
I’m not disputing that. The assumption and argument here is that the calculations come out positive: we have world A where people get a few more blood clots from a very rare side effect of a vaccine, but a lot fewer blood clots as a Covid symptom, versus world B where people get no blood clots from the vaccine but a lot more people get blood clots as a Covid symptom. That’s what I meant by calling the policy of choosing world B penny-wise but pound-foolish.
What I’m saying is that the Danish government did these sorts of calculations at the time and found the trade-off to be worthwhile; the COVID death rate at the time was extremely low, the restrictions were very loose, and the available alternatice vaccines were plentiful, so avoiding AstraZeneca would according to their calculations have lowish costs, such that it is worth it.
Meanwhile, I don’t see any calculations by Zvi showing that it Denmark was obviously stupid; it seems like Zvi was mostly just pattern-matching (as did I when I originally shared Zvi’s article).
Private COVID testing, in early 2020, ran the risk of diverting millions of people away from public testing sites, which, in early 2020, may have been the only way for the military to get data about the pandemic (which was poorly understood at the time, and at the time it was right for biodefense officials to be paranoid about it).
The military had no reason to trust state governments to use all the PPE they ordered in a manner that was optimal for national security, especially if the military was always the first to get solid info about COVID.
They wanted citizens to choose not to buy PPE, which again makes sense with the limited stockpiles of early 2020.
I don’t mean to give anyone any hope where hope isn’t due. Nate’s arguments about incompetence and non-coordination generally hold pretty strongly, including with COVID, even if these aren’t the best examples (resource/information hoarding).
However, I think this highlights a common problem among alignment thinkers where Government is assumed to be totally incompetent, when in reality there is an extremely complicated mix of competence disguised as incompetence and incompetence disguised as competence (and, of course, plenty of undisguised incompetence). This especially differs depending on the area; for example, in China during the Cultural Revolution, the nuclear weapons program was relatively untouched by the mass purges that devastated almost everything else in the government and severely disrupted the military.
Goodhart’s Law is not to be taken lightly, and there’s tons of people in the National Security Establishment who take Goodhart’s Law extremely seriously. Maybe not plenty, certainly not enough, but definitely a lot.
Nope, I don’t buy it. Having read Zvi’s Covid posts and having a sense of how much better policy was possible (and was already being advocated at the time), I just don’t buy a framing where government Covid policy can be partly considered as competent. I’m also dubious of treating “the military” as a monolithic black box entity with coherent behavior and goals, rather than consisting of individual decision-makers who follow the same dysfunctional incentives as everyone else.
If you have sources that e.g. corroborate sane military policy during the early Covid months, feel free to provide it, but for now I’m unconvinced.
Zvi depends on Twitter for a lot of analysis, which is a really serious Goodhart’s law situation, and it resulted in him being really wrong about Goodhart-intensive areas like predicting that the Shanghai lockdown would fail. Again, his analysis is still generally top-notch but it’s far from perfect. It’s more about outperforming large popular news outlets by as much as possible, which he does plenty of. But international affairs is a different beast entirely when it comes to systematic error.
Economic priorities can be considered more important than public health, especially with Omicron and variants, and even more so during a major war, and yet even more so when both sides deliberately use the threat of unpredictably spiralling economic crisis to erode the opposite side’s consensus to maintain force. Persuading people to go to work, instead of not going to work (e.g. due to fear of brain damage), seems to be a good idea that powerful people can rally behind right now.
I never stated that the military is a monolithic entity, it is consensus-driven depending on whatever consensus is possible in the context of compartmentalization e.g. what actions seem obvious to a lot of different people. Governments too corrupt to occasionally rally behind a goal have not been a dominant force in global politics for centuries. Any sort of effective central command structure is anti-inductive to study because it’s built to fool foreign intelligence agencies and domestic business interests. And, obviously, partisan gridlock is usually not the dominant factor influencing core national security priorities.
It is also really difficult to disentangle ordinary bureaucratic actions with military operations carried out through bureaucracies. This is actually extremely common. Among other things, it helps protect military agencies from blame since it’s really important that they remain as popular as possible, for several reasons.
The “pure incompetence” model appears extremely naive compared to the “mixed competence-incompetence” model, and the CDC’s lack of Disease Control and Prevention is only a single data point in a large-n system; even if that was entirely a convulsion, which I never ruled out in the CDC’s case.
I don’t trust Zvi’s COVID posts after sharing one of them with people who were much more clueful than me and getting embarrassed by the resulting critique.
However I also don’t trust the government response to have been even close to optimal.
I would appreciate this kind of reply, except I can’t update on it if the critique isn’t public.
For now, I don’t think basic notions like “all governments were incompetent on covid” are particularly easy to dispute?
To provide two examples:
The only country I know of to a do human challenge trial for Covid was the UK, and as I understand it, that trial only began in February 2021.
I’m not aware of any attempts to ban gain-of-function research, let alone of any bans that were actually implemented.
During COVID, Denmark and a number of other European countries suspended dispensing the AstraZeneca vaccine over worries that it would lead to blood clots. Zvi yelled that all of them were “ludicrously stupid several times over” for doing that, because he did not believe the blood clots were real. However, according to Denmark’s official calculations, given the amount of alternative vaccines they had access to, the relative laxness of their COVID restrictions, etc., it was actually worthwhile to use the other vaccines instead of AstraZeneca. The calculations didn’t look obviously wrong to me, though I got busy before I could check them properly. As far as I understand, it’s currently the general scientific consensus that it can indeed cause blood clots once in a while?
These seem like reasonably plausible examples of cases where countries could do better, but given the blood clot mistake I wouldn’t want to assume that there aren’t problems I’ve missed.
I’d need a source on that. From what I recall, the numbers were small and preliminary but plausibly real, but orders of magnitude below the danger of Covid (which IIRC incidentally also causes blood clots). So one could call the suspension penny-wise but pound-foolish, or some such. Not to mention that IIRC the suspension resulted in a dip in Covid vaccinations, so it’s not clear that it was even the right call in retrospect. I also recall hearing the suspension justified as necessary to preserve trust in the vaccines, when the result seemed to be the opposite.
And remember, even if the calculus was correct for Denmark, didn’t this ultimately result in a temporary suspension of this vaccine throughout Europe?
Based on just what we’ve discussed here, I don’t at all think it warrants being considered an error on Zvi’s part.
And with that being said, I don’t particularly care about policy details like this, when governments failed on the big stuff like human challenge trials or accelerating vaccine discovery or production. We’re discussing all this in the context of AI Safety, after all, and if governments don’t get the big details right, it doesn’t matter if they’re decent or terrible at comparatively minor issues. The latter is ultimately just bike-shedding. (E.g. I can see governments convening panels on algorithmic bias, but at best they’ll be irrelevant to the core issue at hand, and at worst they’ll distract from the important stuff.)
https://www.lesswrong.com/posts/82kxkCdHeSb8smZZq/covid-3-18-an-expected-quantity-of-blood-clots
I’m not sure about the specifics and I’m about to go to bed so I can’t check. But I think the other countries suspended them after Norway did out of a precautionary principle (i.e. if they are not good for Norway then we should probably re-evaluate whether they are good for us), and then often reinstated them after reevaluating.
It seems like a good policy to me for a country to pay attention to findings that a medicine may be surprisingly dangerous.
I feel like it would be exactly the kind of mustache-twirling consequentialism that Eliezer calls out as un-genre-savvy to continue using a vaccination that your calculations say aren’t worth it in terms of naive cost/benefit because switching out the vaccine policy might harm trust. Like I’d rather not have the government decide that I’m too panicky to allow showing doubt.
A dimilar point applies to the suspension in much of Europe. While it might’ve been consequentially better on some level of analysis for Norway/whoever to take into account what effects their suspension would have on other countries, it feels too manipulative to me, and I’d rather they focus on their job.
I wasn’t disputing that Zvi mentioned the blood clot story, I was disputing your characterisation of it. Quoting from literally the first two paragraphs from your link:
This is not consistent with your characterisation that Zvi made an error because “he did not believe the blood clots were real”.
I’m not disputing that. The assumption and argument here is that the calculations come out positive: we have world A where people get a few more blood clots from a very rare side effect of a vaccine, but a lot fewer blood clots as a Covid symptom, versus world B where people get no blood clots from the vaccine but a lot more people get blood clots as a Covid symptom. That’s what I meant by calling the policy of choosing world B penny-wise but pound-foolish.
(EDIT: And in a yet more competent civilization, you wouldn’t even have to make the tradeoff in the first place, because you could just tell your patients “by the way, we’ve discovered another rare side-effect” instead of suspending the vaccine outright.)
You are right also said that it was not worth it to deal with even if they were real (which contradicts what the Danish calculations showed, as I mentioned), but the argument he lead with was that they were not real:
(I had actually initially misremembered him as leading with the argument that they were not worth dealing with, but when I wrote my original post I decided to go back and double-check and found that he was focusing on them not being real instead. So I rewrote it before posting it to say that he didn’t believe them.)
What I’m saying is that the Danish government did these sorts of calculations at the time and found the trade-off to be worthwhile; the COVID death rate at the time was extremely low, the restrictions were very loose, and the available alternatice vaccines were plentiful, so avoiding AstraZeneca would according to their calculations have lowish costs, such that it is worth it.
Meanwhile, I don’t see any calculations by Zvi showing that it Denmark was obviously stupid; it seems like Zvi was mostly just pattern-matching (as did I when I originally shared Zvi’s article).
Mossad was allegedly pretty successful procuring large amounts of PPE from hostile countries: https://www.tandfonline.com/doi/full/10.1080/08850607.2020.1783620. They also had covert contact tracing, and one way or another their case counts seemed pretty low until Omicron.
The first few weeks of COVID lockdowns went extremely well: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7675749/