Reducing costs equally across the board in some domain is bad news in any situation where offense is favored. Reducing costs equally-in-expectation (but unpredictably, with high variance) can be bad even if offense isn’t favored, since you might get unlucky and the payoffs aren’t symmetrical.
(re: recent discourse on bio risks from misuse of future AI systems. I don’t know that I think those risks are particularly likely to materialize, and most of my expected disutility from AI progress doesn’t come from that direction, but I’ve seen a bunch of arguments that seem to be skipping some steps when trying to argue that progress on ability to do generic biotech is positive EV. To be fair, the arguments for why we should expect it to be negative EV are often also skipping those steps. My point is that a convincing argument in either direction needs to justify its conclusion in more depth; the heuristics I reference above aren’t strong enough to carry the argument.)
Reducing costs equally across the board in some domain is bad news in any situation where offense is favored. Reducing costs equally-in-expectation (but unpredictably, with high variance) can be bad even if offense isn’t favored, since you might get unlucky and the payoffs aren’t symmetrical.
(re: recent discourse on bio risks from misuse of future AI systems. I don’t know that I think those risks are particularly likely to materialize, and most of my expected disutility from AI progress doesn’t come from that direction, but I’ve seen a bunch of arguments that seem to be skipping some steps when trying to argue that progress on ability to do generic biotech is positive EV. To be fair, the arguments for why we should expect it to be negative EV are often also skipping those steps. My point is that a convincing argument in either direction needs to justify its conclusion in more depth; the heuristics I reference above aren’t strong enough to carry the argument.)