My comment involves a constant probability of the bad outcome with each draw, and no decreasing benefits. I think this is a good exposition of this portion of the post (which I wrote), if you assume that each unit of bio progress is equally good, but that the goods don’t materialize if we all die of a global pandemic:
Suppose instead that the benefits of thing X grow proportionally to how much it happens: for example, maybe every person who learns about biology makes roughly the same amount of incremental progress in learning how to cure disease and make humans healthier. Also suppose that every person who does thing X has a small probability of causing bad effect Y for everyone that negates all the benefits of X: for example, perhaps 0.01% of people would cause a global pandemic killing everyone if they learned enough about biology.
My comment involves a constant probability of the bad outcome with each draw, and no decreasing benefits. I think this is a good exposition of this portion of the post (which I wrote), if you assume that each unit of bio progress is equally good, but that the goods don’t materialize if we all die of a global pandemic: