Is there a plausible pivotal act that doesn’t amount to some variant of “cripple human civilization so that it can’t make or use computers until it recovers”?
Use AGI to build fast-running high-fidelity human whole-brain emulations. Then run thousands of very-fast-thinking copies of your best thinkers. Seems to me this plausibly makes it realistic to keep tabs on the world’s AGI progress, and locally intervene before anything dangerous happens, in a more surgical way rather than via mass property destruction of any sort.
This is a much less destructive-sounding pivotal act proposal than “melt all GPUs”. I’m trying to figure out why and if it’s actually less destructive...
Does it sound less destructive because it’s just hiding the destructive details behind the decisions of these best thinker simulations? After their, say, 100 subjective years of deliberation do the thinkers just end up with a detailed proposal for how to melt all GPUs”?
I think I give our best thinkers more credit than that. I wouldn’t presume to know in advance the plan that many copies of our best thinkers would come up with after having a long time to deliberate. But I have confidence or at least a hope that they’d come up with something less destructive and at least as effective as “melt all GPUs”.
So this pivotal act proposal puts some distance between us and the messier details of the act. But it does it in a reasonable way, not just by hand-waving or forgetting those details, but instead by deferring them to people who we would most trust to handle them well (many of the best thinkers in the world, overclocked!)
This is an intriguing proposal and because it certainly sounds so much less destructive and horrifying than “melt all GPUs”, I will very much prefer to use and see this used as the go-to theoretical example of a pivotal act until I hear of or think of a better one.
There are maybe ‘plausibly plausible’ (or ‘possibly plausible’) acts that a more ‘adequate’ global civilization might be able to take. But it seems like that hypothetical adequate civilization would have already performed such a pivotal act and the world would look very different than it does now.
It’s (‘strictly’) possible that such a pivotal act has already been performed and that, e.g. the computer hardware currently available isn’t sufficient to build an AGI. It just seems like there’s VERY little evidence that that’s the case.
Is there a plausible pivotal act that doesn’t amount to some variant of “cripple human civilization so that it can’t make or use computers until it recovers”?
Use AGI to build fast-running high-fidelity human whole-brain emulations. Then run thousands of very-fast-thinking copies of your best thinkers. Seems to me this plausibly makes it realistic to keep tabs on the world’s AGI progress, and locally intervene before anything dangerous happens, in a more surgical way rather than via mass property destruction of any sort.
This is a much less destructive-sounding pivotal act proposal than “melt all GPUs”. I’m trying to figure out why and if it’s actually less destructive...
Does it sound less destructive because it’s just hiding the destructive details behind the decisions of these best thinker simulations? After their, say, 100 subjective years of deliberation do the thinkers just end up with a detailed proposal for how to melt all GPUs”?
I think I give our best thinkers more credit than that. I wouldn’t presume to know in advance the plan that many copies of our best thinkers would come up with after having a long time to deliberate. But I have confidence or at least a hope that they’d come up with something less destructive and at least as effective as “melt all GPUs”.
So this pivotal act proposal puts some distance between us and the messier details of the act. But it does it in a reasonable way, not just by hand-waving or forgetting those details, but instead by deferring them to people who we would most trust to handle them well (many of the best thinkers in the world, overclocked!)
This is an intriguing proposal and because it certainly sounds so much less destructive and horrifying than “melt all GPUs”, I will very much prefer to use and see this used as the go-to theoretical example of a pivotal act until I hear of or think of a better one.
I can’t think of any!
There are maybe ‘plausibly plausible’ (or ‘possibly plausible’) acts that a more ‘adequate’ global civilization might be able to take. But it seems like that hypothetical adequate civilization would have already performed such a pivotal act and the world would look very different than it does now.
It’s (‘strictly’) possible that such a pivotal act has already been performed and that, e.g. the computer hardware currently available isn’t sufficient to build an AGI. It just seems like there’s VERY little evidence that that’s the case.