Why does burning all GPUs succeed at preventing unaligned AGI, rather than just delaying it? It seems like you would need to do something more like burning all GPUs now, and also any that get created in the future, and also monitor for any other forms of hardware powerful enough to run AGI, and for any algorithmic progress that allows creating AGI with weaker hardware, and then destroying that other hardware too. Maybe this is what you meant by “burn all GPUs”, but it seems harder to make an AI safely do than just doing that once, because you need to allow the AI to defend itself indefinitely against people who don’t want it to keep destroying GPUs.
Fwiw I think it’s worth the effort to include an extra sentence or two elaborating on this whenever Eliezer or whoever uses it as an example. I don’t think ‘burn all the GPUs’ is obvious enough about what it means
I thought it was clear, given the qualifications already offered, that it was more like a ‘directional example’ than a specific, workable, and concrete example.
I that’s true for people paying attention, but a) it’s just worth being clear, and b) this example is getting repeated a lot, often without the disclaims/setup, and (smart, ingroup) people have definitely been confused/surprised when I said “the GPU thing is obviously meant to include ’continuously surviving counterattacks and dealing with the political fallout.”
I think it’s worth having a compact phrase that’s more likely to survive memetic drift.
Why does burning all GPUs succeed at preventing unaligned AGI, rather than just delaying it? It seems like you would need to do something more like burning all GPUs now, and also any that get created in the future, and also monitor for any other forms of hardware powerful enough to run AGI, and for any algorithmic progress that allows creating AGI with weaker hardware, and then destroying that other hardware too. Maybe this is what you meant by “burn all GPUs”, but it seems harder to make an AI safely do than just doing that once, because you need to allow the AI to defend itself indefinitely against people who don’t want it to keep destroying GPUs.
I think this is basically what the “burn all GPUs” scenario entails, and I agree this is harder.
Fwiw I think it’s worth the effort to include an extra sentence or two elaborating on this whenever Eliezer or whoever uses it as an example. I don’t think ‘burn all the GPUs’ is obvious enough about what it means
I thought it was clear, given the qualifications already offered, that it was more like a ‘directional example’ than a specific, workable, and concrete example.
I that’s true for people paying attention, but a) it’s just worth being clear, and b) this example is getting repeated a lot, often without the disclaims/setup, and (smart, ingroup) people have definitely been confused/surprised when I said “the GPU thing is obviously meant to include ’continuously surviving counterattacks and dealing with the political fallout.”
I think it’s worth having a compact phrase that’s more likely to survive memetic drift.
Updated – thanks!
Do you have any candidates in mind?