I don’t disagree with your analysis. What I’m pointing out is that I haven’t seen any workable proposals to prevent this (or another very disagreeable scenario), except for 1) a singleton AI controlling the effective laws of physics in its light cone, or 2) somehow making sure nobody but a single player (“ruler”) has the ability to create computer hardware and/or software. In which case, the universe will probably be tiled with that ruler. The incentives to create copies and the resulting evolutionary pressures are too great.
And in the context of all this, ideas like democracy are completely unworkable unless one of these restrictions is implemented.
I don’t disagree with your analysis. What I’m pointing out is that I haven’t seen any workable proposals to prevent this (or another very disagreeable scenario), except for 1) a singleton AI controlling the effective laws of physics in its light cone, or 2) somehow making sure nobody but a single player (“ruler”) has the ability to create computer hardware and/or software. In which case, the universe will probably be tiled with that ruler. The incentives to create copies and the resulting evolutionary pressures are too great.
And in the context of all this, ideas like democracy are completely unworkable unless one of these restrictions is implemented.
I’m all for (1). I’ve yet to see a plausible scenario not involving a singleton AI that isn’t, on some level, horrifying.