Paperclips vs obelisks does make the bargaining harder because clippy would be offered fewer expected paperclips.
My current guess is we survive if our CEV puts a steep premium on that. Of course, such hopes of trade ex machina shouldn’t affect how we orient to the alignment problem, even if they affect our personal lives. We should still play to win.
Paperclips vs obelisks does make the bargaining harder because clippy would be offered fewer expected paperclips.
But Clippy also controls fewer expected universes, so the relative bargaining positions of humans VS UFAIs remain the same(compared to a scenario in which all UFAIs had the same value system)
Ah right, because Clippy has less measure, and so has less to offer, so less needs to be offered to it. Nice catch! Guess I’ve been sort of heeding Nate’s advice not to think much about this. :)
Of course, there would still be significant overhead from trading with and/or outbidding sampled plethoras of UFAIs, vs the toy scenario where it’s just Clippy.
I currently suspect we still get more survival measure from aliens in this branch who solved their alignment problems and have a policy of offering deals to UFAIs that didn’t kill their biological boot loaders. Such aliens need not be motivated by compassion to the extent that aboriginals form a Schelling bloc, handwave appendagewave. (But we should still play to win, like they did.)
Paperclips vs obelisks does make the bargaining harder because clippy would be offered fewer expected paperclips.
My current guess is we survive if our CEV puts a steep premium on that. Of course, such hopes of trade ex machina shouldn’t affect how we orient to the alignment problem, even if they affect our personal lives. We should still play to win.
But Clippy also controls fewer expected universes, so the relative bargaining positions of humans VS UFAIs remain the same(compared to a scenario in which all UFAIs had the same value system)
Ah right, because Clippy has less measure, and so has less to offer, so less needs to be offered to it. Nice catch! Guess I’ve been sort of heeding Nate’s advice not to think much about this. :)
Of course, there would still be significant overhead from trading with and/or outbidding sampled plethoras of UFAIs, vs the toy scenario where it’s just Clippy.
I currently suspect we still get more survival measure from aliens in this branch who solved their alignment problems and have a policy of offering deals to UFAIs that didn’t kill their biological boot loaders. Such aliens need not be motivated by compassion to the extent that aboriginals form a Schelling bloc, handwave appendagewave. (But we should still play to win, like they did.)