A greedy/unethical person scenario is not relevant for two reasons:
...
(1) it’s not apparent that an AI can be built that gives significant power, for the hidden complexity of wishes reasons
A powerful machine couldn’t give a human “significant power”?!? Wouldn’t Page and Brin be counter-examples?
(2) if someone has taken over the world, the problem is still the same: what’s next, and how to avoid destroying the world?
One problem with an unethical ruler is that they might trash some fraction in the world in the process of rising to power. For those who get trashed, what the ruler does afterwards may be a problem they are not around to worry about.
...
A powerful machine couldn’t give a human “significant power”?!? Wouldn’t Page and Brin be counter-examples?
One problem with an unethical ruler is that they might trash some fraction in the world in the process of rising to power. For those who get trashed, what the ruler does afterwards may be a problem they are not around to worry about.