David Brin suggests that some kind of political system populated with humans and diverse but imperfectly rational and friendly AIs would evolve in a satisfactory direction for humans.
Brin:
a great diversity of mega minds, contending with each other, politely, and under civil rules, but vigorously nonetheless, holding each other to account and ensuring everything is above-board.
These AIs are all friendly, because they have the constraint of not using force upon each other, i.e. you would have to solve the friendliness structure problem and put in some content about not destroying the other AIs.
Brin:
But the real reason to do this is far more pragmatic. If the new AIs feel vested in a civilization that considers them “human” then they may engage in our give and take process of shining light upon delusion. Each others delusions, above all. Reciprocal accountability—extrapolated to a higher level—may thus maintain the core innovation of our civilization. It’s central and vital insight. And thus, we may find that our new leaders—our godlike grandchildren—will still care about us… and keep trying to explain.
Brin wants to create AI that is conditionally altruistic, and then be nice to it, and then hope that it will be nice back, whilst also having more than one such AI around, each with distinct goals, but also impose the constraint upon them that they have to not use force upon each other.
I’m still trying to figure out what reason the “thus” in “And thus, we may find that our new leaders—our godlike grandchildren—will still care about us...” refers to. When you have two “thus” statements in a row like that, “A thus B thus C”, it means B justifies C. In this case, “Reciprocal accountability may maintain the core innovation of our civilization” is B, which is somehow supposed to justify AIs caring about us.
I’m still trying to figure out what reason the “thus” in “And thus, we may find that our new leaders—our godlike grandchildren—will still care about us...” refers to.
I think the argument is that humans will be instrumentally useful because they will point out machines’ irrationalities (!) plus the fact that the machines come with reciprocal altruism.
Brin:
Source
These AIs are all friendly, because they have the constraint of not using force upon each other, i.e. you would have to solve the friendliness structure problem and put in some content about not destroying the other AIs.
Brin:
Brin wants to create AI that is conditionally altruistic, and then be nice to it, and then hope that it will be nice back, whilst also having more than one such AI around, each with distinct goals, but also impose the constraint upon them that they have to not use force upon each other.
Eliezer, this must be painful for you to read.
I’m still trying to figure out what reason the “thus” in “And thus, we may find that our new leaders—our godlike grandchildren—will still care about us...” refers to. When you have two “thus” statements in a row like that, “A thus B thus C”, it means B justifies C. In this case, “Reciprocal accountability may maintain the core innovation of our civilization” is B, which is somehow supposed to justify AIs caring about us.
I think the argument is that humans will be instrumentally useful because they will point out machines’ irrationalities (!) plus the fact that the machines come with reciprocal altruism.
The word thus does not only mean ‘therefore’. It can also mean ‘in this way’, which I believe is the intended sense here.