We’ve been told that a General AI will have power beyond any despot known to history.
If that will be then we are doomed. Power corrupts. In theory an AI, not being human, might resist the corruption, but I wouldn’t bet on that. I do not think it is a mere peculiarity of humanity that we are vulnerable to corruption.
We humans are kept in check by each other. We might therefore hope, and attempt to engineer, a proliferation of self-improving AIs, to form a society and to keep each other in check. With luck, cooperative AIs might be more successful at improving themselves—just as honest folk are for the most part more successful than criminals—and thus tend for the most part to out-pace the would-be despots.
As far as how a society of AIs would relate to humans, there are various possibilities. One dystopia imagines that humans will be treated like lower animals, but this is not necessarily what will happen. Animals are not merely dumb, but unable to take part in mutual respect of rights. We humans will always be able to and so might well remain forever recipients of an AI respect for our rights however much they evolve past us. We may of course be excluded from aspects of AI society which we humans are not able to handle, just as we exclude animals from rights. We may never earn hyper-rights, whatever those may be. But we might retain our rights.
We’ve been told that a General AI will have power beyond any despot known to history.
If that will be then we are doomed. Power corrupts. In theory an AI, not being human, might resist the corruption, but I wouldn’t bet on that. I do not think it is a mere peculiarity of humanity that we are vulnerable to corruption.
We humans are kept in check by each other. We might therefore hope, and attempt to engineer, a proliferation of self-improving AIs, to form a society and to keep each other in check. With luck, cooperative AIs might be more successful at improving themselves—just as honest folk are for the most part more successful than criminals—and thus tend for the most part to out-pace the would-be despots.
As far as how a society of AIs would relate to humans, there are various possibilities. One dystopia imagines that humans will be treated like lower animals, but this is not necessarily what will happen. Animals are not merely dumb, but unable to take part in mutual respect of rights. We humans will always be able to and so might well remain forever recipients of an AI respect for our rights however much they evolve past us. We may of course be excluded from aspects of AI society which we humans are not able to handle, just as we exclude animals from rights. We may never earn hyper-rights, whatever those may be. But we might retain our rights.