Groups converge as well. We can’t assume AI groups will have the barriers to convergence that human groups currently do (just as we can’t assume that AIs have the barriers to convergence that humans do).
I’m not doubting that groups converge, I am arguing that when a group achieves reflective equilibrium, that is much more meaningful than a singleton doing so, at least as long as there is variation within the group.
What you are trying to do is import positive features from the convergence of human groups (eg the fact that more options are likely to have been considered, the fact that productive discussion is likely to have happened...) into the convergence of AI groups, without spelling them out precisely. Unless we have a clear handle on what, among humans, causes these positive features, we have no real reason to suspect they will happen in AI groups as well.
The two concrete examples you gave weren’t what I had in mind. I was addressing the problem of an AI “losing” values during extrapolation,and it looks like a real reason to me. If you want to prevent an AI undergoing value drift during extrapolation, keep an extrapolated one as a reference. Two is a group minimally.
There may well be other advantages to doing rationality and ethics in groups, and yes, that needs research, and no, that isnt a show stopper.
Groups converge as well. We can’t assume AI groups will have the barriers to convergence that human groups currently do (just as we can’t assume that AIs have the barriers to convergence that humans do).
I’m not doubting that groups converge, I am arguing that when a group achieves reflective equilibrium, that is much more meaningful than a singleton doing so, at least as long as there is variation within the group.
There are bad ways to achieve group convergence.
In absolute terms, maybe, but that doesn’t stop it being relatively better.
What you are trying to do is import positive features from the convergence of human groups (eg the fact that more options are likely to have been considered, the fact that productive discussion is likely to have happened...) into the convergence of AI groups, without spelling them out precisely. Unless we have a clear handle on what, among humans, causes these positive features, we have no real reason to suspect they will happen in AI groups as well.
The two concrete examples you gave weren’t what I had in mind. I was addressing the problem of an AI “losing” values during extrapolation,and it looks like a real reason to me. If you want to prevent an AI undergoing value drift during extrapolation, keep an extrapolated one as a reference. Two is a group minimally.
There may well be other advantages to doing rationality and ethics in groups, and yes, that needs research, and no, that isnt a show stopper.