Thanks. OK I will put some more general thoughts, have to go back a few steps.
To me the more general alignment problem is AI gives humanity ~10,000 years of progress and probably irreversible change in ~1-10 years. To me the issue is how do you raise humans intelligence from that given by biology to that given by the limits of physics in a way that is identify preserving as much as possible. Building AI seems to be the worst way to do that. If I had a fantasy way it would be say increase everyone’s IQ by 10 points per year for 100+ years until we reach the limit.
We can’t do that but that is why I mentioned WBE, my desire would be to stop AGI, get human mind uploading to work, then let those WBE raise their IQ in parallel. Their agreed upon values would be humanities values by definition then.
If our goal is Coherent Extrapolated Volition or something similar for humanity then how can we achieve that if we don’t increase the IQ of humans (or descendants they identify with)? How can we even know what our own desires/values are at increasing IQ’s if we don’t directly experience them.
I have an opinion what successful alignment looks like to me but is it very different for other people? We can all agree what bad is.
Thanks. OK I will put some more general thoughts, have to go back a few steps.
To me the more general alignment problem is AI gives humanity ~10,000 years of progress and probably irreversible change in ~1-10 years. To me the issue is how do you raise humans intelligence from that given by biology to that given by the limits of physics in a way that is identify preserving as much as possible. Building AI seems to be the worst way to do that. If I had a fantasy way it would be say increase everyone’s IQ by 10 points per year for 100+ years until we reach the limit.
We can’t do that but that is why I mentioned WBE, my desire would be to stop AGI, get human mind uploading to work, then let those WBE raise their IQ in parallel. Their agreed upon values would be humanities values by definition then.
If our goal is Coherent Extrapolated Volition or something similar for humanity then how can we achieve that if we don’t increase the IQ of humans (or descendants they identify with)? How can we even know what our own desires/values are at increasing IQ’s if we don’t directly experience them.
I have an opinion what successful alignment looks like to me but is it very different for other people? We can all agree what bad is.