I second the question, all it needs to get superhuman intelligence is reflective decision theory?
Well, that plus a way to efficiently approximately compute it in real life scenarios, of course.
But my question was more given that, I thought there was still work to be done as far as precisely formulating what exactly it is that it should do.
Well, that plus a way to efficiently approximately compute it in real life scenarios, of course.
But my question was more given that, I thought there was still work to be done as far as precisely formulating what exactly it is that it should do.