My point was that no one so deeply understands human value formation that they can confidently rule out the possibility of adapting a similar process to ASI. It seems you agree with that (or at least our lack of understanding)? Do you think our current understanding is sufficient to confidently conclude that human-adjacent / inspired approaches will not scale beyond human level?
I think it depends on which subprocess you consider. Some subprocesses can be ruled out as viable with less information, others require more information.
And yes, without having an enumeration of all the processes, one cannot know that there isn’t some unknown process that scales more easily.
My point was that no one so deeply understands human value formation that they can confidently rule out the possibility of adapting a similar process to ASI. It seems you agree with that (or at least our lack of understanding)? Do you think our current understanding is sufficient to confidently conclude that human-adjacent / inspired approaches will not scale beyond human level?
I think it depends on which subprocess you consider. Some subprocesses can be ruled out as viable with less information, others require more information.
And yes, without having an enumeration of all the processes, one cannot know that there isn’t some unknown process that scales more easily.