Not really. To be clear, I am criticizing the argument Eliezer tends to make. There can be flaws in that argument and we can still be doomed. I am saying his stated confidence is too high because even if alignment is as hard as he thinks, A.I. itself may be harder than he thinks, and this would give us more time to take alignment seriously.
In the second scenario I outlined (say, scenario B) where gains to intelligence feed back into hardware improvements but not drastic software improvements, multiple tries may be possible. On the whole I think that this is not very plausible (1/3 at most), and the other two scenarios look like they only give us one try.
Well, if we only have one try, extra time does not help, unless alignment is only an incremental extra on AI, and not a comparably hard extra effort. If we have multiple tries, yes, there is a chance. I don’t think that at this point we have enough clue as to how it is likely to go. Certainly LLMs have been a big surprise.
Not really. To be clear, I am criticizing the argument Eliezer tends to make. There can be flaws in that argument and we can still be doomed. I am saying his stated confidence is too high because even if alignment is as hard as he thinks, A.I. itself may be harder than he thinks, and this would give us more time to take alignment seriously.
In the second scenario I outlined (say, scenario B) where gains to intelligence feed back into hardware improvements but not drastic software improvements, multiple tries may be possible. On the whole I think that this is not very plausible (1/3 at most), and the other two scenarios look like they only give us one try.
Well, if we only have one try, extra time does not help, unless alignment is only an incremental extra on AI, and not a comparably hard extra effort. If we have multiple tries, yes, there is a chance. I don’t think that at this point we have enough clue as to how it is likely to go. Certainly LLMs have been a big surprise.