Did Eliezer give any details about what exactly was wrong about Paul’s excitement? Might just be an intuition gained from years of experience, but the more details we know the better, I think.
Eliezer has an opaque intuition that weird recursion is hard to get right on the first try. I want to interview him and write this up, but I don’t know if I’m capable of asking the right questions. Probably someone should do it.
Eliezer thinks people tend to be too optimistic in general
I’ve heard other people have an intuition that IDA is unaligned because HCH is unaligned because real human bureaucracies are unaligned
I’ll add that when I asked John Wentworth why he was IDA-bearish, he mentioned the inefficiency of bureaucracies and told me to read the following post to learn why interfaces and coordination are hard: Interfaces as a Scarce Resource.
Did Eliezer give any details about what exactly was wrong about Paul’s excitement? Might just be an intuition gained from years of experience, but the more details we know the better, I think.
Some scattered thoughts in this direction:
this post
Eliezer has an opaque intuition that weird recursion is hard to get right on the first try. I want to interview him and write this up, but I don’t know if I’m capable of asking the right questions. Probably someone should do it.
Eliezer thinks people tend to be too optimistic in general
I’ve heard other people have an intuition that IDA is unaligned because HCH is unaligned because real human bureaucracies are unaligned
I found this comment where Eliezer has detailed criticism of Paul’s alignment agenda including finding problems with “weird recursion”
I’ll add that when I asked John Wentworth why he was IDA-bearish, he mentioned the inefficiency of bureaucracies and told me to read the following post to learn why interfaces and coordination are hard: Interfaces as a Scarce Resource.