I agree this list doesn’t seem to contain much unpublished material, and I think the main value of having it in one numbered list is that “all of it is in one, short place”, and it’s not an “intro to computers can think” and instead is “these are a bunch of the reasons computers thinking is difficult to align”.
The thing that I understand to be Eliezer’s “main complaint” is something like: “why does it seem like No One Else is discovering new elements to add to this list?”. Like, I think Risks From Learned Optimization was great, and am glad you and others wrote it! But also my memory is that it was “prompted” instead of “written from scratch”, and I imagine Eliezer reading it more had the sense of “ah, someone made ‘demons’ palatable enough to publish” instead of “ah, I am learning something new about the structure of intelligence and alignment.”
[I do think the claim that Eliezer ‘figured it out from the empty string’ doesn’t quite jive with the Yudkowsky’s Coming of Age sequence.]
Nearly empty string of uncommon social inputs. All sorts of empirical inputs, including empirical inputs in the social form of other people observing things.
It’s also fair to say that, though they didn’t argue me out of anything, Moravec and Drexler and Ed Regis and Vernor Vinge and Max More could all be counted as social inputs telling me that this was an important thing to look at.
I agree this list doesn’t seem to contain much unpublished material, and I think the main value of having it in one numbered list is that “all of it is in one, short place”, and it’s not an “intro to computers can think” and instead is “these are a bunch of the reasons computers thinking is difficult to align”.
The thing that I understand to be Eliezer’s “main complaint” is something like: “why does it seem like No One Else is discovering new elements to add to this list?”. Like, I think Risks From Learned Optimization was great, and am glad you and others wrote it! But also my memory is that it was “prompted” instead of “written from scratch”, and I imagine Eliezer reading it more had the sense of “ah, someone made ‘demons’ palatable enough to publish” instead of “ah, I am learning something new about the structure of intelligence and alignment.”
[I do think the claim that Eliezer ‘figured it out from the empty string’ doesn’t quite jive with the Yudkowsky’s Coming of Age sequence.]
Nearly empty string of uncommon social inputs. All sorts of empirical inputs, including empirical inputs in the social form of other people observing things.
It’s also fair to say that, though they didn’t argue me out of anything, Moravec and Drexler and Ed Regis and Vernor Vinge and Max More could all be counted as social inputs telling me that this was an important thing to look at.