Can you expand on this concisely inline? I agree with the comment you’re replying to strongly and think it has been one of miri’s biggest weaknesses in the past decade that they didn’t build the fortitude to be able to read existing work without becoming confused by its irrelevance. But I also think your and Abram’s research direction intuitions seem like some of the most important in the field right now, alongside wentworth. I’d like to understand what it is that has held you back from speed reading external work for hunch seeding for so long. To me, it seems like solving from scratch is best done not from scratch, if that makes sense. Don’t defer to what you read.
I’d like to understand what it is that has held you back from speed reading external work for hunch seeding for so long.
Well currently I’m not really doing alignment research. My plans / goals / orientation / thinking style have changed over the years, so I’ve read stuff or tried to read stuff more or less during different periods. When I’m doing my best thinking, yes, I read things for idea seeding / as provocations, but it’s only that—I most certainly am not speed reading, the opposite really: read one paragraph, think for an hour and then maybe write stuff. And I’m obviously not reading some random ML paper, jesus christ. Philosophy, metamathematics, theoretical biology, linguistics, psychology, ethology, … much more interesting and useful.
To me, it seems like solving from scratch is best done not from scratch, if that makes sense.
Absolutely, I 100% agree, IIUC. I also think:
A great majority of the time, when people talk about reading stuff (to “get up to speed”, to “see what other people have done on the subject”, to “get inspiration”, to “become more informed”, to “see what approaches/questions there are”...), they are not doing this “from scratch not from scratch” thing.
“the typical EA / rationalist, especially in AI safety research (most often relatively young and junior in terms of research experience / taste)” is absolutely and pretty extremely erring on the side of failing to ever even try to solve the actual problem at all.
Don’t defer to what you read.
Yeah, I generally agree (https://tsvibt.blogspot.com/2022/09/dangers-of-deferrence.html), though you probably should defer about some stuff at least provisionally (for example, you should probably try out, for a while, the stance of deferring to well-respected philosophers about what questions are interesting).
I think it’s just not appreciated how much people defer to what they read. Specifically, there’s a lot of frame deference. This is usually fine and good in lots of contexts (you don’t need to, like, question epistemology super hard to become a good engineer, or question whether we should actually be basing our buildings off of liquid material rather than solid material or something). It’s catastrophic in AGI alignment, because our frames are bad.
Can you expand on this concisely inline? I agree with the comment you’re replying to strongly and think it has been one of miri’s biggest weaknesses in the past decade that they didn’t build the fortitude to be able to read existing work without becoming confused by its irrelevance. But I also think your and Abram’s research direction intuitions seem like some of the most important in the field right now, alongside wentworth. I’d like to understand what it is that has held you back from speed reading external work for hunch seeding for so long. To me, it seems like solving from scratch is best done not from scratch, if that makes sense. Don’t defer to what you read.
Well currently I’m not really doing alignment research. My plans / goals / orientation / thinking style have changed over the years, so I’ve read stuff or tried to read stuff more or less during different periods. When I’m doing my best thinking, yes, I read things for idea seeding / as provocations, but it’s only that—I most certainly am not speed reading, the opposite really: read one paragraph, think for an hour and then maybe write stuff. And I’m obviously not reading some random ML paper, jesus christ. Philosophy, metamathematics, theoretical biology, linguistics, psychology, ethology, … much more interesting and useful.
Absolutely, I 100% agree, IIUC. I also think:
A great majority of the time, when people talk about reading stuff (to “get up to speed”, to “see what other people have done on the subject”, to “get inspiration”, to “become more informed”, to “see what approaches/questions there are”...), they are not doing this “from scratch not from scratch” thing.
“the typical EA / rationalist, especially in AI safety research (most often relatively young and junior in terms of research experience / taste)” is absolutely and pretty extremely erring on the side of failing to ever even try to solve the actual problem at all.
Yeah, I generally agree (https://tsvibt.blogspot.com/2022/09/dangers-of-deferrence.html), though you probably should defer about some stuff at least provisionally (for example, you should probably try out, for a while, the stance of deferring to well-respected philosophers about what questions are interesting).
I think it’s just not appreciated how much people defer to what they read. Specifically, there’s a lot of frame deference. This is usually fine and good in lots of contexts (you don’t need to, like, question epistemology super hard to become a good engineer, or question whether we should actually be basing our buildings off of liquid material rather than solid material or something). It’s catastrophic in AGI alignment, because our frames are bad.
Not sure I answered your question.