just read “Situational Awareness”—it definitely woke me up. AGI is real, and very plausibly (55%?) happening within this decade. I need to stop sleep walking and get serious about contributing within the next two years.
First, some initial thoughts on the essay
Very “epic” and (self?) aggrandizing. If you believe the conclusions, its not unwarranted, but I worry a bit about narratives that satiate some sense of meaning and self-importance. (That counter-reaction is probably much stronger though, and on the margin it seems really valuable to “full-throatily” take on the prospect of AGI within the next 3-5 years)
I think most of my uncertainty lies in the “unhobbling” type algorithmic progress, this seems especially unpredictable, and may require lots of expensive experimentation if e.g. the relevant capabilities to get some meta-cognitive process to train only emerge at a certain scale. I’m vaguely thinking back to Paul’s post on self-driving cars and AGI timelines. Maybe this is all priced in though—there’s way more research investment, and tech path seems relatively straight forward if we can apply enough experimentation. Still, research is hard, takes a lot of serial time, and is less predictable that e.g. industrial processes. (I’m kind just saying this though, not actually sure how to quantify this, I’m pretty sure people have analsysis of insight generation or whatever, idk...)
just read “Situational Awareness”—it definitely woke me up. AGI is real, and very plausibly (55%?) happening within this decade. I need to stop sleep walking and get serious about contributing within the next two years.
First, some initial thoughts on the essay
Very “epic” and (self?) aggrandizing. If you believe the conclusions, its not unwarranted, but I worry a bit about narratives that satiate some sense of meaning and self-importance. (That counter-reaction is probably much stronger though, and on the margin it seems really valuable to “full-throatily” take on the prospect of AGI within the next 3-5 years)
I think most of my uncertainty lies in the “unhobbling” type algorithmic progress, this seems especially unpredictable, and may require lots of expensive experimentation if e.g. the relevant capabilities to get some meta-cognitive process to train only emerge at a certain scale. I’m vaguely thinking back to Paul’s post on self-driving cars and AGI timelines. Maybe this is all priced in though—there’s way more research investment, and tech path seems relatively straight forward if we can apply enough experimentation. Still, research is hard, takes a lot of serial time, and is less predictable that e.g. industrial processes. (I’m kind just saying this though, not actually sure how to quantify this, I’m pretty sure people have analsysis of insight generation or whatever, idk...)