>If you have technical understanding of current AIs, do you truly believe there are any major obstacles left?
I‘ve been working in AI (on and off) since 1979. I don’t work on it any more, because of my worries about alignment. I think this essay is mostly correct about short timelines.
That said, I do think there is at least one obstacle between us and dangerous superhuman AI. I haven’t seen any good work towards solving it, and I don’t see any way to solve it myself in the short term. That said, I take these facts as pretty weak evidence. Surprising capabilities keep emerging from LLMs and RL, and perhaps we will solve the problem in the next generation without even trying. Also, the argument from personal incomprehension is weak, because there are lots of people working on AI, who are smarter, more creative, and younger.
I’m of mixed feelings about your request not to mention the exact nature of the obstacle. I respect the idea of not being explicit about the nature of the Torment Nexus. But I think we could get more clarity about alignment by discussing it explicitly. I bet there are people working on it already, and I don’t think discussing it here will cause more people to work on it.
There’s no point to my remaining secretive as to my guess at the obstacle between us and superhuman AI. What I was referring to is what Jeffery Ladish called the “Agency Overhang” in his post of the same name. Now that there’s a long and well-written post on the topic, there’s no point in me being secretive about it ☹️.
>If you have technical understanding of current AIs, do you truly believe there are any major obstacles left?
I‘ve been working in AI (on and off) since 1979. I don’t work on it any more, because of my worries about alignment. I think this essay is mostly correct about short timelines.
That said, I do think there is at least one obstacle between us and dangerous superhuman AI. I haven’t seen any good work towards solving it, and I don’t see any way to solve it myself in the short term. That said, I take these facts as pretty weak evidence. Surprising capabilities keep emerging from LLMs and RL, and perhaps we will solve the problem in the next generation without even trying. Also, the argument from personal incomprehension is weak, because there are lots of people working on AI, who are smarter, more creative, and younger.
I’m of mixed feelings about your request not to mention the exact nature of the obstacle. I respect the idea of not being explicit about the nature of the Torment Nexus. But I think we could get more clarity about alignment by discussing it explicitly. I bet there are people working on it already, and I don’t think discussing it here will cause more people to work on it.
There’s no point to my remaining secretive as to my guess at the obstacle between us and superhuman AI. What I was referring to is what Jeffery Ladish called the “Agency Overhang” in his post of the same name. Now that there’s a long and well-written post on the topic, there’s no point in me being secretive about it ☹️.
https://www.lesswrong.com/posts/tqs4eEJapFYSkLGfR/the-agency-overhang