I am not claiming this will be the method used, I am claiming it is obviously achievable and something at least this good will very likely be tried by current AI labs within 3-5 years, conditional on sufficient funding. (if a large llm costs 2 million to train, each AGI candidate would take probably 10 million to train, though I expect many AGI candidates will reuse modules from failed candidates to lower the cost. So a search of 1000 candidates would cost 10 billion, maybe a bit less. Easily possible if AI labs can show revenue in the next 3-5 years)
I do not think this will foom overall, and in the comment thread I explain why, but the intelligence component is self amplifying. It would foom if compute, accurate scientific data, and robotics were all available in unlimited quanties.
I am curious if you think the intelligence self amplifying is possible or not.
In this post below I outline what I think is a grounded, constructible RSI algorithm using current techniques:
https://www.lesswrong.com/posts/Aq82XqYhgqdPdPrBA/?commentId=Mvyq996KxiE4LR6ii
This post I cite the papers I drew from to construct the method: https://www.lesswrong.com/posts/Aq82XqYhgqdPdPrBA/full-transcript-eliezer-yudkowsky-on-the-bankless-podcast?commentId=3AJiGHnweC7z52D6v
I am not claiming this will be the method used, I am claiming it is obviously achievable and something at least this good will very likely be tried by current AI labs within 3-5 years, conditional on sufficient funding. (if a large llm costs 2 million to train, each AGI candidate would take probably 10 million to train, though I expect many AGI candidates will reuse modules from failed candidates to lower the cost. So a search of 1000 candidates would cost 10 billion, maybe a bit less. Easily possible if AI labs can show revenue in the next 3-5 years)
I do not think this will foom overall, and in the comment thread I explain why, but the intelligence component is self amplifying. It would foom if compute, accurate scientific data, and robotics were all available in unlimited quanties.