I would certainly have learned very new and very exciting facts about intelligence, facts which indeed contradict my present model of how intelligences liable to be discovered by present research paradigms work, if you showed me… how can I put this in a properly general way… that problems I thought were about searching for states that get fed into a result function and then a result-scoring function, such that the input gets an output with a high score, were in fact not about search problems like that.
This framing is helpful.
Is that what GPT-4 is doing?
No. GPT-4 is just sampling from a pre-constructed distribution. It’s not (I think) doing any search during inference. There is something more search-y happening during training, but even there we’re just making gradient updates.
Eliezer thinks that that basic model won’t scale to superintelligence. Or more specifically, he thinks that many (all?) of the problems that a superintelligence would be able to solve, to be worth of the name, have a structure such that that requires search-and-evaluation, and not sampling from a distribution.
(Unless I’m confused about something, and actually the sampling process is hiding a search and evaluation process under the hood, in a non-obvious form.)
I don’t know one way or the other, but it does seem like a relevant crux for how hard alignment will be, because it determines if it is technologically possible (socially possible is another matter) to build non-agent-like AIs that can do many of the hard parts of Science and Engineering.
This framing is helpful.
Is that what GPT-4 is doing?
No. GPT-4 is just sampling from a pre-constructed distribution. It’s not (I think) doing any search during inference. There is something more search-y happening during training, but even there we’re just making gradient updates.
Eliezer thinks that that basic model won’t scale to superintelligence. Or more specifically, he thinks that many (all?) of the problems that a superintelligence would be able to solve, to be worth of the name, have a structure such that that requires search-and-evaluation, and not sampling from a distribution.
(Unless I’m confused about something, and actually the sampling process is hiding a search and evaluation process under the hood, in a non-obvious form.)
I don’t know one way or the other, but it does seem like a relevant crux for how hard alignment will be, because it determines if it is technologically possible (socially possible is another matter) to build non-agent-like AIs that can do many of the hard parts of Science and Engineering.