There are two levels of search that we need to think about here: at the outer level, we use machine learning to search for an AI design that works at all. Then, at the inner level, when we deploy this AI into the world, it most likely uses search to find good explanations of its sensor data (i.e. to understand things that we didn’t put in by hand) and most likely also uses search to find plans that lead to fulfilment of its goals.
It seems to me that design at least needs to be part of the story for how we do the outer-level construction of a basic AI architecture. Any good architecture very likely then uses search in some way at the inner level.
OK, well I spend most of my time thinking about a particular AGI architecture (12 etc.) in which the learning algorithm is legible and hand-coded … and let me tell you, in that case, all the problems of AGI safety and alignment are still really really hard, including the “inaccessible information” stuff that Paul was talking about here.
If you’re saying that it would be even worse if, on top of that, the learning algorithm itself is opaque, because it was discovered from a search through algorithm-space … well OK, yeah sure, that does seem even worse.
Thanks for this question. No you’re not confused!
There are two levels of search that we need to think about here: at the outer level, we use machine learning to search for an AI design that works at all. Then, at the inner level, when we deploy this AI into the world, it most likely uses search to find good explanations of its sensor data (i.e. to understand things that we didn’t put in by hand) and most likely also uses search to find plans that lead to fulfilment of its goals.
It seems to me that design at least needs to be part of the story for how we do the outer-level construction of a basic AI architecture. Any good architecture very likely then uses search in some way at the inner level.
Evan wrote a great sequence about inner and outer optimization
OK, well I spend most of my time thinking about a particular AGI architecture (1 2 etc.) in which the learning algorithm is legible and hand-coded … and let me tell you, in that case, all the problems of AGI safety and alignment are still really really hard, including the “inaccessible information” stuff that Paul was talking about here.
If you’re saying that it would be even worse if, on top of that, the learning algorithm itself is opaque, because it was discovered from a search through algorithm-space … well OK, yeah sure, that does seem even worse.