Also related: the best poem-writing AIs are general-purpose language models that have been directed towards writing poems.
Maybe I’m missing something, but this seems like a non-sequitur to me? Or missing the point?
Eliezer expect that that the hypothetical AI that satisfies strawberry alignment will have general enough capabilities to invent novel science for an engineering task (that’s why this task was selected as an example).
Regardless of whether we construct an AI that has “duplicate this strawberry” as fundamental core value or we create a corrigible AGI and instruct it to duplicate a strawberry, the important point is that (Eliezer claims) we don’t know how, to do either, currently, without world-destroying side-effects.
Maybe I’m missing something, but this seems like a non-sequitur to me? Or missing the point?
Eliezer expect that that the hypothetical AI that satisfies strawberry alignment will have general enough capabilities to invent novel science for an engineering task (that’s why this task was selected as an example).
Regardless of whether we construct an AI that has “duplicate this strawberry” as fundamental core value or we create a corrigible AGI and instruct it to duplicate a strawberry, the important point is that (Eliezer claims) we don’t know how, to do either, currently, without world-destroying side-effects.