FWIW, I would distinguish between the conditional task of ‘generating a hand/face accurately matching a particular natural language description’ and the unconditional task of ‘generating hands/faces’. A model can be good at unconditional generation but then bad at conditional generation because they, say, have a weak LLM or they use BPE tokenization or the description is too long. A model may know perfectly well how to model hands in many positions but then just not handle language perfectly well. One interesting recent paper on the sometimes very different levels of capabilities depending on the directions you’re going in modalities: “The Generative AI Paradox: “What It Can Create, It May Not Understand”″, West et al 2023.
FWIW, I would distinguish between the conditional task of ‘generating a hand/face accurately matching a particular natural language description’ and the unconditional task of ‘generating hands/faces’. A model can be good at unconditional generation but then bad at conditional generation because they, say, have a weak LLM or they use BPE tokenization or the description is too long. A model may know perfectly well how to model hands in many positions but then just not handle language perfectly well. One interesting recent paper on the sometimes very different levels of capabilities depending on the directions you’re going in modalities: “The Generative AI Paradox: “What It Can Create, It May Not Understand”″, West et al 2023.