I just tried that, and it kinda worked. Specifically, it worked to get gpt2-small to output text that structurally looks like French, but not to coherently speak French.
Although I then just tried feeding the base gpt2-small a passage in French, and its completions there were also incoherent, so I think it’s just that that version hasn’t seen enough French to speak it very well.
We could not find a “speak in French” vector after about an hour of effort, but it’s possible we missed something straightforward
Did you try 10 or 20 simple French phrases with a positive sign and their translations with a negative sign?
Also try 1000 english words and 1000 french translations in case scale is the problem.
Also try:
“The following text is in English: ′ ”
“The following text is in French: ′ ”
with the second phrase written itself in French.
I just tried that, and it kinda worked. Specifically, it worked to get gpt2-small to output text that structurally looks like French, but not to coherently speak French.
Although I then just tried feeding the base gpt2-small a passage in French, and its completions there were also incoherent, so I think it’s just that that version hasn’t seen enough French to speak it very well.