There are two issues. One is capability. Why do GPTs have the ability to use other languages? The other is why does the RLFH cause InstructGPT to follow instructions in other languages? The tweet is explicitly about the the second and your question seems to be about the second. But the follow-up tweets suggest that Leike is asking about the first (also?). I think the second is not surprising at all, conditional on the first. But the first is quite mysterious.
There are two issues. One is capability. Why do GPTs have the ability to use other languages? The other is why does the RLFH cause InstructGPT to follow instructions in other languages? The tweet is explicitly about the the second and your question seems to be about the second. But the follow-up tweets suggest that Leike is asking about the first (also?). I think the second is not surprising at all, conditional on the first. But the first is quite mysterious.