While Paul was at OpenAI, they accidentally overoptimized a GPT policy against a positive sentiment reward model. This policy evidently learned that wedding parties were the most positive thing that words can describe, because whatever prompt it was given, the completion would inevitably end up describing a wedding party.
In general, the transition into a wedding party was reasonable and semantically meaningful, although there was at least one observed instance where instead of transitioning continuously, the model ended the current story by generating a section break and began an unrelated story about a wedding party.
This example is very interesting to me for a couple of reasons:
Possibly the most interesting thing about this example is that it’s a convergent outcome across (sensory) modes, negative prompting Stable Diffusion on sinister things gives a similar result:
Possibly the most interesting thing about this example is that it’s a convergent outcome across (sensory) modes, negative prompting Stable Diffusion on sinister things gives a similar result:
https://twitter.com/jd_pressman/status/1567571888129605632
Get married, drive a white/silver car, and then buy a house near roads, greenery, and water. Got it.