Mitchell_Porter comments on AI #23: Fundamental Problems with RLHF

[Mitchell_Porter]

send chatGPT the string ` a` repeated 1000 times

This seems to cause some sort of data leak

I don’t think so. It’s just generating a random counterfactual document, as if there was no system prompt.

[nostalgebraist]

Agreed. I wrote some more notes about the phenomenon here.