tailcalled comments on DPO/​PPO-RLHF on LLMs incentivizes sycophancy, exaggeration and deceptive hallucination, but not misaligned powerseeking

[tailcalled]

It depends on both.