It’s not clear that this is undesired behavior from the perspective of OpenAI. They aren’t actually putting GPT in a situation where it will make high-stakes decisions, and upholding deontological principles seems better from a PR perspective than consequentialist reasoning in these cases.
If it is merely “not clear” then this doesn’t seem to be enough for an optimistic inductive inference. I also disagree that this looks good from a PR perspective. It looks even worse than Kant’s infamous example where you allegedly aren’t allowed to lie when hiding someone from a murderer.
At least RLHF is observably generalizing in “catastrophic ways”:
You may argue that this will change in the future, but that isn’t supported by an inductive argument (ChatGPT-3.5 had the same problem).
It’s not clear that this is undesired behavior from the perspective of OpenAI. They aren’t actually putting GPT in a situation where it will make high-stakes decisions, and upholding deontological principles seems better from a PR perspective than consequentialist reasoning in these cases.
If it is merely “not clear” then this doesn’t seem to be enough for an optimistic inductive inference. I also disagree that this looks good from a PR perspective. It looks even worse than Kant’s infamous example where you allegedly aren’t allowed to lie when hiding someone from a murderer.