“OpenAI appears to subscribe to that philosophy [of ‘bothsidesism’]. Also there seems to be a ‘popular opinion determines attention and truth’ thing here?”
OpenAI’s approach is well-intentioned but crude and might be counterproductive. The goal they should be aiming at is something best constructed as “have good moral and political epistemology”, something people are notoriously bad at by default.
Being vaguely both sidesist is a solution you see a lot with human institutions who don’t want to look biased so it’s not an unusually bad solution by any means but not good enough for high stakes situations.
What should the goal be? Instead of just presenting “both sides”, I think we should focus on making the AI acutely aware of the distinction between facts and values and especially in cases where there are values conflicts bringing that up. Making sure the model explicitly identifies and separates empirical claims from value judgments means that we can achieve better epistemics without resorting to false equivalences. Maybe for sufficiently unambiguous values that everyone shares we don’t want to do this but I think you should make the model biased towards saying “if X is what you value then do Y” whenever possible.
“This is weird. Why should the model need to spend tokens affirming that the user can believe what they wish? If information changes someone’s mind, that is a feature.”
Once again I think what they’re getting at is in principle good. I’d phrase it as the model should be biased towards being decision support orientated not persuasive. The strategy of writing persuasive content and then tacking on “but believe what you want!” is indeed a cheap hack that doesn’t solve the underlying issue. It would probably be better for the model to explicitly say when it’s being persuasive and when it’s not and err on the side of not persuading whenever possible but always be “meta honest” and upfront about what it thinks. That way we can at least be more assured it’s just being used for decision assistance when that’s all we want.
OpenAI’s approach is well-intentioned but crude and might be counterproductive. The goal they should be aiming at is something best constructed as “have good moral and political epistemology”, something people are notoriously bad at by default.
Being vaguely both sidesist is a solution you see a lot with human institutions who don’t want to look biased so it’s not an unusually bad solution by any means but not good enough for high stakes situations.
What should the goal be? Instead of just presenting “both sides”, I think we should focus on making the AI acutely aware of the distinction between facts and values and especially in cases where there are values conflicts bringing that up. Making sure the model explicitly identifies and separates empirical claims from value judgments means that we can achieve better epistemics without resorting to false equivalences. Maybe for sufficiently unambiguous values that everyone shares we don’t want to do this but I think you should make the model biased towards saying “if X is what you value then do Y” whenever possible.
Once again I think what they’re getting at is in principle good. I’d phrase it as the model should be biased towards being decision support orientated not persuasive. The strategy of writing persuasive content and then tacking on “but believe what you want!” is indeed a cheap hack that doesn’t solve the underlying issue. It would probably be better for the model to explicitly say when it’s being persuasive and when it’s not and err on the side of not persuading whenever possible but always be “meta honest” and upfront about what it thinks. That way we can at least be more assured it’s just being used for decision assistance when that’s all we want.