I don’t think Sydney poses an existential risk. I don’t think it’s ready for release, in the sense of “is releasing it a good idea from Microsoft’s perspective?”.
Of course, they didn’t actually “release it” in the sense of making it publicly available to everyone at once. I suppose it’s possible OAI / MS had a good understanding of Bing Chat’s misbehaviors and decided that a closed beta was the best way forwards, but it seems unlikely to me.
I suppose it’s possible OAI / MS had a good understanding of Bing Chat’s misbehaviors and decided that a closed beta was the best way forwards, but it seems unlikely to me.
...Why? A closed beta is a closed beta for this exact specific reason, because it’s kind of useful but you need a small group of users to test it and give you feedback first.
Seems like Bing Chat is at the stage where you hire contractors to interact with the AI, and have them agree not to post screenshots of the interaction on Reddit / Twitter. This avoids the reputational risk associated with putting your name on a product that insults, lies to and manipulates its users, and also avoids the risk of people pigeonholing Microsoft’s LM-agumented search products as “that insane offshoot of ChatGPT”.
I don’t think it’s ready for release, in the sense of “is releasing it a good idea from Microsoft’s perspective?”.
You sure about that?
EDIT: to clarify, I don’t claim that this price action is decisive. Hard to attribute price movements to specific events, and the market can be wrong, especially in the short term. But it seems suggestive that the market likes Microsoft’s choice.
I don’t think Sydney poses an existential risk. I don’t think it’s ready for release, in the sense of “is releasing it a good idea from Microsoft’s perspective?”.
It’s not just spooky agentic behavior (though that’s more concerning from an evidentiary point of view), it’s that they’ve built a search product that not-very-infrequently lies to, gaslights, and insults the user. E.g., the post in r/bing I’m disappointed at how bad Bing AI Chat can be, and I think a wide rollout would be a mistake right now.
Of course, they didn’t actually “release it” in the sense of making it publicly available to everyone at once. I suppose it’s possible OAI / MS had a good understanding of Bing Chat’s misbehaviors and decided that a closed beta was the best way forwards, but it seems unlikely to me.
...Why? A closed beta is a closed beta for this exact specific reason, because it’s kind of useful but you need a small group of users to test it and give you feedback first.
Seems like Bing Chat is at the stage where you hire contractors to interact with the AI, and have them agree not to post screenshots of the interaction on Reddit / Twitter. This avoids the reputational risk associated with putting your name on a product that insults, lies to and manipulates its users, and also avoids the risk of people pigeonholing Microsoft’s LM-agumented search products as “that insane offshoot of ChatGPT”.
Mainstream advice for launching new software products is to release quickly and then rapidly iterate based on user feedback.
See also: https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned?commentId=xXcqbXPbGWwcqLskB
You sure about that?
EDIT: to clarify, I don’t claim that this price action is decisive. Hard to attribute price movements to specific events, and the market can be wrong, especially in the short term. But it seems suggestive that the market likes Microsoft’s choice.
I dunno about that. If we’re reading the market like a crystal ball, there’s been interesting movement since:
Fair.