What is respectable to one audience might not be for another. Status is not the concern here; truthfulness is. And all of this might just not be a large update on the probability of existential catasthrophe.
The Bing trainwreck likely tells us nothing about how hard it is to align future models, that we didn’t know earlier. The most reasonable explanation so far points to it being an issue of general accident-prevention abilities, the lack of any meaningful testing, and culture in Microsoft and/or OpenAI.
I genuinely hope that most wrong actions here were made by Microsoft, as their sanity on the technical side should not be a major concern as long as OpenAI makes the calls about training and deployment of their future models. We already knew that Microsoft is far from the level of the most competent actors in the space.
A post-mortem from OpenAI clearly explaining their role in this debacle is definitely required.
(Some people on Twitter suggest that OpenAI might have let this thing happen on purpose. I don’t believe it’s true, but if anyone there is thinking of such dark arts in the future: please don’t. It’s not dignified, and similar actions have the tendency to backfire.)
What is respectable to one audience might not be for another. Status is not the concern here; truthfulness is. And all of this might just not be a large update on the probability of existential catasthrophe.
The Bing trainwreck likely tells us nothing about how hard it is to align future models, that we didn’t know earlier.
The most reasonable explanation so far points to it being an issue of general accident-prevention abilities, the lack of any meaningful testing, and culture in Microsoft and/or OpenAI.
I genuinely hope that most wrong actions here were made by Microsoft, as their sanity on the technical side should not be a major concern as long as OpenAI makes the calls about training and deployment of their future models. We already knew that Microsoft is far from the level of the most competent actors in the space.
A post-mortem from OpenAI clearly explaining their role in this debacle is definitely required.
(Some people on Twitter suggest that OpenAI might have let this thing happen on purpose. I don’t believe it’s true, but if anyone there is thinking of such dark arts in the future: please don’t. It’s not dignified, and similar actions have the tendency to backfire.)