The point of the defection/cooperation thing isn’t just that cooperation is a kindness to Sam Altman personally, which can be overridden by the greater good. The point is that generally cooperative behavior, and generally high amounts of trust, can make everyone better off. If it was true as you said that:
he’s a head state in control of WMDs and should be (and expect to be) treated as such
and as a consequence, he e.g. expected someone to record him during the Q&A, then he would presumably not have done the Q&A in the first place, or would have shared much less information. This would have lead to humanity learning less about OpenAI’s plans.
And this is definitely not a single-shot interaction. This was Sam’s second Q&A at an ACX meetup, and there was no reason to expect it to be the last. Moreover, there’s been a lot of interactions between the alignment community (including some who write on LW) and OpenAI in the past. And given that OpenAI’s decisions about alignment-related things matter a lot (as you say) it seems important to keep up good relations and high degrees of trust.
honestly curious about reasons for downvotes if anyone is willing to share
I initially downvoted, have since retracted it. Since trust/cooperation can be quite fragile and dependent on expectations about how people will behave, I became worried when I read you as basically announcing that you and other people should and will defect in the future. And I wanted a quick way to mark disagreement with that, to communicate that this isn’t a generally accepted point of view. But your point was phrased in a perfectly civil manner, so I should really have just taken the time to write a response, sorry.
The point about it being an iterated prisoner’s dilemma is a good one, and I would rather there be more such ACX instances where he shares even more of his thinking due to our cooperative/trustworthy behavior, than this to be the last one or have the next ones be filtered PR-speak.
A small number of people in the alignment community repeatedly getting access to better information and being able to act on it beats the value of this one single post staying open to the world. And even in the case of “the cat being out of the bag,” hiding/removing the post would probably do good as a gesture of cooperation.
The point of the defection/cooperation thing isn’t just that cooperation is a kindness to Sam Altman personally, which can be overridden by the greater good. The point is that generally cooperative behavior, and generally high amounts of trust, can make everyone better off. If it was true as you said that:
and as a consequence, he e.g. expected someone to record him during the Q&A, then he would presumably not have done the Q&A in the first place, or would have shared much less information. This would have lead to humanity learning less about OpenAI’s plans.
And this is definitely not a single-shot interaction. This was Sam’s second Q&A at an ACX meetup, and there was no reason to expect it to be the last. Moreover, there’s been a lot of interactions between the alignment community (including some who write on LW) and OpenAI in the past. And given that OpenAI’s decisions about alignment-related things matter a lot (as you say) it seems important to keep up good relations and high degrees of trust.
I initially downvoted, have since retracted it. Since trust/cooperation can be quite fragile and dependent on expectations about how people will behave, I became worried when I read you as basically announcing that you and other people should and will defect in the future. And I wanted a quick way to mark disagreement with that, to communicate that this isn’t a generally accepted point of view. But your point was phrased in a perfectly civil manner, so I should really have just taken the time to write a response, sorry.
I appreciate the response and stand corrected.
The point about it being an iterated prisoner’s dilemma is a good one, and I would rather there be more such ACX instances where he shares even more of his thinking due to our cooperative/trustworthy behavior, than this to be the last one or have the next ones be filtered PR-speak.
A small number of people in the alignment community repeatedly getting access to better information and being able to act on it beats the value of this one single post staying open to the world. And even in the case of “the cat being out of the bag,” hiding/removing the post would probably do good as a gesture of cooperation.