Thanks for writing this. I want to push back against a couple of things:
First, I think it’s not at all clear that intellectual progress on alignment is easy, and I think describing the situation in terms of low-hanging fruit is misleading. I think it’s more like there are lots of low-hanging thingies some of which are probably fruit but it’s unclear which ones, and the other ones are poisonous in unknown ways. More concretely, I worry a lot about people making implicit assumptions or adopting frames that seem reasonable but actually actively make it harder to have the right thoughts and go in the right direction. (I have this same kind of worry about bad LW posts.)
Second, I think it’s not at all clear that talking about what Elon Musk said to Demis Hassabis is unimportant; this is closely related to the disagreement I had with Ray about demon threads. Put bluntly, Elon Musk and Demis Hassabis are very powerful people whose actions matter a lot, and it makes sense to keep track of what’s going on with them. The gossip intuitions we evolved for doing this, if anything, underestimate the impact of doing so, since those intuitions are tied to gossip affecting the fate of a few people in a tribe whereas this kind of gossip potentially affects the long-run future of humanity.
Ironic disclaimer: Arguing about whether politics is useful is temptingly distracting in the same way that politics is. I’ll do one more public response clarifiying something if need be, but if this seems like it warrants further discussion would prefer to do so in private channel.
You persuaded me elsewhere that politics/gossip/etc are (at least sometimes, at least reasonably) important to most people. However, the phrasing of this comment feels exactly like what I was trying to caution against.
Ben’s point about “just build the space rocket” is one key point. Another is that if you aren’t in particular circles, Demis and Eloncan’t hear you. And, the set of things you can do that influence them meaningfully are very different from what your intuitions will push you towards. (on average, for most values of “you”)
Yes, there’s something important that needs to be dealt with here. But not the way everyone will do by default as if Elon and Demis were people in the tribe a few hundred feet away.
AI alignment technical progress feels (to the average AI Alignment enthusiast) like something they don’t understand well enough to comment on. So they instead comment on something they think they can comment on, which is what Elon and Demis et all seem to be doing.
I think (this is a bit of an exaggeration but I think close enough to true) that productively engaging in politics should feel about as intimidatingly-opaque as AI Technical Progress does. If it doesn’t feel like you’re solving a complicated problem that your brain didn’t evolve to handle, you probably aren’t doing it right.
You and Ben make fair points. I think I didn’t have a good sense of what level of gossip Ben was pushing back against; I had a sense he was pushing back against “occasionally gossip at parties” which seemed too strong to me (that’s the level of gossip I get exposed to by default), but if Ben is pushing against something more like “15% of my conversations are dominated by gossip by default” that would make more sense to me.
Arguing about whether politics is useful is temptingly distracting in the same way that politics is.
Yeah, this is why I decided that the next post I write on this topic will be more fleshing out of the ‘background models’ section than the heuristic section.
If you work at SpaceX, the decisions Elon Musk makes are very important to you. However, it would be bad if you thus spent a large deal of time discussing Musk’s motives, goals, and personal preferences, instead of building a space rocket. It’s best for the company as a whole to spend most resources building the actual products, and the best way to reliably rise is to reliably create value. It would be really bad if each worker at the company spent 10% of their time gossiping about Musk, rather than building a space rocket.
To give another example, in online forums, there’s a common phenomena whereby a set that is nominally about something (e.g. podcasting equipment) becomes about the forum—who should run it, who gets what rights, etc.
The general point is that there is a constant force to discuss the meta, discuss the community, discuss status, that negatively affects any community that is trying to do something real, be it discuss podcasting gear, anime, or research. The x-risk community is no different.
(I was somewhat sloppy in my words; while I did say “Avoid things that (because they’re social) are fun to argue about”, I also said “Another candidate for a sexy subject that is basically a distraction, is discussion of the high status people in AI e.g. “Did you hear what Elon Musk said to Demis Hassabis?”″. It is actually important, but I want to suggest that most of the time we discuss it we’re likely being motivated by other reasons, and on net we should push against that.)
I think it’s not at all clear that intellectual progress on alignment is easy, and I think describing the situation in terms of low-hanging fruit is misleading
For the most part I was just stating my belief rather than arguing for it (though I pointed at two examples). I’ll think some more about this and maybe write up a post (though I think Ray is also planning such a post).
Thanks for writing this. I want to push back against a couple of things:
First, I think it’s not at all clear that intellectual progress on alignment is easy, and I think describing the situation in terms of low-hanging fruit is misleading. I think it’s more like there are lots of low-hanging thingies some of which are probably fruit but it’s unclear which ones, and the other ones are poisonous in unknown ways. More concretely, I worry a lot about people making implicit assumptions or adopting frames that seem reasonable but actually actively make it harder to have the right thoughts and go in the right direction. (I have this same kind of worry about bad LW posts.)
Second, I think it’s not at all clear that talking about what Elon Musk said to Demis Hassabis is unimportant; this is closely related to the disagreement I had with Ray about demon threads. Put bluntly, Elon Musk and Demis Hassabis are very powerful people whose actions matter a lot, and it makes sense to keep track of what’s going on with them. The gossip intuitions we evolved for doing this, if anything, underestimate the impact of doing so, since those intuitions are tied to gossip affecting the fate of a few people in a tribe whereas this kind of gossip potentially affects the long-run future of humanity.
Ironic disclaimer: Arguing about whether politics is useful is temptingly distracting in the same way that politics is. I’ll do one more public response clarifiying something if need be, but if this seems like it warrants further discussion would prefer to do so in private channel.
You persuaded me elsewhere that politics/gossip/etc are (at least sometimes, at least reasonably) important to most people. However, the phrasing of this comment feels exactly like what I was trying to caution against.
Ben’s point about “just build the space rocket” is one key point. Another is that if you aren’t in particular circles, Demis and Elon can’t hear you. And, the set of things you can do that influence them meaningfully are very different from what your intuitions will push you towards. (on average, for most values of “you”)
Yes, there’s something important that needs to be dealt with here. But not the way everyone will do by default as if Elon and Demis were people in the tribe a few hundred feet away.
AI alignment technical progress feels (to the average AI Alignment enthusiast) like something they don’t understand well enough to comment on. So they instead comment on something they think they can comment on, which is what Elon and Demis et all seem to be doing.
I think (this is a bit of an exaggeration but I think close enough to true) that productively engaging in politics should feel about as intimidatingly-opaque as AI Technical Progress does. If it doesn’t feel like you’re solving a complicated problem that your brain didn’t evolve to handle, you probably aren’t doing it right.
You and Ben make fair points. I think I didn’t have a good sense of what level of gossip Ben was pushing back against; I had a sense he was pushing back against “occasionally gossip at parties” which seemed too strong to me (that’s the level of gossip I get exposed to by default), but if Ben is pushing against something more like “15% of my conversations are dominated by gossip by default” that would make more sense to me.
Yeah, this is why I decided that the next post I write on this topic will be more fleshing out of the ‘background models’ section than the heuristic section.
If you work at SpaceX, the decisions Elon Musk makes are very important to you. However, it would be bad if you thus spent a large deal of time discussing Musk’s motives, goals, and personal preferences, instead of building a space rocket. It’s best for the company as a whole to spend most resources building the actual products, and the best way to reliably rise is to reliably create value. It would be really bad if each worker at the company spent 10% of their time gossiping about Musk, rather than building a space rocket.
To give another example, in online forums, there’s a common phenomena whereby a set that is nominally about something (e.g. podcasting equipment) becomes about the forum—who should run it, who gets what rights, etc.
The general point is that there is a constant force to discuss the meta, discuss the community, discuss status, that negatively affects any community that is trying to do something real, be it discuss podcasting gear, anime, or research. The x-risk community is no different.
(I was somewhat sloppy in my words; while I did say “Avoid things that (because they’re social) are fun to argue about”, I also said “Another candidate for a sexy subject that is basically a distraction, is discussion of the high status people in AI e.g. “Did you hear what Elon Musk said to Demis Hassabis?”″. It is actually important, but I want to suggest that most of the time we discuss it we’re likely being motivated by other reasons, and on net we should push against that.)
For the most part I was just stating my belief rather than arguing for it (though I pointed at two examples). I’ll think some more about this and maybe write up a post (though I think Ray is also planning such a post).