FWIW I think a fairly substantial amount of effort has gone into resolving longstanding disagreements. I think that effort has resulted in a lot of good works and updates from many people reading about the disagreement discussion, but not really changed the mind of the people doing the arguing. (See: the MIRI Dialogues)
And it’s totally plausible to me the answer is “10-100x the amount of work that is gone in so far.”
I maybe agree that people haven’t literally sat and double-cruxed for six months. I don’t know that it’s fair to describe this as “impracticality, coordination difficulties and frustration” instead of “principled epistemics and EV calculations.” Like, if you’ve done a thing a bunch and it doesn’t seem to be working and you feel like you have traction on another thing, it’s not crazy to do the other thing.
(That said, I do still have the gut level feeling of ‘man it’s absolutely bonkers that in the so-called rationality community a lot of prominent thinkers still disagree about such fundamental stuff.’)
Oh sure, I certainly don’t mean to imply that there’s been little effort in absolute terms—I’m very encouraged by the MIRI dialogues, and assume there are a bunch of behind-the-scenes conversations going on. I also assume that everyone is doing what seems best in good faith, and has potentially high-value demands on their time.
However, given the stakes, I think it’s a time for extraordinary efforts—and so I worry that [this isn’t the kind of thing that is usually done] is doing too much work.
I think the “principled epistemics and EV calculations” could perfectly well be the explanation, if it were the case that most researchers put around a 1% chance on [Eliezer/Nate/John… are largely correct on the cruxy stuff].
That’s not the sense I get—more that many put the odds somewhere around 5% to 25%, but don’t believe the arguments are sufficiently crisp to allow productive engagement.
If I’m correct on that (and I may well not be), it does not seem a principled justification for the status-quo. Granted the right course isn’t obvious—we’d need whoever’s on the other side of the double-cruxing to really know their stuff. Perhaps Paul’s/Rohin’s… time is too valuable for a 6 month cost to pay off. (the more realistic version likely involves not-quite-so-valuable people from each ‘side’ doing it)
As for “done a thing a bunch and it doesn’t seem to be working”, what’s the prior on [two experts in a field from very different schools of thought talk for about a week and try to reach agreement]? I’m no expert, but I strongly expect that not to work in most cases.
To have a realistic expectation of its working, you’d need to be doing the kinds of thing that are highly non-standard. Experts having some discussions over a week is standard. Making it your one focus for 6 months is not. (frankly, I’d be over the moon for the one month version [but again, for all I know this may have been tried])
Even more importantly, Aumann’s Agreement Theorem demands that both sides eventually agree on something, and the fact that the AI Alignment field hasn’t agreed yet is concerning.
FWIW I think a fairly substantial amount of effort has gone into resolving longstanding disagreements. I think that effort has resulted in a lot of good works and updates from many people reading about the disagreement discussion, but not really changed the mind of the people doing the arguing. (See: the MIRI Dialogues)
And it’s totally plausible to me the answer is “10-100x the amount of work that is gone in so far.”
I maybe agree that people haven’t literally sat and double-cruxed for six months. I don’t know that it’s fair to describe this as “impracticality, coordination difficulties and frustration” instead of “principled epistemics and EV calculations.” Like, if you’ve done a thing a bunch and it doesn’t seem to be working and you feel like you have traction on another thing, it’s not crazy to do the other thing.
(That said, I do still have the gut level feeling of ‘man it’s absolutely bonkers that in the so-called rationality community a lot of prominent thinkers still disagree about such fundamental stuff.’)
Oh sure, I certainly don’t mean to imply that there’s been little effort in absolute terms—I’m very encouraged by the MIRI dialogues, and assume there are a bunch of behind-the-scenes conversations going on.
I also assume that everyone is doing what seems best in good faith, and has potentially high-value demands on their time.
However, given the stakes, I think it’s a time for extraordinary efforts—and so I worry that [this isn’t the kind of thing that is usually done] is doing too much work.
I think the “principled epistemics and EV calculations” could perfectly well be the explanation, if it were the case that most researchers put around a 1% chance on [Eliezer/Nate/John… are largely correct on the cruxy stuff].
That’s not the sense I get—more that many put the odds somewhere around 5% to 25%, but don’t believe the arguments are sufficiently crisp to allow productive engagement.
If I’m correct on that (and I may well not be), it does not seem a principled justification for the status-quo. Granted the right course isn’t obvious—we’d need whoever’s on the other side of the double-cruxing to really know their stuff. Perhaps Paul’s/Rohin’s… time is too valuable for a 6 month cost to pay off. (the more realistic version likely involves not-quite-so-valuable people from each ‘side’ doing it)
As for “done a thing a bunch and it doesn’t seem to be working”, what’s the prior on [two experts in a field from very different schools of thought talk for about a week and try to reach agreement]? I’m no expert, but I strongly expect that not to work in most cases.
To have a realistic expectation of its working, you’d need to be doing the kinds of thing that are highly non-standard. Experts having some discussions over a week is standard. Making it your one focus for 6 months is not. (frankly, I’d be over the moon for the one month version [but again, for all I know this may have been tried])
Even more importantly, Aumann’s Agreement Theorem demands that both sides eventually agree on something, and the fact that the AI Alignment field hasn’t agreed yet is concerning.
Here’s the link:
https://www.lesswrong.com/tag/aumann-s-agreement-theorem