A lot of those concerns seem valid. I recalled the earlier comment thread and had it in mind while I was writing the response comment. (I agree that “viewpoint X” is a thing, and I don’t even think it’s that uncharitable to call it the MIRI/rationalist viewpoint, although it’s simplified)
Fwiw, while I prefer option #3 (I just added #s to the options for easier reference), #2 and #4 both seem pretty fine. And whichever option one went with, getting representative members seems like an important thing to put a lot of effort into.
My current sense is that AF was aiming to be a place where people-other-than-Paul at OpenAI would feel comfortable participating. I can imagine it turns out “AF already failed to be this sufficiently that if you want that, you need to start over,” but it is moderately expensive to start over. I would agree that this would require a lot of work, but seems potentially quite important and worthwhile.
What are the failure modes you imagine, and/or how much harder do you think it is, to host the review on AF, while aiming for a broader base of participants that AF currently feels oriented towards? (As compared to the “try for a broad base of participants and host it somewhere other than AF”)
I can definitely imagine “it turns out to get people involved you need more anonymity/plausible-deniability than a public forum affords”, so starting from a different vantage point is better.
One of the options someone proposed was “CHAI, MIRI, OpenAI and Deepmind [potentially other orgs] are each sort of treated as an entity in a parliament, with N vote-weight each. It’s up to them how they distribute that vote weight among their internal teams.” I think I’d weakly prefer “actually you just really try to get more people from each team to participate, so you end up with information from 20 individuals rather than 4 opaque orgs”, but I can imagine a few reasons why the former is more practical (with the plausible deniability being a feature/bug combo)
My current sense is that AF was aiming to be a place where people-other-than-Paul at OpenAI would feel comfortable participating.
Agreed that that was the goal; I’m arguing that it has failed at this. (Or, well, maybe they’d be comfortable participating, but they don’t see the value in participating.)
What are the failure modes you imagine, and/or how much harder do you think it is, to host the review on AF, while aiming for a broader base of participants that AF currently feels oriented towards?
Mainly I think it would be really hard to get that broader base of participants. I imagine trying to convince specific people (not going to name names) that they should be participating, and the only argument that I think might be convincing to them would be “if we don’t participate, then our work will be evaluated by MIRI-rationalist standards, and future entrants to the field will forever misunderstand our work in the same way that people forever misunderstand CIRL”. It seems pretty bad to rely on that argument.
I think you might be underestimating how different these two groups are. Like, it’s not just that they work on different things, they also have different opinions on the best ways to publish, what should count as good work, the value of theoretical vs. conceptual vs. empirical work, etc. Certainly most are glad that the other exists in the sense that they think it is better than nothing (but not everyone meets even this low bar), but beyond that there’s not much agreement on anything. I expect the default reaction to be “this review isn’t worth my time”.
I can definitely imagine “it turns out to get people involved you need more anonymity/plausible-deniability than a public forum affords”, so starting from a different vantage point is better.
As above, I expect the default reaction to be “this review isn’t worth my time”, rather than something like “I need plausible deniability to evaluate other people’s work”.
One of the options someone proposed was “CHAI, MIRI, OpenAI and Deepmind [potentially other orgs] are each sort of treated as an entity in a parliament, with N vote-weight each. It’s up to them how they distribute that vote weight among their internal teams.”
This sort of mechanism doesn’t address the “review isn’t worth my time” problem. It would probably give you a more unbiased estimate of what the “field” thinks, but only because e.g. Richard and I would get a very large vote weight. (And even that isn’t unbiased—Richard and I are much closer to the MIRI-rationalist viewpoint than the average for our orgs.)
A lot of those concerns seem valid. I recalled the earlier comment thread and had it in mind while I was writing the response comment. (I agree that “viewpoint X” is a thing, and I don’t even think it’s that uncharitable to call it the MIRI/rationalist viewpoint, although it’s simplified)
Fwiw, while I prefer option #3 (I just added #s to the options for easier reference), #2 and #4 both seem pretty fine. And whichever option one went with, getting representative members seems like an important thing to put a lot of effort into.
My current sense is that AF was aiming to be a place where people-other-than-Paul at OpenAI would feel comfortable participating. I can imagine it turns out “AF already failed to be this sufficiently that if you want that, you need to start over,” but it is moderately expensive to start over. I would agree that this would require a lot of work, but seems potentially quite important and worthwhile.
What are the failure modes you imagine, and/or how much harder do you think it is, to host the review on AF, while aiming for a broader base of participants that AF currently feels oriented towards? (As compared to the “try for a broad base of participants and host it somewhere other than AF”)
Random other things I thought about:
I can definitely imagine “it turns out to get people involved you need more anonymity/plausible-deniability than a public forum affords”, so starting from a different vantage point is better.
One of the options someone proposed was “CHAI, MIRI, OpenAI and Deepmind [potentially other orgs] are each sort of treated as an entity in a parliament, with N vote-weight each. It’s up to them how they distribute that vote weight among their internal teams.” I think I’d weakly prefer “actually you just really try to get more people from each team to participate, so you end up with information from 20 individuals rather than 4 opaque orgs”, but I can imagine a few reasons why the former is more practical (with the plausible deniability being a feature/bug combo)
Agreed that that was the goal; I’m arguing that it has failed at this. (Or, well, maybe they’d be comfortable participating, but they don’t see the value in participating.)
Mainly I think it would be really hard to get that broader base of participants. I imagine trying to convince specific people (not going to name names) that they should be participating, and the only argument that I think might be convincing to them would be “if we don’t participate, then our work will be evaluated by MIRI-rationalist standards, and future entrants to the field will forever misunderstand our work in the same way that people forever misunderstand CIRL”. It seems pretty bad to rely on that argument.
I think you might be underestimating how different these two groups are. Like, it’s not just that they work on different things, they also have different opinions on the best ways to publish, what should count as good work, the value of theoretical vs. conceptual vs. empirical work, etc. Certainly most are glad that the other exists in the sense that they think it is better than nothing (but not everyone meets even this low bar), but beyond that there’s not much agreement on anything. I expect the default reaction to be “this review isn’t worth my time”.
As above, I expect the default reaction to be “this review isn’t worth my time”, rather than something like “I need plausible deniability to evaluate other people’s work”.
This sort of mechanism doesn’t address the “review isn’t worth my time” problem. It would probably give you a more unbiased estimate of what the “field” thinks, but only because e.g. Richard and I would get a very large vote weight. (And even that isn’t unbiased—Richard and I are much closer to the MIRI-rationalist viewpoint than the average for our orgs.)