If everyone calculates 67*23 in their head, they’ll reach a partial consensus. People who disagree with the consensus can ask for an argument, and they’ll get a convincing argument which will convince them of the correct answer; and if the argument is unconvincing, and they present a convincing argument for a different answer, that answer will become the consensus. We thus arrive at consensus with no eigening. If this isn’t how things play out, it’s because there’s something wrong with the consensus / with the people’s epistemics.
Hmm, okay, I think I’ve made an update (not necessarily to agree with you entirely, but still an update on my picture, so thanks).
I was thinking that if a group of people all agree on particular axioms or rules of inferences, etc., then that will be where eigening is occurring even if given sufficiently straightforward axioms, the group members will achieve consensus without further eigening. But possibly you can get consensus on the axioms just via selection and via individuals using their inside-view to adopt them or not. That’s still a degree of “we agreed”, but not eigening.
Huh. Yeah, that’s an interesting case which yeah, plausibly doesn’t require any eigening. I think the plausibility comes from it being a case where someone can so fully do it from their personal inside view (the immediate calculation and also their belief in how the underlying mathematical operations ought to work).
I don’t think it scales to anything interesting (def not alignment research), but it is conceptually interesting for how I’ve been thinking about this.
No, it’s the central example for what would work in alignment. You have to think about the actual problem. The difficulty of the problem and illegibility of intermediate results means eigening becomes dominant, but that’s a failure mode.
I agree that eigening isn’t the key concept for alignment or other scientific process. Sure you could describe any consensus that way, but they could be either very good or just awful depending on how much valid analysis went into each step of doing that eigening. In a really good situation, progress toward consensus is only superficially describable as eigening. The real progress is happening by careful thinking and communicating. The eigening isn’t happening by reputation but by quality of work. In a bad field, eigening is doing most of the work.
Referring to them both as eigening seems to obscure the difference between good and bad science/theory creation.
If everyone calculates 67*23 in their head, they’ll reach a partial consensus. People who disagree with the consensus can ask for an argument, and they’ll get a convincing argument which will convince them of the correct answer; and if the argument is unconvincing, and they present a convincing argument for a different answer, that answer will become the consensus. We thus arrive at consensus with no eigening. If this isn’t how things play out, it’s because there’s something wrong with the consensus / with the people’s epistemics.
Hmm, okay, I think I’ve made an update (not necessarily to agree with you entirely, but still an update on my picture, so thanks).
I was thinking that if a group of people all agree on particular axioms or rules of inferences, etc., then that will be where eigening is occurring even if given sufficiently straightforward axioms, the group members will achieve consensus without further eigening. But possibly you can get consensus on the axioms just via selection and via individuals using their inside-view to adopt them or not. That’s still a degree of “we agreed”, but not eigening.
Huh. Yeah, that’s an interesting case which yeah, plausibly doesn’t require any eigening. I think the plausibility comes from it being a case where someone can so fully do it from their personal inside view (the immediate calculation and also their belief in how the underlying mathematical operations ought to work).
I don’t think it scales to anything interesting (def not alignment research), but it is conceptually interesting for how I’ve been thinking about this.
No, it’s the central example for what would work in alignment. You have to think about the actual problem. The difficulty of the problem and illegibility of intermediate results means eigening becomes dominant, but that’s a failure mode.
Interesting to consider it a failure mode. Maybe it is. Or is at least somewhat.
I’ve got another post on eigening in the works, I think that might provide clearer terminology for talking about this, if you’ll have time to read it.
I agree that eigening isn’t the key concept for alignment or other scientific process. Sure you could describe any consensus that way, but they could be either very good or just awful depending on how much valid analysis went into each step of doing that eigening. In a really good situation, progress toward consensus is only superficially describable as eigening. The real progress is happening by careful thinking and communicating. The eigening isn’t happening by reputation but by quality of work. In a bad field, eigening is doing most of the work.
Referring to them both as eigening seems to obscure the difference between good and bad science/theory creation.
But yeah if you mean “I don’t think it scales to successfully staking out territory around a grift” that seems right.