There’s a fundamental assumption your argument rests on which is a choice of prior: Assume that everyone’s credences in a given proposition is a distribution centered around the correct value an ideal agent would give to that proposition if they had access to all the information that was available and relevant to that proposition and had enough time and capacity to process that information to the fullest extent. Your arguments are sound given that the above is actually the correct prior, but I see most of your essay as arguing why modesty would be the correct stance given that the assumption is true, and less of it about why that prior is the best one to have in the vast majority of situations.
In practice, the kind of modesty we are actually interested in is the case where we belong to group A, and group A has formed some credence in proposition X through a process that involves argumentation, exchange and acquisition of information, verifying each other’s logical steps, and so on (there’s some modesty going on within group A). After a while of this process going on the group consensus around X has converged on a particular value. But there is also another group, group B, which is much larger than A and has been around for longer too. Group B has converged on a very different credence for X than group A. Should group A update their credence in X to be much closer to B’s? Your argument seems to say, yes, basically they should.
I think I would tend to agree with you if the above information was all I had available to me. For all I know, my group A is no more or less effective than group B at reaching the truth, and is not subject to any different systemic biases that would prevent the correct credences from being reached. Since B has been around for longer, and has more members, possibly with experts, I should consider them to have more likely converged to the correct credence in X than group A has.
However, in the cases that we tend to really care about, the places where we think immodesty might be reasonable, group A usually does have some reason to believe that group B is less effective at converging to the correct value. It could be due to systemic issues, inadequate equilibria, Moloch, whatever you want to call it. In other words, we have some reason to think that throwing out the default prior is ok.
So this is where I think the crux of the argument is: How strongly do we expect the distribution of credences on X for group B to not be centered around the correct value? In general I think the key is that A tends to search for the X that maximizes the above statement, in other words, the X that B is most likely to be wrong about. After A thinks it’s found the best X, then it asks if immodesty is ok in this specific case. When you factor in that search process, it makes it a little more likely to actually find a bias that makes the prior wrong. I think there’s an unstated assumption in your essay that implies that X is chosen randomly with respect to the likelihood of bias.
There’s a fundamental assumption your argument rests on which is a choice of prior: Assume that everyone’s credences in a given proposition is a distribution centered around the correct value an ideal agent would give to that proposition if they had access to all the information that was available and relevant to that proposition and had enough time and capacity to process that information to the fullest extent. Your arguments are sound given that the above is actually the correct prior, but I see most of your essay as arguing why modesty would be the correct stance given that the assumption is true, and less of it about why that prior is the best one to have in the vast majority of situations.
In practice, the kind of modesty we are actually interested in is the case where we belong to group A, and group A has formed some credence in proposition X through a process that involves argumentation, exchange and acquisition of information, verifying each other’s logical steps, and so on (there’s some modesty going on within group A). After a while of this process going on the group consensus around X has converged on a particular value. But there is also another group, group B, which is much larger than A and has been around for longer too. Group B has converged on a very different credence for X than group A. Should group A update their credence in X to be much closer to B’s? Your argument seems to say, yes, basically they should.
I think I would tend to agree with you if the above information was all I had available to me. For all I know, my group A is no more or less effective than group B at reaching the truth, and is not subject to any different systemic biases that would prevent the correct credences from being reached. Since B has been around for longer, and has more members, possibly with experts, I should consider them to have more likely converged to the correct credence in X than group A has.
However, in the cases that we tend to really care about, the places where we think immodesty might be reasonable, group A usually does have some reason to believe that group B is less effective at converging to the correct value. It could be due to systemic issues, inadequate equilibria, Moloch, whatever you want to call it. In other words, we have some reason to think that throwing out the default prior is ok.
So this is where I think the crux of the argument is: How strongly do we expect the distribution of credences on X for group B to not be centered around the correct value? In general I think the key is that A tends to search for the X that maximizes the above statement, in other words, the X that B is most likely to be wrong about. After A thinks it’s found the best X, then it asks if immodesty is ok in this specific case. When you factor in that search process, it makes it a little more likely to actually find a bias that makes the prior wrong. I think there’s an unstated assumption in your essay that implies that X is chosen randomly with respect to the likelihood of bias.