For the most part I think that starts to address it. At the same time, on your last point, there is an important difference between “this is how fully idealized rational agents of a certain sort behave” and “this is how you, a non-fully idealized, partially rational agent should behave, to improve your rationality”.
Someone in perfect physical condition (not just for humans, but for idealized physical beings) has a different optimal workout plan from me, and we should plan differently for various physical activities, even if this person is the ideal towards which I am aiming.
So if we idealize our bayesian models too much, we open up the question: “How does this idealized agent’s behavior relate to how I should behave?” It might be that, were we to design rational agents, it makes sense to use these idealized reasoners as models, but if the goal is personal improvement, we need some way to explain what one might call the Kantian inference from “I am an imperfectly rational being” to “I ought to behave the way such-and-such a perfectly rational being would”.
For the most part I think that starts to address it. At the same time, on your last point, there is an important difference between “this is how fully idealized rational agents of a certain sort behave” and “this is how you, a non-fully idealized, partially rational agent should behave, to improve your rationality”.
Someone in perfect physical condition (not just for humans, but for idealized physical beings) has a different optimal workout plan from me, and we should plan differently for various physical activities, even if this person is the ideal towards which I am aiming.
So if we idealize our bayesian models too much, we open up the question: “How does this idealized agent’s behavior relate to how I should behave?” It might be that, were we to design rational agents, it makes sense to use these idealized reasoners as models, but if the goal is personal improvement, we need some way to explain what one might call the Kantian inference from “I am an imperfectly rational being” to “I ought to behave the way such-and-such a perfectly rational being would”.