If it turns out that moon is made out of cheese, should you focus on optimizing your world, or the world of the philosopher considering the thought experiment?
It seems that from philosopher’s point of view, your world has low measure (moral relevance), but it’s unclear whether it’s tempting to decide so just because it’s not possible to affect the cheese world. From cheese world’s point of view, it’s even less clear, and the difficulty of control is similar.
I wonder how much of a role ability to control plays in these estimates, since decision theoretically, moral value (and so probability) of a fact one can’t control is meaningless, and so one is tempted to assign arbitrary or contradictory values. (A related question is whether it makes sense to have control over probability (or moral value) of a fact without having control over the fact itself (I would guess so), in which case probability (moral value) isn’t meaningless if it itself can be controlled, even if the fact it’s probability (moral value) of can’t.)
If it turns out that moon is made out of cheese, should you focus on optimizing your world, or the world of the philosopher considering the thought experiment?
I first heard this as a half-joking argument by Steve Rayhawk as to why we might not want to push fat people in front of trains.[1] (Of course, second-order consequentialists have to do this kind of reasoning constantly in real life: what precedent am I setting, what rule am I choosing to follow, in making this decision, and do I endorse agents like me following that rule generally?)
Even for me personally I think about this a fair bit in the context of simple moral dilemmas like vegetarianism. Another way to think about it is, If some agent could only simulate me with a coarse-grained simulation such that it only observed my decisions as evidence about morality and not the intricacies and subtleties of the context of my decisions, would I still think of myself as providing reflectively endorsed evidence about morality? If not, how strongly does that indicate that I am using my “subtle” contexts as rationalizations for immoral decisions?
[1] Incidentally, this is what meta-contrarianism or steel men are supposed to look like in practice: take a cached conclusion, look at its opposite, construct your best argument for it, then see if that got you anywhere. Even if you’re just manipulating language, like translating some combination of bland computationalism, distributed/parallel computing and memetics into a half-baked description of Plato’s Forms and Instances across human minds, it still gives you a new perspective on your original ideas from which you can find things that are unnatural or poorly defined, and see new connections that previously hadn’t leapt out. The fact that it’s often fun to do so acts as incentive. (I note that this may be typical mind fallacy: it seems that others are somewhat-to-significantly more attached to their preferred languages than I am, and for them that may be a correct choice in the face of many languages that are optimized for anti-epistemology in subtle or not-so-subtle ways, e.g. academic Christianity. But it is important to explicitly note that doing so may reinforce the algorithm that classifies new information by its literary genre rather than its individual merits: you’re trusting your priors more than your likelihood ratios without training your likelihood estimator.)
If it turns out that moon is made out of cheese, should you focus on optimizing your world, or the world of the philosopher considering the thought experiment?
It seems that from philosopher’s point of view, your world has low measure (moral relevance), but it’s unclear whether it’s tempting to decide so just because it’s not possible to affect the cheese world. From cheese world’s point of view, it’s even less clear, and the difficulty of control is similar.
I wonder how much of a role ability to control plays in these estimates, since decision theoretically, moral value (and so probability) of a fact one can’t control is meaningless, and so one is tempted to assign arbitrary or contradictory values. (A related question is whether it makes sense to have control over probability (or moral value) of a fact without having control over the fact itself (I would guess so), in which case probability (moral value) isn’t meaningless if it itself can be controlled, even if the fact it’s probability (moral value) of can’t.)
I first heard this as a half-joking argument by Steve Rayhawk as to why we might not want to push fat people in front of trains.[1] (Of course, second-order consequentialists have to do this kind of reasoning constantly in real life: what precedent am I setting, what rule am I choosing to follow, in making this decision, and do I endorse agents like me following that rule generally?)
Even for me personally I think about this a fair bit in the context of simple moral dilemmas like vegetarianism. Another way to think about it is, If some agent could only simulate me with a coarse-grained simulation such that it only observed my decisions as evidence about morality and not the intricacies and subtleties of the context of my decisions, would I still think of myself as providing reflectively endorsed evidence about morality? If not, how strongly does that indicate that I am using my “subtle” contexts as rationalizations for immoral decisions?
[1] Incidentally, this is what meta-contrarianism or steel men are supposed to look like in practice: take a cached conclusion, look at its opposite, construct your best argument for it, then see if that got you anywhere. Even if you’re just manipulating language, like translating some combination of bland computationalism, distributed/parallel computing and memetics into a half-baked description of Plato’s Forms and Instances across human minds, it still gives you a new perspective on your original ideas from which you can find things that are unnatural or poorly defined, and see new connections that previously hadn’t leapt out. The fact that it’s often fun to do so acts as incentive. (I note that this may be typical mind fallacy: it seems that others are somewhat-to-significantly more attached to their preferred languages than I am, and for them that may be a correct choice in the face of many languages that are optimized for anti-epistemology in subtle or not-so-subtle ways, e.g. academic Christianity. But it is important to explicitly note that doing so may reinforce the algorithm that classifies new information by its literary genre rather than its individual merits: you’re trusting your priors more than your likelihood ratios without training your likelihood estimator.)