I’ve thought of a new way to think about the general case: call it the Alien Implant case. After people are dead (and only after they are dead) an autopsy of the brain reveals that there is a black box in their brains, thought to be implanted by aliens. There is a dial on it, set to A or B. All the people who have the dial set to A, during their lives chose to smoke, and got cancer. All the people who have the dial set to B, during their lives chose not to smoke, and did not get cancer.
It turns out that the box with the dial set to A causes cancer through a simple physical mechanism. Having the dial set to B does not have this effect.
I prefer smoking to not smoking, in general, but smoking and getting cancer would be worse than not smoking. What should I do?
“Not enough info” is now not a valid response, since I have to decide. I could try to estimate the probability that the aliens are predicting my choice, and the probability that they are causing it, and then use some complicated fake utility calculation (fake since it would not match what we would actually expect to be the outcome).
But it seems evident that is what Eliezer called a “ritual of cognition”; someone who cares only about the outcome, will just not smoke.
I’ve thought of a new way to think about the general case: call it the Alien Implant case. After people are dead (and only after they are dead) an autopsy of the brain reveals that there is a black box in their brains, thought to be implanted by aliens. There is a dial on it, set to A or B. All the people who have the dial set to A, during their lives chose to smoke, and got cancer. All the people who have the dial set to B, during their lives chose not to smoke, and did not get cancer.
It turns out that the box with the dial set to A causes cancer through a simple physical mechanism. Having the dial set to B does not have this effect.
I prefer smoking to not smoking, in general, but smoking and getting cancer would be worse than not smoking. What should I do?
“Not enough info” is now not a valid response, since I have to decide. I could try to estimate the probability that the aliens are predicting my choice, and the probability that they are causing it, and then use some complicated fake utility calculation (fake since it would not match what we would actually expect to be the outcome).
But it seems evident that is what Eliezer called a “ritual of cognition”; someone who cares only about the outcome, will just not smoke.