Yes, that’s exactly what was in my mind, and Raemon expressed it also.
their argument does appear to be correct, so I try to ignore the smell.
I don’t think that’s the right way to resolve the conflict. One person’s taking beliefs seriously is another’s toxic decompartmentalisation, after all. Why should the smell yield to the argument instead of vice versa? Especially when you notice that the part telling you that is the part making the argument, and the cognitive nose is inarticulate. No, what is needed is to resolve the contradiction, not to suppress one side of it and pretend you are no longer confused.
And meanwhile, go and be a banker, or whatever the right answer presently seems to be, because there is no such thing as inaction while you resolve the conundrum: to do nothing is itself to do something. If you later conclude it’s a false path, at least the money will give you the flexibility to switch to whatever you decide you should have been doing instead.
Why should the smell yield to the argument instead of vice versa?
That’s the $64,000(-a-year) question, and I don’t have an answer I’m happy with for the general case. Here’s roughly what I think for this specific situation.
As you say, my nose can’t describe what it smells. It might be a genuine problem or a false alarm. To find out which, I have to consciously poke around for an overlooked counterargument or a weak spot in the original argument, something to corroborate the bad smell. I did that here and couldn’t find a killer gap in 80,000 Hours’ arguments, nor a strong counterargument for why I should disregard them.
For simplicity, consider a stripped down version of the decision problem where I have only two options: becoming a rich banker vs. getting a normal job paying the median salary.
Suppose I disapprove of the banker option for whatever reason. If I hold my nose and become a banker anyway, it seems very likely to me that (1) I would nonetheless prefer that to having someone with different values in my place instead, and (2) that even if taking a banking job worked against my values or goals, I could compensate for that by hiring other people to further them.
I had thought that reference class forecasting might warn against the get-rich-and-give strategy: people with more income give a smaller percentage of it to charity, so by entering banking one might opt into a less generous reference class. But quick Googling reveals that people with higher incomes give more in absolute terms, at least in the UK, the US, and Canada.
Putting aside the chance of my being wrong, what about the disutility of being wrong? Well, I agree with your final paragraph, so that doesn’t seem to weigh heavily against the 80,000 Hours point of view either.
All in all my nose seems to have overreacted on this one. Maybe it raised the alarm because 80,000 Hours’ conclusion failed a quick universalizability test, namely “would this still be the best choice if everyone else in the same boat made it too?” But that test itself seems to fail here.
I doubt my thoughts on this are bulletproof; there’s a good chance I’m missing part of the puzzle or just plain wrong on some fundamental issue. Maybe I’ve built a convenient, clever meta-argument for arguing myself into something I’d probably want to do anyway! Still, ultimately I can devote only so much thought (and self-distrust) to this. I have to make a judgement call, and this is the best one I can make, whatever the risk of motivated cognition.
Yes, that’s exactly what was in my mind, and Raemon expressed it also.
I don’t think that’s the right way to resolve the conflict. One person’s taking beliefs seriously is another’s toxic decompartmentalisation, after all. Why should the smell yield to the argument instead of vice versa? Especially when you notice that the part telling you that is the part making the argument, and the cognitive nose is inarticulate. No, what is needed is to resolve the contradiction, not to suppress one side of it and pretend you are no longer confused.
And meanwhile, go and be a banker, or whatever the right answer presently seems to be, because there is no such thing as inaction while you resolve the conundrum: to do nothing is itself to do something. If you later conclude it’s a false path, at least the money will give you the flexibility to switch to whatever you decide you should have been doing instead.
That’s the $64,000(-a-year) question, and I don’t have an answer I’m happy with for the general case. Here’s roughly what I think for this specific situation.
As you say, my nose can’t describe what it smells. It might be a genuine problem or a false alarm. To find out which, I have to consciously poke around for an overlooked counterargument or a weak spot in the original argument, something to corroborate the bad smell. I did that here and couldn’t find a killer gap in 80,000 Hours’ arguments, nor a strong counterargument for why I should disregard them.
For simplicity, consider a stripped down version of the decision problem where I have only two options: becoming a rich banker vs. getting a normal job paying the median salary.
Suppose I disapprove of the banker option for whatever reason. If I hold my nose and become a banker anyway, it seems very likely to me that (1) I would nonetheless prefer that to having someone with different values in my place instead, and (2) that even if taking a banking job worked against my values or goals, I could compensate for that by hiring other people to further them.
I had thought that reference class forecasting might warn against the get-rich-and-give strategy: people with more income give a smaller percentage of it to charity, so by entering banking one might opt into a less generous reference class. But quick Googling reveals that people with higher incomes give more in absolute terms, at least in the UK, the US, and Canada.
Putting aside the chance of my being wrong, what about the disutility of being wrong? Well, I agree with your final paragraph, so that doesn’t seem to weigh heavily against the 80,000 Hours point of view either.
All in all my nose seems to have overreacted on this one. Maybe it raised the alarm because 80,000 Hours’ conclusion failed a quick universalizability test, namely “would this still be the best choice if everyone else in the same boat made it too?” But that test itself seems to fail here.
I doubt my thoughts on this are bulletproof; there’s a good chance I’m missing part of the puzzle or just plain wrong on some fundamental issue. Maybe I’ve built a convenient, clever meta-argument for arguing myself into something I’d probably want to do anyway! Still, ultimately I can devote only so much thought (and self-distrust) to this. I have to make a judgement call, and this is the best one I can make, whatever the risk of motivated cognition.