It’s obvious but worth saying anyway that pretty much all the decision theory scenarios that people talk about, like Newcomb’s problem, are scenarios where people find themselves unsure what to do, and disagree with each other. Therefore the human brain doesn’t give straight answers—or if it does, the answers are not to be found at the “base algorithm” level, but rather the “learned model” level (which can involve metacognition).
One point I personally put a lot of weight on: while people are unsure/disagree about particular scenarios, people do mostly seem to agree on what the relevant arguments are, or what the main “options” are for how to think about particular scenarios. That suggests that we do share a common underlying decision-making algorithm, but that algorithm itself sometimes produces uncertain answers.
In particular, for a predictive-processing-like decision theory, it makes sense that sometimes there would be multiple possible self-consistent models. In those cases, we should expect humans to be unsure/disagree, but we’d still expect people to agree on what the relevant arguments/options are—i.e. the possible models.
sometimes there would be multiple possible self-consistent models
I’m not sure what you’re getting at here; you may have a different conception of predictive-processing-like decision theory than I do. I would say “I will get up and go to the store” is a self-consistent model, “I will sit down and read the news” is a self-consistent model, etc. etc. There are always multiple possible self-consistent models—at least one for each possible action that you will take.
Oh, maybe you’re taking the perspective where if you’re hungry you put a high prior on “I will eat soon”. Yeah, I just don’t think that’s right, or if there’s a sensible way to think about it, I haven’t managed to get it despite some effort. I think if you’re hungry, you want to eat because it leads to a predicted reward, not because you have a prior expectation that you will eat. After all, if you’re stuck on a lifeboat in the middle of the ocean, you’re hungry but you don’t expect to eat. This is an obvious point, frequently brought up, and Friston & colleagues hold strong that it’s not a problem for their theory, and I can’t make heads or tails of what their counterargument is. I discussed my version (where rewards are also involved) here, and then here I went into more depth for a specific example.
One point I personally put a lot of weight on: while people are unsure/disagree about particular scenarios, people do mostly seem to agree on what the relevant arguments are, or what the main “options” are for how to think about particular scenarios. That suggests that we do share a common underlying decision-making algorithm, but that algorithm itself sometimes produces uncertain answers.
In particular, for a predictive-processing-like decision theory, it makes sense that sometimes there would be multiple possible self-consistent models. In those cases, we should expect humans to be unsure/disagree, but we’d still expect people to agree on what the relevant arguments/options are—i.e. the possible models.
I’m not sure what you’re getting at here; you may have a different conception of predictive-processing-like decision theory than I do. I would say “I will get up and go to the store” is a self-consistent model, “I will sit down and read the news” is a self-consistent model, etc. etc. There are always multiple possible self-consistent models—at least one for each possible action that you will take.
Oh, maybe you’re taking the perspective where if you’re hungry you put a high prior on “I will eat soon”. Yeah, I just don’t think that’s right, or if there’s a sensible way to think about it, I haven’t managed to get it despite some effort. I think if you’re hungry, you want to eat because it leads to a predicted reward, not because you have a prior expectation that you will eat. After all, if you’re stuck on a lifeboat in the middle of the ocean, you’re hungry but you don’t expect to eat. This is an obvious point, frequently brought up, and Friston & colleagues hold strong that it’s not a problem for their theory, and I can’t make heads or tails of what their counterargument is. I discussed my version (where rewards are also involved) here, and then here I went into more depth for a specific example.