I think the smoking lesion problem is one of those intuition pumps that you have to be *very* careful with mathematizing and comparing with other things. Let me just quote myself from the past:
In the Smoking Lesion problem, and in similar cases where you consider an agent to have “urges” or “dispositions” et c., it’s important to note that these are pre-mathematical descriptions of something we’d like our decision theory to consider, and that to try to directly apply them to a mathematical theory is to commit a sort of type error.
Specifically, a decision-making procedure that “has a disposition to smoke” is not FDT. It is some other decision theory that has the capability to operate in uncertainty about its own dispositions.
I think it’s totally reasonable to say that we want to research decision theories that are capable of this, because this epistemic state of not being quite sure of your own mind is something humans have to deal with all the time. But one cannot start with a mathematically specified decision theory like proof-based UDT or causal-graph-based CDT and then ask “what it would do if it had the smoking lesion.” It’s a question that seems intuitively reasonable but, when made precise, is nonsense.
I think what this feels like to philosophers is giving the verbal concepts primacy over the math. (With positive associations to “concepts” and negative associations to “math” implied). But what it leads to in practice is people saying “but what about the tickle defense?” or “but what about different formulations of CDT” as if they were talking about different facets of unified concepts (the things that are supposed to have primacy), when these facets have totally distinct mathematizations.
At some point, if you know that a tree falling in the forest makes the air vibrate but doesn’t lead to auditory experiences, it’s time to stop worrying about whether it makes a sound.
So obviously I (and LW orthodoxy) are on the pro-math side, and I think most philosophers are on the pro-concepts side (I’d say “pro-essences,” but that’s a bit too on the nose). But, importantly, if we agree that this descriptive difference exists, then we can at least work to bridge it by being clear about whether were’s using the math perspective or the concept perspective. Then we can keep different mathematizations strictly separate when using the math perspective, but work to amalgamate them when talking about concepts.
If there was one criticism I’d like to repeat, it’s that framing the smoking lesion problem in terms of clean decisions between counterfactuals is already missing something from the pre-mathematical description of the problem. The problem is interesting because we as humans sometimes have to worry that we’re running on “corrupted hardware”—it seems to me that mathematization of this idea requires us to somehow mutilate the decision theories we’re allowed to consider.
To look at this from another angle: I’m agreeing that the counterfactuals are “socio-linguistic conventions”—and I want to go even further and place the entire problem within a context that allows it to have lots of unique quirks depending on the ideas it’s expressing, rather than having only the straightforward standardized interpretation. I see this as a feature, not a bug, and think that we can afford to be “greedy” in trying to hang on to the semantics of the problem statement rather than “lazy” in trying to come up with an efficient model.
Yeah, I agree that I haven’t completely engaged with the issue of “corrupted hardware”, but it seems like any attempt to do this would require so much interpretation that I wouldn’t expect to obtain agreement over whether I had interpreted it correctly. In any case, my aim is purely to solve counterfactuals for non-corrupted agents, at least for now. But glad to see that someone agrees with me about socio-linguistic conventions :-)
Sure. I have this sort of instinctive mental pushback because I think of counterfactuals primarily as useful tools for a planning agent, but I’m assuming that you don’t mean to deny this, and are just applying different emphasis.
Yeah, there’s definitely a tension between being a social-linguistic construct and being pragmatically useful (such as what you might need for a planning agent). I don’t completely know how to resolve this yet, but this post makes a start by noting that in additional to the social linguistic elements, the strength of the physical linkage between elements is important as well. My intuition is that there are a bunch of properties that make something more or less counterfactual and the social-linguistic conventions are about a) which of these properties are present when the problem is ambiguous b) which of these properties need to be satisfied before we accept a counterfactual as valid.
I think the smoking lesion problem is one of those intuition pumps that you have to be *very* careful with mathematizing and comparing with other things. Let me just quote myself from the past:
Do you have a specific objection to what I wrote?
Oh, I more or less agree :P
If there was one criticism I’d like to repeat, it’s that framing the smoking lesion problem in terms of clean decisions between counterfactuals is already missing something from the pre-mathematical description of the problem. The problem is interesting because we as humans sometimes have to worry that we’re running on “corrupted hardware”—it seems to me that mathematization of this idea requires us to somehow mutilate the decision theories we’re allowed to consider.
To look at this from another angle: I’m agreeing that the counterfactuals are “socio-linguistic conventions”—and I want to go even further and place the entire problem within a context that allows it to have lots of unique quirks depending on the ideas it’s expressing, rather than having only the straightforward standardized interpretation. I see this as a feature, not a bug, and think that we can afford to be “greedy” in trying to hang on to the semantics of the problem statement rather than “lazy” in trying to come up with an efficient model.
Yeah, I agree that I haven’t completely engaged with the issue of “corrupted hardware”, but it seems like any attempt to do this would require so much interpretation that I wouldn’t expect to obtain agreement over whether I had interpreted it correctly. In any case, my aim is purely to solve counterfactuals for non-corrupted agents, at least for now. But glad to see that someone agrees with me about socio-linguistic conventions :-)
Sure. I have this sort of instinctive mental pushback because I think of counterfactuals primarily as useful tools for a planning agent, but I’m assuming that you don’t mean to deny this, and are just applying different emphasis.
Yeah, there’s definitely a tension between being a social-linguistic construct and being pragmatically useful (such as what you might need for a planning agent). I don’t completely know how to resolve this yet, but this post makes a start by noting that in additional to the social linguistic elements, the strength of the physical linkage between elements is important as well. My intuition is that there are a bunch of properties that make something more or less counterfactual and the social-linguistic conventions are about a) which of these properties are present when the problem is ambiguous b) which of these properties need to be satisfied before we accept a counterfactual as valid.