Thoughts on Modeling Naturalized Logic Decision Theory Problems in Linear Logic
I hadn’t heard of linear logic before—it seems like a cool formalisation—although I tend to believe that formalisations are overrated as unless they are used very carefully they can obscure more than they reveal.
I believe that spurious counterfactuals are only an issue with the 5 and 10 problem because of an attempt to hack logical-if to substitute for counterfactual-if in such a way that we can reuse proof-based systems. It’s extremely cool that we can do as much as we can working in that fashion, but there’s no reason why we should be surprised that it runs into limits.
So I don’t see inventing alternative formalisations that avoid the 5 and 10 problem as particularly hard as the bug is really quite specific to systems that try to utilise this kind of hack. I’d expect that almost any other system in design space will avoid this. So if, as I claim, attempts at formalisation will avoid this issue by default, the fact that any one formalisation avoids this problem shouldn’t give us too much confidence in it being a good system for representing counterfactuals in general.
Instead, I think it’s much more persuasive to ground any proposed system with philosophical arguments (such as your first post was focusing on), rather than mostly just posting a system and observing it has a few nice properties. I mean, your approach in this article certainly a valuable thing to do, but I don’t see it as getting all the way to the heart of the issue.
One way is by asserting that the logic is about the territory, while the proof system is about the map; so, counterfactuals are represented in the map, even though the map itself asserts that there is only a singular territory.
Interestingly enough, this mirrors my position in Why 1-boxing doesn’t imply backwards causation where I distinguish between Raw Reality (the territory) and Augmented Reality (the territory augmented by counterfactuals). I guess I put more emphasis on delving into the philosophical reasons for such a view and I think that’s what this post is a bit short on.
I’m not sure where you got the idea that this was to solve the spurious counterfactuals problem, that was in the appendix because I anticipated that a MIRI-adjacent person would want to know how it solves that problem.
The core problem it’s solving is that it’s a well-defined mathematical framework in which (a) there are, in some sense, choices, and (b) it is believed that these choices correspond to the results of a particular Turing machine. It goes back to the free will vs determinism paradox, and shows that there’s a formalism that has some properties of “free will” and some properties of “determinism”.
A way that EDT fails to solve 5 and 10 is that it could believe with 100% certainty that it takes $5 so its expected value for $10 is undefined. (I wrote previously about a modification of EDT to avoid this problem.)
CDT solves it by constructing physically impossible counterfactuals which has other problems, e.g. suppose there’s a Laplace’s demon that searches for violations of physics and destroys the universe if physics is violated; this theoretically shouldn’t make a difference but it messes up the CDT counterfactuals.
It does look like your post overall agrees with the view I presented. I would tend to call augmented reality “metaphysics” in that it is a piece of ontology that goes beyond physics. I wrote about metaphysical free will a while ago and didn’t post it on LW because I anticipated people would be allergic to the non-physicalist philosophical language.
I’m not sure where you got the idea that this was to solve the spurious counterfactuals problem, that was in the appendix because I anticipated that a MIRI-adjacent person would want to know how it solves that problem.
Thanks for that clarification.
A way that EDT fails to solve 5 and 10 is that it could believe with 100% certainty that it takes $5 so its expected value for $10 is undefined
I suppose that demonstrates that the 5 and 10 problem is a broader problem than I realised. I still think that it’s only a hard problem within particular systems that have a vulnerability to it.
It does look like your post overall agrees with the view I presented. I would tend to call augmented reality “metaphysics” in that it is a piece of ontology that goes beyond physics
Yeah, we have significant agreement, but I’m more conservative in my interpretations. I guess this is a result of me being, at least in my opinion, more skeptical of language. Like I’m very conscious of arguments where someone says, “X could be described by phrase Y” and then later they rely on connations of Y that weren’t proven.
For example, you write, “From the AI’s perspective, it has a choice among multiple actions, hence in a sense “believing in metaphysical free will”. I would suggest it would be more accurate to write: “The AI models the situation as though it had free will” which leaves open the possibility that it is might be just a pragmatic model, rather than the AI necessarily endorsing itself as possessing free will.
Another way of framing this: there’s an additional step in between observing that an agent acts or models a situation as it believes in freewill and concluding that it actually believes in freewill. For example, I might round all numbers in a calculation to integers in order to make it easier for me, but that doesn’t mean that I believe that the values are integers.
Thoughts on Modeling Naturalized Logic Decision Theory Problems in Linear Logic
I hadn’t heard of linear logic before—it seems like a cool formalisation—although I tend to believe that formalisations are overrated as unless they are used very carefully they can obscure more than they reveal.
I believe that spurious counterfactuals are only an issue with the 5 and 10 problem because of an attempt to hack logical-if to substitute for counterfactual-if in such a way that we can reuse proof-based systems. It’s extremely cool that we can do as much as we can working in that fashion, but there’s no reason why we should be surprised that it runs into limits.
So I don’t see inventing alternative formalisations that avoid the 5 and 10 problem as particularly hard as the bug is really quite specific to systems that try to utilise this kind of hack. I’d expect that almost any other system in design space will avoid this. So if, as I claim, attempts at formalisation will avoid this issue by default, the fact that any one formalisation avoids this problem shouldn’t give us too much confidence in it being a good system for representing counterfactuals in general.
Instead, I think it’s much more persuasive to ground any proposed system with philosophical arguments (such as your first post was focusing on), rather than mostly just posting a system and observing it has a few nice properties. I mean, your approach in this article certainly a valuable thing to do, but I don’t see it as getting all the way to the heart of the issue.
Interestingly enough, this mirrors my position in Why 1-boxing doesn’t imply backwards causation where I distinguish between Raw Reality (the territory) and Augmented Reality (the territory augmented by counterfactuals). I guess I put more emphasis on delving into the philosophical reasons for such a view and I think that’s what this post is a bit short on.
Thanks for reading all the posts!
I’m not sure where you got the idea that this was to solve the spurious counterfactuals problem, that was in the appendix because I anticipated that a MIRI-adjacent person would want to know how it solves that problem.
The core problem it’s solving is that it’s a well-defined mathematical framework in which (a) there are, in some sense, choices, and (b) it is believed that these choices correspond to the results of a particular Turing machine. It goes back to the free will vs determinism paradox, and shows that there’s a formalism that has some properties of “free will” and some properties of “determinism”.
A way that EDT fails to solve 5 and 10 is that it could believe with 100% certainty that it takes $5 so its expected value for $10 is undefined. (I wrote previously about a modification of EDT to avoid this problem.)
CDT solves it by constructing physically impossible counterfactuals which has other problems, e.g. suppose there’s a Laplace’s demon that searches for violations of physics and destroys the universe if physics is violated; this theoretically shouldn’t make a difference but it messes up the CDT counterfactuals.
It does look like your post overall agrees with the view I presented. I would tend to call augmented reality “metaphysics” in that it is a piece of ontology that goes beyond physics. I wrote about metaphysical free will a while ago and didn’t post it on LW because I anticipated people would be allergic to the non-physicalist philosophical language.
Thanks for that clarification.
I suppose that demonstrates that the 5 and 10 problem is a broader problem than I realised. I still think that it’s only a hard problem within particular systems that have a vulnerability to it.
Yeah, we have significant agreement, but I’m more conservative in my interpretations. I guess this is a result of me being, at least in my opinion, more skeptical of language. Like I’m very conscious of arguments where someone says, “X could be described by phrase Y” and then later they rely on connations of Y that weren’t proven.
For example, you write, “From the AI’s perspective, it has a choice among multiple actions, hence in a sense “believing in metaphysical free will”. I would suggest it would be more accurate to write: “The AI models the situation as though it had free will” which leaves open the possibility that it is might be just a pragmatic model, rather than the AI necessarily endorsing itself as possessing free will.
Another way of framing this: there’s an additional step in between observing that an agent acts or models a situation as it believes in freewill and concluding that it actually believes in freewill. For example, I might round all numbers in a calculation to integers in order to make it easier for me, but that doesn’t mean that I believe that the values are integers.