So I think maybe there’s something goin on here where I’m taking too much for granted that formal systems are the best way to figure out what to believe about the world to get an accurate picture of it, and so long as you’re figuring out what to believe using some formal system then you’re forced to ground it in some assumed foundation that is not itself justified. This seems important because it means there’s a lot of interesting stuff going on with those assumptions and how they get chosen such that they cause the rest of the system to produce beliefs that track reality.
But it sounds like you think this is a confused view. I admit I feel a bit differently about things than the way they’re presented in the post, but not entirely, as my comment reflects. I think when I wrote the post I really couldn’t see my way around how to know things without being able to formally justify them. Now I better understand there can be knowing without belief that knowledge is justified or true, but I do think there’s something to the idea that I am still too much captured by my own ontology and not fully seeing it as I see through it, and this causes me to get a bit mixed up in places.
The bit about self-trust is really interesting. To get a little personal and psychological, I think the big difference between me ~4 years ago and me now is that ~4 years ago I learned to trust myself in some big important way that goes deeper than surface level self-trust. On the surface this looks like trusting my feelings and experiences, but to a certain extent anyone can do this if you just give them feedback that says trust those things more than other things. But I’m talking about something a level deeper where I trust the process by which those things get generated to do what they do and that they’re doing their best (in the sense that the universe is deterministic and we’re all always doing our best/worst because it could have been no other way).
If I’m honest I’ve really struggled to understand what’s going on with Lob’s theorem and why it seems like a big deal to everyone. But it seems like from what you’re saying it’s related to the issue that got me started on things, which is how do you deal with the problem of verifying that another agent (which could be yourself at another time) shares your values and is thus in some way aligned with you. If that’s the case maybe I already grasp the fundamentals of this obstacle Lob seems to create and have just failed to realize that’s what folks were talking about when they talk about Lob with respect to aligning AI.
@abramdemski Wanted to say thanks again for engaging with my posts and pointing me towards looking again at Lob. It’s weird: now that I’ve taken so time to understand it, it’s just what in my mind was already the thing going on with Godel, just I wasn’t doing a great job of separating out what Godel proves and what the implications are. As presented on its own, Lob didn’t seem that interesting to me so I kept bouncing off it as something worth looking at, but now I realize it’s just the same thing I learned from GEB’s presentation of Peano arithmetic and Godel when I read it 20+ years ago.
When I go back to make revisions to the book, I’ll have to reconsider including Godel and Lob somehow in the text. I didn’t because I felt like it was a bit complicated and I didn’t really need to dig into it since I think there’s already a bit too many cases where people use Godel to overreach and draw conclusions that aren’t true, but it’s another way to explain these ideas. I just have to think about if Godel and Lob are necessary: that is, do I need to appeal to them to make my key points, or are these things that are better left as additional topics I can point folks at but not key to understanding the intuitions I want them to develop.
I’ve heard Lob remarked that he would never have published if he realized earlier how close his theorem was to just Godel’s second incompleteness theorem; but I can’t seem to entirely agree with Lob there. It does seem like a valuable statement of its own.
I agree, Godel is dangerously over-used, so the key question is whether it’s necessary here. Other formal analogs of your point include Tarski’s undefinability, and the realizablility / grain-of-truth problem. There are many ways to gesture towards a sense of “fundamental uncertainty”, so the question is: what statement of the thing do you want to make most central, and how do you want to argue/illustrate that statement?
Hmm, I’ll have to think about this.
So I think maybe there’s something goin on here where I’m taking too much for granted that formal systems are the best way to figure out what to believe about the world to get an accurate picture of it, and so long as you’re figuring out what to believe using some formal system then you’re forced to ground it in some assumed foundation that is not itself justified. This seems important because it means there’s a lot of interesting stuff going on with those assumptions and how they get chosen such that they cause the rest of the system to produce beliefs that track reality.
But it sounds like you think this is a confused view. I admit I feel a bit differently about things than the way they’re presented in the post, but not entirely, as my comment reflects. I think when I wrote the post I really couldn’t see my way around how to know things without being able to formally justify them. Now I better understand there can be knowing without belief that knowledge is justified or true, but I do think there’s something to the idea that I am still too much captured by my own ontology and not fully seeing it as I see through it, and this causes me to get a bit mixed up in places.
The bit about self-trust is really interesting. To get a little personal and psychological, I think the big difference between me ~4 years ago and me now is that ~4 years ago I learned to trust myself in some big important way that goes deeper than surface level self-trust. On the surface this looks like trusting my feelings and experiences, but to a certain extent anyone can do this if you just give them feedback that says trust those things more than other things. But I’m talking about something a level deeper where I trust the process by which those things get generated to do what they do and that they’re doing their best (in the sense that the universe is deterministic and we’re all always doing our best/worst because it could have been no other way).
If I’m honest I’ve really struggled to understand what’s going on with Lob’s theorem and why it seems like a big deal to everyone. But it seems like from what you’re saying it’s related to the issue that got me started on things, which is how do you deal with the problem of verifying that another agent (which could be yourself at another time) shares your values and is thus in some way aligned with you. If that’s the case maybe I already grasp the fundamentals of this obstacle Lob seems to create and have just failed to realize that’s what folks were talking about when they talk about Lob with respect to aligning AI.
@abramdemski Wanted to say thanks again for engaging with my posts and pointing me towards looking again at Lob. It’s weird: now that I’ve taken so time to understand it, it’s just what in my mind was already the thing going on with Godel, just I wasn’t doing a great job of separating out what Godel proves and what the implications are. As presented on its own, Lob didn’t seem that interesting to me so I kept bouncing off it as something worth looking at, but now I realize it’s just the same thing I learned from GEB’s presentation of Peano arithmetic and Godel when I read it 20+ years ago.
When I go back to make revisions to the book, I’ll have to reconsider including Godel and Lob somehow in the text. I didn’t because I felt like it was a bit complicated and I didn’t really need to dig into it since I think there’s already a bit too many cases where people use Godel to overreach and draw conclusions that aren’t true, but it’s another way to explain these ideas. I just have to think about if Godel and Lob are necessary: that is, do I need to appeal to them to make my key points, or are these things that are better left as additional topics I can point folks at but not key to understanding the intuitions I want them to develop.
I’ve heard Lob remarked that he would never have published if he realized earlier how close his theorem was to just Godel’s second incompleteness theorem; but I can’t seem to entirely agree with Lob there. It does seem like a valuable statement of its own.
I agree, Godel is dangerously over-used, so the key question is whether it’s necessary here. Other formal analogs of your point include Tarski’s undefinability, and the realizablility / grain-of-truth problem. There are many ways to gesture towards a sense of “fundamental uncertainty”, so the question is: what statement of the thing do you want to make most central, and how do you want to argue/illustrate that statement?