That makes sense to me, and it updates me toward your view on the kid-having thing. (Which wasn’t the focus of my comment, but is a thing I was less convinced of before.) I feel sad about that having happened. :( And curious about whether I (or other people I know) are making a similar mistake.
(My personal state re kids is that it feels a bit odd/uncanny when I imagine myself having them, and I don’t currently viscerally feel like I’m giving something up by not reproducing. Though if I lived for centuries, I suspect I’d want kids eventually in the same way I’d want to have a lot of other cool experiences.)
I feel kinda confused about how “political” my AGI-beliefs are. The idea of dying to AGI feels very sensorily-real to me — I feel like my brain puts it in the same reference class as ‘dying from a gunshot wound’, which is something I worry about at least a little in my day-to-day life (even though I live in a pretty safe area by US megalopolis standards), have bad dreams about, semi-regularly idly imagine experiencing, etc. I don’t know how that relates to the “is this belief political?” question, or how to assess that.
Regardless, I like this:
I am worried that people may say this, intend it with some portion of themselves, but have their animal hear “I’m going to abandon these goals and live in belief-in-beliefs.”
I’d also assume by default that ‘the animal discounts more heavily than the philosopher does’ is a factor here...? And/or ‘the animal is better modeled on this particular question as an adaptation-executer rather than a utility-maximizer, such that there’s no trade you can make that will satisfy the animal if it involves trading away having-kids-before-age-45’?
It could be wise to have kids, for the sake of harmony between your parts/subgoals, even if you judge that having kids isn’t directly x-risk-useful and your animal seems to have a pretty visceral appreciation for x-risk—just because that is in fact what an important part of you wants/needs/expects/etc.
❤
That makes sense to me, and it updates me toward your view on the kid-having thing. (Which wasn’t the focus of my comment, but is a thing I was less convinced of before.) I feel sad about that having happened. :( And curious about whether I (or other people I know) are making a similar mistake.
(My personal state re kids is that it feels a bit odd/uncanny when I imagine myself having them, and I don’t currently viscerally feel like I’m giving something up by not reproducing. Though if I lived for centuries, I suspect I’d want kids eventually in the same way I’d want to have a lot of other cool experiences.)
I feel kinda confused about how “political” my AGI-beliefs are. The idea of dying to AGI feels very sensorily-real to me — I feel like my brain puts it in the same reference class as ‘dying from a gunshot wound’, which is something I worry about at least a little in my day-to-day life (even though I live in a pretty safe area by US megalopolis standards), have bad dreams about, semi-regularly idly imagine experiencing, etc. I don’t know how that relates to the “is this belief political?” question, or how to assess that.
Regardless, I like this:
I’d also assume by default that ‘the animal discounts more heavily than the philosopher does’ is a factor here...? And/or ‘the animal is better modeled on this particular question as an adaptation-executer rather than a utility-maximizer, such that there’s no trade you can make that will satisfy the animal if it involves trading away having-kids-before-age-45’?
It could be wise to have kids, for the sake of harmony between your parts/subgoals, even if you judge that having kids isn’t directly x-risk-useful and your animal seems to have a pretty visceral appreciation for x-risk—just because that is in fact what an important part of you wants/needs/expects/etc.