I would say it feels like my brain’s built in values are mostly a big subgoal stomp, of mutually contradictory, inconsistent, and changeable values. [...]
it feels like my brain has this longing to find a small, principled, consistent set of terminal values that I could use to make decisions instead.
Here’s a slate star codex piece on our best guess on how our motivational system works: https://slatestarcodex.com/2018/02/07/guyenet-on-motivation/.It’s essentially just a bunch of small mostly independent modules all fighting for control of the body to act according to what they want.
I don’t think there’s any way out of having “mutually contradictory, inconsistent, and changeable values.” We just gotta negotiate between these as best we can.
There are at least a couple problems with trying to come up with a “small, principled, consistent set of terminal values” you could use to make decisions.
You’re never gonna be able to do it in a way that covered all edge cases.
Even if you were able to come up with the “right” system, you wouldn’t actually be able to follow it. Because our actual motivational systems aren’t simple rule following systems. You’re gonna want what you want, even if your predetermined system says to do otherwise.
You don’t really get to decide what your terminal values are. I mean you can fudge it a bit, but you certainly don’t have complete control over them (and thank god).
Negotiating between competing values isn’t something you can smooth over with a few rules. Instead it requires some degree of finesse and moment to moment awareness.
Do you play any board games? In chess there are a lot of what we can call “values.” Better to keep your king safe, control the center, don’t double your pawns etc. But there’s no “small, principled, consistent set of” rules you can use to negotiate between these. It’s always gotta be felt out in each new situation.
And life is much messier and more complex than something like chess.
It sounds like both of you may have gone through the exercise of find terminal goals that work for you.
I “found terminal goals” in the sense that I tried to figure out what were the main things I wanted in life. I came up with some sort of list (which will probably change in the future). It’s a short list, but definitely not principled or consistent :D. Occasionally it does help to keep me focused on what matters to me. If I find myself spending a lot of time doing stuff that doesn’t go in one of those directions, I try to put myself more on track.
If you want I can try to figure out how I got there. But it seems like your more concerned with the deciding between competing values thing.
Inclusive genetic fitness seems like it may be a reasonable terminal goal to replace the subgoal stomp.
Ya definitely don’t do that. If you did that you’d just spend all your time donating sperm or something.
Thanks Viliam and weathersystems! Sorry it took me a little while to respond, I wanted to make sure I had read and understood the pointers to the related work you guys provided.
Ya definitely don’t do that. If you did that you’d just spend all your time donating sperm or something.
If you liked the article, you might want to try the whole book. It will also answer your question about replacing your goals with inclusive genetic fitness. (Spoiler: bad idea, evolution is not your friend.)
I spent some time digging deeper into inclusive fitness. Social Evolution and Inclusive Fitness Theory: An Introduction by James A. R. Marshall provides a good summary.
There are indeed proofs which show that evolution selects for those individuals that always maximize inclusive genetic fitness in the moment. That said, these proofs assume that maximizing offspring in the moment won’t hurt your chances of creating offspring in the future.
So these proofs don’t really apply to the situations that humans find themselves in. Raping the nearest person may give you the highest inclusive genetic fitness in the moment, but you will go to jail and won’t be able to reproduce any further, so this behavior won’t be favored by evolution in the long run (thank god).
So yeah, definitely don’t maximize short term inclusive genetic fitness.
So what about maximizing inclusive genetic fitness in the long term, say 1 billion years? I couldn’t find any papers analyzing what evolution would do to a strategy like that, but it intuitively sounds a lot better. If that was your terminal goal, you would probably want to push science forward, advance technology, spread the human race as far out into the universe as you can, etc. Honestly, those sound like pretty reasonable things I’d be happy to support.
1. You’re never gonna be able to do it in a way that covered all edge cases.
I completely agree that this is a possible (even likely) scenario. I really don’t know, that’s why I’m asking :)
2. Even if you were able to come up with the “right” system, you wouldn’t actually be able to follow it. Because our actual motivational systems aren’t simple rule following systems. You’re gonna want what you want, even if your predetermined system says to do otherwise.
3. You don’t really get to decide what your terminal values are. I mean you can fudge it a bit, but you certainly don’t have complete control over them (and thank god).
I agree that it’s going to be impossible to completely change my behavior, just because I change my value system. e.g. no matter what the terminal goal would be, I’d still spend many hours a day sleeping, I’d still need narcotics to go through surgery, and I still wouldn’t be able to eat food that tastes absolutely terrible.
That said, I’d say there’s like 80% of my after tax income, 100% of my wealth, and maybe 12 hours a day that I can allocate pretty freely. For example, I think there would be a huge difference between living a life where I donate all my income, where I travel the world, where I play video games all day, where I have 12 kids, where I work for a random company 12 hours a day, or where I live on welfare. All of those are actions that I could absolutely take, if I (and maybe my wife) were convinced that they are the right thing to do.
Do you play any board games? In chess there are a lot of what we can call “values.” Better to keep your king safe, control the center, don’t double your pawns etc. But there’s no “small, principled, consistent set of” rules you can use to negotiate between these. It’s always gotta be felt out in each new situation.
Completely agree, even if you know the terminal goal, that doesn’t mean you’d have the computational capacity to always act optimally. That said, I’d be very happy if I were in a state like chess, i.e. where my terminal goal is very clear, even if I’m not always 100% able to take the perfect action to achieve that goal.
With problems like “my work or my family”, the key is usually to get out of this frame and instead think about specific details.
I love this sentiment. I generally think people underestimate how often you can have the cake and eat it too if you just think a bit outside the box.
Here’s a slate star codex piece on our best guess on how our motivational system works: https://slatestarcodex.com/2018/02/07/guyenet-on-motivation/.It’s essentially just a bunch of small mostly independent modules all fighting for control of the body to act according to what they want.
I don’t think there’s any way out of having “mutually contradictory, inconsistent, and changeable values.” We just gotta negotiate between these as best we can.
There are at least a couple problems with trying to come up with a “small, principled, consistent set of terminal values” you could use to make decisions.
You’re never gonna be able to do it in a way that covered all edge cases.
Even if you were able to come up with the “right” system, you wouldn’t actually be able to follow it. Because our actual motivational systems aren’t simple rule following systems. You’re gonna want what you want, even if your predetermined system says to do otherwise.
You don’t really get to decide what your terminal values are. I mean you can fudge it a bit, but you certainly don’t have complete control over them (and thank god).
Negotiating between competing values isn’t something you can smooth over with a few rules. Instead it requires some degree of finesse and moment to moment awareness.
Do you play any board games? In chess there are a lot of what we can call “values.” Better to keep your king safe, control the center, don’t double your pawns etc. But there’s no “small, principled, consistent set of” rules you can use to negotiate between these. It’s always gotta be felt out in each new situation.
And life is much messier and more complex than something like chess.
I “found terminal goals” in the sense that I tried to figure out what were the main things I wanted in life. I came up with some sort of list (which will probably change in the future). It’s a short list, but definitely not principled or consistent :D. Occasionally it does help to keep me focused on what matters to me. If I find myself spending a lot of time doing stuff that doesn’t go in one of those directions, I try to put myself more on track.
If you want I can try to figure out how I got there. But it seems like your more concerned with the deciding between competing values thing.
Ya definitely don’t do that. If you did that you’d just spend all your time donating sperm or something.
Thanks Viliam and weathersystems! Sorry it took me a little while to respond, I wanted to make sure I had read and understood the pointers to the related work you guys provided.
I spent some time digging deeper into inclusive fitness. Social Evolution and Inclusive Fitness Theory: An Introduction by James A. R. Marshall provides a good summary.
There are indeed proofs which show that evolution selects for those individuals that always maximize inclusive genetic fitness in the moment. That said, these proofs assume that maximizing offspring in the moment won’t hurt your chances of creating offspring in the future.
So these proofs don’t really apply to the situations that humans find themselves in. Raping the nearest person may give you the highest inclusive genetic fitness in the moment, but you will go to jail and won’t be able to reproduce any further, so this behavior won’t be favored by evolution in the long run (thank god).
So yeah, definitely don’t maximize short term inclusive genetic fitness.
So what about maximizing inclusive genetic fitness in the long term, say 1 billion years? I couldn’t find any papers analyzing what evolution would do to a strategy like that, but it intuitively sounds a lot better. If that was your terminal goal, you would probably want to push science forward, advance technology, spread the human race as far out into the universe as you can, etc. Honestly, those sound like pretty reasonable things I’d be happy to support.
I completely agree that this is a possible (even likely) scenario. I really don’t know, that’s why I’m asking :)
I agree that it’s going to be impossible to completely change my behavior, just because I change my value system. e.g. no matter what the terminal goal would be, I’d still spend many hours a day sleeping, I’d still need narcotics to go through surgery, and I still wouldn’t be able to eat food that tastes absolutely terrible.
That said, I’d say there’s like 80% of my after tax income, 100% of my wealth, and maybe 12 hours a day that I can allocate pretty freely. For example, I think there would be a huge difference between living a life where I donate all my income, where I travel the world, where I play video games all day, where I have 12 kids, where I work for a random company 12 hours a day, or where I live on welfare. All of those are actions that I could absolutely take, if I (and maybe my wife) were convinced that they are the right thing to do.
Completely agree, even if you know the terminal goal, that doesn’t mean you’d have the computational capacity to always act optimally. That said, I’d be very happy if I were in a state like chess, i.e. where my terminal goal is very clear, even if I’m not always 100% able to take the perfect action to achieve that goal.
I love this sentiment. I generally think people underestimate how often you can have the cake and eat it too if you just think a bit outside the box.