I don’t think the GPS “searches over all relevant plans”
OK, but you are positing that there’s an argmin, no? That’s a big part of what I’m objecting to. I anticipate that insofar as you’re claiming grader-optimization problems come back, they come back because there’s an AFAICT inappropriate argmin which got tossed into the analysis via the GPS.
But it’s not made of heuristics; rather, it’s something like a systematic way of drawing upon the declarative knowledge/knowledge explicitly represented in the world-model, and that knowledge involves a lot of heuristics.
Sure, sounds reasonable.
Crucially, part of any “problem specification” would be things like “how much time should I spend on thinking about this?” and “how hard should I optimize the plan for doing it?” and “in how much detail should I track the consequences of this decision?”, and if it’s something minor like getting ice cream, then of course you’d spend very little time and use a lot of cached cognitive shortcuts.
Noting that I still feel confused after hearing this explanation. What does it mean to ask “how hard should I optimize”?
If it’s something major, however, like a life-or-death matter, then you’d do high-intensity planning that aims to track what would actually happen in detail, without relying on prior assumptions and vague feelings
Really? I think that people usually don’t do that in life-or-death scenarios. People panic all the time.
What does it mean to ask “how hard should I optimize”?
Satisficing threshold, probability of the plan’s success, the plan’s robustness to unexpected perturbations, etc. I suppose the argmin is somewhat misleading: the GPS doesn’t output the best possible plan for achieving some goal in the world outside the agent, it’s solving the problem in the most efficient way possible, which often means not spending too much time and resources on it. I. e., “mental resources spent” is part of the problem specification, and it’s something it tries to minimize too.
I don’t think this argmin is the central reason for grader-optimization problems here.
Really? I think that people usually don’t do that in life-or-death scenarios. People panic all the time.
I’m assuming no time pressure. Or substitute-in “a matter of grave importance that you nonetheless feel capable of resolving”.
I don’t think this argmin is the central reason for grader-optimization problems here.
I’m going to read the rest of the essay, and also I realize you posted this before myfourpostson “holy cow argmax can blow all your alignment reasoning out of reality all the way to candyland.” But I want to note that including an argmin in the posited motivational architecture makes me extremely nervous / distrusting. Even if this modeling assumption doesn’t end up being central to your arguments on how shard-agents become wrapper-like, I think this assumption should still be flagged extremely heavily.
Mm, I believe that it’s not central because my initial conception of the GPS didn’t include it at all, and everything still worked. I don’t think it serves the same role here as you’re critiquing in the posts you’ve linked; I think it’s inserted at a different abstraction level.
But sure, I’ll wait for you to finish with the post.
OK, but you are positing that there’s an argmin, no? That’s a big part of what I’m objecting to. I anticipate that insofar as you’re claiming grader-optimization problems come back, they come back because there’s an AFAICT inappropriate argmin which got tossed into the analysis via the GPS.
Sure, sounds reasonable.
Noting that I still feel confused after hearing this explanation. What does it mean to ask “how hard should I optimize”?
Really? I think that people usually don’t do that in life-or-death scenarios. People panic all the time.
Satisficing threshold, probability of the plan’s success, the plan’s robustness to unexpected perturbations, etc. I suppose the argmin is somewhat misleading: the GPS doesn’t output the best possible plan for achieving some goal in the world outside the agent, it’s solving the problem in the most efficient way possible, which often means not spending too much time and resources on it. I. e., “mental resources spent” is part of the problem specification, and it’s something it tries to minimize too.
I don’t think this argmin is the central reason for grader-optimization problems here.
I’m assuming no time pressure. Or substitute-in “a matter of grave importance that you nonetheless feel capable of resolving”.
I’m going to read the rest of the essay, and also I realize you posted this before my four posts on “holy cow argmax can blow all your alignment reasoning out of reality all the way to candyland.” But I want to note that including an argmin in the posited motivational architecture makes me extremely nervous / distrusting. Even if this modeling assumption doesn’t end up being central to your arguments on how shard-agents become wrapper-like, I think this assumption should still be flagged extremely heavily.
Mm, I believe that it’s not central because my initial conception of the GPS didn’t include it at all, and everything still worked. I don’t think it serves the same role here as you’re critiquing in the posts you’ve linked; I think it’s inserted at a different abstraction level.
But sure, I’ll wait for you to finish with the post.