How safe I am to say such things about complex environments?
So you use choosement to act and you never vary your comparison-value-generator but just feed it with new kinds of inputs as you encounter new situations. You are in a situation you have essentially been in before. You want to do better than last time (not be mad by expecting different results). This means the tie-breaker on what you will do needs to depend on details that make this situation different from the last one. Luckily the situation is only essentially rather than totally the same so those exist. If your set-in-stone comparison-value-generator picks out the correct inessential detail you do an action better than last time.
So wasn’t it important that the right inessential detail was just the way it was? But that kind of makes it important for control to have been so. So last time step you might have had some other essential choice to make, but you might have also had control over inessential details. So for the sake of the future if the future control detail is among them it is important to set it in the good position. But how can you manage that? The detail is ultimately (objectively) inessential, so you do not have any “proper” motive to pay any attention to it. Well you could be similarly superstisous to have random scratchings help you pick the right one.
All these external dependencies either require all to be unique or be conditional on some other detail also being present. Maybe you get lucky and at some point you can use a detail that is for unrelated reasons a good bet. A lot of your comparison-value-picker probably deals with essential details too. However if it only depends on essential details on some choice, you better make a choice you can repeat for eternity in those situations because there ain’t gonna be improvement.
So a EUM that has a comparison-value-generator that is a function of immediately causable world-state only, learns only to the extent it can use the environment as a map and let it think for it. That such “scratch-pads” would be only-for-this-purpose mindstates still keeps it so that a lot of essential stuff about the policy is not apparent in the comparison-value-generator. And you need at install time make a very detailed module and know even for the late parts that will be used late how they need to be or atleast that they are okay to be as they are (and okay to be in the interrim).
Or you could allow the agent to change comparison-value-generator at will. Then it only needs one superstition at each junction to jump to the next one. Correspondingly any inessential dependence can mean more or different inessential dependencies at future times. Install time is still going to be pretty rough but you only need find one needle, instead of 1 per timestamp. If you manage to verify each “phase” separately you do not need to worry about unused inessential dependencies to be carried over to the next phase. The essential parts can also exists only when they are actually needed.
Choosement is not part of my definition of consequentialism.
Searching based on consequences is part of it, and you are right that in the real world you would want to update your model based on new data you learn. In the EUM framework, these updates are captured by Bayesian conditioning. There are other frameworks which capture the updates in other ways, but the basic points are basically the same.
Linking to totalities of very long posts have downsides comparable to writing wall-of-text replies.
I understand how “searching” can fail to be choosement when it ends up being “solving algebraicly” without actually checking any values of the open variables.
Going from abstract descriptions to more and more concrete solutions is not coupled how many elementary ground-level-concrete solutions get disregarded so it can be fast. I thought part of the worryingness of “checks every option” is that it doesn’t get fooled by faulty (or non-existent) abstractions
So to me it is surprising that an agent that never considers alternative avenues gets under the umbrella “consequentialist”. So an agent that changes policy if it is in pain and keeps policy if it feels pleasure, “is consequentialist” based on that its policy was caused by life-events, even if the policy is pure reflex.
There were vibes also to the effect of “this gets me what I want” is a consequentialist stance because of appearance of “gets”. So
Well so you are right that a functioning consequentialist must either magically have perfect knowledge, or must have some way of observing and understanding the world to improve its knowledge. Since magic isn’t real, in reality for advanced capable agents it must be the latter.
In the EUM framework, the observations and improvements in understanding are captured by Bayesian updating. In different frameworks, it may be captured by different things.
“improve knowledge” here can be “its cognition is more fit to the environment”. Somebody could understand “represent the environment more” which it does not need to be.
With such wide understanding it start to liken to me “the agent isn’t broken” which is not exactly structure-anticipation-limiting.
“improve knowledge” here can be “its cognition is more fit to the environment”. Somebody could understand “represent the environment more” which it does not need to be.
Yes, classical Bayesian decision theory often requires a realizability assumption, which is unrealistic.
With such wide understanding it start to liken to me “the agent isn’t broken” which is not exactly structure-anticipation-limiting.
Realizability is anticipation-limiting but unrealistic.
While EUM captures the core of consequentialism, it does so in a way that is not very computationally feasible and leads to certain paradoxes pushed so far. So yes, EUM is unrealistic. The details are discussed in the embedded agency post.
How safe I am to say such things about complex environments?
So you use choosement to act and you never vary your comparison-value-generator but just feed it with new kinds of inputs as you encounter new situations. You are in a situation you have essentially been in before. You want to do better than last time (not be mad by expecting different results). This means the tie-breaker on what you will do needs to depend on details that make this situation different from the last one. Luckily the situation is only essentially rather than totally the same so those exist. If your set-in-stone comparison-value-generator picks out the correct inessential detail you do an action better than last time.
So wasn’t it important that the right inessential detail was just the way it was? But that kind of makes it important for control to have been so. So last time step you might have had some other essential choice to make, but you might have also had control over inessential details. So for the sake of the future if the future control detail is among them it is important to set it in the good position. But how can you manage that? The detail is ultimately (objectively) inessential, so you do not have any “proper” motive to pay any attention to it. Well you could be similarly superstisous to have random scratchings help you pick the right one.
All these external dependencies either require all to be unique or be conditional on some other detail also being present. Maybe you get lucky and at some point you can use a detail that is for unrelated reasons a good bet. A lot of your comparison-value-picker probably deals with essential details too. However if it only depends on essential details on some choice, you better make a choice you can repeat for eternity in those situations because there ain’t gonna be improvement.
So a EUM that has a comparison-value-generator that is a function of immediately causable world-state only, learns only to the extent it can use the environment as a map and let it think for it. That such “scratch-pads” would be only-for-this-purpose mindstates still keeps it so that a lot of essential stuff about the policy is not apparent in the comparison-value-generator. And you need at install time make a very detailed module and know even for the late parts that will be used late how they need to be or atleast that they are okay to be as they are (and okay to be in the interrim).
Or you could allow the agent to change comparison-value-generator at will. Then it only needs one superstition at each junction to jump to the next one. Correspondingly any inessential dependence can mean more or different inessential dependencies at future times. Install time is still going to be pretty rough but you only need find one needle, instead of 1 per timestamp. If you manage to verify each “phase” separately you do not need to worry about unused inessential dependencies to be carried over to the next phase. The essential parts can also exists only when they are actually needed.
Choosement is not part of my definition of consequentialism.
Searching based on consequences is part of it, and you are right that in the real world you would want to update your model based on new data you learn. In the EUM framework, these updates are captured by Bayesian conditioning. There are other frameworks which capture the updates in other ways, but the basic points are basically the same.
How does “searching based on consequences” fail to ever use choosement?
The possibility of alternatives to choosement is discussed here.
Linking to totalities of very long posts have downsides comparable to writing wall-of-text replies.
I understand how “searching” can fail to be choosement when it ends up being “solving algebraicly” without actually checking any values of the open variables.
Going from abstract descriptions to more and more concrete solutions is not coupled how many elementary ground-level-concrete solutions get disregarded so it can be fast. I thought part of the worryingness of “checks every option” is that it doesn’t get fooled by faulty (or non-existent) abstractions
So to me it is surprising that an agent that never considers alternative avenues gets under the umbrella “consequentialist”. So an agent that changes policy if it is in pain and keeps policy if it feels pleasure, “is consequentialist” based on that its policy was caused by life-events, even if the policy is pure reflex.
There were vibes also to the effect of “this gets me what I want” is a consequentialist stance because of appearance of “gets”. So
is consequentialist because it projects winning.
Well so you are right that a functioning consequentialist must either magically have perfect knowledge, or must have some way of observing and understanding the world to improve its knowledge. Since magic isn’t real, in reality for advanced capable agents it must be the latter.
In the EUM framework, the observations and improvements in understanding are captured by Bayesian updating. In different frameworks, it may be captured by different things.
“improve knowledge” here can be “its cognition is more fit to the environment”. Somebody could understand “represent the environment more” which it does not need to be.
With such wide understanding it start to liken to me “the agent isn’t broken” which is not exactly structure-anticipation-limiting.
Yes, classical Bayesian decision theory often requires a realizability assumption, which is unrealistic.
Realizability is anticipation-limiting but unrealistic.
While EUM captures the core of consequentialism, it does so in a way that is not very computationally feasible and leads to certain paradoxes pushed so far. So yes, EUM is unrealistic. The details are discussed in the embedded agency post.