Do you think you can deny existence of an outcome with infinite utility? The fact that things “break down” is not a valid argument. If you cannot deny—it’s possible. And it it’s possible—alignment impossible.
Do you think you can deny existence of an outcome with infinite utility?
To me, according to my preferences/goals/inclinations, there are conceivable outcomes with infinite utility/disutility.
But I think it is possible (and feasible) for a program/mind to be extremely capable, and affect the world, and not “care” about infinite outcomes.
The fact that things “break down” is not a valid argument.
I guess that depends on what’s being discussed. Like, it is something to take into account/consideration if you want to prove something while referencing utility-functions that reference infinities.
But I think it is possible (and feasible) for a program/mind to be extremely capable, and affect the world, and not “care” about infinite outcomes.
As I understand you do not agree with
If an outcome with infinite utility is presented, then it doesn’t matter how small its probability is: all actions which lead to that outcome will have to dominate the agent’s behavior.
from Pascal’s Mugging, not with me. Do you have any arguments for that?
I do have arguments for that, and I have already mentioned some of them earlier in our discussion (you may not share that assesment, despite us being relatively close in mind-space compared to most possible minds, but oh well).
Some of the more relevant comments from me are on one of the posts that you deleted.
As I mention here, I think I’ll try to round off this discussion. (Edit: I had a malformed/misleading sentence in that comment that should be fixed now.)
Do you think you can deny existence of an outcome with infinite utility? The fact that things “break down” is not a valid argument. If you cannot deny—it’s possible. And it it’s possible—alignment impossible.
To me, according to my preferences/goals/inclinations, there are conceivable outcomes with infinite utility/disutility.
But I think it is possible (and feasible) for a program/mind to be extremely capable, and affect the world, and not “care” about infinite outcomes.
I guess that depends on what’s being discussed. Like, it is something to take into account/consideration if you want to prove something while referencing utility-functions that reference infinities.
As I understand you do not agree with
from Pascal’s Mugging, not with me. Do you have any arguments for that?
I do have arguments for that, and I have already mentioned some of them earlier in our discussion (you may not share that assesment, despite us being relatively close in mind-space compared to most possible minds, but oh well).
Some of the more relevant comments from me are on one of the posts that you deleted.
As I mention here, I think I’ll try to round off this discussion. (Edit: I had a malformed/misleading sentence in that comment that should be fixed now.)