Exploitable? Please explain!
Heighn
Karma: 285
Ah, I just read your substack post on this, and you’ve referenced two pieces I’ve already reacted to (and in my view debunked) before. Seems like we could have a good debate on this :)
I would love to debate you on this. My view: there is no single known problem in which FDT makes an incorrect decision. I have thought about FDT a lot and it seems quite obviously correct to me.
Ah, so your complaint is that the author is ignoring evidence pointing to shorter timelines. I understand your position better now :)
“Insofar as your distribution has a faraway median, that means you have close to certainty that it isn’t happening soon. And that, I submit, is ridiculously overconfident and epistemically unhumble.”
Why? You can say a similar thing about any median anyone ever has. Why is this median in particular overconfident?
“And not only do I not expect the trained agents to not maximize the original “outer” reward signal”
Nitpick: one “not” too many?
I apologize, Said; I misinterpreted your (clearly written) comment.
Reading your newest comment, it seems I actually largely agree with you—the disagreement lies in whether farm animals have sentience.
(No edit was made to the original question.)
Thanks for your answer!
I (strongly) disagree that sentience is uniquely human. It seems to me a priori very unlikely that this would be the case, and evidence does exist to the contrary. I do agree sentience is an important factor (though I’m unsure it’s the only one).
“but certainly none of the things that we (legally) do with animals are bad for any of the important reasons why torture of people is bad.”
That seems very overconfident to me. What are your reasons for believing this, if I may ask? What quality or qualities do humans have that animals lack that makes you certain of this?
One-boxing on Newcomb’s Problem is good news IMO. Why do you believe it’s bad?
Of course! Thanks for your time.
I can, although I indeed don’t think it is nonsense.
What do you think our (or specifically my) viewpoint is?
Hmm, interesting. I don’t know much about UDT. From and FDT perspective, I’d say that if you’re in the situation with the bomb, your decision procedure already Right-boxed and therefore you’re Right-boxing again, as logical necessity. (Making the problem very interesting.)
Sorry, I’m having trouble understanding your point here. I understand your analogy (I was a developer), but am not sure what you’re drawing the analogy to.
I’ve been you ten years ago.
Just… no. Don’t act like you know me, because you don’t. I appreciate you trying to help, but this isn’t the way.
Seems to me Yudkowsky was (way) too pessimistic about OpenAI there. They probably knew something like this would happen.
To explain my view more, the question I try to answer in these problems is more or less: if I were to choose a decision theory now to strictly adhere to, knowing I might run into the Bomb problem, which decision theory would I choose?
“But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot.”
This is what we agree on. If you’re in the situation with a bomb, all that matters is the bomb.
My stance is that Left-boxers virtually never get into the situation to begin with, because of the prediction Omega makes. So with probability close to 1, they never see a bomb.
Your stance (if I understand correctly) is that the problem statement says there is a bomb, so, that’s what’s true with probability 1 (or almost 1).
And so I believe that’s where our disagreement lies. I think Newcomblike problems are often “trick questions” that can be resolved in two ways, one leaning more towards your interpretation.
In spirit of Vladimir’s points, if I annoyed you, I do apologize. I can get quite intense in such discussions.
I see your point, although I have entertained Said’s view as well. But yes, I could have done better. I tend to get like this when my argumentation is being called crazy, and I should have done better.
You could have just told me this instead of complaining about me to Said though.
Wait, why does the real you get nothing? It’s specified you get $200. What am I missing?