anon85

Karma: 18

anon85 13 Oct 2015 21:57 UTC
0 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers

I see you’ve not bothered reading any of my replies and instead just made up your own version in your head.

I read all of your replies. What are you referring to? Also, this is uncharitable/insulting.

Believe it or not there are a lot of people who’ll do things like insist that that’s not the case or insist that you just have to wish carefully enough hence the need for the article.

To be honest, I’m not sure what we’re even disagreeing about. Like, sure, some genies are unsafe no matter how you phrase your wish. For other genies, you can just wish for “whatever I ought to wish for”. For still other genies, giving some information about your wish helps.

If EY’s point was that the first type of genies exist, then yes, he’s made it convincingly. If his point is that you never need to specify a wish other than “whatever I ought to wish for” (assuming a genie is powerful enough), then he failed to provide arguments for this claim (and the claim is probably false).

anon85 13 Oct 2015 17:38 UTC
0 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers

If you have to speak “carefully enough” then you’re taking a big risk though you may luck out and get what you want, they’re not safe.

If your argument is that unless a powerful being is extremely safe, then they’re not extremely safe, this is true by definition. Obviously, if a genie sometimes doesn’t give you what you want, there is some risk that the genie won’t give you what you want. I thought a more substantial argument was being made, though—it sounded like EY was claiming that saying “I wish for whatever I should wish for” is supposed to always be better than every other wish. This claim is certainly false, due to the “mom” example. So I guess I’m left being unsure what the point is.

anon85 13 Oct 2015 2:47 UTC
0 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers
Examples of what? Of hypothetical intelligent minds? I feel like there are examples all over fiction; consider genies themselves, which often grant wishes in a dangerous way (but you can sometimes get around it by speaking carefully enough). Again, I agree that some genies are never safe and some are always safe, but it’s easy to imagine a genie which is safe if and only if you specify your wish carefully.

Anyway, do you concede the point that EY’s article contains no arguments?

anon85 12 Oct 2015 16:16 UTC
0 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers
I’m making 2 points:
1. His metaphor completely fails conceptually, because I’m perfectly capable of imagining genies that fall outside the three categories.
2. Perhaps the classification works in some other setting, such as AIs. However, the article never provided any arguments for this (or any arguments at all, really). Instead, there was one single example (seriously, just one example!) which was then extrapolated to all genies.

anon85 12 Oct 2015 10:25 UTC
0 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers

At age 5 you could safely wish for “I wish for you to do what I should wish for” and at worst you’d be a little disappointed if what she came up with wasn’t as fun as you’d have liked.

I would have gotten the wrong flavor of ice cream. It was strictly better to specify the flavor of ice cream I preferred. Therefore, the statement about the 3 types of genies is simply false. It might be approximately true in some sense, but even if it is, the article never gives any arguments in favor of that thesis, it simply gives one example.

anon85 12 Oct 2015 9:36 UTC
3 points
in reply to: turchin’s comment on: Simulations Map: what is the most probable type of the simulation in which we live?
That sounds pretty similar to a Deist’s God, which created the universe but does not interfere thereafter. Personally, I’d just shave it off with Ocam’s razor.

Also, it seems a little absurd to try to infer things about our simulators, even supposing they exist. After all, their universe can be almost arbitrarily different from ours.

anon85 11 Oct 2015 21:10 UTC
2 points
on: Simulations Map: what is the most probable type of the simulation in which we live?
Does the simulation hypothesis have any predictive power? If so, what does it predict? Is there any way to falsify it?

anon85 11 Oct 2015 18:14 UTC
−2 points
in reply to: gjm’s comment on: A few misconceptions surrounding Roko’s basilisk
Oh, yes, me too. I want to engage in one-shot PD games with entirelyuseless (as opposed to other people), because he or she will give me free utility if I sell myself right. I wouldn’t want to play one-shot PDs against myself, in the same way that I wouldn’t want to play chess against Kasparov.

By the way, note that I usually cooperate in repeated PD games, and most real-life PDs are repeated games. In addition, my utility function takes other people into consideration; I would not screw people over for small personal gains, because I care about their happiness. In other words, defecting in one-shot PDs is entirely consistent with being a decent human being.

anon85 11 Oct 2015 14:45 UTC
−1 points
in reply to: entirelyuseless’s comment on: A few misconceptions surrounding Roko’s basilisk
Cool, so in conclusion, if we met in real life and played a one-shot PD, you’d (probably) cooperate and I’d defect. My strategy seems superior.

anon85 11 Oct 2015 0:30 UTC
2 points
in reply to: HungryHobo’s comment on: [Link] Stephen Hawking AMA answers
I never liked that article. It says “there are three types of genies”, and then, rather than attempting to prove the claim or argue for it, it just provides an example of a genie for which no wish is safe. I mean, fine, I’m convinced that specific genie sucks. But there may well be other genies that don’t know what you want but have the ability to give it to you if you ask (when I was 5 years old, my mom was such a genie).

anon85 10 Oct 2015 22:11 UTC
0 points
in reply to: entirelyuseless’s comment on: A few misconceptions surrounding Roko’s basilisk

But since you’re making it clear that your code is quite different, and in a particular way, I would defect against you.

You don’t know who I am! I’m anonymous! Whoever you’d cooperate with, I might be that person (remember, in real life I pretend to have a completely different philosophy on this matter). Unless you defect against ALL HUMANS, you risk cooperating when facing me, since you don’t know what my disguise will be.

anon85 10 Oct 2015 22:02 UTC
0 points
in reply to: entirelyuseless’s comment on: A few misconceptions surrounding Roko’s basilisk

You can see which side of the room you are on, so you know which one you are.

If I can do this, then my clone and I can do different things. In that case, I can’t be guaranteed that if I cooperate, my clone will too (because my decision might have depended on which side of the room I’m on). But I agree that the cloning situation is strange, and that I might cooperate if I’m actually faced with it (though I’m quite sure that I never will).

People don’t actually have the same code, but they have similar code. If the code in some case is similar enough that you can’t personally tell the difference, you should follow the same rule as when you are playing against a clone.

How do you know if people have “similar” code to you? See, I’m anonymous on this forum, but in real life, I might pretend to believe in TDT and pretend to have code that’s “similar” to people around me (whatever that means—code similarity is not well-defined). So you might know me in real life. If so, presumably you’d cooperate if we played a PD, because you’d believe our code is similar. But I will defect (if it’s a one-time game). My strategy seems strictly superior to yours—I always get more utility in one-shot PDs.

anon85 10 Oct 2015 19:36 UTC
−3 points
in reply to: Houshalter’s comment on: A few misconceptions surrounding Roko’s basilisk

Yes. The universe is deterministic. Your actions are completely predictable, in principle. That’s not unique to this thought experiment. That’s true for every thing you do. You still have to make a choice. Cooperate or defect?

Um, what? First of all, the universe is not deterministic—quantum mechanics means there’s inherent randomness. Secondly, as far as we know, it’s consistent with the laws of physics that my actions are fundamentally unpredictable—see here.

Third, if I’m playing against a clone of myself, I don’t think it’s even a valid PD. Can the utility functions ever differ between me and my clone? Whenever my clone gets utility, I get utility, because there’s no physical way to distinguish between us (I have no way of saying which copy “I” am). But if we always have the exact same utility—if his happiness equals my happiness—then constructing a PD game is impossible.

Finally, even if I agree to cooperate against my clone, I claim this says nothing about cooperating versus other people. Against all agents that don’t have access to my code, the correct strategy in a one-shot PD is to defect, but first do/say whatever causes my opponent to cooperate. For example, if I was playing against LWers, I might first rant on about TDT or whatever, agree with my opponent’s philosophy as much as possible, etc., etc., and then defect in the actual game. (Note again that this only applies to one-shot games).

anon85 10 Oct 2015 1:24 UTC
−6 points
in reply to: mwengler’s comment on: A few misconceptions surrounding Roko’s basilisk

I think the reason to cooperate is not to get the best personal outcome, but because you care about the other person.

I just want to make it clear that by saying this, you’re changing the setting of the prisoners’ dilemma, so you shouldn’t even call it a prisoners’ dilemma anymore. The prisoners’ dilemma is defined so that you get more utility by defecting; if you say you care about your opponent’s utility enough to cooperate, it means you don’t get more utility by defecting, since cooperation gives you utility. Therefore, all you’re saying is that you can never be in a true prisoners’ dilemma game; you’re NOT saying that in a true PD, it’s correct to cooperate (again, by definition, it isn’t).

The most likely reason people are evolutionarily predisposed to cooperate in real-life PDs is that almost all real-life PDs are repeated games and not one-shot. Repeated prisoners’ dilemmas are completely different beasts, and it can definitely be correct to cooperate in them.

anon85 9 Oct 2015 12:59 UTC
−2 points
in reply to: Houshalter’s comment on: A few misconceptions surrounding Roko’s basilisk

Well there is no causal influence. Your opponent is deterministic. His choice may have already been made and nothing you do will change it. And yet the best decision is still to cooperate.

If his choice is already made and nothing I do will change it, then by definition my choice is already made and nothing I do will change it. That’s why my “decision” in this setting is not even well-defined—I don’t really have free will if external agents already know what I will do.

anon85 8 Oct 2015 23:02 UTC
0 points
in reply to: Rob Bensinger’s comment on: A few misconceptions surrounding Roko’s basilisk

The most obvious example of cooperating due to acausal dependence is making two atom-by-atom-identical copies of an agent and put them in a one-shot prisoner’s dilemma against each other. But two agents whose decision-making is 90% similar instead of 100% identical can cooperate on those grounds too, provided the utility of mutual cooperation is sufficiently large.

I’m not sure what “90% similar” means. Either I’m capable of making decisions independently from my opponent, or else I’m not. In real life, I am capable of doing so. The clone situation is strange, I admit, but in that case I’m not sure to what extent my “decision” even makes sense as a concept; I’ll clearly decide whatever my code says I’ll decide. As soon as you start assuming copies of my code being out there, I stop being comfortable with assigning me free will at all.

Anyway, none of this applies to real life, not even approximately. In real life, my decision cannot change your decision at all; in real life, nothing can even come close to predicting a decision I make in advance (assuming I put even a little bit of effort into that decision).

If you’re concerned about blushing etc., then you’re just saying the best strategy in a prisoner’s dilemma involves signaling very strongly that you’re trustworthy. I agree that this is correct against most human opponents. But surely you agree that if I can control my microexpressions, it’s best to signal “I will cooperate” while actually defecting, right?

Let me just ask you the following yes or no question: do you agree that my “always defect, but first pretend to be whatever will convince my opponent to cooperate” strategy beats all other strategies for a realistic one-shot prisoners’ dilemma? By one-shot, I mean that people will not have any memory of me defecting against them, so I can suffer no ill effects from retaliation.

anon85 8 Oct 2015 22:46 UTC
−8 points
in reply to: Houshalter’s comment on: A few misconceptions surrounding Roko’s basilisk
If I’m playing my clone, it’s not clear that even saying that I’m making a choice is well-defined. After all, my choice will be what my code dictates it will be. Do I prefer that my code cause me to accept? Sure, but only because we stipulated that the other player shares the exact same code; it’s more accurate to say that I prefer my opponent’s code to cause him to defect, and it just so happens that his code is the same as mine.

In real life, my code is not the same as my opponent’s, and when I contemplate a decision, I’m only thinking about what I want my code to say. Nothing I do changes what my opponent does; therefore, defecting is correct.

Let me restate once more: the only time I’d ever want to cooperate in a one-shot prisoners’ dilemma was if I thought my decision could affect my opponent’s decision. If the latter is the case, though, then I’m not sure if the game was even a prisoners’ dilemma to begin with; instead it’s some weird variant where the players don’t have the ability to independently make decisions.

anon85 8 Oct 2015 2:23 UTC
0 points
in reply to: Rob Bensinger’s comment on: A few misconceptions surrounding Roko’s basilisk
I don’t know enough about this to tell if (2) had more influence than (3) initially. I’m glad you agree that (2) had some influence, at least. That was the main part of my point.

How long did discussion of the Basilisk stay banned? Wasn’t it many years? How do you explain that, unless the influence of (2) was significant?

anon85 8 Oct 2015 2:16 UTC
−6 points
in reply to: Rob Bensinger’s comment on: A few misconceptions surrounding Roko’s basilisk

Defecting gives you a better outcome than cooperating if your decision is uncorrelated with the other players’. Different humans’ decisions aren’t 100% correlated, but they also aren’t 0% correlated, so the rationality of cooperating in the one-shot PD varies situationally for humans.

You’re confusing correlation with causation. Different players’ decision may be correlated, but they sure as hell aren’t causative of each other (unless they literally see each others’ code, maybe).

But part of the reason for cooperation is probably also that we’ve evolved to do a very weak and probabilistic version of ‘source code sharing’: we’ve evolved to (sometimes) involuntarily display veridical evidence of our emotions, personality, etc. -- as opposed to being in complete control of the information we give others about our dispositions.

Calling this source code sharing, instead of just “signaling for the purposes of a repeated game”, seems counter-productive. Yes, I agree that in a repeated game, the situation is trickier and involves a lot of signaling. The one-shot game is much easier: just always defect. By definition, that’s the best strategy.

anon85 8 Oct 2015 2:10 UTC
0 points
in reply to: ChristianKl’s comment on: A few misconceptions surrounding Roko’s basilisk
Just FYI, if you want a productive discussion you should hold back on accusing your opponents of fallacies. Ironically, since I never claimed that you claimed Eliezer engages in habitual banning on LW, your accusation that I made a strawman argument is itself a strawman argument.

Anyway, we’re not getting anywhere, so let’s disengage.