Whoops, box B was supposed to have a thousand in both cases.
I did have in mind the variant where Omega picks the self-consistent case, instead of using only the box A prediction, though.
Strilanc
Karma: 543
Yes, the advantage comes from being hard to predict. I just wanted to find a game where the information denial benefits were counterfactual (unlike poker).
(Note that the goal is not perfect indistinguishability. If it was, then you could play optimally by just flipping a coin when deciding to bet or call.)
The variant with the clear boxes goes like so:
You are going to walk into a room with two boxes, A and B, both transparent. You’ll be given the opportunity to enter a room with both boxes, their contents visible, where can either take both boxes or just box A.
Omega, the superintelligence from another galaxy that is never wrong, has predicted whether you will take one box or two boxes. If it predicted you were going to take just box A, then box A will contain a million dollars and box B will contain a thousand dollars. If it predicted you were going to take both, then box A will be empty and box B will contain a thousand dollars.
If Omega predicts that you will purposefully contradict its prediction no matter what, the room will contain hornets. Lots and lots of hornets.
Case 1: You walk into the room. You see a million dollars in box A. Do you take both, or just A?
Case 2: You walk into the room. You see no dollars in box A. Do you take both, or just A?
If Omega is making its predictions by simulating what you would do in each case and picking a self-consistent prediction, then you can eliminate case 2 by leaving the thousand dollars behind.
edit Fixed not having a thousand in box B in both cases.
Thanks for the clarification. I removed the game tree image from the overview because it was misleading readers into thinking it was the entirety of the content.
Alright, I removed the game tree from the summary.
The −11 was chosen to give a small, but not empty, area of positive-returns in the strategy space. You’re right that it doesn’t affect which strategies are optimal, but in my mind it affects whether finding an optimal strategy is fun/satisfying.
You followed the link? The game tree image is a decent reference, but a bad introduction.
The answer to your question is that it’s a zero sum game. The defender wants to minimize the score. The attacker wants to maximize it.
Sam Harris recently responded to the winning essay of the “moral landscape challenge”.
I thought it was a bit odd that the essay wasn’t focused on the claimed definition of morality being vacuous. “Increasing the well-being of conscious creatures” is the sort of answer you get when you cheat at rationalist taboo. The problem has been moved into the word “well-being”, not solved in any useful way. In practical terms it’s equivalent to saying non-conscious things don’t count and then stopping.
It’s a bit hard to explain this to people. Condensing the various inferential leaps into a single post might make a useful post. On the other hand it’s just repackaging what’s already here. Thoughts?
Arguably the university’s NAT functioned as intended. They did not provide you with internet access for the purpose of hosting games, even if they weren’t actively against it.
The NAT/firewall was there for security reasons, not to police gaming. This was when I lived in residence, so gaming was a legitimate recreational use.
Endpoints not being able to connect to each other makes some functionality costly or impossible. For example, peer to peer distribution systems rely on being able to contact cooperative endpoints. NAT makes that a lot harder, meaning plenty of development and usability costs.
A more mundane example is multiplayer games. When I played warcraft 3, I had lots of issues testing maps I made because no one could connect to games I hosted (I was behind a university NAT, out of my control). I had to rely on sending the map to friends and having them host.
hrm. I’m trying to replace the word “God” in that sentence with something less incoherent but containing the same sense of emphasis, and coming up blank. I blame Monday. Suggestions, anyone?
Cute Kittens I hope that continues.
(Emphasize the kittens like it’s a curse word, or it will sound ridiculous. You are not trying to avoid cursing, you are trying to introduce it. Also it will sound ridiculous anyways.)
Context: I would have downvoted it, if it hadn’t already been so low. I only saw the shortened version.
The basic issue is that you were disputing definitions. You weren’t showing that all software is software that does tasks that historically humans remained better at than computers (i.e. “AI”), you were just redefining the word AI to be a synonym for software. You but didn’t seem to address or realize that fact.
Also, I may be remembering wrong, but the points seemed to be based on a personal sense of aesthetics. That sort of argument doesn’t work well when others don’t agree on what’s elegant. Better to focus on why the existing definition is contrived, how it doesn’t cut reality at its seams, how it leads to misunderstandings, etc.
How have I not addressed the arguments on its own terms? I agree with basically everything you said, except calling it a solution. You’ll run into non-trivial problems when you try to turn it into an algorithm.
For example, the case of there being an actual physical mugger is meant to be an example of the more general problem of programs with tiny priors predicting super-huge rewards. A strategy based on “probability of the mugger lying” has to be translated to the general case somehow. You have to prevent the AI from mugging itself.
I would expect an honest person to deny incorrect guesses, and to give very vague words of encouragement when the other person said something that was close.
No, that’s how an honest person fools themselves. The encouragement gets stronger as you get closer, so finding the solution goes from a brute force search to a simple hill climbing exercise. The answers should all be “No” without any variation. No hints, no “I think you have some of the right ideas”, no “that’s not even close!”, just “No” “No” “No” “No” “No”.
Also it’s important to notice how many guesses you’re making. For example, when you said “I’m imagining palm trees by a lake at sunset.” in the conversation you posted, that was a guess. Actually, more like three guesses...
Psychic researchers focus on individual people sometimes. Finding a single person capable of consistent remote viewing in a controlled setting would be a huge discovery.
[It seems unlikely]. [He has little to gain]. Secondly, [if he cares why fake it]? Thirdly, [...] he basically seems like an honest person.
Some people just enjoy lying. Sometimes so much so that it’s considered a mental illness.
If you think that he is lying, then in what way? How did he convince 31 minus one people to go along with this lie? Most of the people who solved Nu were active members of the community before DavidM ever became a conspiracy nut.
You’ve met all thirty people that solved the puzzle? In person? DavidM could just say there’s thirty people, or could have multiple accounts and actually be those “people”.
You should ask DavidM how he set up the experiment, for an introduction to one of the solvers, and then repeat the experiment. Then if it works, read a book about experimental design and start improving the experiment.
I mean… you basically gave 60% expectation that this experiment will work. So you should expect significant returns on going after Randi’s million dollar challenge, right? If not, why not?
“Solving” Pascal’s Mugging involves giving an explicit reasoning system and showing that it makes the right decision.
It’s not enough to just say “your confidence has to go down more than their claimed reward goes up”. That part is obvious. The hard part is coming up with actual explicit rules that do that. Particularly ones that don’t fall apart in other situations (e.g. the decision system “always do nothing” can’t be pascal-mugged, but has serious problems).
Another thing not addressed here is that the mugger may be a hypothetical. For example, if the AI generates hypotheses where the universe affects 3^^^^3 people then all decisions will be dominated by these hypotheses because their outcomes outweigh their prior by absurd margins. How do you detect these bad hypotheses? How do you penalize them without excluding them? Should you exclude them?
Please give a more concrete situation with actual numbers and algorithms.
I wish for you to tell me [what] you would wish for [in my place] if you had your current intelligence and knowledge, but the same values and desires as me.
“You should wish for me to tell you what you would wish for in your place if I had your current intelligence and knowledge, but the same values and desires as you.”
“I have replaced your values and desires with my own. You should wish to become a genie.”
“Here is your list of all possible wishes.”
“You should wish that genies never existed.”
Every month you have a lot of interesting links, but it’s so many all at once. Is there a way I can get them more gradually?
Wow.
The closest analogue I have to that is grabbing planet positions and velocities from JPL’s HORIZONS system, then doing small time steps holding accelerations constant.
That’s how I know the (mathematical) solar system behaves as claimed. Except that Mercury’s orbit will eventually become so elliptical and gain so much energy that it careens in and out of the solar system until it flies off to infinity (or people are also right about the limitations of the approximation technique I was using).
Is it expected that electrically disabling key parts of the brain will replace anesthetic drugs?