However, for a sufficiently large value of N, there reaches a point when the odds of a person winning the award is more likely -simulated- than actually winning the award. You expect -somebody- to win the award, therefore if somebody wins the award nothing unusual has happened. However, you cannot expect -yourself- to win the award, and if you do you should update your priors to reflect this fact.
Huh? What does simulation have to do with this?
I’ll use your example against you: Mary Panick recently won $250k from a lottery. Should she increase her belief that someone in the lottery commission crooked the lottery to favor her? How much, exactly?
Objectively, no; as previously mentioned, it shouldn’t surprise us that somebody won the lottery. Subjectively, yes; I would certainly update my odds that something other than pure chance is at work if I happened to win the lottery.
And simulation is coming from Robin Hanson’s assertion that if you’re an important person in the world, you should probably update your priors to suggest you are being simulated; it’s a related argument. If the world is ever capable of simulating individual people, any given important person is more likely a simulation than the real thing—so, given that I’m not particularly important, I can probably assume I’m not simulated, unless something exceptionally unlikely happens to me. But if I were, say, Obama, maybe I -should- think I’m living in a simulation. From the outside, there’s a president of the United States, so it’s not particularly unusual that -somebody- is the president of the United States. From the inside, it would be unusual that -I- am the president of the United States. Same thing.
Objectively, no; as previously mentioned, it shouldn’t surprise us that somebody won the lottery. Subjectively, yes; I would certainly update my odds that something other than pure chance is at work if I happened to win the lottery.
Again, why? Suppose we are comparing two models: in one world, there are 1000 haunted houses which are all explained by gaslamping and sleepwalking etc; in the second world, there are 1000 haunted houses and they are all supernatural etc. Upon encountering a haunted house, would you update in favor of ‘I am in world two and houses are supernatural’? Would someone reading your experience update? I propose that neither would update, because the evidence is equally consistent with both worlds; so far so good.
Now, if in world 1 there are 1000 frightening houses with the mundane explanations mentioned, and in world 2 there are 1000 frightening houses with the mundane explanations (human biology and mentality and the laws of probability etc having not changed) plus 1000 frightening houses due to supernatural influences, upon encountering a frightening house would you update?
Of course; in world 2 there are more frightening houses and you have encountered a frightening house, which is twice as likely in world 2 than in world 1 (2000 houses versus 1000 houses), and so you are now more inclined to think you are in world 2 from whatever you were thinking before. But so would an observer reading your experience!
So where does this unique unconveyable evidence (that your post claims your experience has given you) come from?
And simulation is coming from Robin Hanson’s assertion that if you’re an important person in the world, you should probably update your priors to suggest you are being simulated; it’s a related argument.
Ah. It’s coming from anthropics. You’re making the claim that Aumannian agreement cannot convey anthropic information.
You realize that both the SIA and SSA are hotly debated because either seems to lead to absurd conclusions, right? While Aumann just leads to the conclusion ‘people are irrational’, which certainly doesn’t seem absurd to me.
And since one man’s modus ponens is another man’s modus tollens, why isn’t your post just further evidence that anthropic reasoning as currently understood by most people is completely broken and cannot be trusted in anything?
I think there are non-anthropic problems with even rational!humans communicating evidence.
One is that it’s difficult to communicate that you’re not lying, and it is also difficult to communicate that you’re competent at assessing evidence. A rational agent may have priors saying that OrphanWilde is an average LW member, including the associated wide distribution in propensity to lie and competence at judging evidence. On the other hand, rational!OrphanWilde would (hopefully) have a high confidence assessment of himself (herself?) along both dimensions. However, this assessment is difficult to communicate, since there are strong incentives to lie about these assessments (and also a lot of potential for someone to turn out to not be entirely rational and just get these assessments wrong). So, the rational agent may read this post and update to believing it’s much more likely that OrphanWilde either lies to people for fun (just look at all those improbable details!) or is incompetent at assessing evidence and falls prey to apophenia a lot.
This might not be an issue were it not for the second problem, which is that communication is costly. If communication were free, OrphanWilde could just tell us every single little detail about his life (including in this house and in other houses), and we could then ignore the problem of him potentially being a poor judge of evidence. Alternatively, he could probably perform some very large volume evidence-assessment test to prove that he is, in fact, competent. However, since communication is costly, this seems to be impractical in reality. (The lying issue is slightly different, but could perhaps be overcome with some sort of strong precommitment or an assumption constraining possible motivations combined with a lot of evidence.)
This doesn’t invalidate Aumann agreement as such, but certainly seems to limit its practical applications even for rational agents.
I don’t rule out mundane explanations. Hence my repeated disclaimers on each use of the word “haunted.” If anything “supernatural” exists, it isn’t supernatural, it’s merely natural, and we simply haven’t pinned down what’s going on yet. Empiricism and reductionism don’t get broken.
And anthropic reasoning is -unnecessary- to the logic, it simply provides the simplest examples. I could construct related examples without any anthropic reasoning at all:
You’re shipwrecked on a deserted island with a friend, Johnny. You see a ship in the distance; Johnny’s eyesight is not as good as yours, and he cannot. You’ve been trying to cheer him up for the past three days, because he’s fallen into depression. He doesn’t believe you when you tell him there’s a ship; he cannot see it, and he believes it’s just another attempt by you to cheer him up. -You cannot share the true evidence, because you cannot show him the ship he cannot see-.
Or, to put it in terms of framing:
You flip a coin ten times. It comes up heads each time.
versus
You say a coin will land heads-up ten times. You flip it ten times, and it comes up heads each time.
Even though the odds are strictly speaking equally likely in both cases, the framing of the first proposition in fact makes it [edited: less significant, not more likely]; you would have been similarly impressed if it had come up tails each time. So the second scenario is twice as significant as the first scenario.
The fact that it is happening to me, rather than another person, is a kind of contextual framing, in much the same sense that calling heads first frames the coin-flipping event.
I don’t rule out mundane explanations. Hence my repeated disclaimers on each use of the word “haunted.” If anything “supernatural” exists, it isn’t supernatural, it’s merely natural, and we simply haven’t pinned down what’s going on yet. Empiricism and reductionism don’t get broken.
Fine, replace ‘mundane causes’ with ‘mundane causes minus cause X’ and ‘supernatural with ‘cause X’ in my examples. -_-
I could construct related examples without any anthropic reasoning at all:
And they fail. In the desert island case, Aumann is perfectly applicable: if you have more evidence than he does, then this will be incorporated appropriately; in fact, the desert island case is a great example of Aumann in practice: that you’ve been lying to him merely shows that ‘disagreements are not honest’ (you are the dishonest party here).
The fact that it is happening to me, rather than another person, is a kind of contextual framing, in much the same sense that calling heads first frames the coin-flipping event.
How so? You didn’t predict it would be a haunted house before it went, to point out the most obvious disanalogy.
I feel like your disagreement is getting a little slippery here.
My rejection of Aumann is that there is no common knowledge of our posteriors. It’s not necessary for me to have lied to him before, after all; I could have been trying to cheer him up entirely honestly.
If I -had- predicted it would be a haunted house, I’d be suspicious of any evidence that suggested it was. The point isn’t the prediction—prediction is just one mechanism of framing an outcome. The point is in the priors; my prior of -somebody- experiencing a series of weird events in a given house are pretty high, there’s a lot of people out there to experience such weird events, and some of them will experience several. My prior odds of -me- experiencing a series of weird events in a given house should be pretty low. It’s thus much more significant for -me- to experience a series of weird events in a given house than for some stranger who I wouldn’t have known about except for their reporting such. If I’m not updating my priors after being surprised, what am I doing?
It’s not necessary for me to have lied to him before, after all; I could have been trying to cheer him up entirely honestly.
Then why does he distrust you? If you have never lied and will never lie in trying to cheer you up, then he is wrong to distrust you and this is simply an example of irrationality and not uncommunicable knowledge; if he is right to suspect that you or people like you would lie in such circumstances, then ‘disagreements are not honest’ and this is again not uncommunicable knowledge.
The point is in the priors; my prior of -somebody- experiencing a series of weird events in a given house are pretty high, there’s a lot of people out there to experience such weird events, and some of them will experience several. My prior odds of -me- experiencing a series of weird events in a given house should be pretty low. It’s thus much more significant for -me- to experience a series of weird events in a given house than for some stranger who I wouldn’t have known about except for their reporting such. If I’m not updating my priors after being surprised, what am I doing?
And you talk about me being slippery. We’re right back to where we began:
to the extent you have any knowledge in this case of observing a rare but known event, the knowledge is communicable to an outsider who can also update; it is no more ‘significant’ to you than a stranger, and any significance may just reflect known cognitive biases like anecdotes, salience, base-rate neglect, etc (and fall under the irrationality rubric)
to the extent that this knowledge is anthropic, one can argue that this is uncommunicable but anthropic arguments are so divergent and unreliable that it’s not clear you have learned uncommunicable knowledge rather than found yet another case in which anthropic arguments are unreliable and give absurd conclusions.
You have not shown any examples which simultaneously involve uncommunicable knowledge which does not involve anthropics (what you are claiming is possible) and rationality and honesty on the part of all participants.
You’re presupposing that trustworthiness is communicable, or that rationality demands trusting somebody absent evidence to do otherwise. There’s definitely incommunicable knowledge—what red looks like to me, for example. You’re stretching “rationality” to places it has no business being to defend a proposition, or else adding conditions (honesty) to make the proposition work.
What, exactly, are you calling anthropics? What I’m describing doesn’t depend on either SIA or SSA. If you’re saying that I’m depending upon an argument which treats any given observer as a special case—yes, yes I am, that is in fact the thrust of my argument. Your argument against anthropics was that it leads to “absurd” results. However, your judgment of “absurd” seems tautological; you seem to be treating the idea that being unable to arrive at the same posterior odds as absurd in itself. That’s not an argument. That’s begging the question.
So—where exactly am I absurd in the following statements:
1.) My prior odds of some stranger experiencing highly unlikely circumstances (and me hearing about them—assuming such circumstances are of general interest) should be relatively high
2.) My prior odds of me experiencing highly unlikely circumstances should be low
3.) Per Bayesian inference, given that the likelihood of the two distinct events is different, the posterior distribution is different
4.) Therefore, there is a different amount of information in something happening personally than to something happening to a stranger
Or, hopefully without error, as it’s been a while since I’ve mucked about with this stuff:
M is the event happening to me, O is the event happening to somebody else, X is some idea for which M and O are evidence of, and Z is the population size (assuming random distribution of events:
p(X|M)=p(M|X)*P(X)/P(M)
Assuming X guarantees M and O, we get:
p(X|M)=P(X)/P(M)
p(X|O)=P(X)/P(O)
where p(M) = p(O) / Z
Which means
p(X|M) = p(X|O) * Z
Which is to say, M is stronger evidence than O for Z:
p(X|M, M1) = p(M|X)*P(X|M1)/p(M|M1)
p(X|O, O1) = p(O|X)*P(X|O1)/p(O|O1)
Using the above assumption that X guarantees M and O:
p(X|M, M1) = p(X|M1)/P(M|M1)
p(X|O, O1) = p(X|O1)/P(O|O1)
Substituting, again where P(O1) = p(M1) * Z, and where p(M1|M) and p(O1|O) are both 1:
p(X|M, M1) = p(X|M1)/p(M|M1)
p(X|O, O1) = (P(X)/Z*P(M1)) / (Z*P(M) / Z*P(M1))
= p(X)/(Z*P(M))
Or, in short—the posteriors are different. The information is different. There is a piece of incommunicable evidence when something happens to me as opposed to somebody else.
You’re presupposing that trustworthiness is communicable, or that rationality demands trusting somebody absent evidence to do otherwise. There’s definitely incommunicable knowledge—what red looks like to me, for example. You’re stretching “rationality” to places it has no business being to defend a proposition, or else adding conditions (honesty) to make the proposition work.
The conditions are right there in the Aumann proofs, are they not? I’m not adding anything, I’m dividing up the possible outcomes: anthropics (questionable), communicable knowledge (contra you), or Aumann is inapplicable (honesty etc).
What I’m describing doesn’t depend on either SIA or SSA.
I’d be interested to see if you could prove that the result holds independently of them.
That’s not an argument. That’s begging the question.
That’s the point of the modus tollens vs modus ponens saying. You claim to derive a result, but using premises more questionable than the conclusion, in which case you may have merely disproven the premises via reductio ad absurdum. If this is begging the question (which it isn’t, since that’s when your premise contains the conclusion), then every proof by contradiction or reductio ad absurdum is question-begging.
Or, in short—the posteriors are different. The information is different. There is a piece of incommunicable evidence when something happens to me as opposed to somebody else.
Correct me if I am wrong, but in your example, M is not increased when O fails to happen—more concretely, you assume the number of spooked people you will hear of is constant—when it would be more appropriate to increase the number of observations of O by 1, since if you don’t go into the spooky house someone else does. Then you are merely deriving the uninteresting observation that if there are more events (by 1) consistent with Z, Z will be more likely. Well, yeah. But there is nothing special about one’s own observations in this case; if someone else went into the house and reported observations, you would update a little more, just like if you want into the house, and in both cases, more than if no one went into the house (or they went in and saw nothing).
Also, your equations are messed up. I think you need to escape some stuff.
Aha! I think the issue here is that you’re thinking of it in terms of two identical observers. The observers aren’t identical in my arguments—one is post-hoc. I have realized where the discrepancy between our arguments is coming from with your example, because of the way I keep framing problems as being about the observer. Suppose I and a friend, Bob, are arguing about who goes in the house. In this case, there’s not practical difference between our evidence. The difference isn’t between me and other, the difference is between me and (other who I wouldn’t have known about except for said experience).
Bob and I are identical (I did say this wasn’t necessarily anthropic!) for the purposes of calculation. Bob is included in p(M).
Steve, who wrote a post on a rationality forum describing his experiences, is -not- identical with me for the purposes of calculation. Steve is included in p(O).
Does my argument make more sense now? Bob’s information is fully transferable—in terms of flipping coins, he called heads before flipping ten heads in a row. Steve’s information is -not- - he’s the guy who flipped ten heads in a row without calling anything.
(ETA: I have no idea how to make them look right. How do you escape stuff?)
In certain contexts an asterisk is a magic character and you need to precede it with a backslash to keep it from turning into <em> or </em>. To get
p(X|M, M1) = p(M|X)*P(X|M1)/p(M|M1)
do
p(X|M, M1) = p(M|X)\*P(X|M1)/p(M|M1)
Or you can just put equations in their own paragraphs that are indented by four spaces, in which case no characters will have their magic meaning. (This is how I did the above paragraph where the backslash is visible.)
Suppose i type 693012316 693012316 . Maybe I typed same number twice, maybe I used quantum random number generator and got them separately on the first try. You use the equality as evidence of the former, even if you believe in many worlds, where it is basically a lottery played by parallel yous. Likewise, the winner of the lottery observes the same number twice, which is some evidence for various crazy hypotheses where the selection of “I” necessarily coincides with the winner. edit: you’re totally correct though that such crazy hypotheses are quite improbable to begin with.
Likewise, the winner of the lottery observes the same number twice, which is some evidence for various crazy hypotheses where the selection of “I” necessarily coincides with the winner.
In my example of two worlds, the odds of observing the observed evidence is the same in both worlds and so there is no update.
What set of worlds are you postulating for your “two numbers” example? Because your example, as far as I understand it, doesn’t seem at all analogous.
I’m talking specifically about supernatural explanations for you winning the lottery, I don’t see either why people opt for supernatural explanations for haunting.
Suppose we do something like Solomonoff induction. Dealing with codes that match observations verbatim. There’s a theory that reads bits off the tape to produce the ticket number, then more bits to produce the lottery draw, and there’s a theory that reads bits off the tape and produces both numbers as equal. Suppose the lottery has the size of 2^20, about 1 million. Then the former theory will need 40 lucky bits to match the observation, whereas the latter theory will need only 20 lucky bits to match the observation. For mostly everyone the latter theory will be eliminated, except the lottery winner, for who it will linger, and now, with the required lucky bits, the difference in length between the theories will decrease by 20 bits. S.I. - using learning agent (AIXI and variations of it) which won the lottery will literally expect higher probability of victory on next lottery, because it didn’t eliminate various “I always win” hypotheses. edit: and indeed, given sufficiently big N, the extra code required for “I always win” hack will be smaller than log2(N) so it may well become the dominant hypothesis after a single victory. Things like S.I. are only guaranteed to be eventually correct for almost everyone; if there’s enough instances, the wrongmost ones can be arbitrarily wrong.
At the end of the day it’s just how the agents learn—if you were constantly winning lotteries, at some point you would start believing you got supernatural powers, or MWI is true plus the consciousness preferentially transfers specifically to the happy winner, or the like. Any learning agent is subject to risk of learning wrong things.
edit: more concise explanation: if you choose a person by some unknown method, and then they win the lottery, that’s distinct from you not choosing some person, then someone winning the lottery. Namely, in the former case you got evidence in favour of the hypothesis that “unknown method” picks lottery winners. For a lottery winner, their place in the world was chosen by some unknown method.
You treat a lottery output as a bitstring and ask about SI on it. We can imagine a completely naive agent with no previous observations; what will this ignorant predict? Well, it seems reasonable that one of the top predictions will be for the initial bitstring to be repeated; this seems OK by Occam’s razor (events often repeating are necessary for induction) and I understand that empirically investigating simple Turing machines that many (most? all?) terminating programs will repeat output. It will definitely rank the ‘sequence repeats’ hypotheses above that of possible PRNGs, or very complex physical theories encompassing atmospheric noise and balls dropping into baskets etc.
So far, so good.
I think I lose you when you go on to talk about inferring that you will always win and stuff like that. The repeating hypotheses aren’t contingent on who they happen to. If the particular bitstring emitted by the lottery had also included ‘...and this number was picked by Jain Farstrider’, then SI would seem to then also predict that this Jain will win the next one as well, by the same repeating logic. It certainly will not predict that the agent will win, and the hypothesis ‘the agent (usually) wins’ will drop.
Remember that my trichotomy was that you need to either 1) invoke anthropics; 2) break Aumann via something like dishonesty/incompetence; or 3) you actually do have communicable knowledge.
These SI musings doesn’t seem to invoke anthropics or break Aumannian requirements, and looking at them, they seem communicable. ‘AIXI-MC-MML*, why do you think Jain will win the lottery a second time?’ ‘[translated from minimum-message-length model+message] Well, he won it last time and since I am ignorant of everything in the world, it seems reasonable that he will win it again’. ‘Hmm, that’s a good point.’ And ditto if AIXI-MC-MML happened to be the beneficiary.
* I bring up minimum-message length because Patrick Robotham is supposed to be working on a version of AIXI-MC using MML so one would be able to examine the model of the world(s) a program has devised so far and so one could potentially ask ‘why’ it is making the predictions it is. Having a comprehensible approximation of SI would be pretty convenient for discussing what SI would or would not do.
It will definitely rank the ‘sequence repeats’ hypotheses above that of possible PRNGs,
It doesn’t need PRNGs. The least confusing description of S.I. is as following: the probability of a sequence S is the probability that an universal prefix Turing machine with 3 tapes: input tape which can only be read from, head only advanced in one direction, work tape which can be read from and written to, and is initialized with zeroes, and output tape that can only be written to, will output the sequence S when fed a never-ending string of random bits on the input tape.
The head has such rule set that the program can be loaded via the input tape, and then the program can use the input tape as source of data. This is important because a program can then set up an interpreter emulating other Turing machine (which ensures a constant bound on difference between length of code for different machines).
(We predict using conditional probability—if the machine outputs sequence matching the previous observations, what is the probability that it will produce specific future observations) .
So if we are predicting, for example, perfect coin flips, an input string which begins with code that sets up the working tape so that it will subsequently relay random bits from input to the output, does the trick. This code requires the bits on the input tape to match the observation, meaning that for each observed bit, the length of the input string which has to be correct grows by 1 bit.
Meanwhile a code that sets up the machine to output repeating zeroes does not require any more bits on the input tape to be correct. So when you are getting repeated zeroes, the code relaying random bits is being lowered in weight by factor of 2 with each observed bit, whereas the theory outputting zeroes stays the same (until, of course, you encounter a non zero and it is eliminated).
I think I lose you when you go on to talk about inferring that you will always win and stuff like that. The repeating hypotheses aren’t contingent on who they happen to. If the particular bitstring emitted by the lottery had also included ‘...and this number was picked by Jain Farstrider’, then SI would seem to then also predict that this Jain will win the next one as well, by the same repeating logic.
You scratched your ticket and you seen a number. Correct codes have to match the number on the ticket and the number winning the lottery. Some use same string of input bits to match both, some use different pieces of input string.
(I am assuming that S.I. can not precisely predict the lottery. Even assuming a completely deterministic universe, light from the distant stars, incoming cosmic rays, all of that incoming information ends up mixed in the grand hash of thermal noise and thermal fluctuations)
edit: to make it clearer. Suppose that the lottery has 1000 decimal digits; you scratch one ticket; then later, the winning number is announced, and it matches your ticket. You will conclude that the lottery was rigged, with very good confidence, won’t you? In absence of some rather curious anthropic reasoning, existence or non existence of 10^1000 −1 other tickets, or other conscious players, is entirely irrelevant (and in presence of anthropics you have to figure out which ancestors of h. sapiens will change your answer and which won’t). With regards to Aumann’s agreement theorem, other people would agree that if they were in your shoes (shared the data and the priors) they’d arrive at same conclusions, so it is not at all violated.
If the world is ever capable of simulating individual people, any given important person is more likely a simulation than the real thing—so, given that I’m not particularly important, I can probably assume I’m not simulated, unless something exceptionally unlikely happens to me.
“Important” people in most MMOs tend to be NPCs. You can’t have every PC be King of Orgrimmar or whatever...
Huh? What does simulation have to do with this?
I’ll use your example against you: Mary Panick recently won $250k from a lottery. Should she increase her belief that someone in the lottery commission crooked the lottery to favor her? How much, exactly?
Objectively, no; as previously mentioned, it shouldn’t surprise us that somebody won the lottery. Subjectively, yes; I would certainly update my odds that something other than pure chance is at work if I happened to win the lottery.
And simulation is coming from Robin Hanson’s assertion that if you’re an important person in the world, you should probably update your priors to suggest you are being simulated; it’s a related argument. If the world is ever capable of simulating individual people, any given important person is more likely a simulation than the real thing—so, given that I’m not particularly important, I can probably assume I’m not simulated, unless something exceptionally unlikely happens to me. But if I were, say, Obama, maybe I -should- think I’m living in a simulation. From the outside, there’s a president of the United States, so it’s not particularly unusual that -somebody- is the president of the United States. From the inside, it would be unusual that -I- am the president of the United States. Same thing.
Again, why? Suppose we are comparing two models: in one world, there are 1000 haunted houses which are all explained by gaslamping and sleepwalking etc; in the second world, there are 1000 haunted houses and they are all supernatural etc. Upon encountering a haunted house, would you update in favor of ‘I am in world two and houses are supernatural’? Would someone reading your experience update? I propose that neither would update, because the evidence is equally consistent with both worlds; so far so good.
Now, if in world 1 there are 1000 frightening houses with the mundane explanations mentioned, and in world 2 there are 1000 frightening houses with the mundane explanations (human biology and mentality and the laws of probability etc having not changed) plus 1000 frightening houses due to supernatural influences, upon encountering a frightening house would you update?
Of course; in world 2 there are more frightening houses and you have encountered a frightening house, which is twice as likely in world 2 than in world 1 (2000 houses versus 1000 houses), and so you are now more inclined to think you are in world 2 from whatever you were thinking before. But so would an observer reading your experience!
So where does this unique unconveyable evidence (that your post claims your experience has given you) come from?
Ah. It’s coming from anthropics. You’re making the claim that Aumannian agreement cannot convey anthropic information.
You realize that both the SIA and SSA are hotly debated because either seems to lead to absurd conclusions, right? While Aumann just leads to the conclusion ‘people are irrational’, which certainly doesn’t seem absurd to me.
And since one man’s modus ponens is another man’s modus tollens, why isn’t your post just further evidence that anthropic reasoning as currently understood by most people is completely broken and cannot be trusted in anything?
I think there are non-anthropic problems with even rational!humans communicating evidence.
One is that it’s difficult to communicate that you’re not lying, and it is also difficult to communicate that you’re competent at assessing evidence. A rational agent may have priors saying that OrphanWilde is an average LW member, including the associated wide distribution in propensity to lie and competence at judging evidence. On the other hand, rational!OrphanWilde would (hopefully) have a high confidence assessment of himself (herself?) along both dimensions. However, this assessment is difficult to communicate, since there are strong incentives to lie about these assessments (and also a lot of potential for someone to turn out to not be entirely rational and just get these assessments wrong). So, the rational agent may read this post and update to believing it’s much more likely that OrphanWilde either lies to people for fun (just look at all those improbable details!) or is incompetent at assessing evidence and falls prey to apophenia a lot.
This might not be an issue were it not for the second problem, which is that communication is costly. If communication were free, OrphanWilde could just tell us every single little detail about his life (including in this house and in other houses), and we could then ignore the problem of him potentially being a poor judge of evidence. Alternatively, he could probably perform some very large volume evidence-assessment test to prove that he is, in fact, competent. However, since communication is costly, this seems to be impractical in reality. (The lying issue is slightly different, but could perhaps be overcome with some sort of strong precommitment or an assumption constraining possible motivations combined with a lot of evidence.)
This doesn’t invalidate Aumann agreement as such, but certainly seems to limit its practical applications even for rational agents.
I don’t rule out mundane explanations. Hence my repeated disclaimers on each use of the word “haunted.” If anything “supernatural” exists, it isn’t supernatural, it’s merely natural, and we simply haven’t pinned down what’s going on yet. Empiricism and reductionism don’t get broken.
And anthropic reasoning is -unnecessary- to the logic, it simply provides the simplest examples. I could construct related examples without any anthropic reasoning at all:
You’re shipwrecked on a deserted island with a friend, Johnny. You see a ship in the distance; Johnny’s eyesight is not as good as yours, and he cannot. You’ve been trying to cheer him up for the past three days, because he’s fallen into depression. He doesn’t believe you when you tell him there’s a ship; he cannot see it, and he believes it’s just another attempt by you to cheer him up. -You cannot share the true evidence, because you cannot show him the ship he cannot see-.
Or, to put it in terms of framing:
You flip a coin ten times. It comes up heads each time. versus You say a coin will land heads-up ten times. You flip it ten times, and it comes up heads each time.
Even though the odds are strictly speaking equally likely in both cases, the framing of the first proposition in fact makes it [edited: less significant, not more likely]; you would have been similarly impressed if it had come up tails each time. So the second scenario is twice as significant as the first scenario.
The fact that it is happening to me, rather than another person, is a kind of contextual framing, in much the same sense that calling heads first frames the coin-flipping event.
Fine, replace ‘mundane causes’ with ‘mundane causes minus cause X’ and ‘supernatural with ‘cause X’ in my examples. -_-
And they fail. In the desert island case, Aumann is perfectly applicable: if you have more evidence than he does, then this will be incorporated appropriately; in fact, the desert island case is a great example of Aumann in practice: that you’ve been lying to him merely shows that ‘disagreements are not honest’ (you are the dishonest party here).
How so? You didn’t predict it would be a haunted house before it went, to point out the most obvious disanalogy.
I feel like your disagreement is getting a little slippery here.
My rejection of Aumann is that there is no common knowledge of our posteriors. It’s not necessary for me to have lied to him before, after all; I could have been trying to cheer him up entirely honestly.
If I -had- predicted it would be a haunted house, I’d be suspicious of any evidence that suggested it was. The point isn’t the prediction—prediction is just one mechanism of framing an outcome. The point is in the priors; my prior of -somebody- experiencing a series of weird events in a given house are pretty high, there’s a lot of people out there to experience such weird events, and some of them will experience several. My prior odds of -me- experiencing a series of weird events in a given house should be pretty low. It’s thus much more significant for -me- to experience a series of weird events in a given house than for some stranger who I wouldn’t have known about except for their reporting such. If I’m not updating my priors after being surprised, what am I doing?
Then why does he distrust you? If you have never lied and will never lie in trying to cheer you up, then he is wrong to distrust you and this is simply an example of irrationality and not uncommunicable knowledge; if he is right to suspect that you or people like you would lie in such circumstances, then ‘disagreements are not honest’ and this is again not uncommunicable knowledge.
And you talk about me being slippery. We’re right back to where we began:
to the extent you have any knowledge in this case of observing a rare but known event, the knowledge is communicable to an outsider who can also update; it is no more ‘significant’ to you than a stranger, and any significance may just reflect known cognitive biases like anecdotes, salience, base-rate neglect, etc (and fall under the irrationality rubric)
to the extent that this knowledge is anthropic, one can argue that this is uncommunicable but anthropic arguments are so divergent and unreliable that it’s not clear you have learned uncommunicable knowledge rather than found yet another case in which anthropic arguments are unreliable and give absurd conclusions.
You have not shown any examples which simultaneously involve uncommunicable knowledge which does not involve anthropics (what you are claiming is possible) and rationality and honesty on the part of all participants.
You’re presupposing that trustworthiness is communicable, or that rationality demands trusting somebody absent evidence to do otherwise. There’s definitely incommunicable knowledge—what red looks like to me, for example. You’re stretching “rationality” to places it has no business being to defend a proposition, or else adding conditions (honesty) to make the proposition work.
What, exactly, are you calling anthropics? What I’m describing doesn’t depend on either SIA or SSA. If you’re saying that I’m depending upon an argument which treats any given observer as a special case—yes, yes I am, that is in fact the thrust of my argument. Your argument against anthropics was that it leads to “absurd” results. However, your judgment of “absurd” seems tautological; you seem to be treating the idea that being unable to arrive at the same posterior odds as absurd in itself. That’s not an argument. That’s begging the question.
So—where exactly am I absurd in the following statements: 1.) My prior odds of some stranger experiencing highly unlikely circumstances (and me hearing about them—assuming such circumstances are of general interest) should be relatively high 2.) My prior odds of me experiencing highly unlikely circumstances should be low 3.) Per Bayesian inference, given that the likelihood of the two distinct events is different, the posterior distribution is different 4.) Therefore, there is a different amount of information in something happening personally than to something happening to a stranger
Or, hopefully without error, as it’s been a while since I’ve mucked about with this stuff:
M is the event happening to me, O is the event happening to somebody else, X is some idea for which M and O are evidence of, and Z is the population size (assuming random distribution of events:
Or, in short—the posteriors are different. The information is different. There is a piece of incommunicable evidence when something happens to me as opposed to somebody else.
The conditions are right there in the Aumann proofs, are they not? I’m not adding anything, I’m dividing up the possible outcomes: anthropics (questionable), communicable knowledge (contra you), or Aumann is inapplicable (honesty etc).
I’d be interested to see if you could prove that the result holds independently of them.
That’s the point of the modus tollens vs modus ponens saying. You claim to derive a result, but using premises more questionable than the conclusion, in which case you may have merely disproven the premises via reductio ad absurdum. If this is begging the question (which it isn’t, since that’s when your premise contains the conclusion), then every proof by contradiction or reductio ad absurdum is question-begging.
Correct me if I am wrong, but in your example, M is not increased when O fails to happen—more concretely, you assume the number of spooked people you will hear of is constant—when it would be more appropriate to increase the number of observations of O by 1, since if you don’t go into the spooky house someone else does. Then you are merely deriving the uninteresting observation that if there are more events (by 1) consistent with Z, Z will be more likely. Well, yeah. But there is nothing special about one’s own observations in this case; if someone else went into the house and reported observations, you would update a little more, just like if you want into the house, and in both cases, more than if no one went into the house (or they went in and saw nothing).
Also, your equations are messed up. I think you need to escape some stuff.
Aha! I think the issue here is that you’re thinking of it in terms of two identical observers. The observers aren’t identical in my arguments—one is post-hoc. I have realized where the discrepancy between our arguments is coming from with your example, because of the way I keep framing problems as being about the observer. Suppose I and a friend, Bob, are arguing about who goes in the house. In this case, there’s not practical difference between our evidence. The difference isn’t between me and other, the difference is between me and (other who I wouldn’t have known about except for said experience).
Bob and I are identical (I did say this wasn’t necessarily anthropic!) for the purposes of calculation. Bob is included in p(M).
Steve, who wrote a post on a rationality forum describing his experiences, is -not- identical with me for the purposes of calculation. Steve is included in p(O).
Does my argument make more sense now? Bob’s information is fully transferable—in terms of flipping coins, he called heads before flipping ten heads in a row. Steve’s information is -not- - he’s the guy who flipped ten heads in a row without calling anything.
(ETA: I have no idea how to make them look right. How do you escape stuff?)
In certain contexts an asterisk is a magic character and you need to precede it with a backslash to keep it from turning into
<em>or</em>. To getp(X|M, M1) = p(M|X)*P(X|M1)/p(M|M1)
do
Or you can just put equations in their own paragraphs that are indented by four spaces, in which case no characters will have their magic meaning. (This is how I did the above paragraph where the backslash is visible.)
Is there a reference somewhere on LessWrong or the Wiki for the mark-up used in the comments?
There’s a “Show help” button on the right underneath comment fields. The quick reference it reveals includes a link to the wiki page.
The formatting language used is a (not totally bug-free) subset of Markdown.
Laughs I’m so used to useless help screens I ignored that button, looked for it manually on the wiki, couldn’t find it, and gave up. Thanks!
Suppose i type 693012316 693012316 . Maybe I typed same number twice, maybe I used quantum random number generator and got them separately on the first try. You use the equality as evidence of the former, even if you believe in many worlds, where it is basically a lottery played by parallel yous. Likewise, the winner of the lottery observes the same number twice, which is some evidence for various crazy hypotheses where the selection of “I” necessarily coincides with the winner. edit: you’re totally correct though that such crazy hypotheses are quite improbable to begin with.
In my example of two worlds, the odds of observing the observed evidence is the same in both worlds and so there is no update.
What set of worlds are you postulating for your “two numbers” example? Because your example, as far as I understand it, doesn’t seem at all analogous.
I’m talking specifically about supernatural explanations for you winning the lottery, I don’t see either why people opt for supernatural explanations for haunting.
Suppose we do something like Solomonoff induction. Dealing with codes that match observations verbatim. There’s a theory that reads bits off the tape to produce the ticket number, then more bits to produce the lottery draw, and there’s a theory that reads bits off the tape and produces both numbers as equal. Suppose the lottery has the size of 2^20, about 1 million. Then the former theory will need 40 lucky bits to match the observation, whereas the latter theory will need only 20 lucky bits to match the observation. For mostly everyone the latter theory will be eliminated, except the lottery winner, for who it will linger, and now, with the required lucky bits, the difference in length between the theories will decrease by 20 bits. S.I. - using learning agent (AIXI and variations of it) which won the lottery will literally expect higher probability of victory on next lottery, because it didn’t eliminate various “I always win” hypotheses. edit: and indeed, given sufficiently big N, the extra code required for “I always win” hack will be smaller than log2(N) so it may well become the dominant hypothesis after a single victory. Things like S.I. are only guaranteed to be eventually correct for almost everyone; if there’s enough instances, the wrongmost ones can be arbitrarily wrong.
At the end of the day it’s just how the agents learn—if you were constantly winning lotteries, at some point you would start believing you got supernatural powers, or MWI is true plus the consciousness preferentially transfers specifically to the happy winner, or the like. Any learning agent is subject to risk of learning wrong things.
edit: more concise explanation: if you choose a person by some unknown method, and then they win the lottery, that’s distinct from you not choosing some person, then someone winning the lottery. Namely, in the former case you got evidence in favour of the hypothesis that “unknown method” picks lottery winners. For a lottery winner, their place in the world was chosen by some unknown method.
So let’s see if I’m understanding you here.
You treat a lottery output as a bitstring and ask about SI on it. We can imagine a completely naive agent with no previous observations; what will this ignorant predict? Well, it seems reasonable that one of the top predictions will be for the initial bitstring to be repeated; this seems OK by Occam’s razor (events often repeating are necessary for induction) and I understand that empirically investigating simple Turing machines that many (most? all?) terminating programs will repeat output. It will definitely rank the ‘sequence repeats’ hypotheses above that of possible PRNGs, or very complex physical theories encompassing atmospheric noise and balls dropping into baskets etc.
So far, so good.
I think I lose you when you go on to talk about inferring that you will always win and stuff like that. The repeating hypotheses aren’t contingent on who they happen to. If the particular bitstring emitted by the lottery had also included ‘...and this number was picked by Jain Farstrider’, then SI would seem to then also predict that this Jain will win the next one as well, by the same repeating logic. It certainly will not predict that the agent will win, and the hypothesis ‘the agent (usually) wins’ will drop.
Remember that my trichotomy was that you need to either 1) invoke anthropics; 2) break Aumann via something like dishonesty/incompetence; or 3) you actually do have communicable knowledge.
These SI musings doesn’t seem to invoke anthropics or break Aumannian requirements, and looking at them, they seem communicable. ‘AIXI-MC-MML*, why do you think Jain will win the lottery a second time?’ ‘[translated from minimum-message-length model+message] Well, he won it last time and since I am ignorant of everything in the world, it seems reasonable that he will win it again’. ‘Hmm, that’s a good point.’ And ditto if AIXI-MC-MML happened to be the beneficiary.
* I bring up minimum-message length because Patrick Robotham is supposed to be working on a version of AIXI-MC using MML so one would be able to examine the model of the world(s) a program has devised so far and so one could potentially ask ‘why’ it is making the predictions it is. Having a comprehensible approximation of SI would be pretty convenient for discussing what SI would or would not do.
It doesn’t need PRNGs. The least confusing description of S.I. is as following: the probability of a sequence S is the probability that an universal prefix Turing machine with 3 tapes: input tape which can only be read from, head only advanced in one direction, work tape which can be read from and written to, and is initialized with zeroes, and output tape that can only be written to, will output the sequence S when fed a never-ending string of random bits on the input tape.
The head has such rule set that the program can be loaded via the input tape, and then the program can use the input tape as source of data. This is important because a program can then set up an interpreter emulating other Turing machine (which ensures a constant bound on difference between length of code for different machines).
(We predict using conditional probability—if the machine outputs sequence matching the previous observations, what is the probability that it will produce specific future observations) .
So if we are predicting, for example, perfect coin flips, an input string which begins with code that sets up the working tape so that it will subsequently relay random bits from input to the output, does the trick. This code requires the bits on the input tape to match the observation, meaning that for each observed bit, the length of the input string which has to be correct grows by 1 bit.
Meanwhile a code that sets up the machine to output repeating zeroes does not require any more bits on the input tape to be correct. So when you are getting repeated zeroes, the code relaying random bits is being lowered in weight by factor of 2 with each observed bit, whereas the theory outputting zeroes stays the same (until, of course, you encounter a non zero and it is eliminated).
For more information, see referenced papers in
http://www.scholarpedia.org/article/Algorithmic_probability
You scratched your ticket and you seen a number. Correct codes have to match the number on the ticket and the number winning the lottery. Some use same string of input bits to match both, some use different pieces of input string.
(I am assuming that S.I. can not precisely predict the lottery. Even assuming a completely deterministic universe, light from the distant stars, incoming cosmic rays, all of that incoming information ends up mixed in the grand hash of thermal noise and thermal fluctuations)
edit: to make it clearer. Suppose that the lottery has 1000 decimal digits; you scratch one ticket; then later, the winning number is announced, and it matches your ticket. You will conclude that the lottery was rigged, with very good confidence, won’t you? In absence of some rather curious anthropic reasoning, existence or non existence of 10^1000 −1 other tickets, or other conscious players, is entirely irrelevant (and in presence of anthropics you have to figure out which ancestors of h. sapiens will change your answer and which won’t). With regards to Aumann’s agreement theorem, other people would agree that if they were in your shoes (shared the data and the priors) they’d arrive at same conclusions, so it is not at all violated.
The point is that if the lottery is biased it’s more likely to be biased in such a way that the same number repeats.
“Important” people in most MMOs tend to be NPCs. You can’t have every PC be King of Orgrimmar or whatever...