This argument is irrelevant to the point Eliezer was making in the sequence, as it doesn’t distinguish levels of self-deception possible in normal human experience and those reachable with superoptimization. In effect, you are exploiting the sorites fallacy (or fallacy of gray). That superoptimization might be able to break your mind in a certain way says little about whether your mind can normally break that way.
If you grant me that I am motivated to believe in something false, I think it would not take a super-inteliigent AI to convince me. I could go to a monastery in Tibet, isolate from society and ask the best of them to argue with me every day, study all their books, reading nothing at all that contradicts them. I think it might work. As I pointed out, there are historical examples of people converting to a religion they initially despised. Would my argument not work equally well in this case?
Part of Eliezer’s thesis was that converting to a religion doesn’t qualify if you would still correctly anticipate experimental results that you’d need to explain away.
The claimant must have an accurate model of the situation somewhere in his mind, because he can anticipate, in advance, exactly which experimental results he’ll need to excuse.
Oh, but I have a model of what a creationist believes. I can anticipate what arguments they advance and how to “excuse” them (i.e. explain them away) to some extent. Anyone who changed their belief system has this model for their previous system of belief.
An important distinction between what the post talks about and human arguments/beliefs you refer to is that experimental observations correctly reflect reality, and so ability to anticipate them is ability to model the world, despite the urge to insist on the world working differently than it does.
There are, I think, lots of people who have as good a model of how the world works as any here, who are still religious. In fact, if one is a Deist who believes that God pushed the button to start the Big Bang, they may have a model with an extra node in it subject to Occam’s razor, but it predicts reality equally well, at least until physicists understand the Big Bang better. Many other people have beliefs of purely “spiritual” type, having no observable effects.
But I think a Zoroastrian might not qualify, it’s true. So if I read the book and become one, I might be forced to believe that per [http://en.wikipedia.org/wiki/Zoroastrianism#Basic_beliefs] water was the first element to be created (and that it is in fact an element). I might be clever enough to rationalize it away, like many people do. E.g. water really refers to hydrogen here. If I can make myself believe in Ahura Mazda, I think I can also find a way to fit all the other beliefs in.
When a Zian believes that there is an invisible elephant in the garage, he might not have any explicit beliefs in his system suggesting that it is permeable to sand. However, when an interlocutor arrives and suggests throwing sand at it, the Zian would immediately insist that the elephant is permeable to sand, in order to excuse the experimental result that the sand goes right through where the elephant is purported to be. This only works because the Zian actually has an accurate model of the world which contains “no elephant in the garage”, so that he can correctly anticipate any experimental results he’ll have to excuse when someone proposes one. Thus, though the Zian claims to believe that there is an invisible elephant in the garage, in fact he does not.
We rarely observe Christians trying to walk on water even though they should be able to, given enough faith. In fact they act as if it’s impossible. I assume that this is the sort of thing you are talking about? But we also see people trying faith healing even though it doesn’t work. Their model of the world really is different from yours. Likewise with scientologists and psychiatry. They aren’t faking it. If Z tells me that I must pray in order to be healed, and not take drugs (I have no idea if it does, probably not) and I do in fact do so, being convinced by the book that I must, would that be sufficient?
Well, yes, most Christians believe some parts of Christianity and disbelieve (or only believe they believe) other parts. That any faithful follower of Christ ought to be able to walk on water and command mountains to move are not things they believe; that prayer affects the world and they have souls are things they do believe. This is much like someone who believes Newton’s Laws apply to the world, but doesn’t believe General Relativity applies to the world.
That is, it’s near impossible to not have an accurate model of Newton’s Laws, mountains, and water in order to survive; it’s quite possible to not have an accurate model of relativistic bodies, statistical significance, and strictly-physical worlds and still survive—even thrive.
Faith healing and souls strike me as part of the category of things Christians are “still allowed” to believe:
One who wishes to believe says, “Does the evidence force me to disbelieve?”
That is, it’s near impossible to not have an accurate model of Newton’s Laws, mountains, and water in order to survive;
One needs surprisingly little. There’s a reason Newton’s laws weren’t arrived at until the 1700s. For the vast majority of purposes one can use a pseudo-Aristotleian view of motion and get decent results. But yes, the other two issues could be more immediately fatal.
Citation needed. I can’t recall ever observing behavior like that by a religious person.
What is really happening is that religions have already been selected to be non-falsifiable. The new adherent to an ancient religion doesn’t have to do a lot of work to disqualify observations. The religion is already adapted to be mostly-compatible with the world as currently known, and not to be vulnerable to simple disproofs. When knowledge changes, some clever adherent comes up with a clever explanation, which is quickly disseminated to the faithful.
I don’t know why this was downvoted. As long as the religion has an accurate model of the world and the observations it will need to excuse, the individual adherents do not need one.
This argument is irrelevant to the point Eliezer was making in the sequence, as it doesn’t distinguish levels of self-deception possible in normal human experience and those reachable with superoptimization. In effect, you are exploiting the sorites fallacy (or fallacy of gray). That superoptimization might be able to break your mind in a certain way says little about whether your mind can normally break that way.
If you grant me that I am motivated to believe in something false, I think it would not take a super-inteliigent AI to convince me. I could go to a monastery in Tibet, isolate from society and ask the best of them to argue with me every day, study all their books, reading nothing at all that contradicts them. I think it might work. As I pointed out, there are historical examples of people converting to a religion they initially despised. Would my argument not work equally well in this case?
Part of Eliezer’s thesis was that converting to a religion doesn’t qualify if you would still correctly anticipate experimental results that you’d need to explain away.
I am not completely sure what you mean.
From Belief in Belief:
Oh, but I have a model of what a creationist believes. I can anticipate what arguments they advance and how to “excuse” them (i.e. explain them away) to some extent. Anyone who changed their belief system has this model for their previous system of belief.
An important distinction between what the post talks about and human arguments/beliefs you refer to is that experimental observations correctly reflect reality, and so ability to anticipate them is ability to model the world, despite the urge to insist on the world working differently than it does.
There are, I think, lots of people who have as good a model of how the world works as any here, who are still religious. In fact, if one is a Deist who believes that God pushed the button to start the Big Bang, they may have a model with an extra node in it subject to Occam’s razor, but it predicts reality equally well, at least until physicists understand the Big Bang better. Many other people have beliefs of purely “spiritual” type, having no observable effects.
But I think a Zoroastrian might not qualify, it’s true. So if I read the book and become one, I might be forced to believe that per [http://en.wikipedia.org/wiki/Zoroastrianism#Basic_beliefs] water was the first element to be created (and that it is in fact an element). I might be clever enough to rationalize it away, like many people do. E.g. water really refers to hydrogen here. If I can make myself believe in Ahura Mazda, I think I can also find a way to fit all the other beliefs in.
I think you’re missing the point.
When a Zian believes that there is an invisible elephant in the garage, he might not have any explicit beliefs in his system suggesting that it is permeable to sand. However, when an interlocutor arrives and suggests throwing sand at it, the Zian would immediately insist that the elephant is permeable to sand, in order to excuse the experimental result that the sand goes right through where the elephant is purported to be. This only works because the Zian actually has an accurate model of the world which contains “no elephant in the garage”, so that he can correctly anticipate any experimental results he’ll have to excuse when someone proposes one. Thus, though the Zian claims to believe that there is an invisible elephant in the garage, in fact he does not.
We rarely observe Christians trying to walk on water even though they should be able to, given enough faith. In fact they act as if it’s impossible. I assume that this is the sort of thing you are talking about? But we also see people trying faith healing even though it doesn’t work. Their model of the world really is different from yours. Likewise with scientologists and psychiatry. They aren’t faking it. If Z tells me that I must pray in order to be healed, and not take drugs (I have no idea if it does, probably not) and I do in fact do so, being convinced by the book that I must, would that be sufficient?
Well, yes, most Christians believe some parts of Christianity and disbelieve (or only believe they believe) other parts. That any faithful follower of Christ ought to be able to walk on water and command mountains to move are not things they believe; that prayer affects the world and they have souls are things they do believe. This is much like someone who believes Newton’s Laws apply to the world, but doesn’t believe General Relativity applies to the world.
That is, it’s near impossible to not have an accurate model of Newton’s Laws, mountains, and water in order to survive; it’s quite possible to not have an accurate model of relativistic bodies, statistical significance, and strictly-physical worlds and still survive—even thrive.
Faith healing and souls strike me as part of the category of things Christians are “still allowed” to believe:
The Fourth Sin, Eliezer Yudkowsky
People managed without an accurate model of Newton’s laws for most of human history.
People do occasionally survive some pretty severe memetic immunity failures. One of my own recent ancestors was a snake handler.
Well, they had a decent approximation of Newton’s laws, at least. Otherwise they would struggle to hit things with thrown rocks.
One needs surprisingly little. There’s a reason Newton’s laws weren’t arrived at until the 1700s. For the vast majority of purposes one can use a pseudo-Aristotleian view of motion and get decent results. But yes, the other two issues could be more immediately fatal.
Citation needed. I can’t recall ever observing behavior like that by a religious person.
What is really happening is that religions have already been selected to be non-falsifiable. The new adherent to an ancient religion doesn’t have to do a lot of work to disqualify observations. The religion is already adapted to be mostly-compatible with the world as currently known, and not to be vulnerable to simple disproofs. When knowledge changes, some clever adherent comes up with a clever explanation, which is quickly disseminated to the faithful.
I don’t know why this was downvoted. As long as the religion has an accurate model of the world and the observations it will need to excuse, the individual adherents do not need one.