Long distance: An extrapolated volition your present-day self finds incomprehensible; not outrageous or annoying, but blankly incomprehensible.
This is exactly the argument religious people use to excuse any shortcomings of their personal FAI. Namely, their personal FAI knows better than you what’s best for your AND everyone else.
What average people do is follow what is being taught here on LW. They decide based on their prior. Their probability estimates tell them that their FAI is likely to exist and make up excuses for extraordinary decisions based on the possible existence of it. That is, support their FAI while trying to inhibit other uFAI all in the best interest of the world at large.
It’s not about it following CEV but people believing it, that it acts in their best interest. Reasons are subordinate. It is the similar systematic of positive and negative incentive that I wanted to highlight.
I grew up in a family of Jehovah’s Witnesses. I can assure you that all believed this to be the case.
Faith is considered the way to happiness.
Positive incentive:
“O you who believe! What is the matter with you, that, when you are asked to go forth in the Cause of Allâh, you cling heavily to the earth? Do you prefer the life of this world to the Hereafter? But little is the comfort of this life, as compared with the Hereafter.”
Whoever does right, whether male or female, and is a believer, We will make him live a good life, and We will award them their reward for the best of what they used to do. (Quran, 16:97)
Negative incentive:
“You dissipated the good things you had in your worldly life and enjoyed yourself in it. So today you are being repaid with the punishment of humiliation for being arrogant in the Earth without any right and for being deviators.” (Surat al-Ahqaf: 20)
I could find heaps of arguments for Christianity that highlight the same believe of God knowing what’s the best for you and the world. This is what most people on this planet believe and this is also the underpinning of the rapture of the nerds.
Ah, I understand—except that I think the “negative incentive” element we’re discussing is absurd, would obviously trigger failsafes with CEV as described, etc.
The whole buzz about the removed content is basically about a kind of incompleteness theorem of FAI. Those who seek to represent every element will reach a certain critical point. You can never represent all elements, never make everyone happy. You can maximize friendliness and happiness and continue to do so, but this journey will always be incomplete.
It’s more artful than this and does only affect a tiny minority though. Nothing to worry about anyway, it’s very unlikely in my opinion. So unlikely that I’d deliberately spit it into the face without losing a nights sleep over it.
Yahweh and the associated moral system are far from incomprehensible if you know the cultural context of the Israelites. It’s a recognizably human morality, just a brutal one obsessed with purity of various sorts.
It is not about the moral system being incomprehensible but the acts of the FAI. Whenever something bad happens religious people excuse it with an argument based on “higher intention”. This is the gist of what I wanted to highlight. The similarity between religious people and those true believers into the technological singularity and AI’s. This is not to say it is the same. I’m not arguing about that. I’m saying that this might draw the same kind of people committing the same kind of atrocities. This is very dangerous.
If people don’t like anything happening, i.e. don’t understand it, it’s claimed to be a means to an end that will ultimately benefit their extrapolated volition.
People are not going to claim this in public. But I know that there are people here on LW who are disposed to extensive violence if necessary.
To be clear, I do not doubt the possibilities talked about on LW. I’m not saying they are nonsense like the old religions. What I’m on about is that the ideas the SIAI is based on, while not being nonsense, are posed to draw the same fanatic fellowship and cause the same extreme decisions.
Ask yourself, wouldn’t you fly a plane into a tower if that was the only way to disable Skynet? The difference between religion and the risk of uFAI makes it even more dangerous. This crowd is actually highly intelligent and their incentive based on more than fairy tales told by goatherders. And if dumb people are already able to commit large-scale atrocities based on such nonsense, what are a bunch of highly-intelligent and devoted geeks who see a tangible danger able and willing to do? More so as in this case the very same people who believe it are the ones who think they must act themselves because their God doesn’t even exist yet.
It’s very correct to be suspicious of claims that the stakes are that high, given that irrational memes have a habit of postulating such high stakes. However, assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
...assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
I do not assume that. I’ve always been a vegetarian who’s in favor of animal experiments. I’d drop a nuke to prevent less than what you described above.
Either I misunderstand CEV, or the above statement re: the Abrahamic god following CEV is false.
Coherent Extrapolated Volition
This is exactly the argument religious people use to excuse any shortcomings of their personal FAI. Namely, their personal FAI knows better than you what’s best for your AND everyone else.
What average people do is follow what is being taught here on LW. They decide based on their prior. Their probability estimates tell them that their FAI is likely to exist and make up excuses for extraordinary decisions based on the possible existence of it. That is, support their FAI while trying to inhibit other uFAI all in the best interest of the world at large.
The link and quotation you posted do not seem to back up your argument that the Abrahamic god follows CEV. Could you clarify?
It’s not about it following CEV but people believing it, that it acts in their best interest. Reasons are subordinate. It is the similar systematic of positive and negative incentive that I wanted to highlight.
I grew up in a family of Jehovah’s Witnesses. I can assure you that all believed this to be the case.
Faith is considered the way to happiness.
Positive incentive:
Negative incentive:
I could find heaps of arguments for Christianity that highlight the same believe of God knowing what’s the best for you and the world. This is what most people on this planet believe and this is also the underpinning of the rapture of the nerds.
Ah, I understand—except that I think the “negative incentive” element we’re discussing is absurd, would obviously trigger failsafes with CEV as described, etc.
There’ll always be elements that suffer, that is perceive FAI as uFAI subjectively.
The whole buzz about the removed content is basically about a kind of incompleteness theorem of FAI. Those who seek to represent every element will reach a certain critical point. You can never represent all elements, never make everyone happy. You can maximize friendliness and happiness and continue to do so, but this journey will always be incomplete.
It’s more artful than this and does only affect a tiny minority though. Nothing to worry about anyway, it’s very unlikely in my opinion. So unlikely that I’d deliberately spit it into the face without losing a nights sleep over it.
Yahweh and the associated moral system are far from incomprehensible if you know the cultural context of the Israelites. It’s a recognizably human morality, just a brutal one obsessed with purity of various sorts.
It is not about the moral system being incomprehensible but the acts of the FAI. Whenever something bad happens religious people excuse it with an argument based on “higher intention”. This is the gist of what I wanted to highlight. The similarity between religious people and those true believers into the technological singularity and AI’s. This is not to say it is the same. I’m not arguing about that. I’m saying that this might draw the same kind of people committing the same kind of atrocities. This is very dangerous.
If people don’t like anything happening, i.e. don’t understand it, it’s claimed to be a means to an end that will ultimately benefit their extrapolated volition.
People are not going to claim this in public. But I know that there are people here on LW who are disposed to extensive violence if necessary.
To be clear, I do not doubt the possibilities talked about on LW. I’m not saying they are nonsense like the old religions. What I’m on about is that the ideas the SIAI is based on, while not being nonsense, are posed to draw the same fanatic fellowship and cause the same extreme decisions.
Ask yourself, wouldn’t you fly a plane into a tower if that was the only way to disable Skynet? The difference between religion and the risk of uFAI makes it even more dangerous. This crowd is actually highly intelligent and their incentive based on more than fairy tales told by goatherders. And if dumb people are already able to commit large-scale atrocities based on such nonsense, what are a bunch of highly-intelligent and devoted geeks who see a tangible danger able and willing to do? More so as in this case the very same people who believe it are the ones who think they must act themselves because their God doesn’t even exist yet.
Yes. I would also drop a nuke on New York if it were the only way to prevent global nuclear war. These are both extremely unlikely scenarios.
It’s very correct to be suspicious of claims that the stakes are that high, given that irrational memes have a habit of postulating such high stakes. However, assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
I do not assume that. I’ve always been a vegetarian who’s in favor of animal experiments. I’d drop a nuke to prevent less than what you described above.