Yahweh and the associated moral system are far from incomprehensible if you know the cultural context of the Israelites. It’s a recognizably human morality, just a brutal one obsessed with purity of various sorts.
It is not about the moral system being incomprehensible but the acts of the FAI. Whenever something bad happens religious people excuse it with an argument based on “higher intention”. This is the gist of what I wanted to highlight. The similarity between religious people and those true believers into the technological singularity and AI’s. This is not to say it is the same. I’m not arguing about that. I’m saying that this might draw the same kind of people committing the same kind of atrocities. This is very dangerous.
If people don’t like anything happening, i.e. don’t understand it, it’s claimed to be a means to an end that will ultimately benefit their extrapolated volition.
People are not going to claim this in public. But I know that there are people here on LW who are disposed to extensive violence if necessary.
To be clear, I do not doubt the possibilities talked about on LW. I’m not saying they are nonsense like the old religions. What I’m on about is that the ideas the SIAI is based on, while not being nonsense, are posed to draw the same fanatic fellowship and cause the same extreme decisions.
Ask yourself, wouldn’t you fly a plane into a tower if that was the only way to disable Skynet? The difference between religion and the risk of uFAI makes it even more dangerous. This crowd is actually highly intelligent and their incentive based on more than fairy tales told by goatherders. And if dumb people are already able to commit large-scale atrocities based on such nonsense, what are a bunch of highly-intelligent and devoted geeks who see a tangible danger able and willing to do? More so as in this case the very same people who believe it are the ones who think they must act themselves because their God doesn’t even exist yet.
It’s very correct to be suspicious of claims that the stakes are that high, given that irrational memes have a habit of postulating such high stakes. However, assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
...assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
I do not assume that. I’ve always been a vegetarian who’s in favor of animal experiments. I’d drop a nuke to prevent less than what you described above.
Yahweh and the associated moral system are far from incomprehensible if you know the cultural context of the Israelites. It’s a recognizably human morality, just a brutal one obsessed with purity of various sorts.
It is not about the moral system being incomprehensible but the acts of the FAI. Whenever something bad happens religious people excuse it with an argument based on “higher intention”. This is the gist of what I wanted to highlight. The similarity between religious people and those true believers into the technological singularity and AI’s. This is not to say it is the same. I’m not arguing about that. I’m saying that this might draw the same kind of people committing the same kind of atrocities. This is very dangerous.
If people don’t like anything happening, i.e. don’t understand it, it’s claimed to be a means to an end that will ultimately benefit their extrapolated volition.
People are not going to claim this in public. But I know that there are people here on LW who are disposed to extensive violence if necessary.
To be clear, I do not doubt the possibilities talked about on LW. I’m not saying they are nonsense like the old religions. What I’m on about is that the ideas the SIAI is based on, while not being nonsense, are posed to draw the same fanatic fellowship and cause the same extreme decisions.
Ask yourself, wouldn’t you fly a plane into a tower if that was the only way to disable Skynet? The difference between religion and the risk of uFAI makes it even more dangerous. This crowd is actually highly intelligent and their incentive based on more than fairy tales told by goatherders. And if dumb people are already able to commit large-scale atrocities based on such nonsense, what are a bunch of highly-intelligent and devoted geeks who see a tangible danger able and willing to do? More so as in this case the very same people who believe it are the ones who think they must act themselves because their God doesn’t even exist yet.
Yes. I would also drop a nuke on New York if it were the only way to prevent global nuclear war. These are both extremely unlikely scenarios.
It’s very correct to be suspicious of claims that the stakes are that high, given that irrational memes have a habit of postulating such high stakes. However, assuming thereby that the stakes never could actually be that high, regardless of the evidence, is another way of shooting yourself in the foot.
I do not assume that. I’ve always been a vegetarian who’s in favor of animal experiments. I’d drop a nuke to prevent less than what you described above.