Ok. I don’t think we are disagreeing here much, if at all. I’m not maintaining that there’s no risk from AI, just that the default original AI is likely not to be universe-optimizing in that way. When I said in the bet “without paying attention to Friendliness”, that did not mean without paying attention to risks, since of course programmers even now try to make their programs safe, but just that they would not try to program it to optimize everything for human goals.
Also, I don’t understand why so many people thought my side of the bet was a bad idea, when Eliezer is betting at odds of 100 to 1 against me, and in fact there are plenty of other ways I could win the bet, even if my whole theory is wrong. For example, it is not even specified in the bet that the AI has to be self-modifying, just superintelligent, so it could be that first a human level AI is constructed, not superintelligent and not self-modifying, and then people build a superintelligence simply by adding on lots of hardware. In that case it is not clear at all that it would have any fast way to take over the world, even if it had the ability and desire to optimize the universe. First it would have to acquire the ability to self-modify, which perhaps it could do by convincing people to give it that ability or by taking other actions in the external world to take over first. But that could take a while, which would mean that I would still win the bet—we would still be around acting normally with a superintelligence in the world. Of course, winning the bet wouldn’t do me much good in that particular situation, but I’d still win. And that’s just one example; I can think of plenty of other ways I could win the bet even while being wrong in theory. I don’t see how anyone can reasonably think he’s 99% certain both that my theory is wrong and that none of these other things will happen.
Do you realize you failed to specify any of that? I feel I’m being slightly generous by interpreting “and the world doesn’t end” to mean a causal relationship, e.g. the existence of the first AGI has to inspire someone else to create a more dangerous version if the AI doesn’t do so itself. (Though I can’t pay if the world ends for some other reason, and I might die beforehand.) Of course, you might persuade whatever judge we agree on to rule in your favor before I would consider the question settled.
(In case it’s not clear, the comment I just linked comes from 2010 or thereabouts. This is not a worry I made up on the spot.)
Given the the fact that the bet is 100 to 1 in my favor, I would be happy to let you judge the result yourself.
Or you could agree to whatever result Eliezer agrees with. However, with Eliezer the conditions are specified, and “the world doesn’t end” just means that we’re still alive with the artificial intelligence running for a week.
Ok. I don’t think we are disagreeing here much, if at all. I’m not maintaining that there’s no risk from AI, just that the default original AI is likely not to be universe-optimizing in that way. When I said in the bet “without paying attention to Friendliness”, that did not mean without paying attention to risks, since of course programmers even now try to make their programs safe, but just that they would not try to program it to optimize everything for human goals.
Also, I don’t understand why so many people thought my side of the bet was a bad idea, when Eliezer is betting at odds of 100 to 1 against me, and in fact there are plenty of other ways I could win the bet, even if my whole theory is wrong. For example, it is not even specified in the bet that the AI has to be self-modifying, just superintelligent, so it could be that first a human level AI is constructed, not superintelligent and not self-modifying, and then people build a superintelligence simply by adding on lots of hardware. In that case it is not clear at all that it would have any fast way to take over the world, even if it had the ability and desire to optimize the universe. First it would have to acquire the ability to self-modify, which perhaps it could do by convincing people to give it that ability or by taking other actions in the external world to take over first. But that could take a while, which would mean that I would still win the bet—we would still be around acting normally with a superintelligence in the world. Of course, winning the bet wouldn’t do me much good in that particular situation, but I’d still win. And that’s just one example; I can think of plenty of other ways I could win the bet even while being wrong in theory. I don’t see how anyone can reasonably think he’s 99% certain both that my theory is wrong and that none of these other things will happen.
Do you realize you failed to specify any of that? I feel I’m being slightly generous by interpreting “and the world doesn’t end” to mean a causal relationship, e.g. the existence of the first AGI has to inspire someone else to create a more dangerous version if the AI doesn’t do so itself. (Though I can’t pay if the world ends for some other reason, and I might die beforehand.) Of course, you might persuade whatever judge we agree on to rule in your favor before I would consider the question settled.
(In case it’s not clear, the comment I just linked comes from 2010 or thereabouts. This is not a worry I made up on the spot.)
Given the the fact that the bet is 100 to 1 in my favor, I would be happy to let you judge the result yourself.
Or you could agree to whatever result Eliezer agrees with. However, with Eliezer the conditions are specified, and “the world doesn’t end” just means that we’re still alive with the artificial intelligence running for a week.