There appears to be a lot of logic here that is happening implicitly because I’m not following you.
You wrote:
An AI could not predict its own actions, because any intelligent agent is quite capable of implementing the algorithm: “Take the predictor’s predicted action. Do the opposite.”
Now, this seems like a very narrow sort of AI that would go and then do something else against what was predicted.
For example, it is a logical contradiction for someone to predict my actions in advance (and tell me about it), because my “programming” will lead me to do something else, much like the above algorithm.
You seem to be using “logical contradiction” in a non-standard fashion. Do you mean it won’t happen given how your mind operates? In that case, permit me to make a few predictions about your actions over the next 48 hours (that you could probably predict also): 1) You will sleep at some point in that time period. 2) You will eat at some point in that time period. I make both of those with probability around .98 each. If we extend to one month I’m willing to make a similar confidence prediction that you will make a phonecall or check your email within in that time. I’m pretty sure you are not going to go out of your way as a result of these predictions to try to go do something else.
You also seem to be missing the point about what an AI would actually need to improve. Say for example that the AI has a subroutine for factoring integers. If it comes up with a better algorithm for factoring integers, it can replace the subroutine with the new one. It doesn’t need to think deeply about how this will alter behavior.
I agree with those predictions. However, my point would become clear if you attempted to translate your probability of 0.98 into a bet with me, with me betting $100 and you betting $5000. I would surely win the bet (with at least a probability of 0.98).
I agree with those predictions. However, my point would become clear if you attempted to translate your probability of 0.98 into a bet with me, with me betting $100 and you betting $5000. I would surely win the bet (with at least a probability of 0.98).
No it wouldn’t because that’s a very different situation. My probability estimate for you not eating food in a 48 hour period if you get paid $5000 when you succeed and must pay $100 if you fail is much lower. If I made the bet with some third party I’d be perfectly willing to do so as long as I had some reassurance that the third party isn’t intending to pay you a large portion of the resulting winnings if you win.
There appears to be a lot of logic here that is happening implicitly because I’m not following you.
You wrote:
Now, this seems like a very narrow sort of AI that would go and then do something else against what was predicted.
You seem to be using “logical contradiction” in a non-standard fashion. Do you mean it won’t happen given how your mind operates? In that case, permit me to make a few predictions about your actions over the next 48 hours (that you could probably predict also): 1) You will sleep at some point in that time period. 2) You will eat at some point in that time period. I make both of those with probability around .98 each. If we extend to one month I’m willing to make a similar confidence prediction that you will make a phonecall or check your email within in that time. I’m pretty sure you are not going to go out of your way as a result of these predictions to try to go do something else.
You also seem to be missing the point about what an AI would actually need to improve. Say for example that the AI has a subroutine for factoring integers. If it comes up with a better algorithm for factoring integers, it can replace the subroutine with the new one. It doesn’t need to think deeply about how this will alter behavior.
I agree with those predictions. However, my point would become clear if you attempted to translate your probability of 0.98 into a bet with me, with me betting $100 and you betting $5000. I would surely win the bet (with at least a probability of 0.98).
I am willing to bet, at 10,000 to 1 odds, that you will sleep sometime in the next 2 weeks. The pay out on this bet is not transferable to your heirs.
No it wouldn’t because that’s a very different situation. My probability estimate for you not eating food in a 48 hour period if you get paid $5000 when you succeed and must pay $100 if you fail is much lower. If I made the bet with some third party I’d be perfectly willing to do so as long as I had some reassurance that the third party isn’t intending to pay you a large portion of the resulting winnings if you win.