You’ll find it surprisingly difficult to express what does “AI stops” mean, in terms of AI’s preference. AI always exerts some influence on the world, just by existing. By “AI stops”, you mean a particular kind of influence, but it’s very difficult to formalize what kind of influence, exactly, constitutes “stopping”.
I imagine that an AI would periodically evaluate the state of the world, generate values for many variables that describe that state, identify a set of actions that its programming recommends for those values of the variables, and then take those actions.
For an AI to stop doing something would mean that the state of the world corresponding to an AI having reached maximum acceptable resource usage generates variables that lead to a set of actions that do not include additional use by the AI of those resources.
For example, if the AI is controlling an amount of water that we would think is “too much water,” then the AI would not take actions that involve moving significant amounts of water or significantly affecting its quality. The AI would know that it is “controlling” the water because it would model the world for a few cycles after it took no action, and model the world after it took some action, and notice that the water was in different places in the two models. It would do this a few seconds out, a few minutes out, a few hours out, a few days out, a few months out, and a few years out, doing correspondingly blunter and cruder models each time the period increased to economize on processing power and screen out movements of water that were essentially due to chaos theory rather than proximate causation.
Am I missing something? I realize you probably have a lot more experience specifying AI behavior than I do, but it’s difficult for me to understand the logic behind your insight that specifying “AI stops” is hard. Please explain it to me when you get a chance.
Vladimir, I understand that you’re a respected researcher, but if you keep voting down my comments without explaining why you disagree with me, I’m going to stop talking to you. It doesn’t matter how right you are if you never teach me anything.
If you would like me to stop talking to you, feel free to say so outright, and I will do so, without any hard feelings.
(I didn’t downvote the grandparent, and didn’t even see it downvoted. Your comment is still on my “to reply” list, so when it doesn’t feel like work I might eventually reply (the basic argument is related to “hidden complexity of wishes” and preference is thorough). Also note that I don’t receive a notification when you reply to your own comment, I only saw this because I’m subscribed to wedrifid’s comments.)
We can subscribe to comments? Is this via RSS? And can we do it in bulk? (I suppose that would mean creating many subscriptions and grouping them in the reader.)
What I would like is to be able to subscribe to a feed of “comments that have been upvoted by people who usually upvote the comments that I like and downvote the comments that I dislike”.
Via RSS, I’m subscribed to comments by about 20 people (grouped in a Google Reader folder), which is usually enough to point me to whatever interesting discussions are going on, but doesn’t require looking through tons of relatively low-signal comments in the global comments stream. It’s a good solution, you won’t be able to arrange a similar quality of comment selection automatically.
You’ll find it surprisingly difficult to express what does “AI stops” mean, in terms of AI’s preference. AI always exerts some influence on the world, just by existing. By “AI stops”, you mean a particular kind of influence, but it’s very difficult to formalize what kind of influence, exactly, constitutes “stopping”.
I imagine that an AI would periodically evaluate the state of the world, generate values for many variables that describe that state, identify a set of actions that its programming recommends for those values of the variables, and then take those actions.
For an AI to stop doing something would mean that the state of the world corresponding to an AI having reached maximum acceptable resource usage generates variables that lead to a set of actions that do not include additional use by the AI of those resources.
For example, if the AI is controlling an amount of water that we would think is “too much water,” then the AI would not take actions that involve moving significant amounts of water or significantly affecting its quality. The AI would know that it is “controlling” the water because it would model the world for a few cycles after it took no action, and model the world after it took some action, and notice that the water was in different places in the two models. It would do this a few seconds out, a few minutes out, a few hours out, a few days out, a few months out, and a few years out, doing correspondingly blunter and cruder models each time the period increased to economize on processing power and screen out movements of water that were essentially due to chaos theory rather than proximate causation.
Am I missing something? I realize you probably have a lot more experience specifying AI behavior than I do, but it’s difficult for me to understand the logic behind your insight that specifying “AI stops” is hard. Please explain it to me when you get a chance.
Vladimir, I understand that you’re a respected researcher, but if you keep voting down my comments without explaining why you disagree with me, I’m going to stop talking to you. It doesn’t matter how right you are if you never teach me anything.
If you would like me to stop talking to you, feel free to say so outright, and I will do so, without any hard feelings.
(I didn’t downvote the grandparent, and didn’t even see it downvoted. Your comment is still on my “to reply” list, so when it doesn’t feel like work I might eventually reply (the basic argument is related to “hidden complexity of wishes” and preference is thorough). Also note that I don’t receive a notification when you reply to your own comment, I only saw this because I’m subscribed to wedrifid’s comments.)
All right, I apologize for jumping to conclusions.
We can subscribe to comments? Is this via RSS? And can we do it in bulk? (I suppose that would mean creating many subscriptions and grouping them in the reader.)
What I would like is to be able to subscribe to a feed of “comments that have been upvoted by people who usually upvote the comments that I like and downvote the comments that I dislike”.
Via RSS, I’m subscribed to comments by about 20 people (grouped in a Google Reader folder), which is usually enough to point me to whatever interesting discussions are going on, but doesn’t require looking through tons of relatively low-signal comments in the global comments stream. It’s a good solution, you won’t be able to arrange a similar quality of comment selection automatically.
(I upvoted the grandparent. It seems to ask relevant questions and give clear reasons for asking them.)