Why would something need to be able to follow orders to be useful? Most things in the world do not follow my orders (my furniture, companies that make all my products, most people I know). Like, imagine an AI assistant that’s really good at outputting emails from your inbox that make your company more profitable. You don’t know why it says what it says, but you have learned the empirical fact that as it hires people, fires people, changes their workloads, gives them assignments, that your profits go up a lot. I can’t really tell it what to do, but it sure is useful.
I think nobody knows how to write the code of a fundamentally submissive agent and that other agents are way easier to make, ones that are just optimizing in a way that doesn’t think in terms of submission/dominance. I agree humans exist but nobody understands how they work or how to code one, and you don’t get to count on us learning that before we build super powerful AI systems.
I have no clue why you think that an intelligent Russian is the peak of optimization power. I think that’s a false and wildly anthropomorphic thing to think. Imagine getting 10 Von Neumanns in a locked room with only internet, already it’s more powerful than the Russian, and I bet could do some harm. Now imagine a million. Whatever gets you the assumption that an AI system can’t be more powerful than one human seems wild and I don’t know where you’re getting this idea from.
Btw, unusual ask, but do you want to hop on audio and hash out the debate more sometime? I can make a transcript and can link it here on LW, both posting our own one-paragraph takeaways. I think you’ve been engaging in a broadly good-faith way on the object level in this thread and others and I would be interested in returning the ball.
I think nobody knows how to write the code of a fundamentally submissive agent
Conventional non AI computers are already fundamentally passive. If you boot them up, they just sit there. What’s the problem. The word agent?
Why would something need to be able to follow orders to be useful? Most things in the world do not follow my orders (my furniture, companies that make all my products, most people I know). Like, imagine an AI assistant that’s really good at outputting emails from your inbox that make your company more profitable. You don’t know why it says what it says, but you have learned the empirical fact that as it hires people, fires people, changes their workloads, gives them assignments, that your profits go up a lot. I can’t really tell it what to do, but it sure is useful.
If an AI assistant is replacing a human assistant , it needs to be controllable to the same extent. You don’t expect or want to micromanage a human assistant, but you do expect to set broad parameters.
Sure, if it’s ‘replacing’, but my example isn’t one of replacement, it’s one where it’s useful in a different way to my other products, in a way that I personally suspect is easier to train/build than something that does ‘replacement’.
I disagree with each of your statements.
Why would something need to be able to follow orders to be useful? Most things in the world do not follow my orders (my furniture, companies that make all my products, most people I know). Like, imagine an AI assistant that’s really good at outputting emails from your inbox that make your company more profitable. You don’t know why it says what it says, but you have learned the empirical fact that as it hires people, fires people, changes their workloads, gives them assignments, that your profits go up a lot. I can’t really tell it what to do, but it sure is useful.
I think nobody knows how to write the code of a fundamentally submissive agent and that other agents are way easier to make, ones that are just optimizing in a way that doesn’t think in terms of submission/dominance. I agree humans exist but nobody understands how they work or how to code one, and you don’t get to count on us learning that before we build super powerful AI systems.
I have no clue why you think that an intelligent Russian is the peak of optimization power. I think that’s a false and wildly anthropomorphic thing to think. Imagine getting 10 Von Neumanns in a locked room with only internet, already it’s more powerful than the Russian, and I bet could do some harm. Now imagine a million. Whatever gets you the assumption that an AI system can’t be more powerful than one human seems wild and I don’t know where you’re getting this idea from.
Btw, unusual ask, but do you want to hop on audio and hash out the debate more sometime? I can make a transcript and can link it here on LW, both posting our own one-paragraph takeaways. I think you’ve been engaging in a broadly good-faith way on the object level in this thread and others and I would be interested in returning the ball.
Sure. The best way for me to do that would be through Discord. My id is lone-pine#4172
Would you mind linking the transcript here if you decide to release it publicly? I’d love to hear both of your thoughts expressed in greater detail!
That’d be the plan.
(Ping to reply on Discord.)
Sent you a friend request.
Ooh, I like Ben’s response and am excited about the audio thing happening.
Conventional non AI computers are already fundamentally passive. If you boot them up, they just sit there. What’s the problem. The word agent?
If an AI assistant is replacing a human assistant , it needs to be controllable to the same extent. You don’t expect or want to micromanage a human assistant, but you do expect to set broad parameters.
Yes, the word agent.
Sure, if it’s ‘replacing’, but my example isn’t one of replacement, it’s one where it’s useful in a different way to my other products, in a way that I personally suspect is easier to train/build than something that does ‘replacement’.