It might be that a tool looks like an agent or v.v., according to one’s perspective. I worry that something unexpected could bite us, like while concentrating on generating a certain output, the AI might tend to a different but invisible track that we can’t see because it parallels our own desires. (The “goal” of EFFICIENCY we share with the AI, but if pursued blindly, our own goal will end up killing us!)
For instance, while being pleased that the Map gives us a short route, maybe the AI is actually giving us a solution based instead on a minimum of attention it gives to us, or some similar but different effect. We might blissfully call this perfect success until the time some subtler problem crops up where we realize we’ve been had, not so much by a scheming AI as by our own blindness to subtleties! Thus we served as our own evil agent, using a powerful tool unwisely.
It might be that a tool looks like an agent or v.v., according to one’s perspective. I worry that something unexpected could bite us, like while concentrating on generating a certain output, the AI might tend to a different but invisible track that we can’t see because it parallels our own desires. (The “goal” of EFFICIENCY we share with the AI, but if pursued blindly, our own goal will end up killing us!)
For instance, while being pleased that the Map gives us a short route, maybe the AI is actually giving us a solution based instead on a minimum of attention it gives to us, or some similar but different effect. We might blissfully call this perfect success until the time some subtler problem crops up where we realize we’ve been had, not so much by a scheming AI as by our own blindness to subtleties! Thus we served as our own evil agent, using a powerful tool unwisely.