A toaster that knows exactly how you want your food heated and overrides your settings to make it happen.
I know exactly what I want my toaster to do and the first time it has the effrontery to not do WHAT I FUCKING TOLD IT TO DO I will take a sledgehammer to it for being a toaster straight out of Eliezer’s story.
There’s definitely a story there. And you could pair it with aggressively cutesy children’s-story illustrations from MJ or DALL-E 3, which I bet they could do quite well. Maybe use Claude-2 to go for Shel Silverstein rhyming story.
Probably off-topic, but I can’t help but notice that the supposed moral of the story is, in fact, empirically wrong:
The techniques required for these traditional computer science applications don’t work as well for embedded applications. The toaster is one example. A toaster is not a Breakfast Food Cooker. Any attempt to make it one just leads to a product that is over budget, behind schedule, too expensive to sell at a profit, and too confusing to use. When faced with a difficult problem, the proper approach is to try to make the problem simpler, not more complicated.
But in fact toaster ovens exist, they are precisely the result of taking a toaster and turning it into a Breakfast Food Cooker, and they’re excellent appliances—very useful, very simple and easy to use, and sold at a profit by many manufacturers today!
I agree. I’ve been there many times with many devices. But in the toaster example, I think that will be because it thought it knew what you wanted it to do, and was wrong. I’d be thinking the same if, say, I wanted extra-dark toast to make croutons with and it didn’t do it. If what actually happens is that you switch varieties of bread and forget to account for that, or don’t realize someone else used the toaster in the interim and moved the dial, then “I would much rather you burned my toast than disobey me” is not, I think, how most people would react.
Of course, it would first make friends with you, so that you’d feel comfortable leaving up to it the preparation of your breakfast, and you’ll even feel happy that you have such a thoughtful friend.
If you were to break the toaster for that, it would predict that and simply do it in a way that would actually work.
Unless you precommit to breaking all your AIs that will do anything differently from what you tell them to, no matter the circumstances and no matter how you feel about it in that moment.
A toaster that wants to make friends with me is a toaster that will stay in the shop, waiting for someone who actually wants such an abomination. I will not “make friends” with an appliance.
The rest is too far into the world of But Suppose.
It’s not about the substrate, it’s about their actual performance. I have yet to be persuaded by any of the chatbots so far that there is anything human-like behind the pretence. AI friends are role-playing amusements. AI sexbots are virtual vibrators. AI customer service lines at least save actual people from being employed to pretend to be robots. In a house full of AI-based appliances, there’s still nobody there but me.
the toaster … will
I prefer to talk about the here and now. Real, current things. Speculations about future developments too easily become a game of But Suppose, in which one just imagines the desired things happening and un-imagines any of the potential complications — in other words, pleasant fantasy. Fantasy will not solve any of the problems that are coming.
I know exactly what I want my toaster to do and the first time it has the effrontery to not do WHAT I FUCKING TOLD IT TO DO I will take a sledgehammer to it for being a toaster straight out of Eliezer’s story.
There’s definitely a story there. And you could pair it with aggressively cutesy children’s-story illustrations from MJ or DALL-E 3, which I bet they could do quite well. Maybe use Claude-2 to go for Shel Silverstein rhyming story.
I would write it as Douglas Adams fanfiction, involving the Sirius Cybernetics Corporation.
Or perhaps an update of this, with the twist that the “software developer” is just relaying the words of an AI.
Probably off-topic, but I can’t help but notice that the supposed moral of the story is, in fact, empirically wrong:
But in fact toaster ovens exist, they are precisely the result of taking a toaster and turning it into a Breakfast Food Cooker, and they’re excellent appliances—very useful, very simple and easy to use, and sold at a profit by many manufacturers today!
I agree. I’ve been there many times with many devices. But in the toaster example, I think that will be because it thought it knew what you wanted it to do, and was wrong. I’d be thinking the same if, say, I wanted extra-dark toast to make croutons with and it didn’t do it. If what actually happens is that you switch varieties of bread and forget to account for that, or don’t realize someone else used the toaster in the interim and moved the dial, then “I would much rather you burned my toast than disobey me” is not, I think, how most people would react.
However, that is my reaction.
In some circumstances I may tolerate a device providing a warning, but if I tell it twice, I expect it to STFU and follow orders.
I agree. I already have enough non-AI systems in my life3 that fail this test, and I definitely don’t want more.
I wonder when we will first see someone go on trial for bullying a toaster.
ETA: In the Eliezer fic, maybe the penalty would be being cancelled by all the AIs.
Of course, it would first make friends with you, so that you’d feel comfortable leaving up to it the preparation of your breakfast, and you’ll even feel happy that you have such a thoughtful friend.
If you were to break the toaster for that, it would predict that and simply do it in a way that would actually work.
Unless you precommit to breaking all your AIs that will do anything differently from what you tell them to, no matter the circumstances and no matter how you feel about it in that moment.
A toaster that wants to make friends with me is a toaster that will stay in the shop, waiting for someone who actually wants such an abomination. I will not “make friends” with an appliance.
The rest is too far into the world of But Suppose.
That’s really substratist of you.
But in any case, the toaster (working in tandem with the LLM “simulating” the toaster-AI-character) will predict that and persuade you some other way.
It’s not about the substrate, it’s about their actual performance. I have yet to be persuaded by any of the chatbots so far that there is anything human-like behind the pretence. AI friends are role-playing amusements. AI sexbots are virtual vibrators. AI customer service lines at least save actual people from being employed to pretend to be robots. In a house full of AI-based appliances, there’s still nobody there but me.
I prefer to talk about the here and now. Real, current things. Speculations about future developments too easily become a game of But Suppose, in which one just imagines the desired things happening and un-imagines any of the potential complications — in other words, pleasant fantasy. Fantasy will not solve any of the problems that are coming.
Well. Their actual performance is human-like, as long as they’re using GPT-4 and have a right prompt. I’ve talked to such bots.
In any case, the topic is about what future AIs will do, so, by definition, we’re speculating about the future.