I think it might be a bit dangerous to use the metaphor/terminology of mechanism when talking about the processes that align humans within a society. That is a very complex and complicated environment that I find very poorly described by the term “mechanisms”.
When considering how humans align and how that might inform for the AI alignment what stands out the most for me is that alignment is a learning process and probably needs to start very early in the AI’s development—don’t start training the AI on maximizing things but on learning what it means to be aligned with humans. I’m guessing this has been considered—and probably a bit difficult to implement. It is probably also worth noting that we also have a whole legal system that also serves to reinforce cultural norms along with reactions from other one interacts with.
While commenting on something I really shouldn’t be, if the issue is about the runaway paper clip AI that consumes all resources making paper clips then I don’t really see that as a big problem. It is a design failure but the solution, seems to be, is to not give any AI a single focus for maximization. Make them more like a human consumer who has a near inexhaustible set of things it uses to maximize (and I don’t think they are as closely linked as standard econ describes even if equilibrium condition still holds, the per monetary unit of marginal utilities are equalized). That type of structure also insures that those maximize on one axis results are not realistic. I think the risk here is similar to that of addiction for humans.
While commenting on something I really shouldn’t be, if the issue is about the runaway paper clip AI that consumes all resources making paper clips then I don’t really see that as a big problem. It is a design failure but the solution, seems to be, is to not give any AI a single focus for maximization. Make them more like a human consumer who has a near inexhaustible set of things it uses to maximize
Seems like this wouldn’t really help; the AI would just consume all resources making whichever basket of goods you ask it to maximize.
The problem with a paperclip maximizer isn’t the part where it makes paperclips; making paperclips is OK as paperclips have nonzero value in human society. The problem is the part where it consumes all available resources.
I think that over simplifies what I was saying but accept I did not elaborate either.
The consuming all available resources is not a economically sensible outcome (unless one is defining available resources very narrowly) so saying the AI is not a economically informed AI. That doesn’t seem to be too difficult to address.
If the AI is making output that humans value and follows some simple economic rules then that gross over production and exhausting all available resources is not very likely at all. At some point more is in the basket than wanted so production costs exceed output value and the AI should settle into a steady state type mode.
Now if the AI doesn’t care at all about humans and doesn’t act in anything that resembles what we would understand as normal economic behavior you might get that all resources consumed. But I’m not sure it is correct to think an AI would just not be some type of economic agent given so many of the equilibrating forces in economics seem to have parallel processes in other areas.
Does anyone have a pointer to some argument where the AI does consume all resources and points to why the economics of the environment are not holding? Or, a bit differently, why the economics are so different making the outcome rational?
I think it might be a bit dangerous to use the metaphor/terminology of mechanism when talking about the processes that align humans within a society. That is a very complex and complicated environment that I find very poorly described by the term “mechanisms”.
When considering how humans align and how that might inform for the AI alignment what stands out the most for me is that alignment is a learning process and probably needs to start very early in the AI’s development—don’t start training the AI on maximizing things but on learning what it means to be aligned with humans. I’m guessing this has been considered—and probably a bit difficult to implement. It is probably also worth noting that we also have a whole legal system that also serves to reinforce cultural norms along with reactions from other one interacts with.
While commenting on something I really shouldn’t be, if the issue is about the runaway paper clip AI that consumes all resources making paper clips then I don’t really see that as a big problem. It is a design failure but the solution, seems to be, is to not give any AI a single focus for maximization. Make them more like a human consumer who has a near inexhaustible set of things it uses to maximize (and I don’t think they are as closely linked as standard econ describes even if equilibrium condition still holds, the per monetary unit of marginal utilities are equalized). That type of structure also insures that those maximize on one axis results are not realistic. I think the risk here is similar to that of addiction for humans.
Seems like this wouldn’t really help; the AI would just consume all resources making whichever basket of goods you ask it to maximize.
The problem with a paperclip maximizer isn’t the part where it makes paperclips; making paperclips is OK as paperclips have nonzero value in human society. The problem is the part where it consumes all available resources.
I think that over simplifies what I was saying but accept I did not elaborate either.
The consuming all available resources is not a economically sensible outcome (unless one is defining available resources very narrowly) so saying the AI is not a economically informed AI. That doesn’t seem to be too difficult to address.
If the AI is making output that humans value and follows some simple economic rules then that gross over production and exhausting all available resources is not very likely at all. At some point more is in the basket than wanted so production costs exceed output value and the AI should settle into a steady state type mode.
Now if the AI doesn’t care at all about humans and doesn’t act in anything that resembles what we would understand as normal economic behavior you might get that all resources consumed. But I’m not sure it is correct to think an AI would just not be some type of economic agent given so many of the equilibrating forces in economics seem to have parallel processes in other areas.
Does anyone have a pointer to some argument where the AI does consume all resources and points to why the economics of the environment are not holding? Or, a bit differently, why the economics are so different making the outcome rational?