That’s true, but if it’s “progress” then it must be progress towards something. Will we eventually arrive at our destination, decide society is pretty much perfect, and then stop? Is progress somehow asymptotic so we’ll keep progressing and never quite reach our destination?
It’s quite hard to tell. “Progress” is always relative to the environment you grew up in and on which your ideas and aspirations are based. At the scale of a human life, our trajectory looks a lot like a straight line, but for all we know, it could be circular. At every point on the circle, we would aim to follow the tangent, and it would look like that’s what we are doing. However, as we move along, the tangent would shift ever so subtly and over the course of millennia we would end up doing a roundabout.
I am not saying that’s precisely what we are doing, but there is some truth to it: human goals and values shift. Our environment and upbringing mold us very deeply, in a way that we cannot really abstract away. A big part of what we consider “ideal” is therefore a function of that imprint. However, we rarely ponder the fact that people born and raised in our “ideal world” would be molded differently and thus may have a fundamentally different outlook on life, including wishing for something else. That’s a bit contrived, of course, but it would probably be possible to make a society which wants X when raised on Y, and Y when raised on X, so that it would constantly oscillate between X and Y. We would have enough foresight to figure out a simple oscillator, but if ethics were a kind of semi-random walk, I don’t think it would be obvious. The idea that we are converging towards something might be a bit of an illusion due to underestimating how different future people will be from ourselves.
The thing is, it seems to me that what we’ve been progressing towards is greater expression of our human natures. Greater ability to do what the most positive parts of our natures think we should.
I suspect the negative aspects of our natures occur primarily when access to resources is strained. If every human is sheltered, well fed, has access to plentiful energy, and so on, there aren’t really be any problems to blame on anyone, so everything should work fine (for the most part, anyway). In a sense, this simplifies the task of the AI: you ask it to optimize supply to existing demand and the rest is smooth sailing.
I didn’t literally mean humans, I meant “Creatures with the sorts of goals, values, and personalities that humans have.”
Still, the criterion is explicitly based on human values. Even if not human specifically, you want “human-like” creatures.
Eliezer has suggested that consciousness, sympathy, and boredom are the essential characteristics any intelligent creature should have. I’d love for there to be a wide variety of creatures, but maybe it would be best if they all had those characteristics.
Still fairly anthropomorphic (not necessarily a bad thing, just an observation). In principle, extremely interesting entities could have no conception of self. Sympathy is only relevant to social entities—but why not create solitary ones as well? As for boredom, what makes a population of entities that seek variety in their lives better than one of entities who each have highly specialized interests (all different from each other)? As a whole, wouldn’t the latter display more variation than the former? I mean, when you think about it, in order to bond with each other, social entities must share a lot of preferences, the encoding of which is redundant. Solitary entities with fringe preferences could thus be a cheap and easy way to increase variety.
Or how about creating psychopaths and putting them in controlled environments that they can destroy at will, or creating highly violent entities to throw in fighting pits? Isn’t there a point where this is preferable to creating yet another conscious creature capable of sympathy and boredom?
Sympathy is only relevant to social entities—but why not create solitary ones as well?
A creature that loves solitude might not necessarily be bad to create. But it would still be good to give it capacity for sympathy for pragmatic reasons, to ensure that if it ever did meet another creature it would want to treat it kindly and avoid harming it.
As for boredom, what makes a population of entities that seek variety in their lives better than one of entities who each have highly specialized interests (all different from each other)? As a whole, wouldn’t the latter display more variation than the former?
It’s not about having a specialized interest and exploring it. A creature with no concept of boredom would would, (to paraphrase Eliezer), “play the same screen of the same level of the same fun videogame over and over again.” They wouldn’t be like an autistic savant who knows one subject inside and out. They’d be little better than a wirehead. Someone with narrow interests still explores every single aspect of that interest in great detail. A creature with no boredom would find one tiny aspect of that interest and do it forever.
Or how about creating psychopaths and putting them in controlled environments that they can destroy at will, or creating highly violent entities to throw in fighting pits? Isn’t there a point where this is preferable to creating yet another conscious creature capable of sympathy and boredom?
Yes, I concede that if there is a sufficient quantity of creatures with humane values, it might be good to create other types of creatures for variety’s sake. However, such creatures could be potentially dangerous, we’d have to be very careful.
A creature that loves solitude might not necessarily be bad to create. But it would still be good to give it capacity for sympathy for pragmatic reasons, to ensure that if it ever did meet another creature it would want to treat it kindly and avoid harming it.
Fair enough, though at the level of omnipotence we’re supposing, there would be no chance meetups. You might as well just isolate the creature and be done with it.
A creature with no concept of boredom would would, (to paraphrase Eliezer), “play the same screen of the same level of the same fun videogame over and over again.”
Or it would do it once, and then die happy. Human-like entities might have a lifespan of centuries, and then you would have ephemeral beings living their own limited fantasy for thirty seconds. I mean, why not? We are all bound to repeat ourselves once our interests are exhausted—perhaps entities could be made to embrace death when that happens.
Yes, I concede that if there is a sufficient quantity of creatures with humane values, it might be good to create other types of creatures for variety’s sake. However, such creatures could be potentially dangerous, we’d have to be very careful.
I agree, though an entity with the power to choose the kind of creatures that come to exist probably wouldn’t have much difficulty doing it safely.
It’s quite hard to tell. “Progress” is always relative to the environment you grew up in and on which your ideas and aspirations are based. At the scale of a human life, our trajectory looks a lot like a straight line, but for all we know, it could be circular. At every point on the circle, we would aim to follow the tangent, and it would look like that’s what we are doing. However, as we move along, the tangent would shift ever so subtly and over the course of millennia we would end up doing a roundabout.
I am not saying that’s precisely what we are doing, but there is some truth to it: human goals and values shift. Our environment and upbringing mold us very deeply, in a way that we cannot really abstract away. A big part of what we consider “ideal” is therefore a function of that imprint. However, we rarely ponder the fact that people born and raised in our “ideal world” would be molded differently and thus may have a fundamentally different outlook on life, including wishing for something else. That’s a bit contrived, of course, but it would probably be possible to make a society which wants X when raised on Y, and Y when raised on X, so that it would constantly oscillate between X and Y. We would have enough foresight to figure out a simple oscillator, but if ethics were a kind of semi-random walk, I don’t think it would be obvious. The idea that we are converging towards something might be a bit of an illusion due to underestimating how different future people will be from ourselves.
I suspect the negative aspects of our natures occur primarily when access to resources is strained. If every human is sheltered, well fed, has access to plentiful energy, and so on, there aren’t really be any problems to blame on anyone, so everything should work fine (for the most part, anyway). In a sense, this simplifies the task of the AI: you ask it to optimize supply to existing demand and the rest is smooth sailing.
Still, the criterion is explicitly based on human values. Even if not human specifically, you want “human-like” creatures.
Still fairly anthropomorphic (not necessarily a bad thing, just an observation). In principle, extremely interesting entities could have no conception of self. Sympathy is only relevant to social entities—but why not create solitary ones as well? As for boredom, what makes a population of entities that seek variety in their lives better than one of entities who each have highly specialized interests (all different from each other)? As a whole, wouldn’t the latter display more variation than the former? I mean, when you think about it, in order to bond with each other, social entities must share a lot of preferences, the encoding of which is redundant. Solitary entities with fringe preferences could thus be a cheap and easy way to increase variety.
Or how about creating psychopaths and putting them in controlled environments that they can destroy at will, or creating highly violent entities to throw in fighting pits? Isn’t there a point where this is preferable to creating yet another conscious creature capable of sympathy and boredom?
A creature that loves solitude might not necessarily be bad to create. But it would still be good to give it capacity for sympathy for pragmatic reasons, to ensure that if it ever did meet another creature it would want to treat it kindly and avoid harming it.
It’s not about having a specialized interest and exploring it. A creature with no concept of boredom would would, (to paraphrase Eliezer), “play the same screen of the same level of the same fun videogame over and over again.” They wouldn’t be like an autistic savant who knows one subject inside and out. They’d be little better than a wirehead. Someone with narrow interests still explores every single aspect of that interest in great detail. A creature with no boredom would find one tiny aspect of that interest and do it forever.
Yes, I concede that if there is a sufficient quantity of creatures with humane values, it might be good to create other types of creatures for variety’s sake. However, such creatures could be potentially dangerous, we’d have to be very careful.
Fair enough, though at the level of omnipotence we’re supposing, there would be no chance meetups. You might as well just isolate the creature and be done with it.
Or it would do it once, and then die happy. Human-like entities might have a lifespan of centuries, and then you would have ephemeral beings living their own limited fantasy for thirty seconds. I mean, why not? We are all bound to repeat ourselves once our interests are exhausted—perhaps entities could be made to embrace death when that happens.
I agree, though an entity with the power to choose the kind of creatures that come to exist probably wouldn’t have much difficulty doing it safely.