Quite a good story. But I think at this point I would quite like Eliezer to make some sort of statement about to what degree he endorses Dath Ilan, ethically speaking. As a fictional setting it’s a great machine for fleshing out thought experiments, of course, but it seems downright dystopian in many ways.
(I mean, the fact that they’re cryopreserving everyone and have AGI under control means they’re morally “preferable” to Earth, but that’s sort of a cheat. For example, you could design an alt. history where the world is ruled by a victorious Third Reich, but where Hitler got super into cryopreservation in his old age, and poured a lot of resources and authority into getting the populations under his control to accept it too. Probably in the long run that world is “preferable” to the world where the Allies win but billions more brains rot — but it’s still not much of a utopia in any useful sense.)
Of course, Eliezer previously stuck the non-consensual-sex thing in Three Worlds Collide, as an attempt to simulate the “future societies will likely trivialize things we consider unthinkable and there’s no way to tell what” effect. I suspect — hope? — some of the ickier parts of Dath Ilan are in the same boat.
However, there are also many elements of Dath Ilan that are obviously supposed to come across as straightforwardly aspirational. And I would rather like some clarity on what bits are meant to ring how, both to get a fuller experience out of reading the existing Dath Ilan stuff and any further entries, and to improve my model of Eliezer’s utility function.
…Oh, by the way, I’m posting this here, because I hadn’t previously realized Eliezer was going to keep returning to Dath Ilan, but the central premise of this one isn’t really what sticks out to me the most, “are we sure we want a world like this?”-wise. Mostly it was some of the stuff in the original Q&A.
Of course, Eliezer previously stuck the non-consensual-sex thing in Three Worlds Collide, as an attempt to simulate the “future societies will likely trivialize things we consider unthinkable and there’s no way to tell what” effect.
A surprising number of people seem to have missed what the point of this was in Three Worlds Collide. It’s not a prediction about future (human) societies. AFAICT it’s there to remind people that changing values is actually bad. That when we talk about, for example, AIs getting random values instead of inheriting our human values, that we should not think of this like we think of a foreign country’s cultural quirks, we should think of this as terrifying and revolting. This is a misconception that a lot of people actually have, and TWC as a whole is aimed squarely at dispelling it.
I strongly disagree that this was the point of this in TWC and would be highly surprised if Eliezer agreed with you. For one thing, the parties involved in nonconsensual sex in TWC seem to be having a perfectly fine time. I also wouldn’t be surprised if someone raping an Ancient such that they have a terrible awful no-good time would fall under some other crime and still get the perpetrator arrested.
I don’t think those are contradictory? It can both be “there would be value drift” and “this might be quite bad, actually”. Anyway, whatever the actual actual spirit of that bit in TWC, that doesn’t change my question of wanting some clarity on whether the worse bits of Dath Ilan are intended in the same spirit.
Quite a good story. But I think at this point I would quite like Eliezer to make some sort of statement about to what degree he endorses Dath Ilan, ethically speaking. As a fictional setting it’s a great machine for fleshing out thought experiments, of course, but it seems downright dystopian in many ways.
(I mean, the fact that they’re cryopreserving everyone and have AGI under control means they’re morally “preferable” to Earth, but that’s sort of a cheat. For example, you could design an alt. history where the world is ruled by a victorious Third Reich, but where Hitler got super into cryopreservation in his old age, and poured a lot of resources and authority into getting the populations under his control to accept it too. Probably in the long run that world is “preferable” to the world where the Allies win but billions more brains rot — but it’s still not much of a utopia in any useful sense.)
Of course, Eliezer previously stuck the non-consensual-sex thing in Three Worlds Collide, as an attempt to simulate the “future societies will likely trivialize things we consider unthinkable and there’s no way to tell what” effect. I suspect — hope? — some of the ickier parts of Dath Ilan are in the same boat.
However, there are also many elements of Dath Ilan that are obviously supposed to come across as straightforwardly aspirational. And I would rather like some clarity on what bits are meant to ring how, both to get a fuller experience out of reading the existing Dath Ilan stuff and any further entries, and to improve my model of Eliezer’s utility function.
…Oh, by the way, I’m posting this here, because I hadn’t previously realized Eliezer was going to keep returning to Dath Ilan, but the central premise of this one isn’t really what sticks out to me the most, “are we sure we want a world like this?”-wise. Mostly it was some of the stuff in the original Q&A.
A surprising number of people seem to have missed what the point of this was in Three Worlds Collide. It’s not a prediction about future (human) societies. AFAICT it’s there to remind people that changing values is actually bad. That when we talk about, for example, AIs getting random values instead of inheriting our human values, that we should not think of this like we think of a foreign country’s cultural quirks, we should think of this as terrifying and revolting. This is a misconception that a lot of people actually have, and TWC as a whole is aimed squarely at dispelling it.
I strongly disagree that this was the point of this in TWC and would be highly surprised if Eliezer agreed with you. For one thing, the parties involved in nonconsensual sex in TWC seem to be having a perfectly fine time. I also wouldn’t be surprised if someone raping an Ancient such that they have a terrible awful no-good time would fall under some other crime and still get the perpetrator arrested.
I don’t think those are contradictory? It can both be “there would be value drift” and “this might be quite bad, actually”. Anyway, whatever the actual actual spirit of that bit in TWC, that doesn’t change my question of wanting some clarity on whether the worse bits of Dath Ilan are intended in the same spirit.