do not let anyone else build AGI, plus do not kill anyone, plus do not cause suffering, etc
your problem here is that a good move for the AI is now to anaesthetize everyone, but keep them alive although unconscious until they die naturally.
act in the real world as minimally as possible
I think this might have been one of MIRI’s ideas, but it turns out to be tricky to define what it means. I can’t think what they called it so I can’t find it, but someone will know.
Maybe don’t teletransport anyone until we’ve figured that out?
There may not actually be an answer! I had thought planning for cryonic preservation was a good idea since I was a little boy.
But I found that Eliezer’s arguments in favour of cryonics actually worked backwards on me, and caused me to abandon my previous ideas about what death is and whether I care about entities in the future that remember being me or how many of them there are.
Luckily all that’s replaced them is a vast confusion so I do still have a smoke alarm. Otherwise I ignore the whole problem, go on as usual and don’t bother with cryonics because I’m not anticipating making it to the point of natural death anyway.
OR, if we could at least solve worst-case AI safety (that is, prevent s-risk) it would already be a massive win.
Easy! Build a paperclipper, it kills everyone. We don’t even need to bother doing this, plenty of well funded clever people are working very hard on it on our behalf.
When the brain irreversibly stops you’re dead. It’s clear.
Your problem here is ‘irreversible’, and ‘stops’. How about just slowing it down a really lot?
The problem is that biological violence hurts like hell.
No problem there, I loved rugby and cricket, and they hurt a lot. I’m no masochist! Overcoming the fear and pain and playing anyway is part of the point. What I don’t like is irreversible damage. I have various lifelong injuries (mostly from rugby and cricket...), and various effects of aging preventing me from playing, but if they could be fixed I’d be straight back out there.
But cricket and rugby are no substitute for war, which is what they’re trying to be. And on Mars all injuries heal roughly at the point the pubs open.
have built a bias that “only the mega difficult value alignment will work”
I don’t think so. I think we’d settle for “anything that does better than everyone’s dead”. The problem is that most of the problems look fundamental. If you can do even slightly better than “everyone’s dead”, you can probably solve the whole thing (and build a friendly paperclipper that fills the universe with awesomeness).
So if you do end up coming up with something even slightly better than “everyone’s dead”, do let us know.
I think a lot of the obvious ideas have been thought of before, but I think even then there might still be mileage in making top-level posts about ideas here and letting people take pot-shots at them.
There may well be a nice clear obvious solution to the alignment problem which will make everyone feel a bit silly in retrospect.
It would be ever so undignified if we didn’t think of it because we were convinced we’d already tried everything.
your problem here is that a good move for the AI is now to anaesthetize everyone, but keep them alive although unconscious until they die naturally.
I think this might have been one of MIRI’s ideas, but it turns out to be tricky to define what it means. I can’t think what they called it so I can’t find it, but someone will know.
There may not actually be an answer! I had thought planning for cryonic preservation was a good idea since I was a little boy.
But I found that Eliezer’s arguments in favour of cryonics actually worked backwards on me, and caused me to abandon my previous ideas about what death is and whether I care about entities in the future that remember being me or how many of them there are.
Luckily all that’s replaced them is a vast confusion so I do still have a smoke alarm. Otherwise I ignore the whole problem, go on as usual and don’t bother with cryonics because I’m not anticipating making it to the point of natural death anyway.
Easy! Build a paperclipper, it kills everyone. We don’t even need to bother doing this, plenty of well funded clever people are working very hard on it on our behalf.
Your problem here is ‘irreversible’, and ‘stops’. How about just slowing it down a really lot?
No problem there, I loved rugby and cricket, and they hurt a lot. I’m no masochist! Overcoming the fear and pain and playing anyway is part of the point. What I don’t like is irreversible damage. I have various lifelong injuries (mostly from rugby and cricket...), and various effects of aging preventing me from playing, but if they could be fixed I’d be straight back out there.
But cricket and rugby are no substitute for war, which is what they’re trying to be. And on Mars all injuries heal roughly at the point the pubs open.
I don’t think so. I think we’d settle for “anything that does better than everyone’s dead”. The problem is that most of the problems look fundamental. If you can do even slightly better than “everyone’s dead”, you can probably solve the whole thing (and build a friendly paperclipper that fills the universe with awesomeness).
So if you do end up coming up with something even slightly better than “everyone’s dead”, do let us know.
I think a lot of the obvious ideas have been thought of before, but I think even then there might still be mileage in making top-level posts about ideas here and letting people take pot-shots at them.
There may well be a nice clear obvious solution to the alignment problem which will make everyone feel a bit silly in retrospect.
It would be ever so undignified if we didn’t think of it because we were convinced we’d already tried everything.