Imagine fiancéespace (or fiancéspace) - as in the space of romantic partners that would marry you (assuming you’re not married and you want to be). You can imagine “drawing” from that space, but once you draw nearly all of the work is still ahead of you. Someone that was initially “friendly” wouldn’t necessarily stay that way, and someone that was unfriendly wouldn’t necessarily stay that way. It’s like asking “how do you make sure a human mind stays friendly to you forever?” We can’t solve that with our lowly ape minds, and I’m not sure that we’d want to. The closest solution to that I know if with humans is Williams syndrome, and we probably wouldn’t want an AGI with an analogous handicap. The relationship cultured overtime with other minds is more important in many respects the the initial conditions of the other minds.
Maybe dogs are the better metaphor. We want AGIs to be like very smart Labradors. Random, “feral,” AGIs may be more like wolves. So if we made them so they could be “selectively bred” using something like a genetic algorithm? Select for more Lab-y and less Wolf-y traits.
If a Labrador was like 10 or 100 times smarter than it’s owner, would it still be mostly nice most of the time? I would hope so. Maybe the first AGI works like Garm->Fenrir in God of War (spoiler, sorry).
Imagine fiancéespace (or fiancéspace) - as in the space of romantic partners that would marry you (assuming you’re not married and you want to be). You can imagine “drawing” from that space, but once you draw nearly all of the work is still ahead of you. Someone that was initially “friendly” wouldn’t necessarily stay that way, and someone that was unfriendly wouldn’t necessarily stay that way. It’s like asking “how do you make sure a human mind stays friendly to you forever?” We can’t solve that with our lowly ape minds, and I’m not sure that we’d want to. The closest solution to that I know if with humans is Williams syndrome, and we probably wouldn’t want an AGI with an analogous handicap. The relationship cultured overtime with other minds is more important in many respects the the initial conditions of the other minds.
Maybe dogs are the better metaphor. We want AGIs to be like very smart Labradors. Random, “feral,” AGIs may be more like wolves. So if we made them so they could be “selectively bred” using something like a genetic algorithm? Select for more Lab-y and less Wolf-y traits.
If a Labrador was like 10 or 100 times smarter than it’s owner, would it still be mostly nice most of the time? I would hope so. Maybe the first AGI works like Garm->Fenrir in God of War (spoiler, sorry).
Just thinking out loud a bit...
You can’t selectively breed labradors if the first wolf kills you and everyone else.
Of course you can, you just have to make the first set of wolves very small.