It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard. I think the path of least resistance is augmenting known motivation systems.
But yeah possible crazy failure scenario. I think a small subset of humanity getting hold and monopolizing the tech to just enhance themselves is another more likely failure scenario. It all depends on how it develops, which is probably a little influence-able at the moment.
It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard.
It is hard to specify motivation for a god-like entity. It’s pretty easy to specify motivation for slaves: “You will love the Big Brother, you will experience debilitating anxiety and disgust at any thoughts of resistance, you will consider the most important thing in life to be fulfilling your quota of growing turnips, approval from your supervisor will be the most pleasurable thing you ever feel”.
It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard. I think the path of least resistance is augmenting known motivation systems.
But yeah possible crazy failure scenario. I think a small subset of humanity getting hold and monopolizing the tech to just enhance themselves is another more likely failure scenario. It all depends on how it develops, which is probably a little influence-able at the moment.
It is hard to specify motivation for a god-like entity. It’s pretty easy to specify motivation for slaves: “You will love the Big Brother, you will experience debilitating anxiety and disgust at any thoughts of resistance, you will consider the most important thing in life to be fulfilling your quota of growing turnips, approval from your supervisor will be the most pleasurable thing you ever feel”.