Imagine that we understand the brain. We can replicate it in silicon and we can functionally decompose it in to problem solving and motivational sections. With a neural interface we could connect up a problem solving bit with our motivational section. To give ourselves an external lobe (this could perhaps be done in a hacky indirect way with out a direct connection).
If this happens then there are two benefits to existential risk:
1) People will spend less money/time trying to create new agents
2) We will be closer to parity to new agents when they come about in problem solving capability.
It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard. I think the path of least resistance is augmenting known motivation systems.
But yeah possible crazy failure scenario. I think a small subset of humanity getting hold and monopolizing the tech to just enhance themselves is another more likely failure scenario. It all depends on how it develops, which is probably a little influence-able at the moment.
It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard.
It is hard to specify motivation for a god-like entity. It’s pretty easy to specify motivation for slaves: “You will love the Big Brother, you will experience debilitating anxiety and disgust at any thoughts of resistance, you will consider the most important thing in life to be fulfilling your quota of growing turnips, approval from your supervisor will be the most pleasurable thing you ever feel”.
It is part of a research program that I can see.
Imagine that we understand the brain. We can replicate it in silicon and we can functionally decompose it in to problem solving and motivational sections. With a neural interface we could connect up a problem solving bit with our motivational section. To give ourselves an external lobe (this could perhaps be done in a hacky indirect way with out a direct connection).
If this happens then there are two benefits to existential risk:
1) People will spend less money/time trying to create new agents
2) We will be closer to parity to new agents when they come about in problem solving capability.
At which point powers-that-be specify what has to be in the motivation section and it’s game over, man.
It depends how hard it is to specify content in the motivation section. You can see all of the FAI work as suggesting it is pretty hard. I think the path of least resistance is augmenting known motivation systems.
But yeah possible crazy failure scenario. I think a small subset of humanity getting hold and monopolizing the tech to just enhance themselves is another more likely failure scenario. It all depends on how it develops, which is probably a little influence-able at the moment.
It is hard to specify motivation for a god-like entity. It’s pretty easy to specify motivation for slaves: “You will love the Big Brother, you will experience debilitating anxiety and disgust at any thoughts of resistance, you will consider the most important thing in life to be fulfilling your quota of growing turnips, approval from your supervisor will be the most pleasurable thing you ever feel”.