Wei Dai comments on Evaluating the feasibility of SI’s plan

Wei Dai 11 Jan 2013 22:35 UTC
3 points

where it leaves the Earth alone and so we get a lower bound on how disastrous a mistake can be

I don’t see how to make this work. Do we make the AI indifferent about Earth? If so, Earth will be destroyed as a side effect of its other actions. Do we make it block all causal interactions between Earth and the rest of the universe? Then we’ll be permanently stuck on Earth even if the FAI attempt turns out to be successful in other regards. Any other ideas?
- gwern 11 Jan 2013 22:52 UTC
  0 points
  Parent
  
  Do we make the AI indifferent about Earth? If so, Earth will be destroyed as a side effect of its other actions.
  
  I had a similar qualm about the suicide button
  
  Do we make it block all causal interactions between Earth and the rest of the universe? Then we’ll be permanently stuck on Earth even if the FAI attempt turns out to be successful in other regards.
  
  Nothing comes for free.