orthonormal comments on Would a FAI reward us for helping create it?

orthonormal 30 Dec 2011 18:00 UTC
5 points

its goal would be to maximize something like integral of (fun*population) over time

That’s not what LW refers to as an FAI, but instead a failed FAI. See posts like this one and this one, and this wiki entry.
- Shmi 30 Dec 2011 21:04 UTC
  −1 points
  Parent
  I mean it in this sense.
  - orthonormal 30 Dec 2011 21:24 UTC
    6 points
    Parent
    I would bet US$100 that, if asked, Eliezer would say that
    
    the people with the highest fun/resource ratio would be rewarded, most likely those with the lowest IQ, as they would be happy to be injected with the fun drug and kept in suspended animation for as long as possible.
    
    shows a complete misinterpretation of Fun Theory.
    
    I’m not dismissing the possibility of your scenario, just pointing out that SIAI is explicitly excluding that type of outcome from their definition of “Friendly”.
    - Shmi 30 Dec 2011 21:59 UTC
      −1 points
      Parent
      
      I’m not dismissing the possibility of your scenario, just pointing out that SIAI is explicitly excluding that type of outcome from their definition of “Friendly”.
      
      Only under the unlimited resources assumption, which is not the case here.
      - Eliezer Yudkowsky 8 Oct 2012 10:04 UTC
        1 point
        Parent
        I am explicitly calling that unFriendly given bounded resources.