We’re unFriendly, but we’re unFriendly in a weaker sense than we normally talk about around here: we bear some relationship to the implicit human ethics that we’d want an FAI to uphold, though not a perfect or complete one, and we probably implement a subset of the features that could be used to create a version of Friendliness. Most of us also seem somewhat resistant to the more obvious cognitive traps like wireheading. We’re not there yet, but we’re far further along the road to Friendliness than most points in mind-space.
We also have some built-in limitations that make a hard takeoff difficult for us: though we can self-modify (in a suitably general sense), our architecture is so messy that it’s not fast or easy, especially on individuals. And we run on hardware with a very slow cycle time, although it does parallelize very, very well.
More colloquially, given the kind of power that we talk about FAI eventually having, an arbitrary human or set of humans might use it to make giant golden statues of themselves or carve their dog’s face into the moon, but probably wouldn’t convert the world’s biomass to paperclips. Maybe. I hope.
We’re unFriendly, but we’re unFriendly in a weaker sense than we normally talk about around here: we bear some relationship to the implicit human ethics that we’d want an FAI to uphold, though not a perfect or complete one, and we probably implement a subset of the features that could be used to create a version of Friendliness. Most of us also seem somewhat resistant to the more obvious cognitive traps like wireheading. We’re not there yet, but we’re far further along the road to Friendliness than most points in mind-space.
We also have some built-in limitations that make a hard takeoff difficult for us: though we can self-modify (in a suitably general sense), our architecture is so messy that it’s not fast or easy, especially on individuals. And we run on hardware with a very slow cycle time, although it does parallelize very, very well.
More colloquially, given the kind of power that we talk about FAI eventually having, an arbitrary human or set of humans might use it to make giant golden statues of themselves or carve their dog’s face into the moon, but probably wouldn’t convert the world’s biomass to paperclips. Maybe. I hope.