When a heuristic AI is creating a successor that shares its goals, does it insist on formally-verified self improvements? Does it try understanding its mushy, hazy goal system so as to avoid reifying something it would regret given its current goals? It seems to me like some mind eventually will have to confront the FAI issue, why not humans then?
If you check with Creating Friendly AI you will see that the term is defined by its primary proponent as follows:
The term “Friendly AI” refers to the production of human-benefiting, non-human harming actions in Artificial Intelligence systems that have advanced to the point of making real-world plans in pursuit of goals.
It’s an anthropocentric term. Only humans would care about creating this sort of agent. You would have to redefine the term if you want to use it to refer to something more general.
Half specifically referred to “creating a successor that shares it’s goals”; this is the problem we face when building an FAI. Nobody is saying an agent with arbitrary goals must at some point face the challenge of building an FAI.
(Incidentally, while Friendly is anthropocentric by default, in common usage analogous concepts relating to other species are referred to as “Friendly to X” of “X-Friendly”, just a good is by default used to mean by human standards, but is sometimes used in “good for X”.
does it insist on formally-verified self improvements. Does it try understanding its mushy, hazy goal system so as to avoid reifying something it would regret given its current goals.
Apparently not. If it did do these things perfectly, it would not be what we are here calling the “heuristic AI.”
When a heuristic AI is creating a successor that shares its goals, does it insist on formally-verified self improvements? Does it try understanding its mushy, hazy goal system so as to avoid reifying something it would regret given its current goals? It seems to me like some mind eventually will have to confront the FAI issue, why not humans then?
If you check with Creating Friendly AI you will see that the term is defined by its primary proponent as follows:
It’s an anthropocentric term. Only humans would care about creating this sort of agent. You would have to redefine the term if you want to use it to refer to something more general.
Half specifically referred to “creating a successor that shares it’s goals”; this is the problem we face when building an FAI. Nobody is saying an agent with arbitrary goals must at some point face the challenge of building an FAI.
(Incidentally, while Friendly is anthropocentric by default, in common usage analogous concepts relating to other species are referred to as “Friendly to X” of “X-Friendly”, just a good is by default used to mean by human standards, but is sometimes used in “good for X”.
Apparently not. If it did do these things perfectly, it would not be what we are here calling the “heuristic AI.”