You’re familiar with CEV so I’ll try to reply with the concepts from Eliezer’s CEV document.
Defining Friendliness is not the life-or-death problem on which the survival of humanity depends. It is a life-or-death problem, but not the life-or-death problem. Friendly AI requires:
Solving the technical problems required to maintain a well-specified abstract invariant in a self-modifying goal system. (Interestingly, this problem is relatively straightforward from a theoretical standpoint.)
Choosing something nice to do with the AI. This is about midway in theoretical hairiness between problems 1 and 3.
Designing a framework for an abstract invariant that doesn’t automatically wipe out the human species. This is the hard part.
PhilGoetz does not have a framework for a well specified abstract invariant self-modifying goal system. If Phil was “seeming to be growing too powerful too quickly” then quite likely the same old human problems are occurring and a whole lot more besides.
The problem isn’t with your values, CEV, the problem is that you aren’t a safe system for producing a recursively self improving singularity. Humans don’t even keep the same values when you give them power let alone when they are hacking their brains into unknown territory.
I use ‘extrapolated volition’ when talking about the outcome of the process upon an individual. “Coherent Extrapolated Volition” would be correct but redundant. When speaking of instantiations of CEV with various parameters (of individuals, species or groups) it is practical, technically correct and preferred to write CEV regardless of the count of individuals in the parameter. Partly because it should be clear that CEV and CEV are talking about things very similar in kind. Partly because if people see “CEV” and google it they’ll find out what it means. Mostly because the ‘EV’ acronym is overloaded within the nearby namespace.
AVERAGE(3.1415) works in google docs. It returns 2.1415. If you are comparing a whole heap of aggregations of a feature, some of which only have one value, it is simpler to just use the same formula.
You’re familiar with CEV so I’ll try to reply with the concepts from Eliezer’s CEV document.
PhilGoetz does not have a framework for a well specified abstract invariant self-modifying goal system. If Phil was “seeming to be growing too powerful too quickly” then quite likely the same old human problems are occurring and a whole lot more besides.
The problem isn’t with your values, CEV, the problem is that you aren’t a safe system for producing a recursively self improving singularity. Humans don’t even keep the same values when you give them power let alone when they are hacking their brains into unknown territory.
When talking about one individual, there is no C in CEV.
I use ‘extrapolated volition’ when talking about the outcome of the process upon an individual. “Coherent Extrapolated Volition” would be correct but redundant. When speaking of instantiations of CEV with various parameters (of individuals, species or groups) it is practical, technically correct and preferred to write CEV regardless of the count of individuals in the parameter. Partly because it should be clear that CEV and CEV are talking about things very similar in kind. Partly because if people see “CEV” and google it they’ll find out what it means. Mostly because the ‘EV’ acronym is overloaded within the nearby namespace.
AVERAGE(3.1415) works in google docs. It returns 2.1415. If you are comparing a whole heap of aggregations of a feature, some of which only have one value, it is simpler to just use the same formula.
Seems reasonable.