For a shared and stable value metric to function as a solution to the AI alignment problem it would need also to be:
computable;
computable in new situations where no comparable examples exist;
convergent under self-evaluation.
To illustrate the last requirement, let me make an example.
Let’s suppose that to a new AI is given the task of dividing some fund between the existing four prototype of nuclear fusion plants. It will need to calculate the value of each prototype and their very different supply chains. But it also need to calculate the value of those calculation, since it’s computational power is not infinite, and decide how much to ponder and to what extent calculate the details of those simulation. But it would also need to calculate the value of those calculation, and so on. Only a value that is convergent under self evaluation can be guaranteed to point to an optimal solution.
If that’s what we would have available, then I think FAI would be mostly solved.
How is it going to calculate such things with out a metric for valuation?
Sure, I’m just pointing out that objective and stable are necessary but not sufficient conditions for a value metric to solve the FAI problem, it would also need to have the three features that I detailed, and possibly others. It’s not a refutation, it’s an expansion.
Right but you subtly back handedly agree its a necessary component of AI. If you come back to say “Sure but its not necessarily the ONLY missing component” I will think of you dumb.
For a shared and stable value metric to function as a solution to the AI alignment problem it would need also to be:
computable;
computable in new situations where no comparable examples exist;
convergent under self-evaluation.
To illustrate the last requirement, let me make an example. Let’s suppose that to a new AI is given the task of dividing some fund between the existing four prototype of nuclear fusion plants. It will need to calculate the value of each prototype and their very different supply chains. But it also need to calculate the value of those calculation, since it’s computational power is not infinite, and decide how much to ponder and to what extent calculate the details of those simulation. But it would also need to calculate the value of those calculation, and so on. Only a value that is convergent under self evaluation can be guaranteed to point to an optimal solution.
If that’s what we would have available, then I think FAI would be mostly solved.
How is it going to calculate such things with out a metric for valuation?
Yes so you are seeing the significance of Nash’s proposal, but you don’t believe he is that smart, who is that on?
Sure, I’m just pointing out that objective and stable are necessary but not sufficient conditions for a value metric to solve the FAI problem, it would also need to have the three features that I detailed, and possibly others.
It’s not a refutation, it’s an expansion.
Right but you subtly back handedly agree its a necessary component of AI. If you come back to say “Sure but its not necessarily the ONLY missing component” I will think of you dumb.