I disbelieve the orthonality thesis, but I’m not sure that my poisition is covered by either of your two cases. My position is best described as a statement by Yudkowsky:
“for every X except x0, it is mysteriously impossible to build any computational system which generates a range of actions, predicts the consequences of those actions relative to some ontology and world-model, and then selects among probable consequences using criterion X”
I certainly don’t think AIs become friendly automatically. I agree they have to have the correct goal system X (xo) built-in from the start. My guess is that the AIs without the correct X built-in are not true general intelligences. That is to say, I think they would simply stop functioning correctly (or equivalently, there is an intelligence ceiling past which they cannot go).
Why do you think this, and on a related note why do you think AI’s without X will stop functioning/hit a ceiling (in the sense of what is the causal mechanism)?
Starting from my assumption that concept-free general intelligence is impossible, the implication is that there would be some minimal initial set of concepts required to be built-in for all AGIs.
This minimal set of concepts would imply some necessary cognitive biases/heuristics (because the very definition of a ‘concept’ implies a particular grouping or clustering of data – an initial ‘bias’), which in turn is equivalent to some necessary starting values (a ‘bias’ is in a sense, a type of value judgement).
The same set of heuristics/biases (values) involved in taking actions in the world would also be involved in managing (reorganizing) the internal representational system of the AIs. If the reorganization is not performed in a self-consistent fashion, the AIs stop functioning. Remember: we are talking about a closed loop here….the heuristics/biases used to reorganize the representational system, have to themselves be fully represented in that system.
Therefore, the causal mechanism that stops the uAIs would be the eventual breakdown in their representational systems as the need for ever more new concepts arises, stemming from the inconsistent and/or incomplete initial heuristics/biases being used to manage those representational systems (i.e., failing to maintain a closed loop).
I disbelieve the orthonality thesis, but I’m not sure that my poisition is covered by either of your two cases. My position is best described as a statement by Yudkowsky:
“for every X except x0, it is mysteriously impossible to build any computational system which generates a range of actions, predicts the consequences of those actions relative to some ontology and world-model, and then selects among probable consequences using criterion X”
I certainly don’t think AIs become friendly automatically. I agree they have to have the correct goal system X (xo) built-in from the start. My guess is that the AIs without the correct X built-in are not true general intelligences. That is to say, I think they would simply stop functioning correctly (or equivalently, there is an intelligence ceiling past which they cannot go).
Why do you think this, and on a related note why do you think AI’s without X will stop functioning/hit a ceiling (in the sense of what is the causal mechanism)?
Taking a wild guess I’d say…
Starting from my assumption that concept-free general intelligence is impossible, the implication is that there would be some minimal initial set of concepts required to be built-in for all AGIs.
This minimal set of concepts would imply some necessary cognitive biases/heuristics (because the very definition of a ‘concept’ implies a particular grouping or clustering of data – an initial ‘bias’), which in turn is equivalent to some necessary starting values (a ‘bias’ is in a sense, a type of value judgement).
The same set of heuristics/biases (values) involved in taking actions in the world would also be involved in managing (reorganizing) the internal representational system of the AIs. If the reorganization is not performed in a self-consistent fashion, the AIs stop functioning. Remember: we are talking about a closed loop here….the heuristics/biases used to reorganize the representational system, have to themselves be fully represented in that system.
Therefore, the causal mechanism that stops the uAIs would be the eventual breakdown in their representational systems as the need for ever more new concepts arises, stemming from the inconsistent and/or incomplete initial heuristics/biases being used to manage those representational systems (i.e., failing to maintain a closed loop).
Advanced hard math for all this to follow….