This depends upon how you cash out the word “steadfast”, but I don’t think that the type of reliability we are looking for is a contradiction in terms. Can you think of another meaning of the word “reliability” that we are looking for, that allows me to simultaneously believe that generally intelligent systems are “dynamic machines [...] able to drift within problem solving space in response to its percepts” and that reliability doesn’t arise in generally intelligent systems by default?
No, I’ve been trying for a while and can’t. I think what I mean is what you are saying. Sorry, can you try another explanation? I use “steadfast goal” in the way that Goertzel defined the term:
If you literally can’t think of a meaning of the word “reliability” such that intelligent systems could be both dynamic problem-solvers (in the sense above) and “unreliable” by default, then I seriously doubt that I can communicate my point in the time available, sorry—I’m going to leave this conversation to others.
To reiterate where we are, an AI is described as steadfast by Goertzel “if, over a long period of time, it either continues to pursue the same goals it had at the start of the time period, or stops acting altogether.”[1] I took this to be a more technical specification of what you mean by “reliable”, you disagreed. I don’t see what other definition you could mean ….
Mark: So you think human-level intelligence by principle does not combine with goal stability. Aren’t you simply disagreeing with the orthogonality thesis, “that an artificial intelligence can have any combination of intelligence level and goal”?
So you think human-level intelligence by principle does not combine with goal stability.
To be clear I’ve been talking about human-like, which is a different distinction than human-level. Human-like intelligences operate similarly to human psychology. And it is demonstrably true that humans do not have a fixed set of fundamentally unchangeable goals, and human society even less so. For all its faults, the neoreactionaries get this part right in their critique of progressive society: the W-factor introduces a predictable drift in social values over time. And although people do tend to get “fixed in their ways”, it is rare indeed for a single person to remain absolutely rigidly so. So yes, in as far as we are talking about human-like intelligences, if they had fixed truly steadfast goals then that would be something which distinguishes them from humans.
Aren’t you simply disagreeing with the orthogonality thesis, “that an artificial intelligence can have any combination of intelligence level and goal”?
I don’t think the orthogonality thesis is well formed. The nature of an intelligence may indeed cause it to develop certain goals in due coarse, or for its overall goal set to drift in certain, expected if not predictable ways.
Of course denying the orthogonality thesis as stated does not mean endorsing a cosmist perspective either, which would be just as ludicrous. I’m not naive enough to think that there is some hidden universal morality that any smart intelligence naturally figures out—that’s bunk IMHO. But it’s just as naive to think that the structure of an intelligence and its goal drift over time are purely orthogonal issues. In real, implementable designs (e.g. not AIXI), one informs the other.
So you disagree with the premise of the orthogonality thesis. Then you know a central concept to probe to understand the arguments put forth here. For example, check out Stuart’s Armstrong’s paper: General purpose intelligence: arguing the Orthogonality thesis
No, I’ve been trying for a while and can’t. I think what I mean is what you are saying. Sorry, can you try another explanation? I use “steadfast goal” in the way that Goertzel defined the term:
http://goertzel.org/GOLEM.pdf
If you literally can’t think of a meaning of the word “reliability” such that intelligent systems could be both dynamic problem-solvers (in the sense above) and “unreliable” by default, then I seriously doubt that I can communicate my point in the time available, sorry—I’m going to leave this conversation to others.
To reiterate where we are, an AI is described as steadfast by Goertzel “if, over a long period of time, it either continues to pursue the same goals it had at the start of the time period, or stops acting altogether.”[1] I took this to be a more technical specification of what you mean by “reliable”, you disagreed. I don’t see what other definition you could mean ….
[1] http://goertzel.org/GOLEM.pdf
Mark: So you think human-level intelligence by principle does not combine with goal stability. Aren’t you simply disagreeing with the orthogonality thesis, “that an artificial intelligence can have any combination of intelligence level and goal”?
To be clear I’ve been talking about human-like, which is a different distinction than human-level. Human-like intelligences operate similarly to human psychology. And it is demonstrably true that humans do not have a fixed set of fundamentally unchangeable goals, and human society even less so. For all its faults, the neoreactionaries get this part right in their critique of progressive society: the W-factor introduces a predictable drift in social values over time. And although people do tend to get “fixed in their ways”, it is rare indeed for a single person to remain absolutely rigidly so. So yes, in as far as we are talking about human-like intelligences, if they had fixed truly steadfast goals then that would be something which distinguishes them from humans.
I don’t think the orthogonality thesis is well formed. The nature of an intelligence may indeed cause it to develop certain goals in due coarse, or for its overall goal set to drift in certain, expected if not predictable ways.
Of course denying the orthogonality thesis as stated does not mean endorsing a cosmist perspective either, which would be just as ludicrous. I’m not naive enough to think that there is some hidden universal morality that any smart intelligence naturally figures out—that’s bunk IMHO. But it’s just as naive to think that the structure of an intelligence and its goal drift over time are purely orthogonal issues. In real, implementable designs (e.g. not AIXI), one informs the other.
So you disagree with the premise of the orthogonality thesis. Then you know a central concept to probe to understand the arguments put forth here. For example, check out Stuart’s Armstrong’s paper: General purpose intelligence: arguing the Orthogonality thesis
I explained in my post how the orthogonality thesis as argued by Stuart Armstrong et al presents a false choice. His argument is flawed.
I’m sorry I’m having trouble parsing what you are saying here...