That’s the kind of sentence that I see as arguments for believing your assessment is biased.
Yes, my assessment is certainly biased, I admitted as much in the post. However, I was referring to your claim that LW (in this case, me) was “a failure in rational thinking”, which sounds a lot like Mitchell’s “ungrounded speculations” in my ears.
Of course she gave supporting arguments, you just refuse to hear them
Could you name one? Not any of Mitchell’s argument, but a support for the claim that AI x-risk is just “ungrounded speculation” despite decades of alignment research and lots of papers proving various failures in existing AIs?
In other words you side with Tegmark on insisting to take the question literally, without noticing that both Lecun and Mitchell admit there’s no zero risk
I do side with Tegmark. LeCun compared the risk to an asteroid x-risk, which Tegmark quantified as 1:100,000,000. Mitchell refused to give a number, but it was obvious that she would have put it even below that. If that were true, I’d agree that there is no reason to worry. However, I don’t think it is true. I don’t have a specific estimate, but it is certainly above 1% IMO, high enough to worry about in any case.
As for the style and tone of this exchange, instead of telling me that I’m not listening/not seeing Mitchell’s arguments, it would be helpful if you could tell me what exactly I don’t see.
Thank you for your reply and the clarifications! To briefly comment on your points concerning the examples for blind spots:
superintelligence does not magically solve physical problems
I and everyone I know on LessWrong agree.
evolution don’t believe in instrumental convergence
I disagree. Evolution is all about instrumental convergence IMO. The “goal” of evolution, or rather the driving force behind it, is reproduction. This leads to all kinds of instrumental goals, like developing methods for food acquisition, attack and defense, impressing the opposite sex, etc. “A chicken is an egg’s way of making another egg”, as Samuel Butler put it.
orthogonality thesis equates there’s no impact on intelligence of holding incoherent values
I’m not sure what you mean by “incoherent”. Intelligence tells you what to do, not what to want. Even complicated constructs of seemingly “objective” or “absolute” values in philosophy are really based on the basic needs we humans have, like being part of a social group or caring for our offspring. Some species of octopuses, for example, which are not social animals, might find the idea of caring for others and helping them when in need ridiculous if they could understand it.
the more intelligent human civilization is becoming, the gentler we are
I wish that were so. We have invented some mechanisms to keep power-seeking and deception in check, so we can live together in large cities, but this carries only so far. What I currently see is a global deterioration of democratic values. In terms of the “gentleness” of the human species, I can’t see much progress since the days of Buddha, Socrates, and Jesus. The number of violent conflicts may have decreased, but their scale and brutality have only grown worse. The way we treat animals in today’s factory farms certainly doesn’t speak for general human gentleness.
oilI: Could you name one reason (not from Mitchell) for questioning the validity of many works on x-risk in AIs?
Thanks for that. However, my definition of “intelligence” would be “the ability to find solutions for complex decision problems”. It’s unclear whether the ability of slime molds to find the shortest path through a maze or organize in seemingly “intelligent” ways has anything to do with intelligence, although the underlying principles may be similar.
I haven’t read the article you linked in full, but at first glance, it seems to refer to consciousness, not intelligence. Maybe that is a key to understanding the difference in thinking between me, Melanie Mitchell, and possibly you: If she assumes that for AI to present an x-risk, it has to be conscious in the way we humans are, that would explain Mitchell’s low estimate for achieving this anytime soon. However, I don’t believe that. To become uncontrollable and develop instrumental goals, an advanced AI would probably need what Joseph Carlsmith calls “strategic awareness”—a world model that includes the AI itself as a part of its plan to achieve its goals. That is nothing like human experience, emotions, or “qualia”. Arguably, GPT-4 may display early signs of this kind of awareness.
That’s all important points and I’d glad to discuss them. However I’m also noticing a wave of downvotes, so maybe we should go half private with whoever signal they want to read more? Or you think I should just ignore that and go forward with my answers? Both are ok but I’d like to follow your lead as you know the house better.
I’ve received my fair share of downvotes, see for example this post, which got 15 karma out of 24 votes. :) It’s a signal, but not more than that. As long as you remain respectful, you shouldn’t be discouraged from posting your opinion in comments even if people downvote it. I’m always for open discussions as they help me understand how and why I’m not understood.
Yes, my assessment is certainly biased, I admitted as much in the post. However, I was referring to your claim that LW (in this case, me) was “a failure in rational thinking”, which sounds a lot like Mitchell’s “ungrounded speculations” in my ears.
Could you name one? Not any of Mitchell’s argument, but a support for the claim that AI x-risk is just “ungrounded speculation” despite decades of alignment research and lots of papers proving various failures in existing AIs?
I do side with Tegmark. LeCun compared the risk to an asteroid x-risk, which Tegmark quantified as 1:100,000,000. Mitchell refused to give a number, but it was obvious that she would have put it even below that. If that were true, I’d agree that there is no reason to worry. However, I don’t think it is true. I don’t have a specific estimate, but it is certainly above 1% IMO, high enough to worry about in any case.
As for the style and tone of this exchange, instead of telling me that I’m not listening/not seeing Mitchell’s arguments, it would be helpful if you could tell me what exactly I don’t see.
[canceled]
Thank you for your reply and the clarifications! To briefly comment on your points concerning the examples for blind spots:
I and everyone I know on LessWrong agree.
I disagree. Evolution is all about instrumental convergence IMO. The “goal” of evolution, or rather the driving force behind it, is reproduction. This leads to all kinds of instrumental goals, like developing methods for food acquisition, attack and defense, impressing the opposite sex, etc. “A chicken is an egg’s way of making another egg”, as Samuel Butler put it.
I’m not sure what you mean by “incoherent”. Intelligence tells you what to do, not what to want. Even complicated constructs of seemingly “objective” or “absolute” values in philosophy are really based on the basic needs we humans have, like being part of a social group or caring for our offspring. Some species of octopuses, for example, which are not social animals, might find the idea of caring for others and helping them when in need ridiculous if they could understand it.
I wish that were so. We have invented some mechanisms to keep power-seeking and deception in check, so we can live together in large cities, but this carries only so far. What I currently see is a global deterioration of democratic values. In terms of the “gentleness” of the human species, I can’t see much progress since the days of Buddha, Socrates, and Jesus. The number of violent conflicts may have decreased, but their scale and brutality have only grown worse. The way we treat animals in today’s factory farms certainly doesn’t speak for general human gentleness.
Thanks for that. However, my definition of “intelligence” would be “the ability to find solutions for complex decision problems”. It’s unclear whether the ability of slime molds to find the shortest path through a maze or organize in seemingly “intelligent” ways has anything to do with intelligence, although the underlying principles may be similar.
I haven’t read the article you linked in full, but at first glance, it seems to refer to consciousness, not intelligence. Maybe that is a key to understanding the difference in thinking between me, Melanie Mitchell, and possibly you: If she assumes that for AI to present an x-risk, it has to be conscious in the way we humans are, that would explain Mitchell’s low estimate for achieving this anytime soon. However, I don’t believe that. To become uncontrollable and develop instrumental goals, an advanced AI would probably need what Joseph Carlsmith calls “strategic awareness”—a world model that includes the AI itself as a part of its plan to achieve its goals. That is nothing like human experience, emotions, or “qualia”. Arguably, GPT-4 may display early signs of this kind of awareness.
That’s all important points and I’d glad to discuss them. However I’m also noticing a wave of downvotes, so maybe we should go half private with whoever signal they want to read more? Or you think I should just ignore that and go forward with my answers? Both are ok but I’d like to follow your lead as you know the house better.
I’ve received my fair share of downvotes, see for example this post, which got 15 karma out of 24 votes. :) It’s a signal, but not more than that. As long as you remain respectful, you shouldn’t be discouraged from posting your opinion in comments even if people downvote it. I’m always for open discussions as they help me understand how and why I’m not understood.
[canceled]