And computer scientists haven’t understood the self – because it isn’t about computer science. It’s about the emergent dynamics that happen when you put a whole bunch of general and specialized pattern recognition agents together – a bunch of agents created in a way that they can really cooperate – and when you include in the mix agents oriented toward recognizing patterns in the society as a whole.
and
The goal systems of humans are pretty unpredictable, but a software mind like Novamente is different – the goal system is better-defined. So one reasonable approach is to make the first Novamente a kind of Oracle. Give it a goal system with one top-level goal: To answer peoples’ questions, in a way that’s designed to give them maximum understanding.
It’s possible that with sufficient real-world intelligence tends to come a sense of connectedness with the universe that militates against squashing other sentiences.
I mostly disagree with Ben, but I don’t think judging him based on that paper is fair. It’s pretty bad, but it was also written in 1996. Fourteen years is a lot of time to improve as a thinker.
I had that thought too, and I was thinking of retracting or amending my comment to that effect, but looking at some of his later publications in the same journal(?) suggests that he hasn’t leveled up much since then.
“The Futility Of Emergence” really annoys me. It’s a perfectly useful word. It’s a statement about the map rather than about the territory, but it’s a useful one. Whereas magic means “unknowable unknowns”, emergent means “known unknowns”—the stuff that we know follows, we just don’t know how.
e.g. Chemistry is an emergent property of the Schrodinger equation, but calculating anything useful from that is barely in our grasp. So we just go with the abstraction we know, and they’re separate sciences. But we do know we have that work to do.
Just linking to that essay every time someone you’re disagreeing with says “emergent” is difficult to distinguish from applause lights.
Saying the word “emergent” adds nothing. You’re right that it’s not as bad as calling something magic and declaring that it’s inherently unknowable, but it also offers zero explanatory power. To reword your example:
Chemistry is a property of the Schrodinger equation, but calculating anything useful from that is barely in our grasp. So we just go with the abstraction we know, and they’re separate sciences. But we do know we have that work to do.
There is absolutely no difference in meaning when you take the word “emergent” out. That’s why it isn’t useful, which Eliezer was pointing out.
Nitpick: I don’t think that is exactly what EY was pointing out. Take a look at the comments and the general response of “Huh? Who makes that mistake?” It seems EY was complaining about the tendency of AGI researchers to use “emergence” as if it were an explanation, not ordinary use of the word that doesn’t pretend it is one but just, say, points out that the behavior is surprising given what it’s composed of, or that your current methods aren’t powerful enough to predict the consequences. He didn’t seem to have realized that particular mistake was mostly localized to AGI people.
It seems more likely that when the cited people said “intelligence is an emergent phenomenon”, they were misunderstood as proposing that as a satisfactory explanation of the phenomenon.
There is absolutely no difference in meaning when you take the word “emergent” out. That’s why it isn’t useful, which Eliezer was pointing out.
I’m not entirely sure this is correct. I wouldn’t call the trajectories of planets and galaxies “properties” of Relativity, but I would call it emergent behavior due to Relativity. It’s a stylistic and grammatical choice, like when to use “which” and when to use “that.” They may seem the same to the uninitiated, but there’s a difference and the initiated can tell when you’re doing it wrong.
So, I agree with David Gerard that trying to eradicate the use of the word is misplaced. It’d be like saying “the word ‘which’ is obsolete, we’re only going to use ‘that’ and look down on anyone still using ‘which’.” You lose far more by such a policy than you gain.
From Ten Years to a Positive Singularity:
and
From The Singularity Institute’s Scary Idea (And Why I Don’t Buy It):
From Chance and Consciousness:
And pretty much all of On the Algebraic Structure of Consciousness and Evolutionary Quantum Computation.
This is all just from fifteen minutes of looking around his website. I’m amazed anyone takes him seriously.
Oh...
wow.
I think that paper alone proves your point quite nicely.
I mostly disagree with Ben, but I don’t think judging him based on that paper is fair. It’s pretty bad, but it was also written in 1996. Fourteen years is a lot of time to improve as a thinker.
I had that thought too, and I was thinking of retracting or amending my comment to that effect, but looking at some of his later publications in the same journal(?) suggests that he hasn’t leveled up much since then.
“The Futility Of Emergence” really annoys me. It’s a perfectly useful word. It’s a statement about the map rather than about the territory, but it’s a useful one. Whereas magic means “unknowable unknowns”, emergent means “known unknowns”—the stuff that we know follows, we just don’t know how.
e.g. Chemistry is an emergent property of the Schrodinger equation, but calculating anything useful from that is barely in our grasp. So we just go with the abstraction we know, and they’re separate sciences. But we do know we have that work to do.
Just linking to that essay every time someone you’re disagreeing with says “emergent” is difficult to distinguish from applause lights.
Saying the word “emergent” adds nothing. You’re right that it’s not as bad as calling something magic and declaring that it’s inherently unknowable, but it also offers zero explanatory power. To reword your example:
There is absolutely no difference in meaning when you take the word “emergent” out. That’s why it isn’t useful, which Eliezer was pointing out.
Nitpick: I don’t think that is exactly what EY was pointing out. Take a look at the comments and the general response of “Huh? Who makes that mistake?” It seems EY was complaining about the tendency of AGI researchers to use “emergence” as if it were an explanation, not ordinary use of the word that doesn’t pretend it is one but just, say, points out that the behavior is surprising given what it’s composed of, or that your current methods aren’t powerful enough to predict the consequences. He didn’t seem to have realized that particular mistake was mostly localized to AGI people.
It seems more likely that when the cited people said “intelligence is an emergent phenomenon”, they were misunderstood as proposing that as a satisfactory explanation of the phenomenon.
Nitpick accepted.
I’m not entirely sure this is correct. I wouldn’t call the trajectories of planets and galaxies “properties” of Relativity, but I would call it emergent behavior due to Relativity. It’s a stylistic and grammatical choice, like when to use “which” and when to use “that.” They may seem the same to the uninitiated, but there’s a difference and the initiated can tell when you’re doing it wrong.
So, I agree with David Gerard that trying to eradicate the use of the word is misplaced. It’d be like saying “the word ‘which’ is obsolete, we’re only going to use ‘that’ and look down on anyone still using ‘which’.” You lose far more by such a policy than you gain.
IIRC, that post was adequately dismantled in its comments.