Religion is possibly to blame for the idea that suspended judgment = superiority. Only God is omniscient, so only He knows things for sure, everyone else must act unsure and tentative.
Priests are allowed to pass judgment and still retain their authority, because they are the voice of God on earth. Maybe the idea of judges evolved from priests and retained that immunity.
Value is fragile—isn’t that what conservatives/republicans believe? And the liberal/democrat side believe they can undermine little bits here and there of their society’s value system and not have the whole thing collapse. Who is right?
Evolution (as an algorithm) doesn’t work on the indestructible. Therefore all naturally-evolved beings must be fragile to some extent, and must have evolved to value protecting their fragility.
Yes, a designed life form can have paper clip values, but I don’t think we’ll encounter any naturally occurring beings like this. So our provincial little values may not be so provincial after all, but common on many planets.
How are we meant to interpret the name? At first blush, I would take it to mean “Posts here are less wrong than average, but still wrong,” which is not really encouraging for potential posters...
Also a workaround for anonymous posting might be to make an actual account called “anonymous” and publicize the password.
There were a number of anti-Bush comments in that video. Whatever you thought of him, there were no terrorist attacks for 7 years. Let’s hope Obama can beat that record.
“Why does anything exist in the first place?” or “Why do I find myself in a universe giving rise to experiences that are ordered rather than chaotic?”
So… is cryonics about wanting to see the future, or is it about going to the future to learn the answers to all the “big questions?”
To those who advocate cryonics, if you had all the answers to all the big questions today, would you still use it or would you feel your life “complete” in some way?
I personally will not be using this technique. I will study philosophy and mathematics, and whatever I can find out before I die—that’s it—I just don’t get to know the rest.
The idea of making a mind-design n-space by putting various attributes on the axis, such as humorous/non-humorous, conceptual/perceptual/sensual, etc. -- how much does this tell us about the real possibilites?
What I mean is, for a thing to be possible, there must be some combination of atoms that can fit together to make it work. But merely making an N-space does not tell us about what atoms there are and what they can do.
Come to think of it, how can we assert anything is possible without having already designed it?
But if the brain does not work by magic (of course), then insight does not either. Genius is 99% perspiration, 10,000 failed lightbulbs and all that...
I think the kind of experimental approach Jed Harris was talking about yesterday is where AI will eventually come from. Some researcher who has 10,000 failed AI programs on his hard drive will then have the insight, but not before. The trick is, once he has the insight, to not implement it right away but stop and think about friendliness! But after so many failures how could you resist...
Eliezer, I’m sure if you complete your friendly AI design, there will be multiple honorary PhDs to follow.
“as long as the differences in the new situation are things that were originally allowed to vary”
And all the things that were fixed are still present of course! (since these are what we are presuming are the causal factors)
‘How many new assumptions, exactly, are fatal? How many new terms are you allowed to introduce into an old equation before it becomes “unvetted”, a “new abstraction”?’
Every abstraction is made by holding some things the same and allowing other things to vary. If it allowed nothing to vary it would be a concrete not an abstraction. If it allowed everything to vary it would be the highest possible abstraction—simply “existence.” An abstraction can be reapplied elsewhere as long as the differences in the new situation are things that were originally allowed to vary.
That’s not to say this couldn’t be a black swan, there’s no guarantees, but going purely on evidence what other choice do you have except to do it this way.
“So if you suppose that a final set of changes was enough to produce a sudden huge leap in effective intelligence, it does beg the question of what those changes were.”
Perhaps the final cog was language. The original innovation is concepts: the ability to process thousands of entities at once by forming a class. Major efficiency boost. But chimps can form basic concepts and they didn’t go foom.
Because forming a concept is not good enough—you have to be able to do something useful with it, to process it. Chimps got stuck there, but we passed abstractions through our existing concrete-only processing circuits by using a concrete proxy (a word).
How clear is the distinction between knowledge and intelligence really? The whole genius of the digitial computer is that programs are data. When a human observes someone else doing something, they can copy that action: data seems like programs there too.
And yet “cognitive” is listed as several levels above “knowledge” in the above post, and yesterday CYC was mocked as being not much better than a dictionary. Maybe cognition and knowledge are not so separate, but two perspectives on the same thing.
“Recursion that can rewrite the cognitive level is worth distinguishing.”
Eliezer, would a human that modifies the genes that control how his brain is built qualify as the same class of recursion (but with a longer cycle-time), or is it not quite the same?
‘I found my most productive fifteen minutes were when a friend said, out of nowhere, “want to see who can do the most work in 15 minutes?”’
That’s interesting, because historically great works have been accomplished when a group of really talented people get together in the same place (e.g. Florence, Silicon Valley, Manhattan Project).
The Internet is great in that it enables you to find like minded people and bounce ideas of them. But that’s only half the achievement puzzle. The other half is pestering each other to work, which the Internet is not so good for.
Many rationalists (not saying Eli is one) are of the opinion that introspection is worthless (or at least suspect), so not surprising that trying to predict certain things doesn’t occur to us.
While I totally agree with the sentiment of Eliezer’s prayer, I don’t think saying a prayer on Thanksgiving makes you religious or even implies a belief in God—it’s just tradition. It’s harmless to follow traditions as long as you are epistemologically strong enough not to be in any danger of confusing reality and myth. Just like it’s safe for a person with very strong reason to read a lot of fiction.
Robin’s concept of “Singularity” may be very broad, but your concept of “Optimization Process” is too.
I agree. Creativity is not just being random. The old masters used measurement and perspective when painting their masterpieces, they didn’t just sit there and hum and at the sky and wait for inspiration to strike them.
I think the idea that creativity is somehow mystical comes from a religious model of the human body. If you think your body has causal flesh and a supernatural/acausal soul, and that creativity comes from your soul (the part that is “you”) then it follows that creativity comes from the acausal.
“So do we reason that the most unexpected events, convey the most information, and hence the most surprising acts are those that give us a pleasant shock of creativity—the feeling of suddenly absorbing new information?”
This is very cool.
Stopping aid to Africa? It won’t happen. Even people who fancy themselves rationalist still follow the Christian ethic that it’s better to give something you earn to someone else than to keep it for yourself.
This ethic is irrational because to follow reason is to follow cause and effect, therefore the person who caused the thing to be (who earned it) should suffer the effect (receive the thing).