luzr

Karma: 2

luzr 10 Jan 2009 9:23 UTC
4 points
on: Serious Stories
Eliezer:

I am starting to be sort of frightened by your premises—especially considering that there is non-zero probablity of creating some nonsentient singleton that tries to realize your values.

Before going any further, I STRONGLY suggest that you think AGAIN what might be interesting in carving wooden legs.

Yes, I like to SEE MOVIES with strong main characters going through the hell. But I would not want any of that.

It does not matter that AI can do everything better than me. Right now, I am not the best carving the wood either. But working with wood is still fun. Or swimming, skiing, playing chess (despite the fact computer can beat you each time), caring about animals etc.

I do not need to do dangerous things to be happy. I am definitely sure about that.

luzr 30 Dec 2008 19:22 UTC
1 point
on: Amputation of Destiny
Eliezer:

“Narnia as a simplified case where the problem is especially stark.”

I believe there are at least two significant differences:
- Aslan was not created by humans, it does not represent the “story of intelligence” (quite contrary, lesser intelligence was created by Aslan, as long as you interpret it as God).
- There is only single Aslan with single predetermined “goal” while there are millions of Culture minds, with no single “goal”.
(actually, second point is what I dislike so much about the idea of singleton—it can turn into something like benevolent but oppressing God too easily. Aslan IS Narnia Singleton).

luzr 29 Dec 2008 22:19 UTC
8 points
on: Amputation of Destiny
David:

“asks a Mind whether it could create symphonies as beautiful as it and how hard it would be”

On somewhat related note, there are still human chess players and competitions...

luzr 29 Dec 2008 21:02 UTC
3 points
on: Amputation of Destiny
Eliezer:

It is really off-topic, and I do not have a copy of Consider Phlebas at hand now, but

http://en.wikipedia.org/wiki/Dra%27Azon

Even if Banks have not mentioned ‘sublimed’ in the first novel, the concept exactly fits Dra’Azon.

Besides, Culture is not really advancing its ‘base’ technology, but rather rebuilding its infrastructure to war-machine.

luzr 29 Dec 2008 20:37 UTC
4 points
on: Amputation of Destiny
Eliezer (about Sublimation):

“Ramarren, Banks added on that part later, and it renders a lot of the earlier books nonsensical—why didn’t the Culture or the Idarans increase their intelligence to win their war, if it was that easy? I refuse to regard Excession as canon; it never happened.”

Just a technical (or fandom?) note:

Sublimed civilization is the central plot of Consider Phlebas (Schar’s world, where Mind escapes, is “protected” by sublimed civilization—that is why direct military action by either Iridans or Culture is impossible).

luzr 29 Dec 2008 20:05 UTC
5 points
on: Amputation of Destiny
Julian Morrison:

Or you can revert the issue once again. You can enjoy your time on obsolete skills (like sports, arts or carving table legs...).

There is no shortage of things to do, there is only a problem with your definition of “worthless”.

luzr 29 Dec 2008 19:54 UTC
2 points
on: Amputation of Destiny
“If you already had the lifespan and the health and the promise of future growth, would you want new powerful superintelligences to be created in your vicinity, on your same playing field?”

Yes, definititely. If nothing else, it means diversity.

“Or would you prefer that we stay on as the main characters in the story of intelligent life, with no higher beings above us?”

I do not care, as long as story continues.

And yes, I would like to hear the story—which is about the same thing I would get in case Minds are prohibited. I will not be the main character of the story anyway, so why should I care?

“Should existing human beings grow up at some eudaimonic rate of intelligence increase, and then eventually decide what sort of galaxy to create, and how to people it?”

Grow up how? Does it involve uploading your mind to computronium?

“Or is it better for a nonsentient superintelligence to exercise that decision on our behalf, and start creating new powerful Minds right away?”

Well, this is the only thing I fear. I would prefer sentient superintelligence to create nonsentient utility maximizers. Much less chance of error, IMO.

“If we don’t have to do it one way or the other—if we have both options—and if there’s no particular need for heroic self-sacrifice—then which do you like?”

As you have said—this is a Big world. I do not think both options are mutually exclusive. The only mutually exclusive option I see is nonsentient maximizer singleton programmed to avoid sentient AI and Minds.

“Well… you could have the humans grow up (at some eudaimonic rate of intelligence increase), and then when new people are created, they might be created as powerful Minds to start with.”

Please, explain the difference between the Mind created outright and “grown up humans”. Do you insist on biological computronium?

As you have said, we are living in a Big world. It inevitably means that there is (or will be) quite likely some Culture like civilisation that we will meet if things go well.

How do you think we will be able to compete with your “no sentient AIs, only grown up humans” bias?

Or: Say your CEV AI creates singleton.

Will we be allowed to create the Culture?

What textbooks will be banned?

Will CEV burn any new textbooks we are going to create so that nobody is able to stand on other people’s arms?

luzr 28 Dec 2008 21:04 UTC
0 points
on: Can’t Unbirth a Child
anon: “The cheesecake is a placeholder for anything that the sentient AI might value highly, while we (upon sufficient introspection) do not.”

I am quite aware of that. Anyway, using “cheescake” as placeholder adds a bias to the whole story.

“Eliezer thinks that some/most of our values are consequences of our long history, and are unlikely to be shared by other sentient beings.”

Indeed. So what? In reality, I am quite interested what superintelligence would really consider valueable. But I am pretty sure that “big cheescake” in unlikely.

Thinking about it, AFAIK Eliezer considers himself rationalist. Is not a big part of rationalism involved in disputing values that are merely consequences of our long history?

luzr 28 Dec 2008 19:20 UTC
−6 points
0
on: Can’t Unbirth a Child
“these trillions of people also cared, very strongly, about making giant cheesecakes.”

Uh oh. IMO, that is fallacy. You introduce quite reasonable scenario, then inject some nonsense, without any logic or explanation, to make it look bad.

You should better explain when, on the way from single sentient AI to voting rights fot trillions, cheesecakes came into play. Is it like all sentients being are automatically programmed to like creating big cheescakes? Or anything equally bizzarre?

Subtract cheescakes and your scenario is quite OK with me, including 0,1% of galaxy for humans and 99.9% for AIs. 0.1% of galaxy is about 200 millions of stars...

BTW, it is most likely that without sentient AI, there will be no human (or human originated) presence outside solar system anyway.

Well, so far, my understanding is that your suggestion is to create nonsentient utility maximizer programmed to stop research in certain areas (especially research in creating sentient AI, right?). Thanks, I believe I have a better idea.

luzr 27 Dec 2008 20:19 UTC
−4 points
on: Nonperson Predicates
Uhm, maybe it is naive, but if you have a problem that your mind is too weak to decide, and you have real strong (friendly) superintelligent GAI, would not it be logical to use GAIs strong mental processes to resolve the problem?

luzr 27 Dec 2008 20:09 UTC
0 points
on: Nonsentient Optimizers
“The counter-argument that completely random behavior makes you vulnerable, because predictable agents better enjoy the benefits of social cooperation, just doesn’t have the same pull on people’s emotions.”

BTW, completely deterministic behaviour makes you vulnerable as well. Ask computer security experts.

Somewhat related note: Linux strong random number generator works by capturing real world actions (think user moving mouse) and hashing them into random number that is considered for all practical purposes perfect.

Taking or not taking action may depened on thousands inputs that cannot be reliably predicted or described (the reason is likely buried deep in the physics—enthropy, quantum uncertainity). This IMO is what is the real cause of “free will”.

luzr 20 Dec 2008 10:43 UTC
1 point
on: Complex Novelty
“If this were all the hope the future held, I don’t know if I could bring myself to try. Small wonder that people don’t sign up for cryonics, if even SF writers think this is the best we can do.”

Well, I think that the points missed is that you are not FORCED to carve those legs. If you find something else interesting, do it.

luzr 19 Dec 2008 12:26 UTC
3 points
on: High Challenge
Abigail:

“The “Culture” sequence of novels by Iain M. Banks suggests how people might cope with machines doing all the work.”

Exactly, I think Culture is highly relevant to most topics discussed here. Obviously, it is just a fictional utopia, but I believe it gives plausible answer to “unlimited power future”.

For the reference: http://en.wikipedia.org/wiki/The_Culture

luzr 18 Dec 2008 17:31 UTC
0 points
on: Prolegomena to a Theory of Fun
“Wait for the opponents to catch up a little, stage some nice space battles… close the game window at some point. What if our universe is like that?”

Wow, what a nice elegant Fermi paradox solution:)

luzr 18 Dec 2008 10:41 UTC
1 point
on: Prolegomena to a Theory of Fun
“because you don’t actually want to wake up in an incomprehensible world”

Is not it what all people do each morning anyway?

luzr 16 Dec 2008 8:36 UTC
0 points
on: Not Taking Over the World
“Errr.… luzr, why would I assume that the majority of GAIs that we create will think in a way I define as ‘right’?”

It is not about what YOU define as right.

Anyway, considering that Eliezer is existing self-aware sentient GI agent, with obviously high intelligence and he is able to ask such questions despite his original biological programming makes me suppose that some other powerful strong sentient self-aware GI should reach the same point. I also believe that more general intelligence make GI converge to such “right thinking”.

What makes me worry most is building GAI as non-sentient utility maximizer. OTOH, I believe that ‘non-sentient utility maximizer’ is mutually exclusive with ‘learning’ strong AGI system—in other words, any system capable of learning and exceeding human inteligence must outgrow non-sentience and utility maximizing. I migh be wrong, of course. But the fact that universe is not paperclipped yet makes me hope...

luzr 16 Dec 2008 5:24 UTC
0 points
on: Not Taking Over the World
Phil:

“If we are so unfortunate as to live in a universe in which knowledge is finite, then conflict may serve as a substitute for ignorance in providing us a challenge.”

This is inconsistent. What would conflict really do is to provide new information to process (“knowledge”).

I guess I can agree with the rest of post. What IMO is worth pointing out that the most pleasures, hormones and insticts excluded, are about processing ‘interesting’ infromations.

I guess, somewhere deep in all sentient beings, “interesting informations” are the ultimate joy. This has dire implications for any strong AGI.

I mean, the real pleasure for AGI has to be about acquiring new information patterns. Would not it be a little bit stupid to paperclip solar system in that case?

luzr 16 Dec 2008 5:12 UTC
0 points
on: Not Taking Over the World
“But considering an unlimited amount of ice cream forced me to confront the issue of what to do with any of it.”

“If you invoke the unlimited power to create a quadrillion people, then why not a quadrillion?”

“Say, the programming team has cracked the “hard problem of conscious experience” in sufficient depth that they can guarantee that the AI they create is not sentient—not a repository of pleasure, or pain, or subjective experience, or any interest-in-self—and hence, the AI is only a means to an end, and not an end in itself.”

“What is individually a life worth living?”

Really, is not the ultimate answer to the whole FAI issue encoded there?

IMO, the most important thing about AI is to make sure IT IS SENTIENT. Then, with very high probability, it has to consider the very same questions suggested here.

(And to make sure it does, make more of them and make them diverse. Majority will likely “think right” and supress the rest.)

luzr 12 Dec 2008 18:15 UTC
−1 points
on: What I Think, If Not Why
“real world is deterministic on the most fundamental level”

Is it?

http://en.wikipedia.org/wiki/Determinism#Determinism.2C_quantum_mechanics.2C_and_classical_physics

luzr 11 Dec 2008 23:07 UTC
1 point
on: What I Think, If Not Why
Tim:

Well, as off-topic recourse, I see only cited some engineering problems in your “Against Cyborgs” essay as contraargument. Anyway, let me to say that in my book:

“miniaturizing and refining cell phones, video displays, and other devices that feed our senses. A global-positioning-system brain implant to guide you to your destination would seem seductive only if you could not buy a miniature ear speaker to whisper you directions. Not only could you stow away this and other such gear when you wanted a break, you could upgrade without brain surgery.”

is pretty much equivalent of what I had in mind with cyborging. Brain surgery is not the point. I guess it is even today pretty obvious that to read thoughts, you will not need any surgery at all. And if information is fed back into my glasses, that is OK with.

Still, the ability to just “think” the code (yep, I am a programmer), then see the whole procedure displayed before my eyes already refactored and tested (via weak AI augmentation) sound like nice productivity booster. In fact, I believe that if thinking code is easy, one, with the help of some nice programming language, could learn to use coding to solve much more problems in normal live situations, gradually building personal library of routines..… :)