luzr

Karma: −6

luzr Dec 11, 2008, 10:50 PM
0 points
on: What I Think, If Not Why
Eliezer:

“Will, your example, good or bad, is universal over singletons, nonsingletons, any way of doing things anywhere.”

I guess there is significant difference—for singleton, each mistake can be fatal (and not only for it).

I believe that this is the real part I dislike about the idea, except the part where singleton either cannot evolve or cannot stay singleton (because of speed of light vs locality issue).

luzr Dec 11, 2008, 10:39 PM
2 points
on: What I Think, If Not Why
Eliezer:

“Tim, your page doesn’t say anything about FOR loops or self-optimizing compilers not being able to go a second round, which is the part you got from me and then thought you had invented.”

Well, it certainly does:

“Today, machines already do a lot of programming. They perform refactoring tasks which would once have been delegated to junior programmers. They compile high-level languages into machine code, and generate programs from task specifications. They also also automatically detect programming errors, and automatically test existing programs.”

I guess your claim is only a misunderstaning caused by not understaning CS terminology.

Find a new way how to optimize loops is application of automated refactoring and automated testing and benchmarking.

luzr Dec 11, 2008, 9:26 PM
0 points
on: What I Think, If Not Why
Eliezer:

“Tim probably read my analysis using the self-optimizing compiler as an example, then forgot that I had analyzed it and thought that he was inventing a crushing objection on his own.”

Why do you think it is crushing objection? I believe Tim just repeats his favorite theme (which, in fact, I tend to agree with) where machine augmented humans build better machines. If you can use automated refactoring to improve the way compiler works (and today, you often can), that is in fact pretty cool augmentation of human capabilities. It is recursive FOOM. The only difference of your vision and his is that as long as k < 1 (and perhaps some time after that point), humans are important FOOM agents. Also, humans are getting much more capable in the process. For example, machine augmented human (think weak AI + direct neural interface and all that cyborging whistles + mind drugs) might be quite likely to follow the FOOM.

luzr Dec 11, 2008, 9:10 PM
6 points
on: What I Think, If Not Why
“FOOM that takes two years”

In addition to comments by Robin and Aron, I would also pointed out the possibility that longer the FOOM takes, larger the chance it is not local, regardless of security—somewhere else, there might be another FOOMing AI.

Now as I understand, some consider this situation even more dangerous, but it as well might create “take over” defence.

Another comment to FOOM scenario and this is sort of addition to Tim’s post:

“As machines get smarter, they will gradually become able to improve more and more of themselves. Yes, eventually machines will be able to cut humans out of the loop—but before that there will have been much automated improvement of machines by machines—and after that there may still be human code reviews.”

Eliezer seems to spend a lot of time explaining what happens when “k > 1”—when AI intelligence surpases human and starts selfimproving. But I suspect that the phase 0.3 < k < 1 might be pretty long, maybe decades.

Also, moreover, by the time of FOOM, we should be able to use vast amounts of fast ‘subcritical’ AIs (+ weak AIs) as guardians of process. In fact, by that time, k < 1 AIs might play a pretty important role in world economy and security by that time and it does not take too much pattern recognition power to keep things at bay. (Well, in fact, I believe Eliezer proposes something similar in his thesis, except for locality issue).

luzr Dec 8, 2008, 10:51 AM
0 points
on: Artificial Mysterious Intelligence
Tim Tyler:

As much as I like your posts, one formal note:

If you are responding to somebody else, it is always a good idea to put his name at the beginning of post.

luzr Dec 7, 2008, 5:23 PM
−6 points
on: Sustained Strong Recursion
Lightwave:

But goal system of humans, both as individuals, as social group and as civilization, tends to change over the time, and my undestanding is that it is the result of free will and learning experience. It is in fact basis of adaptibility.

I expect the same must be true for real strong AI. And, in that case, singleton simply does not make too much sense anymore, IMHO (when smaller is faster and local is more capable of reacting..)

luzr Dec 7, 2008, 8:45 AM
−7 points
on: Sustained Strong Recursion
Nick Tarleton:

“If multiple local agents have a common goal system”

I guess, you can largely apply that to current human civilization as well...

I was thinking about the “multimind singleton” a little bit more and came to the conclusion the fundamental question is perhaps:

Is strong AI supposed to have the free will?

luzr Dec 6, 2008, 4:46 PM
−6 points
on: Sustained Strong Recursion
Vladimir Nesov:

“Only few responses to changing context are the right ones”

As long as they are “few” instead of “one”—and these “few” still means basically infinite subset of larger infinite set, differences will accumulate over time, leading to different personality.

Note that such personality might not diverge from the basic goal. But it will inevitable start to ‘disagree’ about choosing one of those “few” good choices because of different learning experience.

This, BTW, is the reason why despite what Tooby & Cosmides says, we have highly diverse ecosystem with very large number of species.

luzr Dec 6, 2008, 11:40 AM
−3 points
on: Sustained Strong Recursion
“because parallelizing is programmatically difficult”

Minor note: “Parallelization is programmatically difficult” is in fact another example of recursion.

The real reason why programming focused on serial execution was the fact that the most hardware was serial. There is not much point learning mysteries of multithreaded development if chances that your SW will run on multicore CPU is close to zero.

Now when multicore CPUs are de facto standard, parallel programming is no longer considered prohibitively difficult, it is just another thing you have to learn. There are new tools, new languages etc..

SW always lags behind HW. Intel had 32-bit CPU since 1986, it took 10 years before 32-bit PC software became mainstream...

luzr Dec 6, 2008, 11:29 AM
−7 points
on: Sustained Strong Recursion
“So an AI that splits itself into two may deviate in capability, but should share the same purpose.”

I cannot see how two initially identical strong AIs sharing the purpose would not start to diverge over time. Strong AI is by definition adaptable (means learning), different set of local conditions (= inputs) must lead to splitting AIs “personality”.

In other words, splitting itself has to end with two “itselfs”, as long as both are “strong”.

luzr Dec 6, 2008, 10:00 AM
−5 points
on: Sustained Strong Recursion
“but I can’t think of anything polite to say.”

Then say something unpolite. It is quite possible there is something I have missed about the concept of singleton. I am taking the definition from Nick Bostrom’s page.

Imagine singleton “taking over” the Mars. Every time intelligent decision is to be made, information has to round-trip with Earth − 6 minutes minimum.

In this case, any less intelligent system, but faster and local, is easily able to prevent such singleton’s expansion.

IMO, the same things happens everywhere, just times are much less than 6 minutes.

To sum it up, light speed limit says two things: “smaller is faster” and “local is more capable of reacting”.

luzr Dec 6, 2008, 8:11 AM
−2 points
on: Sustained Strong Recursion
Nice thread.

Seriously, I guess Eliezer really needs this kind of reality check wakeup, before his whole idea of “FOOM” and “recursion” etc… turns into complete cargo cult science.

While I think the basic premise (strong AI friendliness) is quite concern, many of his recent posts sound like he had read too much science fiction and watched Terminator movie too many times.

There are some very basic issues with the whole recursion and singleton ideas… GenericThinker is right, ‘halting problem’ is very relevant there, in fact it proves that the whole “recursion foom in 48 hours” is completely bogus.

As for ‘singleton’, if nothing else (and there is a lot), speed of light is limiting factor. Therefore, to meaningfully react to local information, you need independent intelligent local agent. No matter what you do, independent intelligent local agent will always diverge from singleton’s global policy. End of story, forget about singletons. Strong AI will be small, fast, and there will be a lot of units.

So, while the basic premise, the concern about strong AI safety, remains, I think we should consider alternative scenario: AI grows relatively slowly (but follows the pattern of current ongoing foom), there is no singleton.

luzr Dec 4, 2008, 9:30 AM
0 points
on: Hard Takeoff

They do that using dedicated hardware. Try to paint Crysis in realtime ‘per pixel’, using a vanilla CPU.

Interestingly, today’s high-end vanilla CPU (quadcore at 3Ghz) would paint 7-8 years old games just fine. Means in another 8 years, we will be capable of doing Crysis without GPU.

luzr Dec 3, 2008, 8:59 AM
−1 points
on: Hard Takeoff
anon:

“You’re currently using a program which can access the internet. Why do you think an AI would be unable to do the same?”

I hope it will. Still, that would get it only to preexisting knowledge.

It can draw many hypothesis, but it will have to TEST them (gain empirical knowledge). Think LHC.

BTW, not that there are problems in quantum physics that do not have analytical solution. Some equations simply cannot be solved. Now of course, perhaps superintelligence will find how to do that, but I believe there are quite solid mathematic proofs that it is not possible.

[quote] Also, computer hardware exists for manipulating objects and acquiring sensory data. Furthermore: by hypothesis, the AI can improve itself better then we can, because, as EY pointed out, we’re not exactly cut out for programming. Also, improving an algorithm does not necessarily increase its complexity. [/quote]

I am afraid that you have missed the part about algorithm being essential, but not the core of AI mind. The mind can as well be data. And it can be unoptimizable, for the same reasons some of equations cannot be analytically solved.

[quote] And you don’t have to simulate reality perfectly to understand it, so there is no showstopper there. [/quote]

To understand certain aspects of reality. All I am saying is that to understand certain aspects might not be enough.

What I suggest is that the “mind” might be something as network of interconnected numerical values. For the outside observer, there will be no order in connections or values. To truly understand the “mind” a poorly as by simulation, you would need much bigger mind, as you would have to simulate and carefully examine each of nodes.

Crude simulation does not help here, because you do not know which aspects to look for. Anything can be important.

luzr Dec 3, 2008, 8:39 AM
0 points
on: Recursive Self-Improvement
“Intelligence tests are timed for a good reason. If you see intelligence as an optimisation process, it is obvious why speed matters—you can do more trials.”

Inteligence tests are designed to measure performance of human brain.

Try this: Strong AI running on 2Ghz CPU. You reduce it to 1Ghz, without changing anything else. Will it make less intelligent? Slower, definitely.

luzr Dec 2, 2008, 10:48 PM
0 points
on: Recursive Self-Improvement
″ The faster they get the smarter they are—since one component of intelligence is speed.”

I think this might be incorrect. The speed means that you can solve the problem faster, not that you can solve more complex problem.

luzr Dec 2, 2008, 9:16 PM
9 points
on: Hard Takeoff
“I suspect that, if correctly designed, a midsize computer cluster would be able to get high-grade thinking done at a serial speed much faster than human, even if the total parallel computing power was less.”

I am glad I can agree for once :)

“The main thing I’ll venture into actually expecting from adding “insight” to the mix, is that there’ll be a discontinuity at the point where the AI understands how to do AI theory, the same way that human researchers try to do AI theory. An AI, to swallow its own optimization chain, must not just be able to rewrite its own source code;”

Anyway, my problem with your speculation about hard takeoff is that you seem to do the same conceptual mistake that you so dislike about Cyc—you seem to thing that AI will be mostly “written in the code”.

I suspect it is very likely that the true working AI code will be relatively small and already pretty well optimized. The “mind” itself will be created from it by some self-learning process (my favorite scenario involves weak AI as initial “tutor”) and in fact will be mostly consist of vast amount of classification coeficients and connections or something like that (think bayesian or neural networks).

While it probably will be in AI power to optimize its “primal algorithm”, gains there will be limited (it will be pretty well optimized by humans anyway). The ability to reorganize its “thinking network” might be severely low. Same as with human—we nearly understand how single neuron work, but are far from understanding the whole network. Also, with any further possible self-improvement, the complexity grows further and it is quite reasonable to predict this complexity will grow faster than AI ability to understand it.

I think it all boils down to very simple showstopper—considering you are building perfect simulation, how many atoms you need to simulate a atom? (BTW, this is also showstopper for “nested virtual reality” idea).

Note however that this whole argument is not really mutually exclusive with hard takeoff. AI still can build next generation AI that is better. But “self” part might not work. (BTW, interesting part is that “parent” AI might then face the same dilemma with descendant’s friendliness ;)

I also thing that in all your “foom” posts, you understimate empirical form of knowledge. It sounds like you expect AI to just sit down in the cellar and think, without much inputs and actions, then invent the theory of everything and take over the world.

That is not going to happen at least for the same reason why the endless chain of nested VRs is unlikely.

luzr Nov 29, 2008, 10:57 AM
4 points
on: Thanksgiving Prayer
Abigail:

“Religion has great benefits for a society and for individuals, even if there is no creator God.”

And these are?

Well, where I live, there is about 70% atheists and 30% believers. I have a couple of believers as friends. From my experience, no, they are not better people than we are.

Grant:

“I’m not sure if this has been discussed here before, but how isn’t atheism as religion? It has to be accepted on faith, because we can’t prove there isn’t a magical space god that created everything. I happen to have more faith in atheism than Christianity, but ultimately its still faith.”

I also believe there isn’t a 1000km wide pink doughnut orbiting Alpha Centauri. Is that a religion too?

BTW, “magical space god that created everything”. How was HE created?

All that said, universe, its beginning, human mind and consciousness and self-awarnes, these all are very fascinating mysteries and “God” is a very tempting and easy answer. But it is pretty much likely wrong.

luzr Nov 28, 2008, 12:47 PM
2 points
on: Thanksgiving Prayer
Abigail:

“Religion might be more understandable to atheists”

What makes you think religion is not understandable to atheists?

As for your explanation, yeah, nice, but do you really think this is what most christians really believe in? (You can try—find some and ask if he thinks that God equals to Society).

Also, what is a use for such abstraction?

luzr Nov 28, 2008, 12:41 PM
−6 points
on: Thanksgiving Prayer
“I’ve seen attempted atheists lapse back into their childhood religions, and it’s a horrible thing to see, a train wreck that you should be able to prevent but can’t.”

But what if my childhood “religion” was atheism?