Hm, that is a better point, it seems then most of my objections are just to the wording. Most intelligent people are also shy etc. and that is why they end up being math researchers instead of being Steve Jobs. If an intelligent person could edit in his mind courage, dedication, charme… that would be powerful.
But I think self-modification would be powerful even without very high IQ, 120 would already make one pretty succesful.
Or is it more IQ being necessary for efficient self-modification?
My point is, this sounds like a powerful combination, but probably not the intelligence explosion kind.
The caste stuff: really elegant steelmanning, congrats. But I think kind of missing the point, probably I explained myself wrong. Basically an IQ-meritocracy requires a market based system, exchanged based one, where what you get is very roughly proportional to what you give to others. However most of the planet is not exchange based but power based. This is where intelligence is less useful. Imagine trying to compete with Stalin for the job of being Lenin’s successor. What traits you need for it? First of all, mountains of courage, that guys is scary. Of course if you can self-edit, that is indeed extremely helpful in it… I did not factor that in. But broadly speaking, you don’t just outsmart him. Power requires other traits. And of course it can very well be that you don’t want power, you want to be a researcher… but in that situation you are forced to take orders from him so you may still want to topple the big boss or something.
Now of course if we see intelligence as simply the amount of sophistication put into self-editing, so seeing a higher intelligence as something that can self-edit e.g. charisma better than a lower intelligence… then these possibilities are indeed there. I am just saying, still no intelligence explosion, more like everything explosion, or maybe everything else but intelligence explosion. Charisma explosion, and so on… but I do agree that this combined can be very powerful.
Or is it more IQ being necessary for efficient self-modification?
Sounds like false dilemma if IQ is one of those things that can be modified. :D
To unpack the word, IQ approximately means how complex concepts can you “juggle” in your head. Without enough IQ, even if you had an easy computer interface to modify your own brain, you wouldn’t understand what exactly you should do to achieve your goals (because you wouldn’t sufficiently understand the concepts and the possible consequences of the changes). That means, you would be making those changes blindly… and you could get lucky and hit the path where your IQ increases so then you can start making reliably the right steps, or you could set yourself on a way towards some self-destructive attractor.
As a simple example, a stupid person could choose to press a button that activates their reward center… and would keep doing that until they die from starvation. Another example would be self-modification where you lose your original goals, or lose the will to further self-improve, etc. This does not have to happen in one obvious step, but could be a subtle cumulative consequence of many seemingly innocent steps. For example, a person could decide that being fit is instrumentally useful for their goals, so they would self-modify to enjoy exercise, but would make a mistake of modifying themselves too much, so now they only want to exercise all day long, and no longer care about their original goals. Then they would either stop self-modifying, or self-modify merely to exercise better.
It also depends on how complex would be the “user interface to modify your own brain”. Maybe IQ 120 would not be enough to understand it. Maybe even IQ 200 wouldn’t. You could just see a huge network of nodes, each connected to hundreds of other nodes, each one with insanely abstract description… and either give up, or start pushing random buttons and most likely hurt yourself.
So basicly the lowest necessary starting IQ is the IQ you need to self-modify to safely enough increase your IQ. This is a very simple model which assumes that if IQ N allows you to get to IQ N+1, then IQ N+1 probably allows you to get to IQ N+2. The way does not have to be this smooth; there may be a smooth increase up to some level, which then requires a radical change to overcome; or maybe the intelligence gains at each step will decrease.
Imagine trying to compete with Stalin for the job of being Lenin’s successor. What traits you need for it?
Courage, social skills, ability to understand how politics really works. You should probably start in some position in army or secret service, some place where you can start building your own network of loyal people, without being noticed by Stalin. Or maybe you should start as a crime boss in hiding, I don’t know.
To unpack the word, IQ approximately means how complex concepts can you “juggle” in your head
I think I agree with this, this is why I don’t understand how can intelligence be defined as goal-achieving ability. When I am struggling with the hardest exercises on the Raven test, what I wish I had more is not some goal-achieving power but something far simpler, something akin to short term memory. So when I wish for more intelligene, I just wish for a bit more RAM in short-term memory, so more detailed, more granular ideas can be uploaded into tumble space. No idea why should it mean goal-achieving or optimizing ability. And for AI IQ sounds like entirely hardware...
Basically an IQ-meritocracy requires a market based system, exchanged based one, where what you get is very roughly proportional to what you give to others. However most of the planet is not exchange based but power based. This is where intelligence is less useful.
Intelligence is very useful in conflicts. If you can calculate beforehand whether you will win or lose a battle you don’t have to fight the battle if you think you will lose it.
In our modern world great intelligence means the ability to hack computers. Getting information and being able to alter email message that person A sends person B. That’s power.
Getting money because you are smart enough to predict stock market movements is another way to get power.
And of course it can very well be that you don’t want power, you want to be a researcher… but in that situation you are forced to take orders from him so you may still want to topple the big boss or something.
Stalin was likely powerful but a lot of today’s political leaders are less powerful.
Peter Thiel made the point that Obama probably didn’t even knew that the US was wiretapping Angela Merkel. It’s something that the nerds in the NSA decided in their Star Trek bridge lookalike.
Hm, that is a better point, it seems then most of my objections are just to the wording. Most intelligent people are also shy etc. and that is why they end up being math researchers instead of being Steve Jobs. If an intelligent person could edit in his mind courage, dedication, charme… that would be powerful.
But I think self-modification would be powerful even without very high IQ, 120 would already make one pretty succesful.
Or is it more IQ being necessary for efficient self-modification?
My point is, this sounds like a powerful combination, but probably not the intelligence explosion kind.
The caste stuff: really elegant steelmanning, congrats. But I think kind of missing the point, probably I explained myself wrong. Basically an IQ-meritocracy requires a market based system, exchanged based one, where what you get is very roughly proportional to what you give to others. However most of the planet is not exchange based but power based. This is where intelligence is less useful. Imagine trying to compete with Stalin for the job of being Lenin’s successor. What traits you need for it? First of all, mountains of courage, that guys is scary. Of course if you can self-edit, that is indeed extremely helpful in it… I did not factor that in. But broadly speaking, you don’t just outsmart him. Power requires other traits. And of course it can very well be that you don’t want power, you want to be a researcher… but in that situation you are forced to take orders from him so you may still want to topple the big boss or something.
Now of course if we see intelligence as simply the amount of sophistication put into self-editing, so seeing a higher intelligence as something that can self-edit e.g. charisma better than a lower intelligence… then these possibilities are indeed there. I am just saying, still no intelligence explosion, more like everything explosion, or maybe everything else but intelligence explosion. Charisma explosion, and so on… but I do agree that this combined can be very powerful.
Sounds like false dilemma if IQ is one of those things that can be modified. :D
To unpack the word, IQ approximately means how complex concepts can you “juggle” in your head. Without enough IQ, even if you had an easy computer interface to modify your own brain, you wouldn’t understand what exactly you should do to achieve your goals (because you wouldn’t sufficiently understand the concepts and the possible consequences of the changes). That means, you would be making those changes blindly… and you could get lucky and hit the path where your IQ increases so then you can start making reliably the right steps, or you could set yourself on a way towards some self-destructive attractor.
As a simple example, a stupid person could choose to press a button that activates their reward center… and would keep doing that until they die from starvation. Another example would be self-modification where you lose your original goals, or lose the will to further self-improve, etc. This does not have to happen in one obvious step, but could be a subtle cumulative consequence of many seemingly innocent steps. For example, a person could decide that being fit is instrumentally useful for their goals, so they would self-modify to enjoy exercise, but would make a mistake of modifying themselves too much, so now they only want to exercise all day long, and no longer care about their original goals. Then they would either stop self-modifying, or self-modify merely to exercise better.
It also depends on how complex would be the “user interface to modify your own brain”. Maybe IQ 120 would not be enough to understand it. Maybe even IQ 200 wouldn’t. You could just see a huge network of nodes, each connected to hundreds of other nodes, each one with insanely abstract description… and either give up, or start pushing random buttons and most likely hurt yourself.
So basicly the lowest necessary starting IQ is the IQ you need to self-modify to safely enough increase your IQ. This is a very simple model which assumes that if IQ N allows you to get to IQ N+1, then IQ N+1 probably allows you to get to IQ N+2. The way does not have to be this smooth; there may be a smooth increase up to some level, which then requires a radical change to overcome; or maybe the intelligence gains at each step will decrease.
Courage, social skills, ability to understand how politics really works. You should probably start in some position in army or secret service, some place where you can start building your own network of loyal people, without being noticed by Stalin. Or maybe you should start as a crime boss in hiding, I don’t know.
I think I agree with this, this is why I don’t understand how can intelligence be defined as goal-achieving ability. When I am struggling with the hardest exercises on the Raven test, what I wish I had more is not some goal-achieving power but something far simpler, something akin to short term memory. So when I wish for more intelligene, I just wish for a bit more RAM in short-term memory, so more detailed, more granular ideas can be uploaded into tumble space. No idea why should it mean goal-achieving or optimizing ability. And for AI IQ sounds like entirely hardware...
Intelligence is very useful in conflicts. If you can calculate beforehand whether you will win or lose a battle you don’t have to fight the battle if you think you will lose it.
In our modern world great intelligence means the ability to hack computers. Getting information and being able to alter email message that person A sends person B. That’s power.
Getting money because you are smart enough to predict stock market movements is another way to get power.
Stalin was likely powerful but a lot of today’s political leaders are less powerful.
Peter Thiel made the point that Obama probably didn’t even knew that the US was wiretapping Angela Merkel. It’s something that the nerds in the NSA decided in their Star Trek bridge lookalike.