Eliezer, you must have lowered your intellectual level because these days I can understand your posts again.
You talk about the friendliness problem as if it can be solved separately from the problem of building an AGI, and in anticipation of that event. I mean that you want to delay the creation of an AGI until friendliness is fully understood. Is that right?
Suppose that we had needed to build jet-planes without ever passing through the stage of propeller-based planes, or if we had needed to build modern computers without first building calculators, 8-bit machines etc. Do you think that would be possible? (It’s not a rhetorical question… I really don’t know the answer).
It seems to me that if we ever build an AGI, there will be many mistakes made along the way. Any traps waiting for us will certainly be triggered.
Perhaps this will all seem clearer when we all have 140 IQ’s. Get to work, Razib! :)
Interesting. And to use a more poignant example: Could we have invented fusion power if we had never invented the atomic bomb?
It does seem plausible to say that technological growth is necessarily (or if not necessarily, at least typically) incremental; it doesn’t proceed in huge leaps and breakthroughs, but in slow accumulations of tinkering.
Of more concern is the equally plausible inference that tinkering requires making mistakes, and as our technology improves, the mistakes will have larger and larger stakes on which to miss.
Eliezer, you must have lowered your intellectual level because these days I can understand your posts again.
You talk about the friendliness problem as if it can be solved separately from the problem of building an AGI, and in anticipation of that event. I mean that you want to delay the creation of an AGI until friendliness is fully understood. Is that right?
Suppose that we had needed to build jet-planes without ever passing through the stage of propeller-based planes, or if we had needed to build modern computers without first building calculators, 8-bit machines etc. Do you think that would be possible? (It’s not a rhetorical question… I really don’t know the answer).
It seems to me that if we ever build an AGI, there will be many mistakes made along the way. Any traps waiting for us will certainly be triggered.
Perhaps this will all seem clearer when we all have 140 IQ’s. Get to work, Razib! :)
Interesting. And to use a more poignant example: Could we have invented fusion power if we had never invented the atomic bomb?
It does seem plausible to say that technological growth is necessarily (or if not necessarily, at least typically) incremental; it doesn’t proceed in huge leaps and breakthroughs, but in slow accumulations of tinkering.
Of more concern is the equally plausible inference that tinkering requires making mistakes, and as our technology improves, the mistakes will have larger and larger stakes on which to miss.