I expect unaligned human-level AIs to try the same thing and have much more success because optimizing code and silicon hardware is easier than optimizing flesh brains.
I agree that human-level AIs will definitely try the same thing, but it’s not obvious to me that it will actually be much easier for them. Current machine learning techniques produce models that are hard to optimize for basically the same reasons that brains are; AIs will be easier to optimize for various reasons but I don’t think it will be nearly as extreme as this sentence makes it sound.
I naively expect the option of “take whatever model constitutes your mind and run it on faster hardware and/or duplicate it” should be relatively easy and likely to lead to fairly extreme gains.
Ruby isn’t saying that computers have faster clock speeds than biological brains (which is definitely true), he’s claiming something like “after we have human-level AI, AIs will be able to get rapidly more powerful by running on faster hardware”; the speed increase is relative to some other computers, so the speed difference between brains and computers isn’t relevant.
Also, running faster and duplicating yourself keeps the model human-level in an important sense. A lot of threat models run through the model doing things that humans can’t understand even given a lot of time, and so those threat models require something stronger than just this.
I think clever duplication of human intelligence is plenty sufficient for general superhuman capacity in the important sense (wherein I mean something like ‘it has capacities such that would be extincion causing if (it believes) minimizing its loss function is achieved by turning off humanity (which could turn it off/ start other (proto-)agis)’).
for one, I don’t think humanity is that robust in the status quo, and 2, a team of internally aligned (because copies) human level intelligence capable of graduate level biology seems plenty existentially scary.
I agree that human-level AIs will definitely try the same thing, but it’s not obvious to me that it will actually be much easier for them. Current machine learning techniques produce models that are hard to optimize for basically the same reasons that brains are; AIs will be easier to optimize for various reasons but I don’t think it will be nearly as extreme as this sentence makes it sound.
I naively expect the option of “take whatever model constitutes your mind and run it on faster hardware and/or duplicate it” should be relatively easy and likely to lead to fairly extreme gains.
I agree we can duplicate models once we’ve trained them, this seems like the strongest argument here.
What do you mean by “run on faster hardware”? Faster than what?
Faster than biological brains, by 6 orders of magnitude.
Ruby isn’t saying that computers have faster clock speeds than biological brains (which is definitely true), he’s claiming something like “after we have human-level AI, AIs will be able to get rapidly more powerful by running on faster hardware”; the speed increase is relative to some other computers, so the speed difference between brains and computers isn’t relevant.
Also, running faster and duplicating yourself keeps the model human-level in an important sense. A lot of threat models run through the model doing things that humans can’t understand even given a lot of time, and so those threat models require something stronger than just this.
I think clever duplication of human intelligence is plenty sufficient for general superhuman capacity in the important sense (wherein I mean something like ‘it has capacities such that would be extincion causing if (it believes) minimizing its loss function is achieved by turning off humanity (which could turn it off/ start other (proto-)agis)’).
for one, I don’t think humanity is that robust in the status quo, and 2, a team of internally aligned (because copies) human level intelligence capable of graduate level biology seems plenty existentially scary.