IMO, a very good response, which Eliezer doesn’t seem to be interested in making as far as I can tell, is that we should not be making the analogy natural selection <--> gradient descent, but rather, human brain learning algorithm <--> gradient descent ; natural selection <--> us trying to build AI.
So here, the striking thing is that evolution failed to solve the alignment problem for humans. I.e. we have a prior example of strongish general intelligence being created, but no prior examples of strongish general intelligence being aligned. Evolution was strong enough to do one but not the other. It’s not hopeless, because we should generally consider ourselves smarter than evolution, but on the other hand, evolution has had a very long time to work and it does frequently manage things that we humans have not been able to replicate. And also, it provides a small amount of evidence against “the problem will be solved with minor tweaks to existing algorithms” since generally minor tweaks are easier for evolution to find than ideas that require many changes at once.
Yeah you can kind of stop at “we are already doing natural selection.” The devs give us random variation. The conferences and the market give us selection. The population is large, the mutation rate is high, the competition is fierce, and replicating costs $0.25 + 10 minutes.
IMO, a very good response, which Eliezer doesn’t seem to be interested in making as far as I can tell, is that we should not be making the analogy
natural selection <--> gradient descent, but rather,human brain learning algorithm <--> gradient descent ; natural selection <--> us trying to build AI.So here, the striking thing is that evolution failed to solve the alignment problem for humans. I.e. we have a prior example of strongish general intelligence being created, but no prior examples of strongish general intelligence being aligned. Evolution was strong enough to do one but not the other. It’s not hopeless, because we should generally consider ourselves smarter than evolution, but on the other hand, evolution has had a very long time to work and it does frequently manage things that we humans have not been able to replicate. And also, it provides a small amount of evidence against “the problem will be solved with minor tweaks to existing algorithms” since generally minor tweaks are easier for evolution to find than ideas that require many changes at once.
Yeah you can kind of stop at “we are already doing natural selection.” The devs give us random variation. The conferences and the market give us selection. The population is large, the mutation rate is high, the competition is fierce, and replicating costs $0.25 + 10 minutes.