I expect unaligned human-level AIs to try the same thing and have much more success because optimizing code and silicon hardware is easier than optimizing flesh brains.
Seems to me that optimizing flesh brains is easier than optimizing code and silicon hardware. It’s so easy, evolution can do it despite being very dumb.
Roughly speaking the part that makes it easy is that the effects of flesh brains are additive with respect to the variables one might modify (standing genetic variation), whereas the effects of hardware and software are very nonlinear with respect to the variables one might modify (circuit connectivity(?) and code characters).
We haven’t made much progress on optimizing humans, but that’s less because optimizing humans is hard and more because humans prefer using the resources that could’ve been used for optimizing humans for self-preservation instead.
For example, if a human says “I’d like to make a similar brain as mine, but with 80% more neurons per cortical minicolumn”, there’s no way to actually do that, at least not without spending decades or centuries on basic bio-engineering research.
By contrast, if an ANN-based AGI says “I’d like to make a similar ANN as mine, but with 80% more neurons per layer”, they can actually do that experiment immediately.
First, some types of software can be largely additive wrt their variables, e.g. neural nets, that’s basically why SGD works. Second, software has lots of other huge advantages like rapid iteration times, copyability and inspectability of intermediate states.
Seems to me that optimizing flesh brains is easier than optimizing code and silicon hardware. It’s so easy, evolution can do it despite being very dumb.
Roughly speaking the part that makes it easy is that the effects of flesh brains are additive with respect to the variables one might modify (standing genetic variation), whereas the effects of hardware and software are very nonlinear with respect to the variables one might modify (circuit connectivity(?) and code characters).
We haven’t made much progress on optimizing humans, but that’s less because optimizing humans is hard and more because humans prefer using the resources that could’ve been used for optimizing humans for self-preservation instead.
Why the disagree vote?
For example, if a human says “I’d like to make a similar brain as mine, but with 80% more neurons per cortical minicolumn”, there’s no way to actually do that, at least not without spending decades or centuries on basic bio-engineering research.
By contrast, if an ANN-based AGI says “I’d like to make a similar ANN as mine, but with 80% more neurons per layer”, they can actually do that experiment immediately.
First, some types of software can be largely additive wrt their variables, e.g. neural nets, that’s basically why SGD works. Second, software has lots of other huge advantages like rapid iteration times, copyability and inspectability of intermediate states.