[to build a Germand-Shepherd-level AI you need to] create something that will learn and improve itself up to the corresponding level and then top out there somehow[.]
I’m not arguing yet, in case I’m missing something, but why do you think that something stupider than a German Shepherd would be better at improving itself up to GSD levels (and stop right there) than a human would be at doing the same job (i.e., improving the potential AGSD, not the human itself).
Or rather, why does it seem like you think it’s obvious? (Again, I’m not arguing, it just sounds counterintuitive and I’m curious what your intuition is.) It sounds a bit like you’re saying something like:
“Hey, I can’t tell, just by looking at my brain-damaged dog, how to built a non-brain-damaged dog. Also, repairing its brain is too hard (many dog experts tried and all failed). I think it’d be easier to make a brain-damaged dog that will fix its own brain damage.”
(Note that AGI in general does not fall under this analogy. Foom scenarios assume the seed is at least human-level, at least at the task of improving its intelligence. The whole premise of fooming is based on that initial advantage. Also note, I’m not saying it’s obviously impossible to make a super-idiot-savant AI that’s stupider than a GSD in general but really good at improving itself, just that’s it goes really hard against my intuition, and I’m curious why yours doesn’t. Don’t feel like you have to justify your intuition to me, but it would be nice to describe it in more detail.)
(Sorry for belated replies, I’ve been completely off LW for a few months and am only now going through my inbox)
I’m not arguing yet, in case I’m missing something, but why do you think that something stupider than a German Shepherd would be better at improving itself up to GSD levels (and stop right there) than a human would be at doing the same job (i.e., improving the potential AGSD, not the human itself).
This is not what I think, or at least not what I expressed. My thoughts are similar, but elaboration later; first, this was an option in parallel with the option where a human designs a complete AGSD and then turns it on, and with the option where a bunch of humans design sub-AGSD iterations up until the point where they obtain a final AGSD.
As for elaboration, I do think it’s easier to build a so-called super-idiot-savant sub-GSD-general-intelligence, post-human-self-improvement AI than building any sort of “out-of-the-box” general intelligence. I don’t currently recall my reasons, since my mind is set in a different mode, but the absurd and extreme case is that of having a human child. A human child is stupider than a GSD, but learns better than adult humans. It is also much simpler to do than any sort of AI programming. ;) But I only say this last in jest, and it isn’t particularly relevant to the discussion.
I’m not arguing yet, in case I’m missing something, but why do you think that something stupider than a German Shepherd would be better at improving itself up to GSD levels (and stop right there) than a human would be at doing the same job (i.e., improving the potential AGSD, not the human itself).
Or rather, why does it seem like you think it’s obvious? (Again, I’m not arguing, it just sounds counterintuitive and I’m curious what your intuition is.) It sounds a bit like you’re saying something like:
“Hey, I can’t tell, just by looking at my brain-damaged dog, how to built a non-brain-damaged dog. Also, repairing its brain is too hard (many dog experts tried and all failed). I think it’d be easier to make a brain-damaged dog that will fix its own brain damage.”
(Note that AGI in general does not fall under this analogy. Foom scenarios assume the seed is at least human-level, at least at the task of improving its intelligence. The whole premise of fooming is based on that initial advantage. Also note, I’m not saying it’s obviously impossible to make a super-idiot-savant AI that’s stupider than a GSD in general but really good at improving itself, just that’s it goes really hard against my intuition, and I’m curious why yours doesn’t. Don’t feel like you have to justify your intuition to me, but it would be nice to describe it in more detail.)
(Sorry for belated replies, I’ve been completely off LW for a few months and am only now going through my inbox)
This is not what I think, or at least not what I expressed. My thoughts are similar, but elaboration later; first, this was an option in parallel with the option where a human designs a complete AGSD and then turns it on, and with the option where a bunch of humans design sub-AGSD iterations up until the point where they obtain a final AGSD.
As for elaboration, I do think it’s easier to build a so-called super-idiot-savant sub-GSD-general-intelligence, post-human-self-improvement AI than building any sort of “out-of-the-box” general intelligence. I don’t currently recall my reasons, since my mind is set in a different mode, but the absurd and extreme case is that of having a human child. A human child is stupider than a GSD, but learns better than adult humans. It is also much simpler to do than any sort of AI programming. ;) But I only say this last in jest, and it isn’t particularly relevant to the discussion.