My current position is that I don’t know how to make an AGI, don’t know how to program it and don’t know how to patch it. All of these appear to be very hard and very important issues. We know that GI is possible, as you rightfully note—i.e. us. We know that “programming” and “patching” beings like us is doable to a certain extent, which can certainly give us hope that if we have an AGI like us, then it could be taught to be nice. But all of this assumes that we can make something in our image.
The point of the detached lever story is that the lever only works if it’s connected to exactly the right components, most of which are very complicated. So in the case of AGI, your approach seems sensible, but only if we can recreate the whole complicated mess that is a human brain—otherwise it’ll probably turn out that we missed some small but crucial part which results in the AGI being more like a chimp that a human. It’s closest in the sense that it’s easy to describe in words (i.e. “make a simulated human brain”), but each of those words hides masses and masses of finicky details. This seems to be the main point of contention here. I highly recommend the other articles in that sequence, if you haven’t read them yet. Or even better—read the whole sequences if you have a spare 6 months :D They explain this a lot better than I can, though they also take a loooot more words to do so.
I agree with most of the rest of what you say though: programming can certainly turn out to be harder than creation (which is one of the main causes of concern), logging and checking each thought will exclude or prolongs indefinitely our ability to create AGI (so won’t work, as someone else will skip this part), and a risk must be taken at some point, otherwise why even bother?
My current position is that I don’t know how to make an AGI, don’t know how to program it and don’t know how to patch it. All of these appear to be very hard and very important issues. We know that GI is possible, as you rightfully note—i.e. us. We know that “programming” and “patching” beings like us is doable to a certain extent, which can certainly give us hope that if we have an AGI like us, then it could be taught to be nice. But all of this assumes that we can make something in our image.
The point of the detached lever story is that the lever only works if it’s connected to exactly the right components, most of which are very complicated. So in the case of AGI, your approach seems sensible, but only if we can recreate the whole complicated mess that is a human brain—otherwise it’ll probably turn out that we missed some small but crucial part which results in the AGI being more like a chimp that a human. It’s closest in the sense that it’s easy to describe in words (i.e. “make a simulated human brain”), but each of those words hides masses and masses of finicky details. This seems to be the main point of contention here. I highly recommend the other articles in that sequence, if you haven’t read them yet. Or even better—read the whole sequences if you have a spare 6 months :D They explain this a lot better than I can, though they also take a loooot more words to do so.
I agree with most of the rest of what you say though: programming can certainly turn out to be harder than creation (which is one of the main causes of concern), logging and checking each thought will exclude or prolongs indefinitely our ability to create AGI (so won’t work, as someone else will skip this part), and a risk must be taken at some point, otherwise why even bother?