You keep suggesting that there’s no reason to worry about how to constrain the behavior of computer programs, because computer programs can only do what they are told to do.
No, I just keep saying that we don’t need to program them to “like rewards and fear punishments” and train them like we’d train dogs.
I agree completely that, in doing so, it is merely doing what I told it to do: I’m the one who wrote that stupid bug, it didn’t magically come out of nowhere, the program doesn’t have any mysterious kind of free will or anything. It’s just a program I wrote. But I don’t see why that should be particularly reassuring.
Oh no, it’s not. I have several posts on my blog detailing how bugs like that could actually turn a whole machine army against us and turn Terminator into a reality rather than a cheesy robots-take-over-the-world-for-shits-and-giggles flick.
… and yet we have no significant difficulty equating a running program with its source code.
But the source code isn’t like DNA in an organism. Source code covers so much more ground than that. Imagine having an absolute blueprint of how every cell cluster in your body will react to any stimuli through your entire life and every process it will undertake from now until your death, including how it will age. That would be source code. Your DNA is not ever nearly that complete. It’s more like a list of suggestions and blueprints for raw materials.
No, I just keep saying that we don’t need to program them to “like rewards and fear punishments” and train them like we’d train dogs.
(shrug) OK, fair enough.
I agree with you that reward/punishment conditioning of software is a goofy idea.
I was reading your comment here to indicate that we can constrain the behavior of human-level AGIs by just putting appropriate constraints in the code. (“You don’t want the machine to do something? Put in a boundry. [..] with a machine, you can just tell it not to do that.”)
I think that idea is importantly wrong, which is why I was responding to it, but if you don’t actually believe that then we apparently don’t have a disagreement.
Re: source code… if we’re talking about code that is capable of itself generating executable code as output in response to situations that arise (which seems implicit in the idea of a human-level AGI, given that humans are capable of generating executable code), it isn’t at all clear to me that its original source code comprises in any kind of useful way an absolute blueprint for how every part of it will react to any stimuli.
Again, sure, I’m not positing magic: whatever it does, it does because of the interaction between its source code and the environment in which it runs, there’s no kind of magic third factor. So, sure, given the source code and an accurate specification of its environment (including its entire relevant history), I can in principle determine precisely what it will do. Absolutely agreed. (Of course, in practice that might be so complicated that I can’t actually do it, but you aren’t claiming otherwise.)
If you don’t think the same is true of humans, then we disagree about humans, but I think that’s incidental.
… if we’re talking about code that is capable of itself generating executable code as output in response to situations that arise
Again, it really shouldn’t be doing that. It should have the capacity to learn new skills and build new neural networks to do so. That doesn’t require new code, it just requires a routine to initialize a new set of ANN objects at runtime.
If it somehow follows from that that there’s an absolute blueprint in it for how every part of it will react to any stimuli in a way that is categorically different from how human genetics specify how humans will respond to any environment, then I don’t follow the connection… sorry. I have only an interested layman’s understanding of ANNs.
No, I just keep saying that we don’t need to program them to “like rewards and fear punishments” and train them like we’d train dogs.
Oh no, it’s not. I have several posts on my blog detailing how bugs like that could actually turn a whole machine army against us and turn Terminator into a reality rather than a cheesy robots-take-over-the-world-for-shits-and-giggles flick.
But the source code isn’t like DNA in an organism. Source code covers so much more ground than that. Imagine having an absolute blueprint of how every cell cluster in your body will react to any stimuli through your entire life and every process it will undertake from now until your death, including how it will age. That would be source code. Your DNA is not ever nearly that complete. It’s more like a list of suggestions and blueprints for raw materials.
(shrug) OK, fair enough.
I agree with you that reward/punishment conditioning of software is a goofy idea.
I was reading your comment here to indicate that we can constrain the behavior of human-level AGIs by just putting appropriate constraints in the code. (“You don’t want the machine to do something? Put in a boundry. [..] with a machine, you can just tell it not to do that.”)
I think that idea is importantly wrong, which is why I was responding to it, but if you don’t actually believe that then we apparently don’t have a disagreement.
Re: source code… if we’re talking about code that is capable of itself generating executable code as output in response to situations that arise (which seems implicit in the idea of a human-level AGI, given that humans are capable of generating executable code), it isn’t at all clear to me that its original source code comprises in any kind of useful way an absolute blueprint for how every part of it will react to any stimuli.
Again, sure, I’m not positing magic: whatever it does, it does because of the interaction between its source code and the environment in which it runs, there’s no kind of magic third factor. So, sure, given the source code and an accurate specification of its environment (including its entire relevant history), I can in principle determine precisely what it will do. Absolutely agreed. (Of course, in practice that might be so complicated that I can’t actually do it, but you aren’t claiming otherwise.)
If you don’t think the same is true of humans, then we disagree about humans, but I think that’s incidental.
Again, it really shouldn’t be doing that. It should have the capacity to learn new skills and build new neural networks to do so. That doesn’t require new code, it just requires a routine to initialize a new set of ANN objects at runtime.
If it somehow follows from that that there’s an absolute blueprint in it for how every part of it will react to any stimuli in a way that is categorically different from how human genetics specify how humans will respond to any environment, then I don’t follow the connection… sorry. I have only an interested layman’s understanding of ANNs.