Now, why exactly should we expect the superintelligence that grows out of the seed to value what we really mean by ‘pleasure’, when all we programmed it to do was X, our probably-failed attempt at summarizing our values?
Maybe we didn’t do it ithat way. Maybe we did it Loosemore’s way, where you code in the high-level sentence, and let the AI figure it out. Maybe that would avoid the problem. Maybe Loosemore has solved FAi much more straightforwardly than EY.
Maybe we told it to. Maybe we gave it the low-level expansion of “happy” that we or our seed AI came up with together with an instruction that it is meant to capture the meaning of the high-level statement, and that the HL statement is the Prime Directive, and that if the AI judges that the expansion is wrong, then it should reject the expansion.
Maybe the AI will value getting things right because it is rational.
“Maybe we gave it the low-level expansion of ‘happy’ that we or our seed AI came up with ‘together with’ an instruction that it is meant to capture the meaning of the high-level statement”
If the AI is too dumb to understand ‘make us happy’, then why should we expect it to be smart enough to understand ‘figure out how to correctly understand “make us happy”, and then follow that instruction’? We have to actually code ‘correctly understand’ into the AI. Otherwise, even when it does have the right understanding, that understanding won’t be linked to its utility function.
“Maybe the AI will value getting things right because it is rational.”
So it’s impossible to directly or indirectly code in the compex thing called semantics, but possible to directly or indirectly code in the compex thing called morality? What? What is your point? You keep talking as if I am suggesting there is someting that can be had for free, without coding. I never even remotely said that.
If the AI is too dumb to understand ‘make us happy’, then why should we expect it to be smart enough to understand ‘figure out how to correctly understand “make us happy”, and then follow that instruction’? We have to actually code ‘correctly understand’ into the AI. Otherwise, even when it does have the right understanding, that understanding won’t be linked to its utility function.
I know. A Loosemore architecture AI has to treat its directives as directives. I never disputed that. But coding “follow these plain English instructions” isn’t obviously harder or more fragile than coding “follow <>”. And it isn’t trivial, and I didn’t say it was.
So it’s impossible to directly or indirectly code in the compex thing called semantics, but possible to directly or indirectly code in the compex thing called morality?
Read the first section of the article you’re commenting on. Semantics may turn out to be a harder problem than morality, because the problem of morality may turn out to be a subset of the problem of semantics. Coding a machine to know what the word ‘Friendliness’ means (and to care about ‘Friendliness’) is just a more indirect way of coding it to be Friendly, and it’s not clear why that added indirection should make an already risky or dangerous project easy or safe. What does indirect indirect normativity get us that indirect normativity doesn’t?
Robb, at the point where Peterdjones suddenly shows up, I’m willing to say—with some reluctance—that your endless willingness to explain is being treated as a delicious free meal by trolls. Can you direct them to your blog rather than responding to them here? And we’ll try to get you some more prestigious non-troll figure to argue with—maybe Gary Drescher would be interested, he has the obvious credentials in cognitive reductionism but is (I think incorrectly) trying to derive morality from timeless decision theory.
Sure. I’m willing to respond to novel points, but at the stage where half of my responses just consist of links to the very article they’re commenting on or an already-referenced Sequence post, I agree the added noise is ceasing to be productive. Fortunately, most of this seems to already have been exorcised into my blog. :)
Agree with Eliezer. Your explanatory skill and patience are mostly wasted on the people you’ve been arguing with so far, though it may have been good practice for you. I would, however, love to see you try to talk Drescher out of trying to pull moral realism out of TDT/UDT, or try to talk Dalyrmple out of his “I’m not partisan enough to prioritize human values over the Darwinian imperative” position, or help Preston Greene persuade mainstream philosophers of “the reliabilist metatheory of rationality” (aka rationality as systematized winning).
Semantcs isn’t optional. Nothing could qualify as an AGI,let alone a super one, unless it could hack natural language. So Loosemore architectures don’t make anything harder, since semantics has to be solved anyway.
It’s a problem of sequence. The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety. Tasking the programmers to work on Semantics-in-General makes things harder if it’s a more complex or roundabout way of trying to address Indirect Normativity; most of the work on understanding what English-language sentences mean can be relegated to the SI, provided we’ve already made it safe to make an SI at all.
It’s worth noting that using an AI’s semantic understanding of ethics to modify it’s motivational system is so unghostly, and unmysterious that it’s actually been done:
But that doesn’t prove much, because it was never—not in 2023, not in 2013 -- the case that that kind of self-correction was necessarily an appeal to the supernatural. Using one part of a software system to modify another is not magic!
The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety.
We have AIs with very good semantic understanding that haven’t killed us, and we are working on safety.
Maybe we didn’t do it ithat way. Maybe we did it Loosemore’s way, where you code in the high-level sentence, and let the AI figure it out. Maybe that would avoid the problem. Maybe Loosemore has solved FAi much more straightforwardly than EY.
Maybe we told it to. Maybe we gave it the low-level expansion of “happy” that we or our seed AI came up with together with an instruction that it is meant to capture the meaning of the high-level statement, and that the HL statement is the Prime Directive, and that if the AI judges that the expansion is wrong, then it should reject the expansion.
Maybe the AI will value getting things right because it is rational.
http://lesswrong.com/lw/rf/ghosts_in_the_machine/
If the AI is too dumb to understand ‘make us happy’, then why should we expect it to be smart enough to understand ‘figure out how to correctly understand “make us happy”, and then follow that instruction’? We have to actually code ‘correctly understand’ into the AI. Otherwise, even when it does have the right understanding, that understanding won’t be linked to its utility function.
http://lesswrong.com/lw/igf/the_genie_knows_but_doesnt_care/
So it’s impossible to directly or indirectly code in the compex thing called semantics, but possible to directly or indirectly code in the compex thing called morality? What? What is your point? You keep talking as if I am suggesting there is someting that can be had for free, without coding. I never even remotely said that.
I know. A Loosemore architecture AI has to treat its directives as directives. I never disputed that. But coding “follow these plain English instructions” isn’t obviously harder or more fragile than coding “follow <>”. And it isn’t trivial, and I didn’t say it was.
Read the first section of the article you’re commenting on. Semantics may turn out to be a harder problem than morality, because the problem of morality may turn out to be a subset of the problem of semantics. Coding a machine to know what the word ‘Friendliness’ means (and to care about ‘Friendliness’) is just a more indirect way of coding it to be Friendly, and it’s not clear why that added indirection should make an already risky or dangerous project easy or safe. What does indirect indirect normativity get us that indirect normativity doesn’t?
Robb, at the point where Peterdjones suddenly shows up, I’m willing to say—with some reluctance—that your endless willingness to explain is being treated as a delicious free meal by trolls. Can you direct them to your blog rather than responding to them here? And we’ll try to get you some more prestigious non-troll figure to argue with—maybe Gary Drescher would be interested, he has the obvious credentials in cognitive reductionism but is (I think incorrectly) trying to derive morality from timeless decision theory.
Sure. I’m willing to respond to novel points, but at the stage where half of my responses just consist of links to the very article they’re commenting on or an already-referenced Sequence post, I agree the added noise is ceasing to be productive. Fortunately, most of this seems to already have been exorcised into my blog. :)
Agree with Eliezer. Your explanatory skill and patience are mostly wasted on the people you’ve been arguing with so far, though it may have been good practice for you. I would, however, love to see you try to talk Drescher out of trying to pull moral realism out of TDT/UDT, or try to talk Dalyrmple out of his “I’m not partisan enough to prioritize human values over the Darwinian imperative” position, or help Preston Greene persuade mainstream philosophers of “the reliabilist metatheory of rationality” (aka rationality as systematized winning).
Semantcs isn’t optional. Nothing could qualify as an AGI,let alone a super one, unless it could hack natural language. So Loosemore architectures don’t make anything harder, since semantics has to be solved anyway.
It’s a problem of sequence. The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety. Tasking the programmers to work on Semantics-in-General makes things harder if it’s a more complex or roundabout way of trying to address Indirect Normativity; most of the work on understanding what English-language sentences mean can be relegated to the SI, provided we’ve already made it safe to make an SI at all.
It’s worth noting that using an AI’s semantic understanding of ethics to modify it’s motivational system is so unghostly, and unmysterious that it’s actually been done:
https://astralcodexten.substack.com/p/constitutional-ai-rlhf-on-steroids
But that doesn’t prove much, because it was never—not in 2023, not in 2013 -- the case that that kind of self-correction was necessarily an appeal to the supernatural. Using one part of a software system to modify another is not magic!
We have AIs with very good semantic understanding that haven’t killed us, and we are working on safety.
Then solve semantics in a seed.
PeterDJones, if you wish to converse further with RobbBB, I ask that you do so on RobbBB’s blog rather than here.