So it’s impossible to directly or indirectly code in the compex thing called semantics, but possible to directly or indirectly code in the compex thing called morality?
Read the first section of the article you’re commenting on. Semantics may turn out to be a harder problem than morality, because the problem of morality may turn out to be a subset of the problem of semantics. Coding a machine to know what the word ‘Friendliness’ means (and to care about ‘Friendliness’) is just a more indirect way of coding it to be Friendly, and it’s not clear why that added indirection should make an already risky or dangerous project easy or safe. What does indirect indirect normativity get us that indirect normativity doesn’t?
Robb, at the point where Peterdjones suddenly shows up, I’m willing to say—with some reluctance—that your endless willingness to explain is being treated as a delicious free meal by trolls. Can you direct them to your blog rather than responding to them here? And we’ll try to get you some more prestigious non-troll figure to argue with—maybe Gary Drescher would be interested, he has the obvious credentials in cognitive reductionism but is (I think incorrectly) trying to derive morality from timeless decision theory.
Sure. I’m willing to respond to novel points, but at the stage where half of my responses just consist of links to the very article they’re commenting on or an already-referenced Sequence post, I agree the added noise is ceasing to be productive. Fortunately, most of this seems to already have been exorcised into my blog. :)
Agree with Eliezer. Your explanatory skill and patience are mostly wasted on the people you’ve been arguing with so far, though it may have been good practice for you. I would, however, love to see you try to talk Drescher out of trying to pull moral realism out of TDT/UDT, or try to talk Dalyrmple out of his “I’m not partisan enough to prioritize human values over the Darwinian imperative” position, or help Preston Greene persuade mainstream philosophers of “the reliabilist metatheory of rationality” (aka rationality as systematized winning).
Semantcs isn’t optional. Nothing could qualify as an AGI,let alone a super one, unless it could hack natural language. So Loosemore architectures don’t make anything harder, since semantics has to be solved anyway.
It’s a problem of sequence. The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety. Tasking the programmers to work on Semantics-in-General makes things harder if it’s a more complex or roundabout way of trying to address Indirect Normativity; most of the work on understanding what English-language sentences mean can be relegated to the SI, provided we’ve already made it safe to make an SI at all.
It’s worth noting that using an AI’s semantic understanding of ethics to modify it’s motivational system is so unghostly, and unmysterious that it’s actually been done:
But that doesn’t prove much, because it was never—not in 2023, not in 2013 -- the case that that kind of self-correction was necessarily an appeal to the supernatural. Using one part of a software system to modify another is not magic!
The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety.
We have AIs with very good semantic understanding that haven’t killed us, and we are working on safety.
Read the first section of the article you’re commenting on. Semantics may turn out to be a harder problem than morality, because the problem of morality may turn out to be a subset of the problem of semantics. Coding a machine to know what the word ‘Friendliness’ means (and to care about ‘Friendliness’) is just a more indirect way of coding it to be Friendly, and it’s not clear why that added indirection should make an already risky or dangerous project easy or safe. What does indirect indirect normativity get us that indirect normativity doesn’t?
Robb, at the point where Peterdjones suddenly shows up, I’m willing to say—with some reluctance—that your endless willingness to explain is being treated as a delicious free meal by trolls. Can you direct them to your blog rather than responding to them here? And we’ll try to get you some more prestigious non-troll figure to argue with—maybe Gary Drescher would be interested, he has the obvious credentials in cognitive reductionism but is (I think incorrectly) trying to derive morality from timeless decision theory.
Sure. I’m willing to respond to novel points, but at the stage where half of my responses just consist of links to the very article they’re commenting on or an already-referenced Sequence post, I agree the added noise is ceasing to be productive. Fortunately, most of this seems to already have been exorcised into my blog. :)
Agree with Eliezer. Your explanatory skill and patience are mostly wasted on the people you’ve been arguing with so far, though it may have been good practice for you. I would, however, love to see you try to talk Drescher out of trying to pull moral realism out of TDT/UDT, or try to talk Dalyrmple out of his “I’m not partisan enough to prioritize human values over the Darwinian imperative” position, or help Preston Greene persuade mainstream philosophers of “the reliabilist metatheory of rationality” (aka rationality as systematized winning).
Semantcs isn’t optional. Nothing could qualify as an AGI,let alone a super one, unless it could hack natural language. So Loosemore architectures don’t make anything harder, since semantics has to be solved anyway.
It’s a problem of sequence. The superintelligence will be able to solve Semantics-in-General, but at that point if it isn’t already safe it will be rather late to start working on safety. Tasking the programmers to work on Semantics-in-General makes things harder if it’s a more complex or roundabout way of trying to address Indirect Normativity; most of the work on understanding what English-language sentences mean can be relegated to the SI, provided we’ve already made it safe to make an SI at all.
It’s worth noting that using an AI’s semantic understanding of ethics to modify it’s motivational system is so unghostly, and unmysterious that it’s actually been done:
https://astralcodexten.substack.com/p/constitutional-ai-rlhf-on-steroids
But that doesn’t prove much, because it was never—not in 2023, not in 2013 -- the case that that kind of self-correction was necessarily an appeal to the supernatural. Using one part of a software system to modify another is not magic!
We have AIs with very good semantic understanding that haven’t killed us, and we are working on safety.
Then solve semantics in a seed.