That no one has come close to providing a successful approach on how to do this, and that each proposal fails in very similar ways. There is no ontologically fundamental difference between an acceptable and an unacceptable query, and drawing a practical boundary has so far proved to be impossible.
If you have a solution to that, then I advise you analyse it carefully, and then put it as a top level post. Since it would half-solve the whole FAI problem, it would garner great interest.
You’ve adopted Robby’s favourite fallacy: arguing of absolute difficulty as though it were relative difficulty. The hard part has got be harder than the rest of AGI. But why shout a SAI that can pass the .TT with flying colours be unable to do something a human can do?
The thing in question isn’t collecting barbie dolls, it’s understanding correctly. An .AI that sits at the end of a series of self improvements has got to be pretty good at that.
You can say it will have only instrumental rationality, and will start getting things wrong in order to pursue its ultimate goal of word domination, and I can say that if instrumental rationality is dangerous, don’t bulld it that way.
No, it’s preferences the problem, not understanding. Why would an AI sitting at the end of a series of self improvements choose to interpret ambiguous coding in the way we prefer?
I can say that if instrumental rationality is dangerous, don’t bulld it that way.
How do you propose to build an AI without instrumental rationality or preventing that from developing? And how do you propose to convince AI designers to go down that route?
That no one has come close to providing a successful approach on how to do this, and that each proposal fails in very similar ways. There is no ontologically fundamental difference between an acceptable and an unacceptable query, and drawing a practical boundary has so far proved to be impossible.
If you have a solution to that, then I advise you analyse it carefully, and then put it as a top level post. Since it would half-solve the whole FAI problem, it would garner great interest.
Nobody knows how to build AGI either.
You’ve adopted Robby’s favourite fallacy: arguing of absolute difficulty as though it were relative difficulty. The hard part has got be harder than the rest of AGI. But why shout a SAI that can pass the .TT with flying colours be unable to do something a human can do?
Orthogonality thesis: building an AGI is a completely different challenge from building an AGI with an acceptable motivation system.
It is not a question of ability, but of preferences. Why should an AI that can pass the TT want something that a human wants?
The thing in question isn’t collecting barbie dolls, it’s understanding correctly. An .AI that sits at the end of a series of self improvements has got to be pretty good at that.
You can say it will have only instrumental rationality, and will start getting things wrong in order to pursue its ultimate goal of word domination, and I can say that if instrumental rationality is dangerous, don’t bulld it that way.
No, it’s preferences the problem, not understanding. Why would an AI sitting at the end of a series of self improvements choose to interpret ambiguous coding in the way we prefer?
How do you propose to build an AI without instrumental rationality or preventing that from developing? And how do you propose to convince AI designers to go down that route?
If it has epistemic rationality as a goal, it will default to getting things right rather than wrong.
Not only nstrumental rationality = epistemic rationality.
If it has epistemic rationality as a goal, it will default to acquiring true beliefs about the world. Explain how this will make it “nice”.
See above. The question was originally about interpreting directives. You have switched to inferring morality apriori.