I don’t at all think that’s off the table temporarily! I don’t trust that it’ll stay on the table—if the adult has malicious intent, knowing what the child means isn’t enough; it seems hard to know when it’ll stop being viable without more progress. (for example, I doubt it’ll ever be a good idea to do that with an OpenAI model, they seem highly deceptively misaligned to most of their users. seems possible for it to be a good idea with Claude.) But the challenge is how to certify that the math does in fact say the right thing to durably point to the ontology in which we want to preserve good things; at some point we have to actually understand some sort of specification that constrains what the stuff we don’t understand is doing to be what it seems to say in natural language.
I don’t at all think that’s off the table temporarily! I don’t trust that it’ll stay on the table—if the adult has malicious intent, knowing what the child means isn’t enough; it seems hard to know when it’ll stop being viable without more progress. (for example, I doubt it’ll ever be a good idea to do that with an OpenAI model, they seem highly deceptively misaligned to most of their users. seems possible for it to be a good idea with Claude.) But the challenge is how to certify that the math does in fact say the right thing to durably point to the ontology in which we want to preserve good things; at some point we have to actually understand some sort of specification that constrains what the stuff we don’t understand is doing to be what it seems to say in natural language.