Ah, I see your point. So the less misleading thing to say might be something roughly like: “We don’t yet know how to find or reason about our values, but we have notions of where we might start, and we can expect that whatever methods do end up making headway are going to have to be non-stupid in at least as many ways as our existing methods of solving hard problems are non-stupid.”
Ah, I see your point. So the less misleading thing to say might be something roughly like: “We don’t yet know how to find or reason about our values, but we have notions of where we might start, and we can expect that whatever methods do end up making headway are going to have to be non-stupid in at least as many ways as our existing methods of solving hard problems are non-stupid.”