I haven’t yet thought about this in much detail, but here is what I have:
I will assume you can avoid getting “hacked” while overseeing the debate. If you don’t assume that, then it might be important whether you can differentiate between arguments that are vs aren’t relevant to the question at hand. (I suppose that it is much harder to get hacked when strictly sticking to a specific subject-matter topic. And harder yet if you are, e.g., restricted to answering in math proofs, which might be sufficient for some types of questions.)
As for the features of safe questions, I think that one axis is the potential impact of the answer and an orthogonal one is the likelihood that the answer will be undesirable/misaligned/bad.
My guess is that if you can avoid getting hacked, then the lower-impact-of-downstream-consequences questions are inherently safer (from the trivial reason of being less impactful).
But this feels like a cheating answer, and the second axis seems more interesting.
My intuition about the “how likely are we to get an aligned answer” axis is this:
There questions where I am fairly confident in our judging skills (for example, math proofs). Many of those could fall into the “definitely safe” category.
Then there is the other extreme of questions where our judgement might be very fallible—things that are too vague or that play into our biases. (For example hard philosophical questions and problems whose solutions depend on answers to such questions. E.g., I wouldn’t trust myself to be a good judge of “how should we decide on the future of the universe” or “what is the best place for me to go for a vacation”.) I imagine these are “very likely unsafe”.
And as a general principle, where there are two extremes, there often will be a continuum inbetween. Maybe “what is a reasonable way of curing cancer?” could fall here? (Being probably safe, but I wouldn’t bet all my money on it.)
I haven’t yet thought about this in much detail, but here is what I have:
I will assume you can avoid getting “hacked” while overseeing the debate. If you don’t assume that, then it might be important whether you can differentiate between arguments that are vs aren’t relevant to the question at hand. (I suppose that it is much harder to get hacked when strictly sticking to a specific subject-matter topic. And harder yet if you are, e.g., restricted to answering in math proofs, which might be sufficient for some types of questions.)
As for the features of safe questions, I think that one axis is the potential impact of the answer and an orthogonal one is the likelihood that the answer will be undesirable/misaligned/bad. My guess is that if you can avoid getting hacked, then the lower-impact-of-downstream-consequences questions are inherently safer (from the trivial reason of being less impactful). But this feels like a cheating answer, and the second axis seems more interesting.
My intuition about the “how likely are we to get an aligned answer” axis is this: There questions where I am fairly confident in our judging skills (for example, math proofs). Many of those could fall into the “definitely safe” category. Then there is the other extreme of questions where our judgement might be very fallible—things that are too vague or that play into our biases. (For example hard philosophical questions and problems whose solutions depend on answers to such questions. E.g., I wouldn’t trust myself to be a good judge of “how should we decide on the future of the universe” or “what is the best place for me to go for a vacation”.) I imagine these are “very likely unsafe”. And as a general principle, where there are two extremes, there often will be a continuum inbetween. Maybe “what is a reasonable way of curing cancer?” could fall here? (Being probably safe, but I wouldn’t bet all my money on it.)