Raising awareness about the problem isn’t just about getting would-be uFAI’ers to mend their sinful ways, you know. It’s absolutely necessary if you’re convinced you need help with it. As you said, inaction is untenable. If you’re certain that a goal of this magnitude is basically impossible given the status quo, taking some initial risks is a trivial decision. It doesn’t follow that additional risks share the same justification.
I’m also not convinced we understand the boundaries between “intelligent” and “self-improving” well enough to assume we can experiment with one and not the other. What sort of “practical tasks” do you have in mind that don’t involve potentially intelligent information-processing systems, and why do you think they’ll be at all relevant to the “real” work ahead?
Raising awareness about the problem isn’t just about getting would-be uFAI’ers to mend their sinful ways, you know. It’s absolutely necessary if you’re convinced you need help with it. As you said, inaction is untenable. If you’re certain that a goal of this magnitude is basically impossible given the status quo, taking some initial risks is a trivial decision. It doesn’t follow that additional risks share the same justification.
I’m also not convinced we understand the boundaries between “intelligent” and “self-improving” well enough to assume we can experiment with one and not the other. What sort of “practical tasks” do you have in mind that don’t involve potentially intelligent information-processing systems, and why do you think they’ll be at all relevant to the “real” work ahead?