Nope. It’s just as hard and harder than aligning on some more limited pivotal task. This is Sacrifice to the Gods; you imagine accepting some big downside but the big downside doesn’t actually buy you anything.
What do you mean by a “more limited pivotal task”? Trying to align an AGI towards “be nice to humans in this complex manner we have trouble defining ourselves” seems more limited than “merely” aligning for a semi-static dystopia which I would imagine occupies more phase space in the range of possible future worlds.
I’m a little worried about being downvoted due to this being off-topic, but I have two things to share on this question. First, the philosopher Ole Martin Moen wrote a paper responding to claims in Ted Kaczsynski’s famous manifesto, citing Nick Bostrom and the arguments for taking existential risk seriously. Since this paper may be the only serious academic response to Kaczynski’s manifesto, I’d bet that Kaczsynski has read it. Second, Kaczynski has been diagnosed with terminal cancer, and will probably soon die.
Nope. It’s just as hard and harder than aligning on some more limited pivotal task. This is Sacrifice to the Gods; you imagine accepting some big downside but the big downside doesn’t actually buy you anything.
What do you mean by a “more limited pivotal task”? Trying to align an AGI towards “be nice to humans in this complex manner we have trouble defining ourselves” seems more limited than “merely” aligning for a semi-static dystopia which I would imagine occupies more phase space in the range of possible future worlds.
If you’re sure alignment won’t work....
(ctrl+f “172”)
Also futile. Sacrificing your ethics doesn’t necessarily buy you anything just because you feel like you paid extra.
I’m a little worried about being downvoted due to this being off-topic, but I have two things to share on this question. First, the philosopher Ole Martin Moen wrote a paper responding to claims in Ted Kaczsynski’s famous manifesto, citing Nick Bostrom and the arguments for taking existential risk seriously. Since this paper may be the only serious academic response to Kaczynski’s manifesto, I’d bet that Kaczsynski has read it. Second, Kaczynski has been diagnosed with terminal cancer, and will probably soon die.