If we have total conviction that the end of the world is nigh, isn’t it rational to consider even awful, unpalatable options for extending the timeline before we “achieve” AGI?
It’s not strictly necessary that a pivotal act is powered by AI itself.
Avoiding explicit details for obvious reasons and trusting that it’s understood. But surely it’s within the realm of possibility to persecute, terrorize or sabotage the progression of AI research; and plausibly for a long enough time to solve alignment first.
Curious to know the “dignity” calculation here. Presumably almost any pivotal act with an aligned-AGI is forgivable because it would be specific and the dawn of a Utopia. But what if horrible things are done only to buy more time into a still uncertain future?
If we have total conviction that the end of the world is nigh, isn’t it rational to consider even awful, unpalatable options for extending the timeline before we “achieve” AGI?
Eliezer has been very clear that he thinks this is a bad idea, see e.g. Q2 of this post.
Also, keep in mind that a single instance of one AI safety person doing something criminal has the potential for massively damaging the public standing of the community. I think this should dominate the calculation; even if you think the probability that [the arguments from the current post are totally wrong] is low, it’s not that low.
I’d be mindful of information hazards. All you need is one person doing this too soon, and likely failing, for talking about the dangers of AI to become taboo in the public eye.
If we have total conviction that the end of the world is nigh, isn’t it rational to consider even awful, unpalatable options for extending the timeline before we “achieve” AGI?
It’s not strictly necessary that a pivotal act is powered by AI itself.
Avoiding explicit details for obvious reasons and trusting that it’s understood. But surely it’s within the realm of possibility to persecute, terrorize or sabotage the progression of AI research; and plausibly for a long enough time to solve alignment first.
Curious to know the “dignity” calculation here. Presumably almost any pivotal act with an aligned-AGI is forgivable because it would be specific and the dawn of a Utopia. But what if horrible things are done only to buy more time into a still uncertain future?
Eliezer has been very clear that he thinks this is a bad idea, see e.g. Q2 of this post.
Also, keep in mind that a single instance of one AI safety person doing something criminal has the potential for massively damaging the public standing of the community. I think this should dominate the calculation; even if you think the probability that [the arguments from the current post are totally wrong] is low, it’s not that low.
I’d be mindful of information hazards. All you need is one person doing this too soon, and likely failing, for talking about the dangers of AI to become taboo in the public eye.