If you think the coming of AGI is inevitable, but you think that surviving AGI is hard and you might be able to help with it, then you should do everything you can to make the transition to a safe AGI future go well. Including possibly sacrificing your own life, if you value the lives of your loved ones in aggregate more than your life alone. In a sense, working hard to make AGI go well is ‘risk aversion’ on a society-wide basis, but I’d call the attitude of the agentic actors in this scenario more one of ‘ambition maximizing’ rather than ‘personal risk aversion’.
If you think the coming of AGI is inevitable, but you think that surviving AGI is hard and you might be able to help with it, then you should do everything you can to make the transition to a safe AGI future go well. Including possibly sacrificing your own life, if you value the lives of your loved ones in aggregate more than your life alone. In a sense, working hard to make AGI go well is ‘risk aversion’ on a society-wide basis, but I’d call the attitude of the agentic actors in this scenario more one of ‘ambition maximizing’ rather than ‘personal risk aversion’.