In fact, their bodies are possibly so optimized for their current hunting strategy that higher intelligence might only trip them up.
It is much more likely that intelligence beyond this point simply costs too much relative to the benefit. Brains use a lot of energy.
“The order in which the conscion visits your person-slices makes no difference to what it’s like to be you”. Then how does it make a difference? What does it even mean for a conscion to visit someone before someone else? If it makes no difference, then you should adapt the theory to reflect that. And then we are left with two sets of points of spacetime (those visited by the conscion and those not), which sounds rather epiphenomenal.
The fact that—unlike the case of the nuclear war where the quality of the threat was visible to politicians and the public alike—alignment seems to be a problem which not even all AI researchers understand is worth mentioning. That in itself probably excludes the possibility of a direct political solution. But even politics in the narrow sense can be utilized with a bit of creativity (e.g. by providing politicians a motivation more direct than saving the world, grounded on things they can understand without believing weird-sounding claims of cultish-looking folks).
How do I put it, so as not to offend anyone… I think this is the right discussion for me to say that although I percieve this comment as positive, this definitely is not one I would wish to allocate my attention to, given the choice. I would have expected such posts to get downvoted. I suggest two separate systems of voting: one for positive fuzzy feelings, one for worthiness of attention. What I hope is that it would mitigate the reluctance to downvote (or to not upvote) stemming from the social nature of humans. I.e. we could continue not discouraging each other while still having a useful conversation.