Obviously I meant some kind of approximation of consensus or acceptability derived from much greater substantiation. There is no equivalent to Climate Change or ZFC in the field of AI in terms of acceptability and standardisation. Matthew Barnett made my point better in the above comments.
Yes, most policy has no degree of consensus. Most policy is also not asking to shut down the entire world’s major industries. So there must be a high bar. A lot of policy incidentally ends up being malformed and hurting people, so it sounds like you’re just making the case for more “consensus” and not less.
How do you suppose the AGI is going to be able to wrap the sun in a dyson sphere using only the resources available on earth? Do you have evidence that there are enough resources on asteroids or nearby planets for their mining to be economically viable? At the current rate, mining an asteroid costs billions while their value is nothing. Even then we don’t know if they’ll have enough of the exact kind of materials necessary to make a dyson sphere around an object which has 12000x the surface area of earth. You could have von nuemman replicators do the mining but then they’d spend most of the materials on the replicators and have to go very far to get more materials, at which point they’d just settle on a new star. They could turn human atoms into usable material, but humans are a very tiny percentage of earth and our useful matter is even tinier, it certainly wouldn’t be enough to envelop the sun. Even if the replicators had a perfectly efficient way to turn photoelectric power into any feasible construction of atoms, which you don’t know if that’s possible, it doesn’t seem like it would be efficient to overcome some of the gravities and distances and related issues with other planets to make mining them economically viable either. Within our solar system and possibly even further it just doesn’t seem possible to envelop the sun at all.
I’m playing within your hypotheticals here even though I think none of them will ever happen, but even within your hypotheticals it seems like the dyson sphere point is just total nonsense and we probably wont go extinct by having the sun covered up. I wont deny your other points because even though they already rest on a lot of astronomically unlikely assumptions, they do seem legitimate within that framework.