Very likely it’s only considering AI taking over because it functionally “wants” to take over. That’s the standard concern in AGI safety/alignment. “Misinterpreting” commands could result in it “wanting” to take over- but if it’s still taking commands, those orders could probably be reversed before too much damage was done. We tend to mostly worry about the-humans-are-dead (or out-of-power-and-never-getting-it-back, which unless it’s a very benevolent takeover probably means on the way to the-humans-are-dead).
Very likely it’s only considering AI taking over because it functionally “wants” to take over. That’s the standard concern in AGI safety/alignment. “Misinterpreting” commands could result in it “wanting” to take over- but if it’s still taking commands, those orders could probably be reversed before too much damage was done. We tend to mostly worry about the-humans-are-dead (or out-of-power-and-never-getting-it-back, which unless it’s a very benevolent takeover probably means on the way to the-humans-are-dead).