If we knew how to build a machine that chooses its outputs as to maximize some property of the surrounding universe, such a machine would be very dangerous, because maximizing almost any easily defined property leads to a worthless universe (without humans, or with humans living pointless lives, etc.)
A sufficiently anachronistic such machine, yes, i.e. a maximizer much more powerful than the tech level of the world in which it exists should be able to support. It’s like asking whether a microwave oven would be dangerous if its power output was, say, 10^50 watts instead of the more typical 800 watts. The answer is that such a device would probably be safe for a civilization capable of building it, but if somehow created in today’s world it would indeed be dangerous.
That leads to the answer to one of your questions:
Do you pursue alternate avenues like uploads, or focus only on FAI?
Are you postulating a world where anachronistic technology in general is easy, or where AI is easy but other things like uploads remain realistically difficult? If the former, uploads might well be worth going for; if the latter, you’re probably better off to forget about everything else and focus only on FAI.
A sufficiently anachronistic such machine, yes, i.e. a maximizer much more powerful than the tech level of the world in which it exists should be able to support. It’s like asking whether a microwave oven would be dangerous if its power output was, say, 10^50 watts instead of the more typical 800 watts. The answer is that such a device would probably be safe for a civilization capable of building it, but if somehow created in today’s world it would indeed be dangerous.
That leads to the answer to one of your questions:
Are you postulating a world where anachronistic technology in general is easy, or where AI is easy but other things like uploads remain realistically difficult? If the former, uploads might well be worth going for; if the latter, you’re probably better off to forget about everything else and focus only on FAI.