The main problem that remains unsolved is that somebody else can still build a fully self-improving UFAI. And that AI, being a lot smarter and more free than your limited AI, will crush it. The best way to prevent that from happening is to have a system that can monitor and understand pretty much everything, and that’s pretty much FAI.
The main problem that remains unsolved is that somebody else can still build a fully self-improving UFAI. And that AI, being a lot smarter and more free than your limited AI, will crush it. The best way to prevent that from happening is to have a system that can monitor and understand pretty much everything, and that’s pretty much FAI.
Is it really the case that an unrestricted growing entity will outcompete a restricted growing entity?
i.e. something like Cancer vs the Immune System.
I’m not convinced that the unrestricted growing entity will always win.
Didn’t you just answer your own question?
And the problem isn’t that unrestricted growing entity will always win. One win will be enough to end it all.