(Defining Tool AI as a program that would evaluate the answer to a question given available data without seeking to obtain any new data, and then shut down after having discovered the answer) While those arguments (if successful) argue that it’s harder to program a Tool AI than it might look at first, so AI alignment research is still something that should be actively researched (and I doubt Tegmark think AI alignment research is useless), they don’t really address the point that making aligned Tool AIs are still in some sense “inherently safer” than making Friendly AGI because the lack of a singleton scenario mean you don’t need to solve all moral and political philosophy from first principles in your garage in 5 years and hope you “get it right” the first time.
(Defining Tool AI as a program that would evaluate the answer to a question given available data without seeking to obtain any new data, and then shut down after having discovered the answer) While those arguments (if successful) argue that it’s harder to program a Tool AI than it might look at first, so AI alignment research is still something that should be actively researched (and I doubt Tegmark think AI alignment research is useless), they don’t really address the point that making aligned Tool AIs are still in some sense “inherently safer” than making Friendly AGI because the lack of a singleton scenario mean you don’t need to solve all moral and political philosophy from first principles in your garage in 5 years and hope you “get it right” the first time.