if someone is smart enough to recognize that self-improving agents need stable utility-functions and have to be friendly with respect to the values of their lower-level-self’s, then it is very unlikely that the same person fails to recognize the need for human-friendliness.
I’m not disputing the above statement itself, but it implies that you are counting a friendly AI recursively improving to superintelligence quickly as not being a FOOM. If you call a friendly FOOM a FOOM, then P6 is irrelevant.
I’m not disputing the above statement itself, but it implies that you are counting a friendly AI recursively improving to superintelligence quickly as not being a FOOM. If you call a friendly FOOM a FOOM, then P6 is irrelevant.