I don’t have a valid proof for you. Omega is typically defined like that (arbitrarily powerful and completely trustworthy), but a number of the problems I’ve seen of this type tend to just say ‘Omega appears’ and assume that you know Omega is the defined entity simply because it self-identifies as Omega, so I felt the need to specify that in this instance, Omega has just proved itself.
Theoretically, you could verify the trustworthiness of a superintelligence by examining its code… but even if we ignore the fact that you’re probably not equipped to comprehend the code of a superintelligence (really, you’ll probably need another completely trustworthy superintelligence to interpret the code for you, which rather defeats the point), there’s still the problem that an untrustworthy superintelligence could provide you with a completely convincing forgery, which could potentially be designed in such a way that it would performs every action in the same way as the real one would (in that way being evaluated as ‘trustworthy’ under simulation)… except the one for which the untrustworthy superintelligence is choosing to deceive you on. Accordingly, I think that even a superintelligence probably can’t be sure about the trustworthiness of another superintelligence, regardless of evidence.
I don’t have a valid proof for you. Omega is typically defined like that (arbitrarily powerful and completely trustworthy), but a number of the problems I’ve seen of this type tend to just say ‘Omega appears’ and assume that you know Omega is the defined entity simply because it self-identifies as Omega, so I felt the need to specify that in this instance, Omega has just proved itself.
Theoretically, you could verify the trustworthiness of a superintelligence by examining its code… but even if we ignore the fact that you’re probably not equipped to comprehend the code of a superintelligence (really, you’ll probably need another completely trustworthy superintelligence to interpret the code for you, which rather defeats the point), there’s still the problem that an untrustworthy superintelligence could provide you with a completely convincing forgery, which could potentially be designed in such a way that it would performs every action in the same way as the real one would (in that way being evaluated as ‘trustworthy’ under simulation)… except the one for which the untrustworthy superintelligence is choosing to deceive you on. Accordingly, I think that even a superintelligence probably can’t be sure about the trustworthiness of another superintelligence, regardless of evidence.