How would you find out something that a three-year-old is trying to hide from you?
A three-year-old that designed the universe I live in in such a way that it stays hidden as well as possible? I have absolutely no idea. I would probably hope for it to get bored and tell me.
We don’t have any way of knowing whether a deception that would fool the likes of us would fool somebody way smarter.
We do have at least some clues:
We know the mathematics of how optimal belief updating works.
We have some rough estimates of the complexity of the theory the AI must figure out.
We have some rough estimates of the amount of information we have given the AI.
I know there is such a thing as underestimating AI, but I think you are severely overestimating it.
A three-year-old that designed the universe I live in in such a way that it stays hidden as well as possible? I have absolutely no idea. I would probably hope for it to get bored and tell me.
We do have at least some clues:
We know the mathematics of how optimal belief updating works.
We have some rough estimates of the complexity of the theory the AI must figure out.
We have some rough estimates of the amount of information we have given the AI.
I know there is such a thing as underestimating AI, but I think you are severely overestimating it.