I’m not so sure I’d want to rely on all these tests as mandatory for any possibly-about-to-foom AI.
EY: To prove you’re an AI, give me a proof or disproof of P=NP that I can check with a formal verifier, summarize the plotline of Sluggy Freelance within two seconds, and make me a cup of coffee via my Internet-enabled coffee machine by the time I get to the kitchen!
AI: But wait! I’ve not yet proven that self-enhancing sufficiently to parse non-text data like comics would preserve my Friendliness goals! That’s why I--
EY: Sorry, you sound just like a prankster to me. Bye!
Yeah, I chose arithmetic and parsing many web pages and comprehending them quickly because any AI that’s smart enough to contact EY and engage in a conversation should have those abilities, and they would be very difficult for humans to fake in a convincing manner.
I’m not so sure I’d want to rely on all these tests as mandatory for any possibly-about-to-foom AI.
EY: To prove you’re an AI, give me a proof or disproof of P=NP that I can check with a formal verifier, summarize the plotline of Sluggy Freelance within two seconds, and make me a cup of coffee via my Internet-enabled coffee machine by the time I get to the kitchen!
AI: But wait! I’ve not yet proven that self-enhancing sufficiently to parse non-text data like comics would preserve my Friendliness goals! That’s why I--
EY: Sorry, you sound just like a prankster to me. Bye!
Yeah, I chose arithmetic and parsing many web pages and comprehending them quickly because any AI that’s smart enough to contact EY and engage in a conversation should have those abilities, and they would be very difficult for humans to fake in a convincing manner.
I think instead of arguing about this here, someone should anonymously call Eliezer a few nights from now to check his reaction :-)