If we can put high confidence in #1 via testing, can we still put high confidence in it on based on a general impression or a conversation, or even on the basis of mysterious evidence? e.g. This quote: “(Interesting question: If I’m not judging Brooks by the goodness of his AI theories, what is it that made him seem smart to me? I don’t remember any stunning epiphanies in his presentation at the Summit. I didn’t talk to him very long in person. He just came across as… formidable, somehow.)”
I don’t think you can. A conversation or ‘general impression’ is going to be based on interpersonal skills, and unless it is a highly technical conversation, be based mostly on verbal sorts of skills. Asking whether an IQ test would be less reliable than a conversation is a little like asking ‘if we drop the SAT Math section and just use Verbal, is that better than using both the Math and Verbal sections?’ No one item loads very heavily on g which is why IQ tests typically have a bunch of subtests.
I don’t think you can. A conversation or ‘general impression’ is going to be based on interpersonal skills, and unless it is a highly technical conversation, be based mostly on verbal sorts of skills. Asking whether an IQ test would be less reliable than a conversation is a little like asking ‘if we drop the SAT Math section and just use Verbal, is that better than using both the Math and Verbal sections?’ No one item loads very heavily on g which is why IQ tests typically have a bunch of subtests.