In her latest post, Sarah Constantin writes:
Whatever ability IQ tests and math tests measure, I believe that lacking that ability doesn’t have any effect on one’s ability to make a good social impression or even to “seem smart” in conversation.
I remember that I made the opposite prediction a few months ago, while I was studying psychometrics more actively. In particular, I had the sense that because of the positive manifold (i.e. almost all positive mental attributes quite strongly correlate), it seems like judging someone’s IQ should be relatively easy, and also quite valuable in assessing a large number of other positive qualities, which should make it quite evolutionarily advantageous to be able to assess it.
Two concrete experiments I’ve thought about:
1. Take a group of people whose IQ you now, then just take pictures of their faces (or short videos of them saying something), and then have a group of participants rate them on their intelligence (probably in order). See how strong they correlate.
2. Throw a bunch of participants into a group, have them talk to each other, then afterwards ask them about the relative judgement of the intelligence of the other members of the group, see how predictive they are.
These seem like very straightforward experiments, that I can imagine someone having already run, and where I would be pretty interested in the results. If only so that I can be better calibrated on how much to trust my gut judgement of someone’s intelligence.
Does anyone know of any similar experiments that have been run?
Googling around phrases like ‘perception of intelligence’ seems to be a keyword for a relevant literature. On a very cursory skim (i.e. no more than what you see here) it seems to suggest “people can estimate intelligence of strangers better than chance (but with plenty of room for error and bias), even with limited exposure”. E.g.:
Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women (Note in this study the assessment was done purely on looking at a photograph of someone’s face)
Accurate Intelligence Assessments in Social Interactions: Mediators and Gender Effects (Abstract starts with: “Research indicates that people can assess a stranger’s measured intelligence more accurately than expected by chance, based on minimal information involving appearance and behavior.”)
Thin Slices of Behavior as Cues of Personality and Intelligence. (Short 1-2min slices of behaviour in a variety of contexts leads to assessments by strangers that positively correlate with administered test scores for IQ and big 5)
“Better than chance” doesn’t really mean much. The question is how well, quantitatively. Just briefly scanning over your studies:
The first one seems kinda sketchy because it has a smallish sample size and then it splits it up by sex, which leads to a truly small sample size within each sex. The difference between men and women seems unrealistically big to me, so I wanna say that to get less biased results, we should probably average the answers, yielding a correlation of about 0.24 for appearance alone. Though this is probably an underestimate due to range restriction. Also they didn’t report the raw correlation coefficients so I did some quick kind of sketchy calculations and they might be wrong.
Second study is kind of hard to interpret because they give correlation with sound, without sound, and with only a transcript. Range of correlations go from 0.37 (for video with sound) to 0.04 (for transcript only), but that’s with about 80 participants, so the standard errors are fairly large.
I don’t have time to parse the entirety of the third study because I need to go to bed soon, but it looks to me like the correlations there were about 0.2?
Note that it might be tempting to think that you can just average effects over a longer time to get more accurate results, but that is not always true; because of correlated measurement error, you might get stuck at some accuracy. For instance you could imagine that people would use academic interests as a proxy for intelligence; in such a case, they cannot exceed the prediction accuracy determined by the correlation between academic interests and intelligence.
The amount of effort people invest in seeming smarter than they are, for example in situations like hiring or sales, suggests to me that people’s gut judgments are easy enough to fool to be worth it.
I expect that what people are actually doing is using a few heuristics to make the judgment. I would also expect that courtesy of things like Dunning-Kruger, people towards the bottom will be as bad at estimating IQ as they are competence at any particular thing.
If we just stick with the intuition provided by fact that some people are really terrible at guessing this sort of thing and some people are not, I further expect that what people use as a heuristic changes with ability level. In example: people at the bottom might base it on whether someone agrees with them about stuff; those in the middle might focus heavily on whether someone has a smart-person job like doctor or lawyer; people towards the top might look more at the way someone does things, like some sign that they chose to think carefully about it or apply some technique known to them.
FWIW, the original Dunning-Kruger study did not show the effect that it’s become known for. See: https://danluu.com/dunning-kruger/
In particular:
The weaker claim is what holds in my example: in 4 out of 4 cases, the bottom quartile was very wrong in their assessment. I was speculating as to possible causal explanations; I think it is how they do the assessment.
Looking at those graphs, I can think of one rule that explains all four groups: say you are a above average when asked does the trick.
Faception is a startup which claims to have developed machine learning algorithms that can classify IQ and other things (including “academic researcher”, “professional poker player”, and “terrorist”).
If it is the case that AI can actually do this well, then I’d take that as evidence that humans might be capable of it too.
(From a quick look it’s unclear how successful they are. They seemed to be focused on military/security applications, i.e. detecting terrorists.)
″ Does anyone know of any similar experiments that have been run? ”
I can say with certainty that it has been done on a small scale, because I once saw a German TV documentary for which they had also gotten a small group (6-10) of both males and females and gotten them tested and then shown them the pictures of the others. Later they also gave them some facts, showed videos and asked them again.
The outcome was that there was a clear correlation between the guesses and the IQ-Test results and it got better the more information people had.
But the people were not representative but of above average education. In my own experience it is not the case that people always have a good intuition. I would guess that people are bad as soon as the person judged is much more intelligent than the judge. I think very smart people are much better at recognizing each other.
Research on how humans attribute intelligence to each other suggests that most people attribute higher intelligence to efficient agents, i.e., agents whose actions most resemble what we believe is the best course of action in an uncertain situation. A minority attributes intelligence to agents based on the outcome of their actions.
People attribute intelligence to efficiency in proportion to their own ability to plan. Less competent planners attribute intelligence to outcome. To elaborate, a competent planner would evaluate the behavior of an agent and understand whether it acted rationally, regardless of its success. On the other hand, a bad planner cannot infer the reasoning of an agent; since he himself cannot foresee the best course of action, he would not understand why an agent made a particular decision, and would likely attribute high intelligence to an irrational but lucky agent since its behavior would result in a good outcome.
My interpretation of these results is that intelligent people are better at evaluating the intelligence of others. I believe that given enough time (and thus enough information to evaluate) an intelligent person can adequately estimate a peer’s intelligence. It could be argued that “seeming” smart requires a sufficient degree of awareness and intelligence, although of a different kind; perhaps IQ tests do not have a wide enough scope to accurately measure different intelligence in its different forms.
Source:
Kryven, Marta. “Attributed Intelligence.” (2018).
Look into the history of EQ and specifically the Mayer Salovey-Caruso work. Truly observed and assessed IQ was random and compounded by the observers own sense of IQ. Primary school teachers were the most accurate at assessing IQ but that was a low correlation and still less than coin toss. The Johari window also comes into play. The more educated you are AND the higher your IQ, the lower you assess your own IQ...i.e. you actually know what you don’t know...you are smart but you know you are not the smartest...make sense?
So who is anyone to judge another person’s IQ? No body. Test it, don’t ask others to judge it.