That’s part of the problem, often the bad middle ground looks superficially plausible, so it’s very sticky and hard to get rid of, because it’s not exactly that people get told the wrong things but rather that they spontaneously develop the wrong ideas.
The three basic issues with this viewpoint are:
IQ test batteries do not measure even close to all cognitive abilities and realistically could never do that.
Many of the abilities that IQ scores weight highly are practically unimportant.
Differential-psychology tests are in practice more like log scales than like linear scales, so “sums” are more like products than like actual suns; even if you are absurdly good at one thing, you’re going to have a hard time competing with someone in IQ if they are moderately better at many things.
1. IQ scores do not measure even close to all cognitive abilities and realistically could never do that.
Well, the original statement was “sums together cognitive abilities” and didn’t use the word “all”, and I, at least, saw no reason to assume it. If you’re going to say something along the lines of “Well, I’ve tried to have reasonable discussions with these people, but they have these insane views”, that seems like a good time to be careful about how you represent those views.
2. Many of the abilities that IQ scores weight highly are practically unimportant.
Are you talking about direct measurement, or what they correlate with? Because, certainly, things like anagramming a word have almost no practical application, but I think it’s intended to (and does) correlate with language ability. But in any case, the truth value of the statement that IQ is “an index that sums together cognitive abilities” is unaffected by whether those abilities are useful ones.
Perhaps you have some idea of a holistic view, of which that statement is only a part, and maybe that holistic view contains other statements which are in fact insane, and you’re attacking that view, but… in the spirit of this post, I would recommend confining your attacks to specific statements rather than to other claims that you think correlate with those statements.
3. Differential-psychology tests are in practice more like log scales than like linear scales, so “sums” are more like products than like actual suns; even if you are absurdly good at one thing, you’re going to have a hard time competing with someone in IQ if they are moderately better at many things.
I wonder how large a difference this makes in practice. So if we run with your claim here, it seems like your conclusion would be… that IQ tests combine the subtest scores in the wrong way, and are less accurate than they should be for people with very uneven abilities? Is that your position? At any rate, even if the numbers are logarithms, it’s still correct to say that the test is adding them up, and I don’t consider that good grounds for calling it “insane” for people to consider it addition.
(Fun tangent, not directly addressing this argument thread.)
There’s a trio of great posts from 2015 by @JonahS : The Truth About Mathematical Ability ; Innate Mathematical Ability ; Is Scott Alexander bad at math? which (among other things) argues that you can be “good at math” along the dimension(s) of noticing patterns very quickly, AND/OR you can be “good at math” along the dimension(s) of an “aesthetic” sense for concepts being right and sensible. (My summary, not his.)
The “aesthetics” is sorta a loss function that provides a guidestar for developing good deep novel understanding—but that process may take a very long time. He offers Scott Alexander, and himself, and Alexander Grothendieck as examples of people with lopsided profiles—stronger on “aesthetics” than they are on “fast pattern-recognition”.
I found it a thought-provoking hypothesis. I wish JonahS had written more.
The analogy that I’m objecting to is, if you looked at e.g. the total for a ledger or a budget, it is an index that sums together expenses in a much more straightforward way. For instance if there is a large expense, the total is large.
Meanwhile, IQ scores are more like the geometric mean of the entries on such an entity. The geometric mean tells you whether the individual items tend to be large or small, which gives you broad-hitting information that distinguishes e.g. people who live in high-income countries from people who live in low-income countries, or large organizations from individual people; but it won’t inform you if someone got hit by a giant medical bill or if they managed to hack themselves to an ultra-cheap living space. These pretty much necessarily have to be low-rank mediators (like in the g model) rather than diverse aggregates (like in the sum model).
(Well, a complication in this analogy is that a ledger can vary not just in the magnitude of the transfers but also qualitatively in the kinds of transfers that are made, whereas IQ tests fix the variables, making it more analogous to a standardized budget form (e.g. for tax or loan purposes) broken down by stuff like “living space rent”, “food”, “healthcare”, etc..)
So, the arithmetic and geometric mean agree when the inputs are equal, and, the more unequal they are, the lower the geometric mean is.
I note that the subtests have ceilings, which puts a limit on how much any one can skew the result. Like, if you have 10 subtests, and the max score is something like 150, then presumably each test has a max score of 15 points. If we imagine someone gets five 7s and five 13s (a moderately unbalanced set of abilities), then the geometric mean is 9.54, while the arithmetic mean is 10. So, even if someone were confused about whether the IQ test was using a geometric or an arithmetic mean, does it make a large difference in practice?
The people you’re arguing against, is it actually a crux for them? Do they think IQ tests are totally invalid because they’re using an arithmetic mean, but actually they should realize it’s more like a geometric mean and then they’d agree IQ tests are great?
If you consider the “true ability” to be the exponential of the subtest scores, then the extent to which the problem I mention applies depends on the base of the exponential. In the limiting case where the base goes to infinity, only the highest ability matter, whereas in the limiting case where the base goes to 1, you end up with something basically linear.
As for whether it’s a crux, approximately nobody has thought about this deeply enough that they would recognize it, but I think it’s pretty foundational for a lot of disagreements about IQ.
That’s part of the problem, often the bad middle ground looks superficially plausible, so it’s very sticky and hard to get rid of, because it’s not exactly that people get told the wrong things but rather that they spontaneously develop the wrong ideas.
The three basic issues with this viewpoint are:
IQ test batteries do not measure even close to all cognitive abilities and realistically could never do that.
Many of the abilities that IQ scores weight highly are practically unimportant.
Differential-psychology tests are in practice more like log scales than like linear scales, so “sums” are more like products than like actual suns; even if you are absurdly good at one thing, you’re going to have a hard time competing with someone in IQ if they are moderately better at many things.
Well, the original statement was “sums together cognitive abilities” and didn’t use the word “all”, and I, at least, saw no reason to assume it. If you’re going to say something along the lines of “Well, I’ve tried to have reasonable discussions with these people, but they have these insane views”, that seems like a good time to be careful about how you represent those views.
Are you talking about direct measurement, or what they correlate with? Because, certainly, things like anagramming a word have almost no practical application, but I think it’s intended to (and does) correlate with language ability. But in any case, the truth value of the statement that IQ is “an index that sums together cognitive abilities” is unaffected by whether those abilities are useful ones.
Perhaps you have some idea of a holistic view, of which that statement is only a part, and maybe that holistic view contains other statements which are in fact insane, and you’re attacking that view, but… in the spirit of this post, I would recommend confining your attacks to specific statements rather than to other claims that you think correlate with those statements.
I wonder how large a difference this makes in practice. So if we run with your claim here, it seems like your conclusion would be… that IQ tests combine the subtest scores in the wrong way, and are less accurate than they should be for people with very uneven abilities? Is that your position? At any rate, even if the numbers are logarithms, it’s still correct to say that the test is adding them up, and I don’t consider that good grounds for calling it “insane” for people to consider it addition.
(Fun tangent, not directly addressing this argument thread.)
There’s a trio of great posts from 2015 by @JonahS : The Truth About Mathematical Ability ; Innate Mathematical Ability ; Is Scott Alexander bad at math? which (among other things) argues that you can be “good at math” along the dimension(s) of noticing patterns very quickly, AND/OR you can be “good at math” along the dimension(s) of an “aesthetic” sense for concepts being right and sensible. (My summary, not his.)
The “aesthetics” is sorta a loss function that provides a guidestar for developing good deep novel understanding—but that process may take a very long time. He offers Scott Alexander, and himself, and Alexander Grothendieck as examples of people with lopsided profiles—stronger on “aesthetics” than they are on “fast pattern-recognition”.
I found it a thought-provoking hypothesis. I wish JonahS had written more.
The analogy that I’m objecting to is, if you looked at e.g. the total for a ledger or a budget, it is an index that sums together expenses in a much more straightforward way. For instance if there is a large expense, the total is large.
Meanwhile, IQ scores are more like the geometric mean of the entries on such an entity. The geometric mean tells you whether the individual items tend to be large or small, which gives you broad-hitting information that distinguishes e.g. people who live in high-income countries from people who live in low-income countries, or large organizations from individual people; but it won’t inform you if someone got hit by a giant medical bill or if they managed to hack themselves to an ultra-cheap living space. These pretty much necessarily have to be low-rank mediators (like in the g model) rather than diverse aggregates (like in the sum model).
(Well, a complication in this analogy is that a ledger can vary not just in the magnitude of the transfers but also qualitatively in the kinds of transfers that are made, whereas IQ tests fix the variables, making it more analogous to a standardized budget form (e.g. for tax or loan purposes) broken down by stuff like “living space rent”, “food”, “healthcare”, etc..)
So, the arithmetic and geometric mean agree when the inputs are equal, and, the more unequal they are, the lower the geometric mean is.
I note that the subtests have ceilings, which puts a limit on how much any one can skew the result. Like, if you have 10 subtests, and the max score is something like 150, then presumably each test has a max score of 15 points. If we imagine someone gets five 7s and five 13s (a moderately unbalanced set of abilities), then the geometric mean is 9.54, while the arithmetic mean is 10. So, even if someone were confused about whether the IQ test was using a geometric or an arithmetic mean, does it make a large difference in practice?
The people you’re arguing against, is it actually a crux for them? Do they think IQ tests are totally invalid because they’re using an arithmetic mean, but actually they should realize it’s more like a geometric mean and then they’d agree IQ tests are great?
If you consider the “true ability” to be the exponential of the subtest scores, then the extent to which the problem I mention applies depends on the base of the exponential. In the limiting case where the base goes to infinity, only the highest ability matter, whereas in the limiting case where the base goes to 1, you end up with something basically linear.
As for whether it’s a crux, approximately nobody has thought about this deeply enough that they would recognize it, but I think it’s pretty foundational for a lot of disagreements about IQ.