Given these, plus Miller’s sloppiness in using “costly signaling” verbiage, I must agree with Nesov’s objections. The post wants to lead us to some conclusions, but these are only vaguely implied not stated, and the heavy use of quotations from the book, wall-o-text fashion, distracts from noticing that the post doesn’t really get to any particular point.
Which ones specifically seem weak? The problems with “garbage in, garbage out” and the lack of explanatory power of factor analysis seem serious enough to me. Basically, the WP page says that there is no “Big Five personality theory”, contrary to what the comment by Mass_Driver suggests.
What there is is a bunch of questionnaire items, the answers to which seem to somewhat reliably predict the answers to other questionnaire items when subjected to CFA; the label “Openness” for instance is arbitrarily given to one of those five “factors”, and if you look at the “Sample openness items” from the WP page it’s not clear that any of these questions in fact have to do with a person’s being open to new experiences.
Critics argue that there are limitations to the scope of Big Five as an explanatory or predictive theory. It is argued that the Big Five does not explain all of human personality.
If people claim that it is an all encompassing model, then that would be a serious criticism. I don’t actually know if researchers claim that, but it seems unlikely to me.
The methodology used to identify the dimensional structure of personality traits, factor analysis, is often challenged for not having a universally-recognized basis for choosing among solutions with different numbers of factors.
This is a weaker criticism than it seems. It stems from using classical stats. Latent factor (such as FA) models are less confusing from a Bayesian point of view. The particular indeterminacy they are talking about disappears when you specify a model and prior more clearly and try to update in a principled manner. A better criticism would be that the results of personality models might be sensitive to model assumptions. However, that sensitivity is an empirical matter and isn’t established by referencing the indeterminacy of FA. Another good criticism of Big Five research might be that factor analysis is not a good tool because it could be hard to figure out how sensitive your results are to the particular analysis.
Another frequent criticism is that the Big Five is not theory-driven. It is merely a data-driven investigation of certain descriptors that tend to cluster together under factor analysis.
What I’d like to know, primarily, is whether the Big Five traits correlate with anything other than answers to questionnaire items: in particular, actual behavior.
Pointer to studies appreciated; Google hasn’t turned up much. Some studies of job performance have C having the kind of influence you’d expect, then there’s things like this.
I think Bryan Caplan talks about the Big Five some and is usually pretty clear thinking. He has a paper on the topic which gives some references for correlations between the Big Five and other things. For example:
A particularly noteworthy aspect of the Conscientiousness—job performance link is that Conscientiousness is highly correlated (.5 to .6) with various measures of educational achievement but uncorrelated with measured intelligence. (Barrick and Mount 1991, p.5) Conscientious people are more successful in both school and work. In consequence, rate of return to education estimates that fail to control for Conscientiousness are likely to be biased upwards.
And
Criminals are on average markedly lower in both Conscientiousness and Agreeableness than non-criminals, even holding other variables fixed.1 (Costa and Widiger 1994; Wilson and Herrnstein 1985)
I’d be interested in hearing if you find interesting results.
I am also interested in that. My impression from hearing people talk about such things is that yes, they do, but I don’t actually know much about this area. That result is interesting.
Not just weak, but confused, and not even clearly criticisms, as they look more like snarky suggestions for improvement (might be Wikipedia page’s framing).
Objections have been voiced and not addressed. Not just here but on the WP page as well.
Given these, plus Miller’s sloppiness in using “costly signaling” verbiage, I must agree with Nesov’s objections. The post wants to lead us to some conclusions, but these are only vaguely implied not stated, and the heavy use of quotations from the book, wall-o-text fashion, distracts from noticing that the post doesn’t really get to any particular point.
Those WP criticisms seem surprisingly weak.
Which ones specifically seem weak? The problems with “garbage in, garbage out” and the lack of explanatory power of factor analysis seem serious enough to me. Basically, the WP page says that there is no “Big Five personality theory”, contrary to what the comment by Mass_Driver suggests.
What there is is a bunch of questionnaire items, the answers to which seem to somewhat reliably predict the answers to other questionnaire items when subjected to CFA; the label “Openness” for instance is arbitrarily given to one of those five “factors”, and if you look at the “Sample openness items” from the WP page it’s not clear that any of these questions in fact have to do with a person’s being open to new experiences.
If people claim that it is an all encompassing model, then that would be a serious criticism. I don’t actually know if researchers claim that, but it seems unlikely to me.
This is a weaker criticism than it seems. It stems from using classical stats. Latent factor (such as FA) models are less confusing from a Bayesian point of view. The particular indeterminacy they are talking about disappears when you specify a model and prior more clearly and try to update in a principled manner. A better criticism would be that the results of personality models might be sensitive to model assumptions. However, that sensitivity is an empirical matter and isn’t established by referencing the indeterminacy of FA. Another good criticism of Big Five research might be that factor analysis is not a good tool because it could be hard to figure out how sensitive your results are to the particular analysis.
This doesn’t seems like a criticism to me.
What I’d like to know, primarily, is whether the Big Five traits correlate with anything other than answers to questionnaire items: in particular, actual behavior.
Pointer to studies appreciated; Google hasn’t turned up much. Some studies of job performance have C having the kind of influence you’d expect, then there’s things like this.
I think Bryan Caplan talks about the Big Five some and is usually pretty clear thinking. He has a paper on the topic which gives some references for correlations between the Big Five and other things. For example:
And
I’d be interested in hearing if you find interesting results.
I am also interested in that. My impression from hearing people talk about such things is that yes, they do, but I don’t actually know much about this area. That result is interesting.
Not just weak, but confused, and not even clearly criticisms, as they look more like snarky suggestions for improvement (might be Wikipedia page’s framing).