I am willing to bet money (up to €100, I am a student) that these will not, in fact, be the best predictors, just as grades in college have been proven not to be good predictors of how well a Google employee does, IQ tests have been proven not to be a good predictor of how well a salesman does, and higher salaries have proven to, past a certain point, be detrimental to productivity for employees whose jobs require creativity, imagination or otherwise intellectual exertion.
My hypothesis is that a <10 min IQ test will be so inaccurate as to be useless, and that the current school system already favors people who do what they are told as they are told without getting too far ahead or behind, with the destructive and wasteful results we all know: they are the ones it grades best, and yet, time and again, tested against the workplace, those grades have proven little.
IQ test scores are massively correlated with workplace performance.
I find it plausible that among the set of people Google has hired grades don’t predict workplace success, but I bet if Google were to randomly select employees from among the U.S. population (or even from just among computer science majors) grades would be hugely correlated with performance.
What are the terms of the bet? I would be willing to bet €100 depending on the terms. Full disclosure: I read a lot of academic research on IQ to help me write this book, one of whose chapters is about IQ.
There is a lot of silly stuff written about IQ. No offense, but do you think you have the needed background to separate the high from the low quality IQ scholarship?
I only read Talent is Overrated, which cited a few studies to the effect that IQ was a bad predictor of professional performance beyond the short term; to become a world-class performer at some specific skill, you need to spend 20000 hours of your life deliberately practicing it (trying things outside your comfort zone, failing to get it right, figuring out what to improve and how, getting it right, trying something harder, rinse and repeat, and it’s a pain in the neck). Most people, no matter how smart, tend to stop improving at their jobs after around one year and a half of practice: this includes highly skilled professionals like doctors and lawyers.
And, on the anecdotal side, in my own experience and what I could get from the people that frequent Lesswrong and TVTropes, a high IQ commonly results in huge childhood issues, a miserable social life, and a boatpload of akrasia, which seriously hamper whatever base utility it might have. Perhaps there’s a selection bias and only the incompetent brains have the time to hang out here, while competent smart people are too busy actually getting things done?
in my own experience and what I could get from the people that frequent Lesswrong and TVTropes, a high IQ commonly results in [...] a miserable social life, and a boatpload of akrasia [...] Perhaps there’s a selection bias and only the incompetent brains have the time to hang out here[?]
There is massive selection bias going on here.
I don’t think it has as much to do with free time as with target audience, though. LW attracts a few different clusters of people, but the ones you’re seeing in this context are those who feel their thinking is flawed in some way, and who believe they have a decent chance of fixing it with a cognitive science toolset and vocabulary. The site’s native idiom and interaction style—basically a founder effect—imposes a few more filters, underrepresenting some problems and overrepresenting others. Akrasia and social problems are precisely the issues I’d expect to see a lot of, given those constraints.
TV Tropes… well, that might have more to do with free time. Almost everyone likes media, but if you want to make many original contributions, you need unusual knowledge of media and an analytical attitude towards it. Moreover, the media best represented there tend to be the most time-consuming ones—TV, anime, doorstopper fantasy novels. I don’t think I need to go into too much detail regarding the people most likely to share those requirements.
Offtopic post, but a discussion I wish to pursue nonetheless:
Regarding TVT: it used to be so. Nowadays school study media and the “literary canon” are beginning to find their way in… and all those tropes with silly names, built from mass media, are proving their usefulness as tools of analysis. Of course, getting a movie adaptation or a TV miniseries is one of the best ways to draw troper attention to a work, but classics always end up getting those with some regularity. So let’s just say that the user base has widened. Oh, and many classics are as doorstoppery as modern fantasy sagas, especially stuff from the XIXth century, when novels where published as long-running serials in magazines and authors were paid by the word. When people call Eliezer Yudkowsky a terrible writer because of MoR’s lack of tightness or his using it as a vehicle for ideas and lectures, I feel half-tempted to point at the likes of Victor Hugo or Alexandre Dumas or Dickens or Benito Pérez Galdós, just to name a few… surely if those are the traits of terrible writing, it means that those books have room for improvement, if only by way of abridging them?
Regarding LW: The filters the site imposes on its demographics (language and mode of interaction) worry me: what’s the point of translating MoR to Spanish or French, if afterwards hispanophones and francophones don’t get to read their sequences and have a site to be a community in? I’m trying to seed a rationalist community at my school, and the language barrier is proving to be troublesome; I often find myself not even knowing how a bias would be called in Spanish.
what’s the point of translating MoR to Spanish or French, if afterwards hispanophones and francophones don’t get to read their sequences and have a site to be a community in?
There have been a fewSequencestranslationprojects discussed, proposed, or started, in various languages, plus a few more for MoR. Of these, only one (diegocaleiro’s, in Portugese) seems aimed at a persistent community hub. I might have missed some, though.
With regard to the specific issue of bias names, if you don’t have access to cognitive science faculty, I think a good place to start might be finding a translation of Daniel Kahneman’s Thinking, Fast and Slow; Kahneman is a major source for the Sequences, his book’s been widely translated, and there should be a lot of overlap. After that, you might want to try digging into XiXiDu’s resource list. Though there are a few original LW coinages floating around, and I’m afraid you’re on your own with those.
Well thank goodness I’m on a holiday and can actually afford to read all that. Nevertheless, such a violent flipflop of opinions would make me dangerously closer to epistemic learned helplessness, i.e. giving up on learning new stuff because of not feeling confident that I can tell good arguments from bad ones when they come from experts after getting burned one time too many.
That is a dangerous state for a rationalist to be in. so would you please be a dear and have a look at that book to figure out how precisely it junk-scienced me and the rest of its readers? It would really help me out and I’d really be grateful for that.
Also, I have a vested interest in believing in the “only deliberate practice matters” thesis. I used to feel having a high IQ obliged me to get excellent results with little effort, and every time that didn’t happen I felt disappointed in myself and that I didn’t live up to some weird standard. Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.
That is a dangerous state for a rationalist to be in. so would you please be a dear and have a look at that book to figure out how precisely it junk-scienced me and the rest of its readers? It would really help me out and I’d really be grateful for that.
I’m afraid I haven’t read that exact popularization, but if it’s drawing on Ericsson’s research as it sounds like, the explanation is easy enough: Ericsson’s points are valid largely because the studies are correlational, do not control for underlying factors or Matthew effects, and suffer from heavy range restriction in he’s already looking at people who are selected or self-selected to be elites.
(ie. suppose someone studied MIT physicists with a mean IQ of 150 and discovered that in this group of physicists, Conscientiousness predicted better than IQ which would go on to win Nobels. This is a possible result, and what this has actually demonstrated is “you have to be incredibly brainy to be a MIT physicist in the first place, but once you’ve gotten that, then other things are also important; which is another way of saying that if we look at the general population, like all the people from IQ 60 to 150, IQ is the overwhelming most important trait” but this is easily popularized to “IQ doesn’t matter!”)
Also, I have a vested interest in believing in the “only deliberate practice matters” thesis. I used to feel having a high IQ obliged me to get excellent results with little effort, and every time that didn’t happen I felt disappointed in myself and that I didn’t live up to some weird standard. Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.
As a practical matter, it’s probably a good idea to believe hard work matters and deliberate practice matters. If you could somehow improve your intelligence, then it might be important to hold correct beliefs about IQ being far more important than practice; but unfortunately, IQ is pretty much fixed and all that’s left is to make the best use possible. Given people matched on IQ and other traits, and Conscientiousness will be pretty important.
However, in other contexts, it’s very important to hold the correct beliefs about the relative value of intelligence and ‘just work harder’ - if we were discussing iodization or immigration or whether someone should go into debt for college, for example. Many population-level questions will rest far more on intelligence than other traits.
Understanding which context we’re in can be a hard balancing act, and especially difficult when reading papers making statistical claims (did they control for IQ? Should they control? Or for education? Or for range restriction? Or for reliability of their metrics?); I try to be consistent and clear in any discussions of Conscientiousness or IQ which context we’re in and which we should value or ignore, but I don’t think I always succeed.
Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.
Imagine a world where 50% of your results are genetically determined and 50% of your results are hard work. What would be the best strategy for success in that world, assuming that you already have decent genes? It would be working hard. Not working 50% hard, but working 100% hard.
Seems like you found the right strategy for the wrong reasons. You can keep the strategy; you don’t have to blindly reverse your decisions.
I dunno, at a certain point the marginal utility of one unit of hard work will be less than the marginal utility of one unit of leisure, and it’s well possible that the point at which that happens depends on how genetically good you are.
You should work 100% hard on whatever you’re working on when you’re working on, but there might still be cases where you should think about the nature/nurture ratio to get the best outcome.
If outcomes are all about hard work, doggedly aiming for a rare high-reward position that requires a large amount of skill, like a quantitative analyst on Wall Street or a professional athlete, can be a good high risk / high reward strategy. But the more you know outcomes to be affected by genetic talent, the faster you’d want to recognize that some goals are beyond you and direct your 100% effort elsewhere if you find your genetic talent lacking, because then the people who also put in 100% effort but have more genetic talent than you will take all the positions no matter how much effort you put in.
I’d be curious to see that research that proves IQ test scores are correlated with workplace performance. All jobs or just some? How strongly correlated? How is workplace performance measured? Is it discussed in your book?
A good starting point might be this 2004 review by Schmidt & Hunter. It tabulates correlations of 0.23 to 0.58 between “general mental ability” and “supervisory ratings of job performance”, with the correlation increasing monotonically with a five-level rating of job complexity.
Edit: I should probably add Schmidt & Hunter’s note that those averages “are corrected for measurement error in the dependent variable and for range restriction but not for measurement error in the GMA measure; hence, these are estimates of operational validities, not construct-level correlations.”
I am willing to bet money (up to €100, I am a student) that these will not, in fact, be the best predictors, just as grades in college have been proven not to be good predictors of how well a Google employee does, IQ tests have been proven not to be a good predictor of how well a salesman does, and higher salaries have proven to, past a certain point, be detrimental to productivity for employees whose jobs require creativity, imagination or otherwise intellectual exertion.
My hypothesis is that a <10 min IQ test will be so inaccurate as to be useless, and that the current school system already favors people who do what they are told as they are told without getting too far ahead or behind, with the destructive and wasteful results we all know: they are the ones it grades best, and yet, time and again, tested against the workplace, those grades have proven little.
IQ test scores are massively correlated with workplace performance.
I find it plausible that among the set of people Google has hired grades don’t predict workplace success, but I bet if Google were to randomly select employees from among the U.S. population (or even from just among computer science majors) grades would be hugely correlated with performance.
How much are you betting?
I can cite my studies, if you want. It may take me some time to find them (an afternoon or two). Can you cite yours?
What are the terms of the bet? I would be willing to bet €100 depending on the terms. Full disclosure: I read a lot of academic research on IQ to help me write this book, one of whose chapters is about IQ.
There is a lot of silly stuff written about IQ. No offense, but do you think you have the needed background to separate the high from the low quality IQ scholarship?
I only read Talent is Overrated, which cited a few studies to the effect that IQ was a bad predictor of professional performance beyond the short term; to become a world-class performer at some specific skill, you need to spend 20000 hours of your life deliberately practicing it (trying things outside your comfort zone, failing to get it right, figuring out what to improve and how, getting it right, trying something harder, rinse and repeat, and it’s a pain in the neck). Most people, no matter how smart, tend to stop improving at their jobs after around one year and a half of practice: this includes highly skilled professionals like doctors and lawyers.
And, on the anecdotal side, in my own experience and what I could get from the people that frequent Lesswrong and TVTropes, a high IQ commonly results in huge childhood issues, a miserable social life, and a boatpload of akrasia, which seriously hamper whatever base utility it might have. Perhaps there’s a selection bias and only the incompetent brains have the time to hang out here, while competent smart people are too busy actually getting things done?
There is massive selection bias going on here.
I don’t think it has as much to do with free time as with target audience, though. LW attracts a few different clusters of people, but the ones you’re seeing in this context are those who feel their thinking is flawed in some way, and who believe they have a decent chance of fixing it with a cognitive science toolset and vocabulary. The site’s native idiom and interaction style—basically a founder effect—imposes a few more filters, underrepresenting some problems and overrepresenting others. Akrasia and social problems are precisely the issues I’d expect to see a lot of, given those constraints.
TV Tropes… well, that might have more to do with free time. Almost everyone likes media, but if you want to make many original contributions, you need unusual knowledge of media and an analytical attitude towards it. Moreover, the media best represented there tend to be the most time-consuming ones—TV, anime, doorstopper fantasy novels. I don’t think I need to go into too much detail regarding the people most likely to share those requirements.
Offtopic post, but a discussion I wish to pursue nonetheless:
Regarding TVT: it used to be so. Nowadays school study media and the “literary canon” are beginning to find their way in… and all those tropes with silly names, built from mass media, are proving their usefulness as tools of analysis. Of course, getting a movie adaptation or a TV miniseries is one of the best ways to draw troper attention to a work, but classics always end up getting those with some regularity. So let’s just say that the user base has widened. Oh, and many classics are as doorstoppery as modern fantasy sagas, especially stuff from the XIXth century, when novels where published as long-running serials in magazines and authors were paid by the word. When people call Eliezer Yudkowsky a terrible writer because of MoR’s lack of tightness or his using it as a vehicle for ideas and lectures, I feel half-tempted to point at the likes of Victor Hugo or Alexandre Dumas or Dickens or Benito Pérez Galdós, just to name a few… surely if those are the traits of terrible writing, it means that those books have room for improvement, if only by way of abridging them?
Regarding LW: The filters the site imposes on its demographics (language and mode of interaction) worry me: what’s the point of translating MoR to Spanish or French, if afterwards hispanophones and francophones don’t get to read their sequences and have a site to be a community in? I’m trying to seed a rationalist community at my school, and the language barrier is proving to be troublesome; I often find myself not even knowing how a bias would be called in Spanish.
There have been a few Sequences translation projects discussed, proposed, or started, in various languages, plus a few more for MoR. Of these, only one (diegocaleiro’s, in Portugese) seems aimed at a persistent community hub. I might have missed some, though.
With regard to the specific issue of bias names, if you don’t have access to cognitive science faculty, I think a good place to start might be finding a translation of Daniel Kahneman’s Thinking, Fast and Slow; Kahneman is a major source for the Sequences, his book’s been widely translated, and there should be a lot of overlap. After that, you might want to try digging into XiXiDu’s resource list. Though there are a few original LW coinages floating around, and I’m afraid you’re on your own with those.
That is junk science.
The correlation between IQ and workplace performance is extremely robust and very well established.
See Intelligence: Knowns and Unknowns
Well thank goodness I’m on a holiday and can actually afford to read all that. Nevertheless, such a violent flipflop of opinions would make me dangerously closer to epistemic learned helplessness, i.e. giving up on learning new stuff because of not feeling confident that I can tell good arguments from bad ones when they come from experts after getting burned one time too many.
That is a dangerous state for a rationalist to be in. so would you please be a dear and have a look at that book to figure out how precisely it junk-scienced me and the rest of its readers? It would really help me out and I’d really be grateful for that.
Also, I have a vested interest in believing in the “only deliberate practice matters” thesis. I used to feel having a high IQ obliged me to get excellent results with little effort, and every time that didn’t happen I felt disappointed in myself and that I didn’t live up to some weird standard. Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.
I’m afraid I haven’t read that exact popularization, but if it’s drawing on Ericsson’s research as it sounds like, the explanation is easy enough: Ericsson’s points are valid largely because the studies are correlational, do not control for underlying factors or Matthew effects, and suffer from heavy range restriction in he’s already looking at people who are selected or self-selected to be elites.
(ie. suppose someone studied MIT physicists with a mean IQ of 150 and discovered that in this group of physicists, Conscientiousness predicted better than IQ which would go on to win Nobels. This is a possible result, and what this has actually demonstrated is “you have to be incredibly brainy to be a MIT physicist in the first place, but once you’ve gotten that, then other things are also important; which is another way of saying that if we look at the general population, like all the people from IQ 60 to 150, IQ is the overwhelming most important trait” but this is easily popularized to “IQ doesn’t matter!”)
As a practical matter, it’s probably a good idea to believe hard work matters and deliberate practice matters. If you could somehow improve your intelligence, then it might be important to hold correct beliefs about IQ being far more important than practice; but unfortunately, IQ is pretty much fixed and all that’s left is to make the best use possible. Given people matched on IQ and other traits, and Conscientiousness will be pretty important.
However, in other contexts, it’s very important to hold the correct beliefs about the relative value of intelligence and ‘just work harder’ - if we were discussing iodization or immigration or whether someone should go into debt for college, for example. Many population-level questions will rest far more on intelligence than other traits.
Understanding which context we’re in can be a hard balancing act, and especially difficult when reading papers making statistical claims (did they control for IQ? Should they control? Or for education? Or for range restriction? Or for reliability of their metrics?); I try to be consistent and clear in any discussions of Conscientiousness or IQ which context we’re in and which we should value or ignore, but I don’t think I always succeed.
Well thank you very much!
.. sigh This may be the first time I find some actual use for compartmentalization, and I can’t do it anymore...
Imagine a world where 50% of your results are genetically determined and 50% of your results are hard work. What would be the best strategy for success in that world, assuming that you already have decent genes? It would be working hard. Not working 50% hard, but working 100% hard.
Seems like you found the right strategy for the wrong reasons. You can keep the strategy; you don’t have to blindly reverse your decisions.
I dunno, at a certain point the marginal utility of one unit of hard work will be less than the marginal utility of one unit of leisure, and it’s well possible that the point at which that happens depends on how genetically good you are.
You should work 100% hard on whatever you’re working on when you’re working on, but there might still be cases where you should think about the nature/nurture ratio to get the best outcome.
If outcomes are all about hard work, doggedly aiming for a rare high-reward position that requires a large amount of skill, like a quantitative analyst on Wall Street or a professional athlete, can be a good high risk / high reward strategy. But the more you know outcomes to be affected by genetic talent, the faster you’d want to recognize that some goals are beyond you and direct your 100% effort elsewhere if you find your genetic talent lacking, because then the people who also put in 100% effort but have more genetic talent than you will take all the positions no matter how much effort you put in.
See Hambrick et al, Deliberate practice: Is that all it takes to become an expert?, for a recent critical assessment of the claim that deliberate practice is sufficient for attaining expertise. Gwern provides some quotes.
I’d be curious to see that research that proves IQ test scores are correlated with workplace performance. All jobs or just some? How strongly correlated? How is workplace performance measured? Is it discussed in your book?
A good starting point might be this 2004 review by Schmidt & Hunter. It tabulates correlations of 0.23 to 0.58 between “general mental ability” and “supervisory ratings of job performance”, with the correlation increasing monotonically with a five-level rating of job complexity.
Edit: I should probably add Schmidt & Hunter’s note that those averages “are corrected for measurement error in the dependent variable and for range restriction but not for measurement error in the GMA measure; hence, these are estimates of operational validities, not construct-level correlations.”