Are successes at the IMO a reliable and objective measure of the skills you need?
Well, of course, this is a major filter for intelligence, creativity and plain math “basic front kick” proficiency. However, you also exclude lots and lots of people who may possess the necessary skills but did not choose to enter IMO, were not aware of it, had other personal commitments or simply procrastinated too much etc. Also, the skills IMO medalists have acquired so far are in no way a guarantee that more skills will follow, and may be the result of good teachers or enthusiastic parents. Let me present you some weak evidence, behold, a personal anecdote!
On behalf of my chemistry teacher and owed to my school’s policy in general, I entered one of these sciency olympiads a while ago. I procrastinated over the first round, which was a homework assignment, got it in just in time not quite completed, but was allowed to the next round. From there on, I made it to the national team and won a medal at the international competition. Looking back on what I did back then, I’d say the questions were quite easy, not at a level I’d call requiring serious skill. Of course I’ve continued to learn much since then, but it could’ve gone another way. I would not say such medal winners are overly altruistic or more determined to their cause than others are (the two of my teammates that I know about are now studying medicine, “to make money” as they told me).
I’d conclude that if you want bright mathematicians, you might be well off taking IMO medalists. But if you want people who are also trustworthy, altruistic and deeply committed to AGI, let alone especially rational, you should probably widen the filter for intelligence and math skills (if IMO medals are a good measure of this) a lot. Perhaps ask Mensa, take applications and filter from those, reach out to the best 1% on some college entry test or something. Just wild, uneducated guesses. But focussing on IMO medalists, which should give you less than 500 potential candidates a year, doesn’t sound like a good strategy. More so since those people are approached not only by you but also by quite a few companies.
Perhaps I misunderstood what you mean with “at medal-winning level”, but since you’re doing this SPARC camp and I can’t think of another way of attracting such people, I assumed you were reaching out to people who actually compete.
More so since those people are approached not only by you but also by quite a few companies.
Unless we have different understandings of what it means to be ‘approached’ this statement clashes quite strongly with my experience in the year since I won my IMO silver.
Another data point: I have a gold IMO medal (only 4 people in my country ever had it), and in the following 20 years I never had a phone call or an e-mail saying: “We have this project where we need math talents, and I saw your name online, so if you are interested I would like to tell you more details.”
No one cares. Seems to me the only predictor companies use is: “Did you do the same kind of work at some previous company? How many years?” and then the greater number wins. (Which effectively means that one is paid for their age and their ability to pick the right technology when they finish university and stick with it. Kinda depressing.)
EDIT: As a sidenote, I did not use my math skills significantly since university, so in my case the IMO medal is probably not so good predictor now.
No one cares. Seems to me the only predictor companies use is: “Did you do the same kind of work at some previous company? How many years?” and then the greater number wins. (Which effectively means that one is paid for their age and their ability to pick the right technology when they finish university and stick with it. Kinda depressing.)
Cynic. You’re neglecting the influence of a pretty face, good clothes, and a bit of charm!
I have been pretty continuously approached throughout my four years of undergrad. Probably this also has a good deal to do with social connections formed during contest participation, not just the contest performance itself.
It probably has something to do with the fact that Paul Christiano was a silver medalist. (Paul is currently just a Research Associate, but I think he is much more involved with SI than the others. See here for recent discussion of some of his work.)
If people know of stronger predictors of raw math ability than IMO performance, Putnam performance, and early-age 800 on Math SAT, I’d like to know what they are.
This is generally not an indicator available for young people (and I think it’s reasonable for MIRI recruiting efforts to target young people), and when it is, it isn’t obviously a good idea to use. There are research opportunities available at the high school and undergraduate level, but they are not universally available, and I have been told by people who sit on graduate admissions committees that most of the research that gets produced by these opportunities is bad. Among the research that is not bad, I think it’s likely to be unclear to what extent quality of the work is due to the student’s efforts or the program / advisor’s. (I have heard rumors that in one particular such program, the advisor plots out the course of the research in advance and leads students through it as something like a series of guided exercises.) Edit: I worded that somewhat poorly; I didn’t mean to suggest that the work has been done by someone other than the student, but a nontrivial portion of the success of a research project at this level is due to the careful selection of a research problem and careful guidance on the part of the program / advisor, which is not what we want to select for.
By contrast, tests like the AMC (which leads to the IMO in the US), the Putnam, and the SAT are widely available and standardized.
The fact that you are looking for “raw” math ability seems questionable. If their most recent achievements are IMO/SAT, you’re looking at high schoolers or early undergrads (Putnam winners have their tickets punched at top grad schools and will be very hard to recruit). Given that, you’ll have at least a 5-10 year lag while they continue learning enough to do basic research.
The IMO/IOI and qualification processes for them seem to be useful as early indicators of general intelligence; they obviously don’t capture everyone or even a huge fraction of all comparably smart people, but they seem to have fewer false positives by far than almost any other external indicators until research careers begin in earnest.
We used contests heavily in the screening process for SPARC in part for this reason, and in part because there is a community surrounding contests which the SPARC instructors understand and have credibility with, and which looks like it could actually benefit from exposure to (something like) rationality, which seems like an awesome opportunity.
“IMO medal-winning level” is (I presume) intended to refer to a level of general intelligence / affinity for math. As I said, the majority of people at this level don’t in fact have IMO medals, and some IMO medalists aren’t at this level. The fact that this descriptor gets used, instead of something like “top 0.01%”, probably comes down to a combination of wanting to avoid precision (both about what is being measured and how high the bar is), and wanting to use a measure which reflects well on the current state of affairs. There may be similar, but I expect (and hope) much smaller, effects on thinking in addition to talking.
I don’t think I’ve had a large effect on the way SI folks view possible researchers. I don’t know how large an effect I’ve had on the way Luke talks about possible researchers.
Also note: I’m a very marginal IMO medalist. People analogous to me don’t have medals in most nearby possible worlds.
What’s your evidence that you’re a marginal IMO medalist?
I only ask because I’ve noticed that my perception of a person’s actual ability and my perception of their ego seem to be negatively correlated among the people I’ve met, including Less Wrong users. For example, I once met a guy at a party who told me he wasn’t much of a coder; next semester he left undergrad to be the CTO of a highly technical Y Combinator startup.
This is part of the reason why I’m a little skeptical of SI’s of telling people “send us an e-mail if you did well on the Putnam”—I would guess a large fraction of those who did well on the Putnam think they did well by pure luck. (Imposter syndrome.) SI might be better off trying to collect info on everyone who thinks they might want to work on FAI, no matter how untalented, and judge relative competence for themselves instead of letting FAI contributor wannabes judge themselves. (Or at least specify a score above which one should definitely contact them, regardless of how lucky one feels one got.)
One of the main effects of illusory superiority in IQ is the Downing effect. This describes the tendency of people with a below average IQ to overestimate their IQ, and of people with an above average IQ to underestimate their IQ.
(I personally am a very good example of this, because although I think I’m not terribly bright, I am in fact a genius.)
For what it’s worth, I have no maths (well, at best I had Maths A level and now I have simple calculus), but my experience from university was that it seemed that most of those selected to study maths (at a maths-oriented college in Cambridge, so this is highly selective) were Maths Olympiads. So it’s obviously not just a Singularity Institute thing. In the case of unis, I suspect it’s used largely because the general academic qualifications don’t differentiate enough for the best people. I suppose the question is whether it remains a useful indicator later on in life.
The weirder question is why they think the potential donors shouldn’t apply same sort of criteria to the sellers of the AI risk reduction. Say, you want photo-realistic ray traced photon mapped mmorpg. The hardware doesn’t support this, but I’m sure there’s a few startups making something like that, talking to investors, drafting plans how they must hire topcoder and other programming contest winners and optics experts… hell I myself have been approached by such folks at least 4 times that I remember, with ‘job offers’. The common feature of such startups? They aren’t founded by awesome tech guys themselves, because those tech guys not only are tech guys but also have much better grasp of the big-picture issues and see that its too early.
What is with you guys and the math olympiad?
Are successes at the IMO a reliable and objective measure of the skills you need?
Well, of course, this is a major filter for intelligence, creativity and plain math “basic front kick” proficiency. However, you also exclude lots and lots of people who may possess the necessary skills but did not choose to enter IMO, were not aware of it, had other personal commitments or simply procrastinated too much etc. Also, the skills IMO medalists have acquired so far are in no way a guarantee that more skills will follow, and may be the result of good teachers or enthusiastic parents. Let me present you some weak evidence, behold, a personal anecdote!
On behalf of my chemistry teacher and owed to my school’s policy in general, I entered one of these sciency olympiads a while ago. I procrastinated over the first round, which was a homework assignment, got it in just in time not quite completed, but was allowed to the next round. From there on, I made it to the national team and won a medal at the international competition. Looking back on what I did back then, I’d say the questions were quite easy, not at a level I’d call requiring serious skill. Of course I’ve continued to learn much since then, but it could’ve gone another way. I would not say such medal winners are overly altruistic or more determined to their cause than others are (the two of my teammates that I know about are now studying medicine, “to make money” as they told me).
I’d conclude that if you want bright mathematicians, you might be well off taking IMO medalists. But if you want people who are also trustworthy, altruistic and deeply committed to AGI, let alone especially rational, you should probably widen the filter for intelligence and math skills (if IMO medals are a good measure of this) a lot. Perhaps ask Mensa, take applications and filter from those, reach out to the best 1% on some college entry test or something. Just wild, uneducated guesses. But focussing on IMO medalists, which should give you less than 500 potential candidates a year, doesn’t sound like a good strategy. More so since those people are approached not only by you but also by quite a few companies.
Perhaps I misunderstood what you mean with “at medal-winning level”, but since you’re doing this SPARC camp and I can’t think of another way of attracting such people, I assumed you were reaching out to people who actually compete.
Unless we have different understandings of what it means to be ‘approached’ this statement clashes quite strongly with my experience in the year since I won my IMO silver.
Another data point: I have a gold IMO medal (only 4 people in my country ever had it), and in the following 20 years I never had a phone call or an e-mail saying: “We have this project where we need math talents, and I saw your name online, so if you are interested I would like to tell you more details.”
No one cares. Seems to me the only predictor companies use is: “Did you do the same kind of work at some previous company? How many years?” and then the greater number wins. (Which effectively means that one is paid for their age and their ability to pick the right technology when they finish university and stick with it. Kinda depressing.)
EDIT: As a sidenote, I did not use my math skills significantly since university, so in my case the IMO medal is probably not so good predictor now.
Cynic. You’re neglecting the influence of a pretty face, good clothes, and a bit of charm!
I’m an IOI silver medalist and have been approached many times by companies, mostly quantitative finance / tech start ups.
Interesting. If I may ask, how many of those were within the first year?
I have been pretty continuously approached throughout my four years of undergrad. Probably this also has a good deal to do with social connections formed during contest participation, not just the contest performance itself.
I said “the IMO medal-winning level.”
IMO performance isn’t the only metric which strongly predicts elite math ability.
It probably has something to do with the fact that Paul Christiano was a silver medalist. (Paul is currently just a Research Associate, but I think he is much more involved with SI than the others. See here for recent discussion of some of his work.)
No, we were IMO fetishists before we met Paul.
If people know of stronger predictors of raw math ability than IMO performance, Putnam performance, and early-age 800 on Math SAT, I’d like to know what they are.
Past achievement in original math research, of course.
This is generally not an indicator available for young people (and I think it’s reasonable for MIRI recruiting efforts to target young people), and when it is, it isn’t obviously a good idea to use. There are research opportunities available at the high school and undergraduate level, but they are not universally available, and I have been told by people who sit on graduate admissions committees that most of the research that gets produced by these opportunities is bad. Among the research that is not bad, I think it’s likely to be unclear to what extent quality of the work is due to the student’s efforts or the program / advisor’s. (I have heard rumors that in one particular such program, the advisor plots out the course of the research in advance and leads students through it as something like a series of guided exercises.) Edit: I worded that somewhat poorly; I didn’t mean to suggest that the work has been done by someone other than the student, but a nontrivial portion of the success of a research project at this level is due to the careful selection of a research problem and careful guidance on the part of the program / advisor, which is not what we want to select for.
By contrast, tests like the AMC (which leads to the IMO in the US), the Putnam, and the SAT are widely available and standardized.
The fact that you are looking for “raw” math ability seems questionable. If their most recent achievements are IMO/SAT, you’re looking at high schoolers or early undergrads (Putnam winners have their tickets punched at top grad schools and will be very hard to recruit). Given that, you’ll have at least a 5-10 year lag while they continue learning enough to do basic research.
Yes. So? During that time, you can get them interested in rationality and x-risk.
The IMO/IOI and qualification processes for them seem to be useful as early indicators of general intelligence; they obviously don’t capture everyone or even a huge fraction of all comparably smart people, but they seem to have fewer false positives by far than almost any other external indicators until research careers begin in earnest.
We used contests heavily in the screening process for SPARC in part for this reason, and in part because there is a community surrounding contests which the SPARC instructors understand and have credibility with, and which looks like it could actually benefit from exposure to (something like) rationality, which seems like an awesome opportunity.
“IMO medal-winning level” is (I presume) intended to refer to a level of general intelligence / affinity for math. As I said, the majority of people at this level don’t in fact have IMO medals, and some IMO medalists aren’t at this level. The fact that this descriptor gets used, instead of something like “top 0.01%”, probably comes down to a combination of wanting to avoid precision (both about what is being measured and how high the bar is), and wanting to use a measure which reflects well on the current state of affairs. There may be similar, but I expect (and hope) much smaller, effects on thinking in addition to talking.
I don’t think I’ve had a large effect on the way SI folks view possible researchers. I don’t know how large an effect I’ve had on the way Luke talks about possible researchers.
Also note: I’m a very marginal IMO medalist. People analogous to me don’t have medals in most nearby possible worlds.
What’s your evidence that you’re a marginal IMO medalist?
I only ask because I’ve noticed that my perception of a person’s actual ability and my perception of their ego seem to be negatively correlated among the people I’ve met, including Less Wrong users. For example, I once met a guy at a party who told me he wasn’t much of a coder; next semester he left undergrad to be the CTO of a highly technical Y Combinator startup.
This is part of the reason why I’m a little skeptical of SI’s of telling people “send us an e-mail if you did well on the Putnam”—I would guess a large fraction of those who did well on the Putnam think they did well by pure luck. (Imposter syndrome.) SI might be better off trying to collect info on everyone who thinks they might want to work on FAI, no matter how untalented, and judge relative competence for themselves instead of letting FAI contributor wannabes judge themselves. (Or at least specify a score above which one should definitely contact them, regardless of how lucky one feels one got.)
Less Wrong post on mathematicians and status:
http://lesswrong.com/lw/2vb/vanity_and_ambition_in_mathematics/
IAWYC, and so does Wikipedia:
(I personally am a very good example of this, because although I think I’m not terribly bright, I am in fact a genius.)
Sure, but with the obsession as the antecedent.
Seriously. What is with the IMO fetish?
For what it’s worth, I have no maths (well, at best I had Maths A level and now I have simple calculus), but my experience from university was that it seemed that most of those selected to study maths (at a maths-oriented college in Cambridge, so this is highly selective) were Maths Olympiads. So it’s obviously not just a Singularity Institute thing. In the case of unis, I suspect it’s used largely because the general academic qualifications don’t differentiate enough for the best people. I suppose the question is whether it remains a useful indicator later on in life.
The weirder question is why they think the potential donors shouldn’t apply same sort of criteria to the sellers of the AI risk reduction. Say, you want photo-realistic ray traced photon mapped mmorpg. The hardware doesn’t support this, but I’m sure there’s a few startups making something like that, talking to investors, drafting plans how they must hire topcoder and other programming contest winners and optics experts… hell I myself have been approached by such folks at least 4 times that I remember, with ‘job offers’. The common feature of such startups? They aren’t founded by awesome tech guys themselves, because those tech guys not only are tech guys but also have much better grasp of the big-picture issues and see that its too early.