On the other hand, that does not mean that those fields are going to recognize [a smart outsider] as a thought-leader or whatever.
Could this be a more general problem that people in academia are very unlikely to recognize anything that originates outside of academia?
I mean, first you get your degree by listening to people in academia, and reading textbooks written by people in academia. Then you learn to solve problems, defined by people in academia, under the leadership of people in academia, mostly by reading a lot of papers written and reviewed by people in academia. Etc. I think it is very easy to forget that the universe outside of academia exists at all.
And even if you remember that the universe outside of academia exists, interacting with it is potentially dangerous for your reputation. If you cite something from an academically respected source, then… even if it later turns out to be completely wrong, it kinda wasn’t your mistake, was it? Compare to the situation where you cite something from a smart outsider… who also turns out to be completely wrong. Now you look like an idiot… and you can’t shift the blame to him (because he is outside the system) nor to his reviewers (because either he had none, or they were also outside the system).
Similarly, suppose that two respected scientists have expressed a similar idea, and you prefer one of the formulations over the other. No problem; just cite the one you prefer. Now suppose that a respected scientist wrote something about a topic, but a smart outsider wrote about something slightly more relevant for your specific case and/or he expressed it better. Citing the outsider instead of the respected scientist… will make you seem like you are not familiar with the existing research in your field. (Chances are, the respected scientist or his colleague will review your paper.)
So it’s part familiarity, part incentives.
Epistemic status: just a guess; I am not working in academia, so I only imagine what the incentives are
.
Now assuming that this is true, or at least pointing in the right direction, the problem seems solvable, but it would require some work. For example, you can’t expect people in academia to go outside and read your research on their own initiative. But maybe if you publish it as a book that is easy to read, and donate them the book? Or maybe you could academia-wash your research by making someone inside the system a co-author.
Perhaps you could do both of these at the same time, like publish a popular science book that would contain articles both from LW researchers and academic researchers. And then the book would kinda be okay to cite.
From my experience, professors are very open to discuss ideas and their work with anyone who seems serious, interested, and knowledgeable. Even someone inside academia will face skepticism if their work uses completely different methods. They will have to very convincingly prove the methods are valid.
I agree, but there are two different perspectives:
whether the outsider wants to be taken seriously by academia
whether the people in academia want to collect knowledge efficiently
From the first perspective, of course, if you want to be taken seriously, you need to play by their rules. And if you don’t, then… those are your revealed preferences, I guess.
It is the second perspective I was concerned about. I agree that the outsiders are often wrong. But, consider the tweet you linked:
If you never published your research but somehow developed it into a product, you might die rich. But you’ll still be a bit bitter and largely forgotten.
It seems to me that from the perspective of a researcher, taking ideas of the outsiders who have already developed successful products based on them, and examining them scientifically (and maybe rejecting them afterwards), should be a low-hanging fruit.
I am not suggesting to treat the ideas of the outsiders as scientific. I am suggesting to treat them as “hypotheses worth examining”.
Refusing to even look at a hypothesis because it is not scientifically proven yet, that’s putting the cart before the horse. Hypotheses are considered first, scientifically proved later; not the other way round. All scientific theories were non-scientific hypotheses first, at the moment they were conceived.
Choosing the right hypothesis to examine, is an art. Not a science yet; that is what it becomes after we examine it. In theory, any (falsifiable) hypothesis could be examined scientifically, and afterwards confirmed or rejected. In practice, testing completely random hypotheses would be a waste of time; they are 99.9999% likely to be wrong, and if you don’t find at least one that is right, your scientific career is over. (You won’t become famous by e.g. testing million random objects and scientifically confirming that none of them defies gravity. Well, you probably would become famous actually, but in the bad way.)
From the Bayesian perspective, what you need to do is test hypotheses that have a non-negligible prior probability of being correct. From the perspective of the truth-seeker, that’s because both the success and the (more likely) failure contribute non-negligibly to our understanding of the world. From the perspective of a scientific career-seeker, because finding the correct one is the thing that is rewarded. The incentives are almost aligned here.
I think that the opinions of smart outsiders have maybe 10% probability of being right, which makes them hypotheses worth examining scientifically. (The exact number would depend on what kind of smart outsiders are we talking about here.) Even if 10% right is still 90% wrong. Why do I claim that 10% is a good deal? Because when you look at the published results (the actual “Science according to the checklist”) that passed the p=0.05 threshold… and later half of them failed to replicate… then the math says that their prior probability was less than 10%.
(Technically, with prior probability 10%, and 95% chance of a wrong hypothesis being rejected, out of 1000 original hypotheses, 100 would be correct and published, 900 would be incorrect and 45 of them published. Which means, out of 145 published scientific findings, only about a third would fail to replicate.)
So we have a kind of motte-and-bailey situation here. The motte is that opinions of smart outsiders, no matter how popular, now matter how commercially successful, should not be treated as scientific. The bailey is that the serious researchers should not even consider them seriously as hypotheses; in other words that their prior probability is significantly lower than 10% (because hypotheses with prior probability about 10% are actually examined by serious researchers all the time).
And what I suggest here is that maybe the actual problem is not that the hypotheses of smart and successful outsiders are too unlikely, but rather that exploring hypotheses with 10% prior probability is a career-advancing move if those hypotheses originate within academia, but a career-destroying move if they originate outside of it. With the former, you get a 10% chance of successfully publishing a true result (plus a 5% chance of successfully publishing a false result), and 85% chance of being seen as a good scientist who just wasn’t very successful so far. With the latter, you get a 90% chance of being seen as a crackpot.
Returning to Yann LeCun’s tweet… if you invent some smart ideas outside of academia, and you build a successful product out of them, but the academia refuses to even look at them because the ideas are now coded as “non-scientific” and anyone who treats them seriously would lose their academic status… and therefore we will never have those ideas scientifically confirmed or rejected… that’s not just a loss for you, for also for the science.
In entrepreneurship, there is the phrase “ideas are worthless”. This is because everyone already has lots of ideas they believe are promising. Hence, a pre-business idea is unlikely to be stolen.
Similarly, every LLM researcher already has a backlog of intriguing hypotheses paired with evidence. So an outside idea would have to seem more promising than the backlog. Likely this will require the proposer to prove something beyond evidence.
For example, Krizhevsky/Sutskever/Hinton had the idea of applying then-antiquated neural nets to recognize images. Only when they validated this in the ImageNet competition did their idea attract more researchers.
This is why ideas/hypotheses—even with a bit of evidence—are not considered very useful. What would be useful is to conclusively prove an idea true. This would attract lots of researchers … but it turns out to be incredibly difficult to do, and in some cases requires sophisticated techniques. (The same applies to entrepreneurship. Few people will join/invest until you validate the idea and lower the risk.)
Sidenote: There are countless stories of academics putting forth original, non-mainstream ideas, only to be initially rejected by their peers (e.g. Cantor’s infinity). I believe this not to be an issue just with outsiders, but merely that extraordinary claims require extraordinary proof. ML is an interesting example, because lots of so-called outsiders without a PhD now present at conferences!
Could this be a more general problem that people in academia are very unlikely to recognize anything that originates outside of academia?
I mean, first you get your degree by listening to people in academia, and reading textbooks written by people in academia. Then you learn to solve problems, defined by people in academia, under the leadership of people in academia, mostly by reading a lot of papers written and reviewed by people in academia. Etc. I think it is very easy to forget that the universe outside of academia exists at all.
And even if you remember that the universe outside of academia exists, interacting with it is potentially dangerous for your reputation. If you cite something from an academically respected source, then… even if it later turns out to be completely wrong, it kinda wasn’t your mistake, was it? Compare to the situation where you cite something from a smart outsider… who also turns out to be completely wrong. Now you look like an idiot… and you can’t shift the blame to him (because he is outside the system) nor to his reviewers (because either he had none, or they were also outside the system).
Similarly, suppose that two respected scientists have expressed a similar idea, and you prefer one of the formulations over the other. No problem; just cite the one you prefer. Now suppose that a respected scientist wrote something about a topic, but a smart outsider wrote about something slightly more relevant for your specific case and/or he expressed it better. Citing the outsider instead of the respected scientist… will make you seem like you are not familiar with the existing research in your field. (Chances are, the respected scientist or his colleague will review your paper.)
So it’s part familiarity, part incentives.
Epistemic status: just a guess; I am not working in academia, so I only imagine what the incentives are
.
Now assuming that this is true, or at least pointing in the right direction, the problem seems solvable, but it would require some work. For example, you can’t expect people in academia to go outside and read your research on their own initiative. But maybe if you publish it as a book that is easy to read, and donate them the book? Or maybe you could academia-wash your research by making someone inside the system a co-author.
Perhaps you could do both of these at the same time, like publish a popular science book that would contain articles both from LW researchers and academic researchers. And then the book would kinda be okay to cite.
If an outsider’s objective is to be taken seriously, they should write papers and submit them to peer review (e.g. conferences and journals).
Yann LeCun has gone so far to say that independent work only counts as “science” if submitted to peer review:
“Without peer review and reproducibility, chances are your methodology was flawed and you fooled yourself into thinking you did something great.”—https://x.com/ylecun/status/1795589846771147018?s=19.
From my experience, professors are very open to discuss ideas and their work with anyone who seems serious, interested, and knowledgeable. Even someone inside academia will face skepticism if their work uses completely different methods. They will have to very convincingly prove the methods are valid.
I agree, but there are two different perspectives:
whether the outsider wants to be taken seriously by academia
whether the people in academia want to collect knowledge efficiently
From the first perspective, of course, if you want to be taken seriously, you need to play by their rules. And if you don’t, then… those are your revealed preferences, I guess.
It is the second perspective I was concerned about. I agree that the outsiders are often wrong. But, consider the tweet you linked:
It seems to me that from the perspective of a researcher, taking ideas of the outsiders who have already developed successful products based on them, and examining them scientifically (and maybe rejecting them afterwards), should be a low-hanging fruit.
I am not suggesting to treat the ideas of the outsiders as scientific. I am suggesting to treat them as “hypotheses worth examining”.
Refusing to even look at a hypothesis because it is not scientifically proven yet, that’s putting the cart before the horse. Hypotheses are considered first, scientifically proved later; not the other way round. All scientific theories were non-scientific hypotheses first, at the moment they were conceived.
Choosing the right hypothesis to examine, is an art. Not a science yet; that is what it becomes after we examine it. In theory, any (falsifiable) hypothesis could be examined scientifically, and afterwards confirmed or rejected. In practice, testing completely random hypotheses would be a waste of time; they are 99.9999% likely to be wrong, and if you don’t find at least one that is right, your scientific career is over. (You won’t become famous by e.g. testing million random objects and scientifically confirming that none of them defies gravity. Well, you probably would become famous actually, but in the bad way.)
From the Bayesian perspective, what you need to do is test hypotheses that have a non-negligible prior probability of being correct. From the perspective of the truth-seeker, that’s because both the success and the (more likely) failure contribute non-negligibly to our understanding of the world. From the perspective of a scientific career-seeker, because finding the correct one is the thing that is rewarded. The incentives are almost aligned here.
I think that the opinions of smart outsiders have maybe 10% probability of being right, which makes them hypotheses worth examining scientifically. (The exact number would depend on what kind of smart outsiders are we talking about here.) Even if 10% right is still 90% wrong. Why do I claim that 10% is a good deal? Because when you look at the published results (the actual “Science according to the checklist”) that passed the p=0.05 threshold… and later half of them failed to replicate… then the math says that their prior probability was less than 10%.
(Technically, with prior probability 10%, and 95% chance of a wrong hypothesis being rejected, out of 1000 original hypotheses, 100 would be correct and published, 900 would be incorrect and 45 of them published. Which means, out of 145 published scientific findings, only about a third would fail to replicate.)
So we have a kind of motte-and-bailey situation here. The motte is that opinions of smart outsiders, no matter how popular, now matter how commercially successful, should not be treated as scientific. The bailey is that the serious researchers should not even consider them seriously as hypotheses; in other words that their prior probability is significantly lower than 10% (because hypotheses with prior probability about 10% are actually examined by serious researchers all the time).
And what I suggest here is that maybe the actual problem is not that the hypotheses of smart and successful outsiders are too unlikely, but rather that exploring hypotheses with 10% prior probability is a career-advancing move if those hypotheses originate within academia, but a career-destroying move if they originate outside of it. With the former, you get a 10% chance of successfully publishing a true result (plus a 5% chance of successfully publishing a false result), and 85% chance of being seen as a good scientist who just wasn’t very successful so far. With the latter, you get a 90% chance of being seen as a crackpot.
Returning to Yann LeCun’s tweet… if you invent some smart ideas outside of academia, and you build a successful product out of them, but the academia refuses to even look at them because the ideas are now coded as “non-scientific” and anyone who treats them seriously would lose their academic status… and therefore we will never have those ideas scientifically confirmed or rejected… that’s not just a loss for you, for also for the science.
In entrepreneurship, there is the phrase “ideas are worthless”. This is because everyone already has lots of ideas they believe are promising. Hence, a pre-business idea is unlikely to be stolen.
Similarly, every LLM researcher already has a backlog of intriguing hypotheses paired with evidence. So an outside idea would have to seem more promising than the backlog. Likely this will require the proposer to prove something beyond evidence.
For example, Krizhevsky/Sutskever/Hinton had the idea of applying then-antiquated neural nets to recognize images. Only when they validated this in the ImageNet competition did their idea attract more researchers.
This is why ideas/hypotheses—even with a bit of evidence—are not considered very useful. What would be useful is to conclusively prove an idea true. This would attract lots of researchers … but it turns out to be incredibly difficult to do, and in some cases requires sophisticated techniques. (The same applies to entrepreneurship. Few people will join/invest until you validate the idea and lower the risk.)
Sidenote: There are countless stories of academics putting forth original, non-mainstream ideas, only to be initially rejected by their peers (e.g. Cantor’s infinity). I believe this not to be an issue just with outsiders, but merely that extraordinary claims require extraordinary proof. ML is an interesting example, because lots of so-called outsiders without a PhD now present at conferences!