First, I’m somewhat sympathetic to the idea of “be a nerd even if it doesn’t pay off in your lifetime because it will make society better”. That said, I think it’s all the more reason to be skeptical about the argument here.
Just to zoom in on one thing, I doubt the story you tell about the discovery of astronomy. As you say, it’s illustrative and not historical, but there is some historical and anthropological evidence about this, and I’m not sure your story is very believable against that evidence. Skimming the book at that link and knowing as a general prior that humans tend to overfit data (i.e. find patterns in everything), it would be surprising to me if it took some special person with special skills to produce the sort of basic astronomy you’re suggesting. If anything, I’d expect humans to rapidly reinvent astronomy based on noticing patterns in the stars and seeing them as representative of animals, plants, etc. if you wiped the memories of every human living today and asked them to start over from scratch.
You then go on to say some things about bravery and working on neglected topics that might turn out to be hits, but this seems to have nothing to do with neurodivergence other than a kind of backwards causation where neurodivergent folks are more likely to work on neglected things, but not because they are neglected but because the space of things one might care about is wide and someone who isn’t interested in popular things will most likely not accidentally land on caring about a thing that is incidentally popular.
So overall I’m not convinced that your arguments hold up for saying anything more than that there is value in being brave and working on neglected things if you have good reason to think they are important or because you are pursuing a high-variance strategy; the rest is just incidental.
Sometimes I get an impression that people on autistic spectrum “have outlived their usefulness” (TV Tropes) from the perspective of the society. There was a time, not that long ago, when normies didn’t care about computers, because using them required esoteric knowledge of things such as binary numbers, and they didn’t care about internet, because it was mostly a way to interact with people who cared about these esoteric things. To become good with computers, you had to spend a lot of time studying obsessively something that didn’t have much value in the eyes of most people.
Then it became common knowledge that IT is where the money is, and also working with computers became easier. Suddenly people with no intrinsic interest in esotetic knowledge started paying attention to IT. And now you have students of computer science who freely admit that they actually don’t like programming and consider it boring… but they are willing to do it for money (because presumably all other jobs are boring, too).
The weirdoes became a minority in the field they have created, and the social norms are turning against them. Caring about the craft already became low-status; if you care about clean code and algorithmic complexity, you are obviously not paying attention to the larger picture i.e. the buzzwords the management is most happy about recently. There are not enough resources to do anything properly (although there sometimes are resources to do the same thing over and over again as the old solutions keep falling apart under their technical debt). The social skills are more important than the technical ones. Even in open source people are kicked out of projects for being bad at political games.
Of course, there is a value in social skills, and there is a harm in excessive weirdness. People can have long unproductive wars about minutiae of formatting the source code. Lack of communication within the project can waste lot of resources. Documentation sucks when it is written by people who hate talking to others. Introducing social skills to the project should be good… if we could keep the balance. If the people with social skills could respect the people with technical skills, and vice versa. But it seems to me that after the initial resistance is broken, the pendulum swings to the opposite extreme, and suddenly we have a formerly nerdy profession where people are regularly reminded that nerds suck.
Normie-ness is a positive feedback loop; the more normies you have, the greater the pressure to eliminate the non-normies. People with better social skills will almost by definition succeed at pushing the narrative that what we really need is to give even more power to people with social skills. And when things start falling apart, instead of shutting up and fixing the code, more and more meetings are scheduled, because for a normie, talking endlessly is the preferred (and the only known) way to solve all problems.
To some degree, this is not as bad as it sounds. Software is easy to copy. You could have 99% of software projects completely dysfunctional, and the remaining 1% would still move the planet forward. Similarly, you can have million anti-vaxers, but as long as you have one Einstein, science can still move forward. One person doing the right thing is more important than millions wasting time, if the solution can be copied.
But ultimately, the resources are scarce, and the people pretending to care are competing against the people who actually care. When you get to the point where the Einstein can’t get a job, because he is outcompeted at every position by people with better social skills, then—unless he is independently wealthy (but how could he save for early retirement if he can’t get a good job?) or he has a generous sponsor (but here he also competes against people who have better social skills) -- he will not be able to work on his theory of relativity. And if only 1% of programmers care about clean code, you won’t get clean code in 1% of projects; it will be much less, because most projects are developed by teams, and you would need a majority of the team to actually care.
Lucky individuals often find something which has a long inference chain. Not a one-step improvement to existing theory, but many steps further. To the experts, it may look like a few new steps, combined masterfully. To the non-expert, they may need to learn about 100 new concepts for it to make sense. That’s why 100 non-experts can’t just invent general relativity, they need to take a 100 steps, 1 step each, but all in the same direction.
How many dimensions is inference space? How many duck-sized horses do we need, to have a 2⁄3 chance of taking those steps? And are they being modeled as duck-sized monkeys with typewriters, or are they closer to a proper mini-Einstein, who is likely to go the correct direction?
But we can build institutions that allow weirdos with strange obsessions to work on their obsessions.
I suspect this is far easier said than done and that for various reasons any such institution is unstable. Either it will fail to produce anything, in which case it will die out. Or it will become extremely successful and then get co-opted and by agents who enforce conformance norms. I don’t fully understand why this is the case, but I suspect it’s because conformance norms are an excellent way for people to coordinate, establish and maintain power.
First, I’m somewhat sympathetic to the idea of “be a nerd even if it doesn’t pay off in your lifetime because it will make society better”. That said, I think it’s all the more reason to be skeptical about the argument here.
Just to zoom in on one thing, I doubt the story you tell about the discovery of astronomy. As you say, it’s illustrative and not historical, but there is some historical and anthropological evidence about this, and I’m not sure your story is very believable against that evidence. Skimming the book at that link and knowing as a general prior that humans tend to overfit data (i.e. find patterns in everything), it would be surprising to me if it took some special person with special skills to produce the sort of basic astronomy you’re suggesting. If anything, I’d expect humans to rapidly reinvent astronomy based on noticing patterns in the stars and seeing them as representative of animals, plants, etc. if you wiped the memories of every human living today and asked them to start over from scratch.
You then go on to say some things about bravery and working on neglected topics that might turn out to be hits, but this seems to have nothing to do with neurodivergence other than a kind of backwards causation where neurodivergent folks are more likely to work on neglected things, but not because they are neglected but because the space of things one might care about is wide and someone who isn’t interested in popular things will most likely not accidentally land on caring about a thing that is incidentally popular.
So overall I’m not convinced that your arguments hold up for saying anything more than that there is value in being brave and working on neglected things if you have good reason to think they are important or because you are pursuing a high-variance strategy; the rest is just incidental.
Sometimes I get an impression that people on autistic spectrum “have outlived their usefulness” (TV Tropes) from the perspective of the society. There was a time, not that long ago, when normies didn’t care about computers, because using them required esoteric knowledge of things such as binary numbers, and they didn’t care about internet, because it was mostly a way to interact with people who cared about these esoteric things. To become good with computers, you had to spend a lot of time studying obsessively something that didn’t have much value in the eyes of most people.
Then it became common knowledge that IT is where the money is, and also working with computers became easier. Suddenly people with no intrinsic interest in esotetic knowledge started paying attention to IT. And now you have students of computer science who freely admit that they actually don’t like programming and consider it boring… but they are willing to do it for money (because presumably all other jobs are boring, too).
The weirdoes became a minority in the field they have created, and the social norms are turning against them. Caring about the craft already became low-status; if you care about clean code and algorithmic complexity, you are obviously not paying attention to the larger picture i.e. the buzzwords the management is most happy about recently. There are not enough resources to do anything properly (although there sometimes are resources to do the same thing over and over again as the old solutions keep falling apart under their technical debt). The social skills are more important than the technical ones. Even in open source people are kicked out of projects for being bad at political games.
Of course, there is a value in social skills, and there is a harm in excessive weirdness. People can have long unproductive wars about minutiae of formatting the source code. Lack of communication within the project can waste lot of resources. Documentation sucks when it is written by people who hate talking to others. Introducing social skills to the project should be good… if we could keep the balance. If the people with social skills could respect the people with technical skills, and vice versa. But it seems to me that after the initial resistance is broken, the pendulum swings to the opposite extreme, and suddenly we have a formerly nerdy profession where people are regularly reminded that nerds suck.
Normie-ness is a positive feedback loop; the more normies you have, the greater the pressure to eliminate the non-normies. People with better social skills will almost by definition succeed at pushing the narrative that what we really need is to give even more power to people with social skills. And when things start falling apart, instead of shutting up and fixing the code, more and more meetings are scheduled, because for a normie, talking endlessly is the preferred (and the only known) way to solve all problems.
To some degree, this is not as bad as it sounds. Software is easy to copy. You could have 99% of software projects completely dysfunctional, and the remaining 1% would still move the planet forward. Similarly, you can have million anti-vaxers, but as long as you have one Einstein, science can still move forward. One person doing the right thing is more important than millions wasting time, if the solution can be copied.
But ultimately, the resources are scarce, and the people pretending to care are competing against the people who actually care. When you get to the point where the Einstein can’t get a job, because he is outcompeted at every position by people with better social skills, then—unless he is independently wealthy (but how could he save for early retirement if he can’t get a good job?) or he has a generous sponsor (but here he also competes against people who have better social skills) -- he will not be able to work on his theory of relativity. And if only 1% of programmers care about clean code, you won’t get clean code in 1% of projects; it will be much less, because most projects are developed by teams, and you would need a majority of the team to actually care.
Lucky individuals often find something which has a long inference chain. Not a one-step improvement to existing theory, but many steps further. To the experts, it may look like a few new steps, combined masterfully. To the non-expert, they may need to learn about 100 new concepts for it to make sense. That’s why 100 non-experts can’t just invent general relativity, they need to take a 100 steps, 1 step each, but all in the same direction.
How many dimensions is inference space? How many duck-sized horses do we need, to have a 2⁄3 chance of taking those steps? And are they being modeled as duck-sized monkeys with typewriters, or are they closer to a proper mini-Einstein, who is likely to go the correct direction?
It seems like Patreon is sort-of fixing this, at least when the thing people are obsessed with is interesting enough.
I suspect this is far easier said than done and that for various reasons any such institution is unstable. Either it will fail to produce anything, in which case it will die out. Or it will become extremely successful and then get co-opted and by agents who enforce conformance norms. I don’t fully understand why this is the case, but I suspect it’s because conformance norms are an excellent way for people to coordinate, establish and maintain power.