I like to start by trying to find one author who has excellent thinking and see what they cite — this works for both papers and books with bibliographies, but increasingly other forms of media.
For instance, Dan Carlin of the (exceptional and highly recommended) Hardcore History podcast cites all the sources he uses when he does a deep investigation of a historical era, which is a good jumping-off point if you want to go deep.
The hard part is finding that first excellent thinker, especially in a domain where you can’t differentiate quality in a field yet. But there’s some general conventions of how smart thinkers tend to write and reason that you can learn to spot. There’s a certain amount of empathy, clarity, and — for lack of a better word — “good aesthetics” that, if they’re present, the author tends to be smart and trustworthy.
The opposite isn’t necessarily the case — there are good thinkers who don’t follow those practices and are hard to follow (say, Laozi or Wittgenstein maybe) — but when those factors are present, I tend to weight the thinking well.
Even if you have no technical background at all, this piece by Paul Graham looks credible (emphasis added) —
“What does addn look like in C? You just can’t write it.
You might be wondering, when does one ever want to do things like this? Programming languages teach you not to want what they cannot provide. You have to think in a language to write programs in it, and it’s hard to want something you can’t describe. When I first started writing programs—in Basic—I didn’t miss recursion, because I didn’t know there was such a thing. I thought in Basic. I could only conceive of iterative algorithms, so why should I miss recursion?
If you don’t miss lexical closures (which is what’s being made in the preceding example), take it on faith, for the time being, that Lisp programmers use them all the time. It would be hard to find a Common Lisp program of any length that did not take advantage of closures. By page 112 you will be using them yourself.”
When I spot that level of empathy/clarity/aesthetics, I think, “Ok, this person likely knows what they’re talking about.”
So, me, I start by looking for someone like Paul Graham or Ray Dalio or Dan Carlin, and then I look at who they cite and reference when I want to go deeper.
My experience is that readability doesn’t translate much to quality and might even be negatively correlated, because reality is messy and simplifications are easier to read. I do think works that make themselves easy to double check are probably higher quality on average, but haven’t rigorously tested this.
It’s hard to nail down; it’d probably be a very long essay to even try.
And it’s not a perfect predictor, alas — just evidence.
But I believe there’s a certain way to spot “good reasoning” and “having thoroughly worked out the problem” from one’s writing. It’s not the smoothness of the words, nor the simplicity.
I’s hard to describe, but it seems somewhat consistently recognizable. Yudkowsky has it, incidentally.
It seems like your approach would work well in fields like programming. It’s a practical skill with a lot of people working in it and huge amounts of money at stake to figure out best practices. Plus, the issue he’s addressing doesn’t seem to be controversial.
Outside that safe zone, prose quality isn’t a proxy for the truth. And I think it’s these issues that Elizabeth’s worried about.
For example, how many windows are there in your house? If you wanted to answer that question without getting out of your chair, you’d probably form a mental image of the house, then “walk around” and count up the windows.
Reading their diametrically opposed papers on the same topic, I’m sure I couldn’t tell who’s right based on their prose. It’s formal academic writing, and the issue is nuanced.
I like to start by trying to find one author who has excellent thinking and see what they cite — this works for both papers and books with bibliographies, but increasingly other forms of media.
For instance, Dan Carlin of the (exceptional and highly recommended) Hardcore History podcast cites all the sources he uses when he does a deep investigation of a historical era, which is a good jumping-off point if you want to go deep.
The hard part is finding that first excellent thinker, especially in a domain where you can’t differentiate quality in a field yet. But there’s some general conventions of how smart thinkers tend to write and reason that you can learn to spot. There’s a certain amount of empathy, clarity, and — for lack of a better word — “good aesthetics” that, if they’re present, the author tends to be smart and trustworthy.
The opposite isn’t necessarily the case — there are good thinkers who don’t follow those practices and are hard to follow (say, Laozi or Wittgenstein maybe) — but when those factors are present, I tend to weight the thinking well.
Even if you have no technical background at all, this piece by Paul Graham looks credible (emphasis added) —
https://sep.yimg.com/ty/cdn/paulgraham/acl1.txt?t=1593689476&
When I spot that level of empathy/clarity/aesthetics, I think, “Ok, this person likely knows what they’re talking about.”
So, me, I start by looking for someone like Paul Graham or Ray Dalio or Dan Carlin, and then I look at who they cite and reference when I want to go deeper.
My experience is that readability doesn’t translate much to quality and might even be negatively correlated, because reality is messy and simplifications are easier to read. I do think works that make themselves easy to double check are probably higher quality on average, but haven’t rigorously tested this.
The quality I’m describing isn’t quite “readability” — it overlaps, but that’s not quite it.
Feynman has it —
http://www.faculty.umassd.edu/j.wang/feynman.pdf
It’s hard to nail down; it’d probably be a very long essay to even try.
And it’s not a perfect predictor, alas — just evidence.
But I believe there’s a certain way to spot “good reasoning” and “having thoroughly worked out the problem” from one’s writing. It’s not the smoothness of the words, nor the simplicity.
I’s hard to describe, but it seems somewhat consistently recognizable. Yudkowsky has it, incidentally.
It seems like your approach would work well in fields like programming. It’s a practical skill with a lot of people working in it and huge amounts of money at stake to figure out best practices. Plus, the issue he’s addressing doesn’t seem to be controversial.
Outside that safe zone, prose quality isn’t a proxy for the truth. And I think it’s these issues that Elizabeth’s worried about.
For example, how many windows are there in your house? If you wanted to answer that question without getting out of your chair, you’d probably form a mental image of the house, then “walk around” and count up the windows.
At least, that’s what the picture theorists think. Others think there’s some other process underlying this cognition, perhaps linguistic in nature.
Reading their diametrically opposed papers on the same topic, I’m sure I couldn’t tell who’s right based on their prose. It’s formal academic writing, and the issue is nuanced.