My impression of the proposed idea is to create a “hard” intellectual accountability system for intellectuals by sampling some “falsifiable” subspace of idea space, similar to what exists for superpredictors and athletes. This certainly seems helpful in some areas, and I think is similar to the purpose of politifact.
But then there’s the risk of falling into the Marxist mistake: that if something isn’t quantifiable or “hard” it is not useful. The idea that “hard” production matters (farmers, factory workers, etc.) while “soft” production (merchants, market researchers, entertainers) does not, which is at the base of Marxism, has been disgraced by modern economics. But this is kind of hard to explain, and Marxism seemed “obviously correct” to the proto-rationalists of the early 20th century.
The sphere of intellectual expertise, especially the even “softer” side of people who’ve taken it on themselves to digest ideas for the public, is much harder still to get right than economics. No matter how many parameters an analysis like this tries to take into account, it is likely to miss something important. While I like the idea of using something like this on the margin, to boost lower-status people who get an unusually high number of things right or flag people who are obviously full of it, I think it would be a bad idea to use something like this to replace existing systems.
My impression of the proposed idea is to create a “hard” intellectual accountability system for intellectuals by sampling some “falsifiable” subspace of idea space, similar to what exists for superpredictors and athletes. This certainly seems helpful in some areas, and I think is similar to the purpose of politifact.
But then there’s the risk of falling into the Marxist mistake: that if something isn’t quantifiable or “hard” it is not useful. The idea that “hard” production matters (farmers, factory workers, etc.) while “soft” production (merchants, market researchers, entertainers) does not, which is at the base of Marxism, has been disgraced by modern economics. But this is kind of hard to explain, and Marxism seemed “obviously correct” to the proto-rationalists of the early 20th century.
The sphere of intellectual expertise, especially the even “softer” side of people who’ve taken it on themselves to digest ideas for the public, is much harder still to get right than economics. No matter how many parameters an analysis like this tries to take into account, it is likely to miss something important. While I like the idea of using something like this on the margin, to boost lower-status people who get an unusually high number of things right or flag people who are obviously full of it, I think it would be a bad idea to use something like this to replace existing systems.