It occurs to me that the world could benefit from more affirmative fact checker. Existing fact checkers are appropriately rude to people who publicly make false claims, but there’s not much in the way of celebration of people who make difficult true claims. For example, Politifact awards “Pants on Fire” for bald lies, but only “True” for bald truths. I think there should be an even higher-status classification for true claims that run counter to the interests of the speaker. For example, we could award “Bayesian Stars” to figures who publicly update on new evidence, or “Bullets Bitten” to public figures who promulgate true evidence that weakens their arguments.
AABoyles
Karma: 494
It occurs to me that “Following one’s passion” is terrible advice at least in part because of the lack of diversity in the activities we encourage children to pursue. It follows that encouraging children to participate in activities with very high-competition job markets (e.g. sports, the arts) may be a substantial drag on economic growth. After 5 minutes of search, I could not find research on this relationship. (It seems the state of scholarship on the topic is restricted to models in which participation in extracurriculars early in childhood leads to better metrics later in childhood.) This may merit a more careful assessment.
Attention Conservation Warning: I envision a model which would demonstrate something obvious, and decide the world probably wouldn’t benefit from its existence.
The standard publication bias is that we must be 95% certain a described phenomenon exists before a result is publishable (at which time it becomes sufficiently “confirmed” to treat the phenomenon as a factual claim). But the statistical confidence of a phenomenon conveys interesting and useful information regardless of what that confidence is.
Consider the space of all possible relationships: most of these are going to be absurd (e.g. the relationship between number of minted pennies and number of atoms in moons of Saturn), and exhibit no correlation. Some will exhibit weak correlations (in the range of p = 0.5). Those are still useful evidence that a pathway to a common cause exists! The universal prior on random relationships should be roughly zero, because most relationships will be absurd.
What would science look like if it could make efficient use of the information disclosed by presently unpublishable results? I think I can generate a sort of agent-based model to imagine this. Here’s the broad outline:
Create a random DAG representing some complex related phenomena.
Create an agent which holds beliefs about the relationship between nodes in the graph, and updates its beliefs when it discovers a correlation with p > 0.95.
Create a second agent with the same belief structure, but which updates on every experiment regardless of the correlation.
On each iteration have each agent select two nodes in the graph, measure their correlation, and update their beliefs. Then have them compute the DAG corresponding to their current belief matrix. Measure the difference between the DAG they output and the original DAG created in step 1.
I believe that both agents will converge on the correct DAG, but the un-publication-biased agent will converge much more rapidly. There are a bunch of open parameters that need careful selection and defense here. How do the properties of the original DAG affect the outcome? What if agents can update on a relationship multiple times (e.g. run a test on 100 samples, then on 10,000)?
Given defensible positions on these issues, I suspect that such a model would demonstrate that publication bias reduces scientific productivity by roughly an order of magnitude (and perhaps much more).
But what would the point be? No one will be convinced by such a thing.
AABoyles’s Shortform
Please please please make this happen!
Me too!
Definitely 2. If you start messing with reality, things get really boring really quickly.
Thank you! I don’t have a good way to test Apple products (so the fix won’t be quick), but I’ll look into it.
Thanks for letting me know. I use [Calibre](https://calibre-ebook.com/about) to test the files, and it opens the file without complaint. What are you using (and on what platform) to read it?
My pleasure!
Also See Also: Simulate and defer to more rational selves.
I have converted Rationality Abridged to EPUB and MOBI formats. The code to accomplish this is stored in this repository.
I’m on it!
- A roundworm has been uploaded to a Lego body (http://edition.cnn.com/2015/01/21/tech/mci-lego-worm/index.html)
This happened in 2015.
Another Case Study of Inadequate Equilibrium in Medicine
The Wikipedia page on Resurrection contains some scattershot content (including links the H+Pedia article lacks) which might be useful to assimilate.
The fronted plugin worked when I submitted my comment, but I’m getting the “refresh to render LaTex” message as well. Neither refreshes nor fresh browser sessions seem to yield latex.
It’s time to test the Grue Hypothesis! Anyone have some Emeralds handy?