savvy don’t (just) have more skills to extract signal from a “naturally” occurring source of lies. They’re collaborating with it!
This (from your tweet) is false, and your post here even has a straightforward argument against it (with the honest cop and the corrupt cop both savvy enough to discern a lightly veiled bribery attempt). Savviness about extracting information from a source does not imply complicity with the source.
Trying to make this more intuitive: consider a prediction market which is currently priced at x, where each share will pay out $1 if it resolves as True.
If you think it’s underpriced because your probability is y, where y>x, then your subjective EV from buying a share is y-x. e.g., If it’s priced at $0.70 and you think p=0.8, your subjective EV from buying a share is $0.10.
If you think it’s overpriced because your probability is z, where z<x, then your subjective EV from selling a share is x-z. e.g., If it’s priced at $0.70 and you think p=0.56, your subjective EV from selling a share is $0.14.
Those two will be equal if x is halfway between y and z, at their arithmetic mean.
So if two people disagree on whether the price should be y or z, then they will have equal EV by setting a price at the arithmetic mean of y & z, and trading some number of prediction market shares at that price. i.e., The fair (equal subjective EV) betting odds are at the arithmetic mean of their probabilities.
This is a bet at 30% probability, as 42.86/142.86 = .30001.
That is the average of Alice’s probability and Bob’s probability. The fair bet according to equal subjective EV is at the average of the two probabilities; previous discussion here.
I don’t buy the way that Spencer tried to norm the ClearerThinking test. It sounds like he just assumed that people who took their test and had a college degree as their highest level of education had the same IQ as the portion of the general population with the same educational level, and similarly for all other education levels. Then he used that to scale how scores on the ClearerThinking test correspond to IQs. That seems like a very strong and probably inaccurate assumption.
Much of what this post and Scott’s post are using the ClearerThinking IQ numbers for relies on this norming.
It occurs to me that the ClearerThinking data provides a way to check this assumption. It included data from 2 different groups, crowdworkers and people in Spencer’s social network. If college-degree-level crowdworkers did just as well on the ClearerThinking test as college-degree-level people in Spencer’s network, then it becomes more plausible that both did about as well as college-degree-level people in the general population would have. Whereas if the college-degree-level crowdworkers and Spencer’s network people scored differently, then obviously they can’t both match the college-degree-level general population, so there’d be an open question about how the groups compare and direct evidence against the accuracy of Spencer’s method of norming the test.
I think that the way that Scott estimated IQ from SAT is flawed, in a way that underestimates IQ, for reasons given in comments like this one. This post kept that flaw.
“Can crimes be discussed literally?”:
some kinds of hypocrisy (the law and medicine examples) are normalized
these hypocrisies are / the fact of their normalization is antimemetic (OK, I’m to some extent interpolating this one based on familiarity with Ben’s ideas, but I do think it’s both implied by the post, and relevant to why someone might think the post is interesting/important)
the usage of words like ‘crime’ and ‘lie’ departs from their denotation, to exclude normalized things
people will push back in certain predictable ways on calling normalized things ‘crimes’/‘lies’, related to the function of those words as both description and (call for) attack
“There is a clear conflict between the use of language to punish offenders, and the use of language to describe problems, and there is great need for a language that can describe problems. For instance, if I wanted to understand how to interpret statistics generated by the medical system, I would need a short, simple way to refer to any significant tendency to generate false reports. If the available simple terms were also attack words, the process would become much more complicated.”
Does it bother you that this is not what’s happening in many of the examples in the post? e.g., With “the American hospital system is built on lies.”
This post reads like it’s trying to express an attitude or put forward a narrative frame, rather than trying to describe the world.
Many of these claims seem obviously false, if I take them at face value at take a moment to consider what they’re claiming and whether it’s true.
e.g., On the first two bullet points it’s easy to come up with counterexamples. Some successful attempts to steer the future, by stopping people from doing locally self-interested & non-violent things, include: patent law (“To promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right to their respective writings and discoveries”) and banning lead in gasoline. As well as some others that I now see that other commenters have mentioned.
In America, people shopped at Walmart instead of local mom & pop stores because it had lower prices and more selection, so Walmart and other chain stores grew and spread while lots of mom & pop stores shut down. Why didn’t that happen in Wentworld?
I made a graph of this and the unemployment rate, they’re correlated at r=0.66 (with one data point for each time Gallup ran the survey, taking the unemployment rate on the closest day for which there’s data). You can see both lines spike with every recession.
Are you telling me 2008 did actual nothing?
It looks like 2008 led to about a 1.3x increase in the number of people who said they were dissatisfied with their life.
It’s common for much simpler Statistical Prediction Rules, such as linear regression or even simpler models, to outperform experts even when they were built to predict the experts’ judgment.
Or “Defense wins championships.”
With the ingredients he has, he has gotten a successful Barkskin Potion:
1 of the 1 times (100%) he brewed together Crushed Onyx, Giant’s Toe, Ground Bone, Oaken Twigs, Redwood Sap, and Vampire Fang.
19 of the 29 times (66%) he brewed together Crushed Onyx, Demon Claw, Ground Bone, and Vampire Fang.
Only 2 other combinations of the in-stock ingredients have ever produced Barkskin Potion, both at under a 50% rate (4/10 and 18⁄75).
The 4-ingredient, 66% success rate potion looks like the best option if we’re just going to copy something that has worked. That’s what I’d recommend if I had to make the decision right now.
Many combinations that used currently-missing ingredients reliably (100%) produced Barkskin Potion many times (up to 118⁄118). There may be a variant on one of those, which he has never tried, that could work better than 66% of the time using ingredients that he has. Or there may be information in there about the reliability of the 6-ingredient combination which worked once.
Being Wrong on the Internet: The LLM generates a flawed forum-style comment, such that the thing you’ve been wanting to write is a knockdown response to this comment, and you can get a “someone”-is-wrong-on-the-internet drive to make the points you wanted to make. You can adjust how thoughtful/annoying/etc. the wrong comment is.
Target Audience Personas: You specify the target audience that your writing is aimed at, or a few different target audiences. The LLM takes on the persona of a member of that audience and engages with what you’ve written, with more explicit explanation of how that persona is reacting and why than most actual humans would give. The structure could be like comments on google docs.
Heat Maps: Color the text with a heat map of how interested the LLM expects the reader to be at each point in the text, or how confused, how angry, how amused, how disagreeing, how much they’re learning, how memorable it is, etc. Could be associated with specific target audiences.
I don’t think that the key element in the aging example is ‘being about value claims’. Instead, it’s that the question about what’s healthy is a question that many people wonder about. Since many people wonder about that question, some people will venture an answer. Even if humanity hasn’t yet built up enough knowledge to have an accurate answer.
Thousands of years ago many people wondered what the deal is with the moon and some of them made up stories about this factual (non-value) question whose correct answer was beyond them. And it plays out similarly these days with rumors/speculation/gossip about the topics that grab people’s attention. Where curiosity & interest exceeds knowledge, speculation will fill the gaps, sometimes taking on a similar presentation to knowledge.
Note the dynamic in your aging example: when you’re in a room with 5+ people and you mention that you’ve read a lot about aging, someone asks the question about what’s healthy. No particular answer needs to be memetic because it’s the question that keeps popping up and so answers will follow. If we don’t know a sufficiently good/accurate/thorough answer then the answers that follow will often be bullshit, whether that’s a small number of bullshit answers that are especially memetically fit or whether it’s a more varied and changing froth of made-up answers.
There are some kinds of value claims that are pretty vague and floaty, disconnected from entangled truths and empirical constraints. But that is not so true of instrumental claims about things like health, where (e.g.) the claim that smoking causes lung cancel is very much empirical & entangled. You might still see a lot of bullshit about these sorts of instrumental value claims, because people will wonder about the question even if humanity doesn’t have a good answer. It’s useful to know (e.g.) what foods are healthy, so the question of what foods are healthy is one that will keep popping up when there’s hope that someone in the room might have some information about it.
7% of the variance isn’t negligible. Just look at the pictures (Figure 1 in the paper):
I got the same result: DEHK.
I’m not sure that there are no patterns in what works for self-taught architects, and if we were aiming to balance cost & likelihood of impossibility then I might look into that more (since I expect A,L,N to be the the cheapest options with a chance to work), but since we’re prioritizing impossibility I’ll stick with the architects with the competent mentors.
As I understand it, Scott’s post was making basically the same conceptual distinction as this Andrew Gelman post, where Gelman writes:
Scott labels the first two of Gelman’s categories “clueless” and the third “savvy”.