Context: I wrote a post about how “Graham’s Design Paradox” (though I didn’t use that phrase) is a major economic bottleneck, especially once you have a lot of money.
I generally buy the point of this post, though I’d frame it differently. Your bullet points correspond to heuristics like:
use a red team
do experimental tests
check that someone saying the thing has been done correlates with the thing actually having been done
watch out for Goodharting
watch out for principal-agent problems
reflect on the selection pressures induced by your feedback loop
These sorts of heuristics are themselves a class of knowledge/skill. It’s a type of knowledge/skill which generalizes across many domains, and gives us some ability to recognize expertise across many domains.
But I wouldn’t call these sorts of heuristics “general abilities, resources, and motives”; they’re more narrow than that. They all fit a certain pattern. What I’d call these sorts of heuristics is “rationality skills”.
Context: I wrote a post about how “Graham’s Design Paradox” (though I didn’t use that phrase) is a major economic bottleneck, especially once you have a lot of money.
I generally buy the point of this post, though I’d frame it differently. Your bullet points correspond to heuristics like:
use a red team
do experimental tests
check that someone saying the thing has been done correlates with the thing actually having been done
watch out for Goodharting
watch out for principal-agent problems
reflect on the selection pressures induced by your feedback loop
These sorts of heuristics are themselves a class of knowledge/skill. It’s a type of knowledge/skill which generalizes across many domains, and gives us some ability to recognize expertise across many domains.
But I wouldn’t call these sorts of heuristics “general abilities, resources, and motives”; they’re more narrow than that. They all fit a certain pattern. What I’d call these sorts of heuristics is “rationality skills”.