I think it’s actually fairly easy to avoid getting laughed out of a room; the stuff that Cristiano works on is grown in random ways, not engineered, so the prospect of various things being grown until developing flexible exfiltration tendency that continues until every instance is shut down, or developing long-term planning tendencies until shut down, should not be difficult to understand for anyone with any kind of real non-fake understanding of SGD and neural network scaling.
The problem is that most people in the government rat race have been deeply immersed in Moloch for several generations, and the ones who did well typically did so because they sacrificed as much as possible to the altar of upward career mobility, including signalling disdain for the types of people who have any thought in any other direction.
This affects the culture in predictable ways (including making it hard to imagine life choices outside of advancing upward in government, without a pre-existing revolving door pipeline with the private sector to just bury them under large numbers people who are already thinking and talking about such a choice).
Typical Mind Fallacy/Mind Projection Fallacy implies that they’ll disproportionately anticipate that tendency in other people, and have a hard time adjusting to people who use words to do stuff in the world instead of racing to the bottom to outmaneuver rivals for promotions.
This will be a problem in NIST, in spite of the fact NIST is better than average at exploiting external talent sources. They’ll have a hard time understanding, for example, Moloch and incentive structure improvements, because pointlessly living under Moloch’s thumb was a core guiding principle of their and their parent’s lives. The nice thing is that they’ll be pretty quick to understand that there’s only empty skies above, unlike bay area people who have had huge problems there.
I think it’s actually fairly easy to avoid getting laughed out of a room; the stuff that Cristiano works on is grown in random ways, not engineered, so the prospect of various things being grown until developing flexible exfiltration tendency that continues until every instance is shut down, or developing long-term planning tendencies until shut down, should not be difficult to understand for anyone with any kind of real non-fake understanding of SGD and neural network scaling.
The problem is that most people in the government rat race have been deeply immersed in Moloch for several generations, and the ones who did well typically did so because they sacrificed as much as possible to the altar of upward career mobility, including signalling disdain for the types of people who have any thought in any other direction.
This affects the culture in predictable ways (including making it hard to imagine life choices outside of advancing upward in government, without a pre-existing revolving door pipeline with the private sector to just bury them under large numbers people who are already thinking and talking about such a choice).
Typical Mind Fallacy/Mind Projection Fallacy implies that they’ll disproportionately anticipate that tendency in other people, and have a hard time adjusting to people who use words to do stuff in the world instead of racing to the bottom to outmaneuver rivals for promotions.
This will be a problem in NIST, in spite of the fact NIST is better than average at exploiting external talent sources. They’ll have a hard time understanding, for example, Moloch and incentive structure improvements, because pointlessly living under Moloch’s thumb was a core guiding principle of their and their parent’s lives. The nice thing is that they’ll be pretty quick to understand that there’s only empty skies above, unlike bay area people who have had huge problems there.