I overall agree with this comment, but do want to push back on this sentence. I don’t really know what it means to “invent AI governance” or “invent AI strategy”, so I don’t really know what it means to “reinvent AI governance” or “reinvent AI strategy”.
By reinventing it, I means, for example, asking questions like “how to influence the dynamic between AI labs in a way which allows everyone to slow down at critical stage”, “can we convince some actors about AI risk without the main effect being they will put more resources into the race”, “what’s up with China”, “how to generally slow down things when necessary” and similar, and attempts to answer them.
I do agree that reading a lot of policy papers is of limited direct use in terms of direct hypothesis forming: in my experience the more valuable parts often have the form of private thoughts or semi-privately shared thinking.
On the other hand… in my view, if people have a decent epistemic base, they often should engage with the stuff you dislike, but from a proper perspective: not “this is the author attempting to literally communicate what they believe”, but more of “this is a written speech-act which probably makes some sense and has some purpose”. In other words… people who want to work on strategy unfortunately eventually need to be able to operate in epistemically hostile environments. They should train elsewhere, and spent enough time elsewhere to stay sane, but they need to understand e.g. how incentive landscapes influence what people think and write, and this is not possible to get good without dipping your feet in the water.
By reinventing it, I means, for example, asking questions like “how to influence the dynamic between AI labs in a way which allows everyone to slow down at critical stage”, “can we convince some actors about AI risk without the main effect being they will put more resources into the race”, “what’s up with China”, “how to generally slow down things when necessary” and similar, and attempts to answer them.
I do agree that reading a lot of policy papers is of limited direct use in terms of direct hypothesis forming: in my experience the more valuable parts often have the form of private thoughts or semi-privately shared thinking.
On the other hand… in my view, if people have a decent epistemic base, they often should engage with the stuff you dislike, but from a proper perspective: not “this is the author attempting to literally communicate what they believe”, but more of “this is a written speech-act which probably makes some sense and has some purpose”. In other words… people who want to work on strategy unfortunately eventually need to be able to operate in epistemically hostile environments. They should train elsewhere, and spent enough time elsewhere to stay sane, but they need to understand e.g. how incentive landscapes influence what people think and write, and this is not possible to get good without dipping your feet in the water.