I recently published an article in the Georgetown Security Studies Review (GSSR) on the use of “argument management systems” (e.g., Kialo) for the complex debates that arise in fields where it’s often impractical to resolve disagreements through standard empirical methods (e.g., RCTs). I’ve long been confused why this method of discussion is not more widely supported in EA and Rationalist circles, and am considering adapting my GSSR article to AI policy/safety research (or Longtermism more generally, which various people have criticized for being too speculative/theoretical rather than being based on empirical tests) and posting it here. However, before doing that I would love to get a sense of people’s reasons for skepticism or apathy towards such methods, so I could potentially address them in the post.
For what it’s worth, I have seen Leverage Research’s report on the topic, and I am aware of the criticism that “argument mapping” (in some formats) is overly formal and too complicated. (I plan to respond to these points)
In short, I expect my argument to be fairly similar to what I laid out in my GSSR article: the way we currently present arguments (i.e., predominantly through prose/paragraph text) seems rife with points of failure and inefficiencies, especially given how debates are often not linear but rather branch and have cross-cutting points. In fields like international relations and peace/conflict studies I’ve repeatedly encountered instances where people fail to (seriously) address existing counterarguments, and more generally it is hard for audiences to determine who has or hasn’t addressed counterarguments. In contrast, I think that making one’s arguments more explicit and keeping track of the arguments in a more-searchable and more-permanent format than “memory” or prose would help to mitigate some of these problems.
To me, better methods of argumentation seems like a natural extension of norms that promote statistics/experimental methods in science, but thus far I’ve found the EA/Rationalist communities fairly lukewarm towards the ideas (even if they are more receptive on average than the general public).
Argument mapping systems exist , in the sense that code for them exists, but have repeatedly suffered from lack of quality content.
When I was considering whether to try to make one happen the main reason was that users would need to treat it like learning a new language. That makes it rather high barriers and high payoff.
The other thing was that it would be murky whether “lets just gather more data / user activity” is done enough or will never get to a critical point. Narrowness makes it so that it actually gets used but makes it less exciting as a thing to have.
I really like the idea of such an argument mapping system. In the early days of wikipedia I thought it might be a natural extension of the wikipedia project to have a debate-plattform attatched to it. So its interesting for me to see that a platform like Kialo exists—I didnt know about it before your post, thanks.
I hold the same opinion as you do, that many arguments are very complex and when you try to get a good impression of the state of the debate it is a complicated task, because its hard to find good, exhaustive and well structured debates. And a platform that systematizes this important task is very valueable for people and society in my opinion.