the way to get the AGI doom argument off the ground with the least amount of contingent propositions
Well, if you’re really shooting for the least, you’ve already found the structure: just frame the argument for non-doom in terms of a lot of conjunctions (you have to build AI that we understand how to give inputs to, and also you have to solve various coordination and politics problems, and also you have to be confident that it won’t have bugs, and also you have to solve various philosophical problems about moral progress, etc.), and make lots of independent arguments for why non-doom won’t pan out. This body of arguments will touch on lots of assumptions, but very few will actually be load-bearing. (Or if you actually wrote things in Aristotelian logic, it would turn out that there were few contingent propositions, but some of those would be really long disjunctions.)
By actually making arguments you sidestep flaw #1 (symmetry), and somewhat lessen #3 (difficulty of understanding conditional probabilities). But #2 remains in full force, which is a big reason why this doesn’t lead to One Objectively Best Argument Everyone Agrees On.
(Other big reasons include the fact that arguing from less information isn’t always good—when you know more contingent facts about the world you can make better predictions—and counting number of propositions isn’t always a good measure of amount of information assumed, often leaving people disagreeing about which premises are “actually simpler”)
Well, if you’re really shooting for the least, you’ve already found the structure: just frame the argument for non-doom in terms of a lot of conjunctions (you have to build AI that we understand how to give inputs to, and also you have to solve various coordination and politics problems, and also you have to be confident that it won’t have bugs, and also you have to solve various philosophical problems about moral progress, etc.), and make lots of independent arguments for why non-doom won’t pan out. This body of arguments will touch on lots of assumptions, but very few will actually be load-bearing. (Or if you actually wrote things in Aristotelian logic, it would turn out that there were few contingent propositions, but some of those would be really long disjunctions.)
By actually making arguments you sidestep flaw #1 (symmetry), and somewhat lessen #3 (difficulty of understanding conditional probabilities). But #2 remains in full force, which is a big reason why this doesn’t lead to One Objectively Best Argument Everyone Agrees On.
(Other big reasons include the fact that arguing from less information isn’t always good—when you know more contingent facts about the world you can make better predictions—and counting number of propositions isn’t always a good measure of amount of information assumed, often leaving people disagreeing about which premises are “actually simpler”)