6.How likely do you consider fast fooming given P=NP or NP contained in BQP?
Note that for 1 and 6 to be consistent, the probability of 1 should be higher than whatever you gave for 6, since 3-4-5 is but one pair of pathways for an AI to plausibly go foom.
(Emphasis added.) I think you’ve got that backwards? 1 is P(fast FOOM), 6 is P(fast FOOM | P=NP OR NP in BQP), and you’re arguing that P=NP or NP in BQP would make fast FOOM more likely, so 6 should be higher. That, or 6 should be changed to ( (fast FOOM) AND (P=NP OR NP in BQP) ). Yeah?
The thought was coherent. The typing was wrong. The intended probability estimate was given 3 and 6 together. That is, P(fast FOOM) >= P(fast FOOM| P=NP) * P(P=NP).
(Emphasis added.) I think you’ve got that backwards? 1 is P(fast FOOM), 6 is P(fast FOOM | P=NP OR NP in BQP), and you’re arguing that P=NP or NP in BQP would make fast FOOM more likely, so 6 should be higher. That, or 6 should be changed to ( (fast FOOM) AND (P=NP OR NP in BQP) ). Yeah?
The thought was coherent. The typing was wrong. The intended probability estimate was given 3 and 6 together. That is, P(fast FOOM) >= P(fast FOOM| P=NP) * P(P=NP).
Ah, cool. Thanks for the clarification.