Prediction Thread: Make Predictions About How Different Factors Affect AGI X-Risk.
In this post you can make several predictions for how different factors affect the probability that the creation of AGI leads to an extinction level catastrophe. This might be useful for planning.
Please let me know if you have other ideas for questions that could be valuable to ask.
Predictions based on who develops AGI:
Predictions based on technology used for developing AGI:
Prediction based on approach for creating AGI:
Predictions on how money affects probability of AGI X-risk:
I’m moderately confident that money would just make things worse, but there’s no option for increase in catastrophe risk from increased donations.
Sadly I could only create questions between 1-99 for some reason, I guess we should interpret 1% to mean 1% or less (including negative).
What makes you think more money would be net negative?
Do you think that it would also be negative if you had 100% of how the money was spent, or would it only apply if other AI Alignment researchers were responsible for the strategy to donate?
I think more money spent right now, even with the best of intentions, is likely to increase capabilities much faster than it reduces risk. I think OpenAI and consequent capability races are turning out to be an example of this.
There are hypothetical worlds where spending an extra ten billion (or a trillion) dollars on AI research with good intentions doesn’t do this, but I don’t think they’re likely to be our world. I don’t think that directing who gets the money is likely to prevent it, without pretty major non-monetary controls in addition.
I do agree that OpenAI is an example of good intentions going wrong, however I think we could learn from that and top researchers would be vary of such risks.
Nevertheless I do think your concerns are valid and is important not to dismiss.
Disclaimer: my answers are wild guesses & poorly calibrated, I spent ~5s on each of them.
The odds of any particular AGI destroying the world are << than the odds of AGI as a whole destroying the world, so the questions in “Prediction based on approach for creating AGI:” may be miscalibrated.
I interpreted that question as a conditional probability, and so not required to be strictly less. In particular, P(X causes catastrophe | X was of the given type, and the first AGI developed, on the given date).
Nothing requires that this conditional probability should be less than “some AGI destroys the world conditional on any AGI being developed”.
Excellent point.
I do think that the first AGI developed will have a big effect on the probability of doom, so hopefully it will be some value possible to derive from the question. But it would be interesting to control for what other AIs do, in order to get better calibrated statistics.