When I think about takeoffs, I notice that I’m less interested in GDP or how fast the AI’s cognition improves, and more on how AI will affect me, and how quickly. More plainly, how fast will shit go crazy for me, and how does that change my ability to steer events?
For example, assume unipolarity. Let architecture Z be the architecture which happens to be used to train the AGI.
How long is the duration between “architecture Z is published / seriously considered” and “the AGI kills me, assuming alignment fails”?
How long do I have, in theory, to learn about this architecture and its inductive biases, before I counterfactually die?
How many non-AGI experiments will the alignment community run on architecture Z, keeping in mind that we won’t actually know ahead of time that this is the one?
These questions are important, or would be if I had a sufficiently sharply defined research agenda such that knowing the answer to (3) would change my current time allocation (e.g. if (3) were particularly short, I’d try a bit harder to understand the principles which govern which architectures have which inductive biases, instead of getting good at studying a single architecture). I think (3) wouldn’t actually help me right now because I’m not in fact that precise. But in theory, it’s decision-relevant.
Seems like e.g. intelligence explosion microeconomic considerations become relevant as subquestions for answering the above decision-relevant questions, but not because those microecon questions are my main focus of inquiry.
EDIT: Seems that Paul Christiano happened to write a takeoff speed comment just before I wrote this one; my comment is not a reply to his.
When I think about takeoffs, I notice that I’m less interested in GDP or how fast the AI’s cognition improves, and more on how AI will affect me, and how quickly. More plainly, how fast will shit go crazy for me, and how does that change my ability to steer events?
For example, assume unipolarity. Let architecture Z be the architecture which happens to be used to train the AGI.
How long is the duration between “architecture Z is published / seriously considered” and “the AGI kills me, assuming alignment fails”?
How long do I have, in theory, to learn about this architecture and its inductive biases, before I counterfactually die?
How many non-AGI experiments will the alignment community run on architecture Z, keeping in mind that we won’t actually know ahead of time that this is the one?
These questions are important, or would be if I had a sufficiently sharply defined research agenda such that knowing the answer to (3) would change my current time allocation (e.g. if (3) were particularly short, I’d try a bit harder to understand the principles which govern which architectures have which inductive biases, instead of getting good at studying a single architecture). I think (3) wouldn’t actually help me right now because I’m not in fact that precise. But in theory, it’s decision-relevant.
Seems like e.g. intelligence explosion microeconomic considerations become relevant as subquestions for answering the above decision-relevant questions, but not because those microecon questions are my main focus of inquiry.
EDIT: Seems that Paul Christiano happened to write a takeoff speed comment just before I wrote this one; my comment is not a reply to his.