Great post! I especially liked that you outlined potential emerging technologies and the economic considerations.
Having looked a bit into this when writing my TAI and Compute sequence, I agree with your main takeaways. In particular, I’d like to see more work on DRAM and the interconnect trends and potential emerging paradigms.
I’d be interested in you compute forecasts to inform TAI timelines. For example Cotra’s draft report assumes a doubling time of 2.5 years for the FLOPs/$ but acknowledges that this forecast could be easily improved by someone with more domain knowledge—that could be you.
I’d be interested in you compute forecasts to inform TAI timelines.
I’m not all that sure this is going to give you anything more useful than what you have already. Around the end of this decade my compute predictions detach from any strict timeline, and the pace of progress within this decade is smaller than the potential range of money spent on the problem, so even if you draw a sure line around where TAI happens with today’s AI techniques, you don’t gain all that much from better guesses about hardware progress.
Put another way, if you assume current connectivist architectures scaled up to ~brain parity buys you most of the tools you need to build TAI, then you don’t need to worry about longer term hardware progress. If you don’t assume that, then you don’t have a meaningful anchor to use these longer term predictions with anyway. If I had strong timelines for physical technology progress you could at least say, architectures like P will be tried around the 20X0s, and architectures like Q will be tried around the 20Y0s, but I don’t have strong timelines for progress that goes that far out.
I do think understanding longer term tech progress is relevant, because I think that current AI systems do seem to keep buying relevant cognitive abilities as they scale, and having a long roadmap implies that we’ll keep doing that until the trick stops working or we hit AGI. But I don’t know how to put a date on that, at least one that’s more informative than ‘it’s technically plausible, and could come moderately soon if things go fast’.
Great post! I especially liked that you outlined potential emerging technologies and the economic considerations.
Having looked a bit into this when writing my TAI and Compute sequence, I agree with your main takeaways. In particular, I’d like to see more work on DRAM and the interconnect trends and potential emerging paradigms.
I’d be interested in you compute forecasts to inform TAI timelines. For example Cotra’s draft report assumes a doubling time of 2.5 years for the FLOPs/$ but acknowledges that this forecast could be easily improved by someone with more domain knowledge—that could be you.
I’m not all that sure this is going to give you anything more useful than what you have already. Around the end of this decade my compute predictions detach from any strict timeline, and the pace of progress within this decade is smaller than the potential range of money spent on the problem, so even if you draw a sure line around where TAI happens with today’s AI techniques, you don’t gain all that much from better guesses about hardware progress.
Put another way, if you assume current connectivist architectures scaled up to ~brain parity buys you most of the tools you need to build TAI, then you don’t need to worry about longer term hardware progress. If you don’t assume that, then you don’t have a meaningful anchor to use these longer term predictions with anyway. If I had strong timelines for physical technology progress you could at least say, architectures like P will be tried around the 20X0s, and architectures like Q will be tried around the 20Y0s, but I don’t have strong timelines for progress that goes that far out.
I do think understanding longer term tech progress is relevant, because I think that current AI systems do seem to keep buying relevant cognitive abilities as they scale, and having a long roadmap implies that we’ll keep doing that until the trick stops working or we hit AGI. But I don’t know how to put a date on that, at least one that’s more informative than ‘it’s technically plausible, and could come moderately soon if things go fast’.