Any guesses at the difficulty? My first impression was that this is not going to be solved any time soon. I just don’t think current techniques are good enough to write flawless lean code given a difficult objective.
I think grand challenges in AI are are sometimes useful, but when they are at this level I am a bit pessimistic. I don’t think this is necessarily AI-complete, but it’s perhaps close.
Can you quantify soon :) ? For example, I’d be willing to bet at 1⁄3 odds that this will be solved in the next 10 years conditional on a certain amount of effort being put in and more like 1⁄1 odds for the next 20 years. It’s hard to quantify the conditional piece but I’d cash it out as something like if researchers put in the same amount of effort into this that they put into NLP/image recognition benchmarks. I don’t think that’ll happen, so this is purely a counterfactual claim, but maybe it will help ground any subsequent discussion with some sort of concrete claim?
By soon I mean 5 years. Interestingly, I have a slightly higher probability that it will be solved within 20 years, which highlights the difficulty of saying ambiguous things like “soon.”
That is interesting! I should be clear that my odds ratios are pretty tentative given the uncertainty around the challenge. For example, I literally woke up this morning and thought that my 1⁄3 odds might be too conservative given recent progress on 8th grade science tests and theorem proving.
I created three PredictionBook predictions to track this if anyone’s interested (5 years, 10 years, 20 years).
Any guesses at the difficulty? My first impression was that this is not going to be solved any time soon. I just don’t think current techniques are good enough to write flawless lean code given a difficult objective.
I think grand challenges in AI are are sometimes useful, but when they are at this level I am a bit pessimistic. I don’t think this is necessarily AI-complete, but it’s perhaps close.
Can you quantify soon :) ? For example, I’d be willing to bet at 1⁄3 odds that this will be solved in the next 10 years conditional on a certain amount of effort being put in and more like 1⁄1 odds for the next 20 years. It’s hard to quantify the conditional piece but I’d cash it out as something like if researchers put in the same amount of effort into this that they put into NLP/image recognition benchmarks. I don’t think that’ll happen, so this is purely a counterfactual claim, but maybe it will help ground any subsequent discussion with some sort of concrete claim?
By soon I mean 5 years. Interestingly, I have a slightly higher probability that it will be solved within 20 years, which highlights the difficulty of saying ambiguous things like “soon.”
That is interesting! I should be clear that my odds ratios are pretty tentative given the uncertainty around the challenge. For example, I literally woke up this morning and thought that my 1⁄3 odds might be too conservative given recent progress on 8th grade science tests and theorem proving.
I created three PredictionBook predictions to track this if anyone’s interested (5 years, 10 years, 20 years).