I skimmed this, but I get the sense that you’re interpreting Hanson’s predictions in ways that he would not have agreed with. My cached thoughts suggest that Hanson’s model predicts deep learning couldn’t possibly work, because creating “intelligence” will require lots of custom engineering for different skills instead of “GPU go brr”. Hence his admiration of Cyc: it is focusing on implemeting a whole host of skills with lots of integrated knowledge.
See his post “I heart CYC”. Here’s a quote form it, which I think highlights Hanson’s own interpretation of “architecture is overrated”:
The lesson Lenat took from EURISKO is that architecture is overrated; AIs learn slowly now mainly because they know so little. So we need to explicitly code knowledge by hand until we have enough to build systems effective at asking questions, reading, and learning for themselves. Prior AI researchers were too comfortable starting every project over from scratch; they needed to join to create larger integrated knowledge bases. This still seems to me a reasonable view, and anyone who thinks Lenat created the best AI system ever should consider seriously the lesson he thinks he learned.
That sure doesn’t look like “just do next token prediction with a simple architecture and lots of compute”.
EDIT: I didn’t realize this, but “I heart CYC” was written in 2008, around the time of the FOOM debate. So it is directly relevant to how we should interpret, and score, the incidental predictions you mention. As you can tell, I advocate a contextual reading of old predictions. Yes, that leaves some degrees of freedom to twist words any way you wish, but not that much if we’re looking at simple interpretations alongside other relevant statements by the predictor. Use all the data you have available. Plus, we can just ask Hanson.
I do in fact include the same quote you include in the section titled “Cyc is not a Promising Approach to Machine Intelligence.” That’s part of the reason why that section resolves in favor of Yudkowsky.
I agree that Hanson thinks skills in general will be harder to acquire than Yudkowsky thinks. I think that could easily be another point for Yudkowsky in the “human content vs right architecture.” Like many points there in that section, I don’t think it’s operationalized particularly well, which is why I don’t call it either way.
I skimmed this, but I get the sense that you’re interpreting Hanson’s predictions in ways that he would not have agreed with. My cached thoughts suggest that Hanson’s model predicts deep learning couldn’t possibly work, because creating “intelligence” will require lots of custom engineering for different skills instead of “GPU go brr”. Hence his admiration of Cyc: it is focusing on implemeting a whole host of skills with lots of integrated knowledge.
See his post “I heart CYC”. Here’s a quote form it, which I think highlights Hanson’s own interpretation of “architecture is overrated”:
That sure doesn’t look like “just do next token prediction with a simple architecture and lots of compute”.
EDIT: I didn’t realize this, but “I heart CYC” was written in 2008, around the time of the FOOM debate. So it is directly relevant to how we should interpret, and score, the incidental predictions you mention. As you can tell, I advocate a contextual reading of old predictions. Yes, that leaves some degrees of freedom to twist words any way you wish, but not that much if we’re looking at simple interpretations alongside other relevant statements by the predictor. Use all the data you have available. Plus, we can just ask Hanson.
EDIT^2: I asked Hanson and Yud on Twitter. Let’s see if they reply. https://twitter.com/Algon_33/status/1664383482884239365
I do in fact include the same quote you include in the section titled “Cyc is not a Promising Approach to Machine Intelligence.” That’s part of the reason why that section resolves in favor of Yudkowsky.
I agree that Hanson thinks skills in general will be harder to acquire than Yudkowsky thinks. I think that could easily be another point for Yudkowsky in the “human content vs right architecture.” Like many points there in that section, I don’t think it’s operationalized particularly well, which is why I don’t call it either way.