0:20:04.4 Vael: Yeah, it seems right. I would kind of expect that if people really got stuck, they would start pouring effort into interpretability work for other types of things.
I think I don’t have this impression? I think when something ‘gets stuck’, progress normally comes from one of the side branches that turns out not to have the problem that the main branch has, rather than people really doubling down on understanding the main branch and figuring out how far back they have to roll back / what they have to be doing instead.
Like, when I think about Chess and Go bots, there’s some stuff you get from ‘better transparency’ on the old paradigms, but I’m guessing it mostly wouldn’t have given you the advances that we actually got. My impression of things in stats/ml more broadly is that we have lots of “Bayesian understandings” of techniques which point out the optimal way to do a technique, and something like 2/3rds of the time they’re “here’s the story behind the optimality of the technique someone found by futzing around” and only about 1/3rd of the time they’re “here’s a new technique that we found by thinking about what techniques are good.” Somehow this paper comes to mind.
Commentary on the cvgig transcript:
I think I don’t have this impression? I think when something ‘gets stuck’, progress normally comes from one of the side branches that turns out not to have the problem that the main branch has, rather than people really doubling down on understanding the main branch and figuring out how far back they have to roll back / what they have to be doing instead.
Like, when I think about Chess and Go bots, there’s some stuff you get from ‘better transparency’ on the old paradigms, but I’m guessing it mostly wouldn’t have given you the advances that we actually got. My impression of things in stats/ml more broadly is that we have lots of “Bayesian understandings” of techniques which point out the optimal way to do a technique, and something like 2/3rds of the time they’re “here’s the story behind the optimality of the technique someone found by futzing around” and only about 1/3rd of the time they’re “here’s a new technique that we found by thinking about what techniques are good.” Somehow this paper comes to mind.