Something that came up a bit, but felt like too in-the-weeds for the main post, is a number of people expressing skepticism about transfer-learning that would carry over to new domains.
I haven’t looked deeply into it yet, but my understanding is something like “past attempts at training transfer learning haven’t really worked or replicated”, or “complicated schemes to do so don’t seem better than simple ones.” This seems important to look into and understand the context of. It’s not exactly a crux for me – I’m driven more by a general intuition that “surely deliberate practicing thinking is at least somewhat useful, and we should find out how useful?” than about my specific model of how you could train transfer thinking-about-novel-domains.
But I also just… roll to disbelieve on this sort of thing not working? It just seems so fucking weird to me if deliberate practice + metacognition + learning general tools couldn’t enable you to improve in detectable ways. (it’s not weird to me if plenty of people vaguely try deliberate practice + metacognition and fail, and it’s not too weird if it really does take like 20 years to figure it out, but it’d be quite weird to me if an adequately resourced group who were intelligently iterating on their approach couldn’t figure it out)
Some reasons I believe transfer learning can happen:
When I did martial arts as an adult, the instructors could spot people with previous experience with martial arts, dance, yoga, or gymnastics. I remember at least three true positive predictions, and no false positives or negatives, although I could have missed them.
This wasn’t true for other physical activities, like football or track, so it wasn’t raw athleticism.
As a child my karate teacher advised us to try gymnastics, to improve karate.
I hear about football players being sent to do dance to aid football, but not the reverse.
Inside view, I know I learned skills like and “balance” from yoga and then applied it in dance and martial arts.
Dancers report their skills transfer between dances, but not symmetrically, some dances are much better foundations than others.
Everytime I hear a polyglot talk, they say languages start to get easier around the fourth one, and by 10 they’re collecting them like pokemon.
Some things I think are broadly useful, although you can argue about if they are literally transfer learning or merely very widely applicable skills:
thinking through problems step by step
chunking problems to reduce RAM demands
noticing when I actually understand something, versus merely think I do or could pass a test
knowing how to space repetition
somatic skills:
balance
Is this good, muscle-building pain, or bad, tendon-tearing pain?
Is this gentle stretching or am I gonna tear a tendon again?
watch a novel movement you haven’t seen before and turn it into muscle commands
copy a novel movement maintaining L-R instead of mirroring
translate verbal instructions into novel muscle movements
Update: I tested snakebird on three people: one hardcore math person who delights in solving math puzzles in his head (but hadn’t done many puzzle games), one unusually mathy social science type, one generalist (who had played snakebird before). Of these, the hardcore math guy blew the others away. He picked up rules faster, had more endurance, and was much more likely to actually one shot, including after skipping 20 levels ahead.
But, also maybe more interestingly/importantly: I’m interested in having the Real Smart People walk through what their process is actually like, and see if they’re doing things differently that other people can learn. (Presumably this is also something there’s some literature on?)
Came here to comment that. It seems much more efficient to learn the cognitive strategies smart people use than to try to figure them out from scratch. Ideally, you would have people of different skill levels solve problems (and maybe even do research) while thinking out loud and describing or drawing the images they are manipulating. I know this has been done at least for chess, and it would be nice to have it for domains with more structure. Then you could catalog these strategies and measure the effectiveness of teaching the system 2 process (the whole process they use, not only the winning path) and explicitly train in isolation the individual system 1 steps that make it up.
Yeah, although notably: the goal here is to become confidently good at solving domains where there are no established experts (with the motivating case being AI alignment, though I think lots of high-impact-but-vague fields are relevant). I think this does require developing the ability to invent new ways of thinking, and check for yourself which ways of thinking apply to a situation.
I think the optimal curriculum will include some amount of learning-for-yourself and some amount of learning from others.
This might be confusing the cart with the horse though, since this doesn’t control for IQ. A person with a high IQ might be more attracted to math because of it’s relative ease and also be able to pick up specific cognitive skills faster (i.e. being able to play snakebird well). In other words, correlation doesn’t imply causation.
Transfer learning isn’t what is controversial, it is far and/or general transfer to many different domains which is controversial. There is no verified method of raising general intelligence, for example.
Do you have any pointers to what you mean? (i.e. sources that demonstrate “not particularly general transfer?” or “explicitly not working in the general case”)
Part of why I feel optimistic is I’m specifically trying to learn/teach/enable skills in a set-of-domains that seem at least fairly related, i.e. research taste in novel, technical domains, and I’d expect “weak transfer learning” to be good enough to matter without making any claims about “general transfer learning.”
(I separately guess it should be possible to train at general transfer learning but it should require training at a pretty wide variety of skills, at which point it’s actually kinda unclear whether mechanistically what’s happening is “lots of transfer between related skills” vs “raising general intelligence factor”)
Even if transfer learning is a thing that could work, in any given domain that doesn’t have terrible feedback loops, would it not be more efficient to just apply the deliberate practice and metacognition to the domain itself? Like, if I’m trying to learn how to solve puzzle games, would it not be more efficient to just practice solving puzzle games than to do physics problems and try to generalise? Or if you think that this sort of general rationality training is only important for ‘specialising in problems we don’t understand’ type stuff with bad feedback loops, how would you even figure out whether or not it’s working given the bad feedback loops? Like sure, maybe you measure how well people perform at some legibly measurable tasks after the rationality training and they perform a bit better, but the goal in the first place was to use the rationality training’s good feedback loops to improve in domains with bad feedback loops, and those domains seem likely to be different enough that a lot of rationality lessons or whatever just don’t generalise well.
It just feels to me like the world where transfer learning works well enough to be worth the investment looks a lot different wrt how specialised the people who are best at X are for any given X. I can’t off the top of my head think of anyone who became the best at their thing by learning very general skills first and then applying them to their domain, rather than just focusing really hard on whatever their thing was.
would it not be more efficient to just apply the deliberate practice and metacognition to the domain itself
Yes, if that’s the only thing you want to learn. The more domains you want to understand the more it makes sense to invest in cross-domain meta skills.
Something that came up a bit, but felt like too in-the-weeds for the main post, is a number of people expressing skepticism about transfer-learning that would carry over to new domains.
I haven’t looked deeply into it yet, but my understanding is something like “past attempts at training transfer learning haven’t really worked or replicated”, or “complicated schemes to do so don’t seem better than simple ones.” This seems important to look into and understand the context of. It’s not exactly a crux for me – I’m driven more by a general intuition that “surely deliberate practicing thinking is at least somewhat useful, and we should find out how useful?” than about my specific model of how you could train transfer thinking-about-novel-domains.
But I also just… roll to disbelieve on this sort of thing not working? It just seems so fucking weird to me if deliberate practice + metacognition + learning general tools couldn’t enable you to improve in detectable ways. (it’s not weird to me if plenty of people vaguely try deliberate practice + metacognition and fail, and it’s not too weird if it really does take like 20 years to figure it out, but it’d be quite weird to me if an adequately resourced group who were intelligently iterating on their approach couldn’t figure it out)
Some reasons I believe transfer learning can happen:
When I did martial arts as an adult, the instructors could spot people with previous experience with martial arts, dance, yoga, or gymnastics. I remember at least three true positive predictions, and no false positives or negatives, although I could have missed them.
This wasn’t true for other physical activities, like football or track, so it wasn’t raw athleticism.
As a child my karate teacher advised us to try gymnastics, to improve karate.
I hear about football players being sent to do dance to aid football, but not the reverse.
Inside view, I know I learned skills like and “balance” from yoga and then applied it in dance and martial arts.
Dancers report their skills transfer between dances, but not symmetrically, some dances are much better foundations than others.
Everytime I hear a polyglot talk, they say languages start to get easier around the fourth one, and by 10 they’re collecting them like pokemon.
Some things I think are broadly useful, although you can argue about if they are literally transfer learning or merely very widely applicable skills:
thinking through problems step by step
chunking problems to reduce RAM demands
noticing when I actually understand something, versus merely think I do or could pass a test
knowing how to space repetition
somatic skills:
balance
Is this good, muscle-building pain, or bad, tendon-tearing pain?
Is this gentle stretching or am I gonna tear a tendon again?
watch a novel movement you haven’t seen before and turn it into muscle commands
copy a novel movement maintaining L-R instead of mirroring
translate verbal instructions into novel muscle movements
read quickly without losing comprehension
experimental design + analysis
Update: I tested snakebird on three people: one hardcore math person who delights in solving math puzzles in his head (but hadn’t done many puzzle games), one unusually mathy social science type, one generalist (who had played snakebird before). Of these, the hardcore math guy blew the others away. He picked up rules faster, had more endurance, and was much more likely to actually one shot, including after skipping 20 levels ahead.
Reminds me of video games > IQ tests
But, also maybe more interestingly/importantly: I’m interested in having the Real Smart People walk through what their process is actually like, and see if they’re doing things differently that other people can learn. (Presumably this is also something there’s some literature on?)
Came here to comment that. It seems much more efficient to learn the cognitive strategies smart people use than to try to figure them out from scratch. Ideally, you would have people of different skill levels solve problems (and maybe even do research) while thinking out loud and describing or drawing the images they are manipulating. I know this has been done at least for chess, and it would be nice to have it for domains with more structure. Then you could catalog these strategies and measure the effectiveness of teaching the system 2 process (the whole process they use, not only the winning path) and explicitly train in isolation the individual system 1 steps that make it up.
Yeah, although notably: the goal here is to become confidently good at solving domains where there are no established experts (with the motivating case being AI alignment, though I think lots of high-impact-but-vague fields are relevant). I think this does require developing the ability to invent new ways of thinking, and check for yourself which ways of thinking apply to a situation.
I think the optimal curriculum will include some amount of learning-for-yourself and some amount of learning from others.
This might be confusing the cart with the horse though, since this doesn’t control for IQ. A person with a high IQ might be more attracted to math because of it’s relative ease and also be able to pick up specific cognitive skills faster (i.e. being able to play snakebird well). In other words, correlation doesn’t imply causation.
Transfer learning isn’t what is controversial, it is far and/or general transfer to many different domains which is controversial. There is no verified method of raising general intelligence, for example.
Do you have any pointers to what you mean? (i.e. sources that demonstrate “not particularly general transfer?” or “explicitly not working in the general case”)
Part of why I feel optimistic is I’m specifically trying to learn/teach/enable skills in a set-of-domains that seem at least fairly related, i.e. research taste in novel, technical domains, and I’d expect “weak transfer learning” to be good enough to matter without making any claims about “general transfer learning.”
(I separately guess it should be possible to train at general transfer learning but it should require training at a pretty wide variety of skills, at which point it’s actually kinda unclear whether mechanistically what’s happening is “lots of transfer between related skills” vs “raising general intelligence factor”)
Even if transfer learning is a thing that could work, in any given domain that doesn’t have terrible feedback loops, would it not be more efficient to just apply the deliberate practice and metacognition to the domain itself? Like, if I’m trying to learn how to solve puzzle games, would it not be more efficient to just practice solving puzzle games than to do physics problems and try to generalise? Or if you think that this sort of general rationality training is only important for ‘specialising in problems we don’t understand’ type stuff with bad feedback loops, how would you even figure out whether or not it’s working given the bad feedback loops? Like sure, maybe you measure how well people perform at some legibly measurable tasks after the rationality training and they perform a bit better, but the goal in the first place was to use the rationality training’s good feedback loops to improve in domains with bad feedback loops, and those domains seem likely to be different enough that a lot of rationality lessons or whatever just don’t generalise well.
It just feels to me like the world where transfer learning works well enough to be worth the investment looks a lot different wrt how specialised the people who are best at X are for any given X. I can’t off the top of my head think of anyone who became the best at their thing by learning very general skills first and then applying them to their domain, rather than just focusing really hard on whatever their thing was.
Yes, if that’s the only thing you want to learn. The more domains you want to understand the more it makes sense to invest in cross-domain meta skills.