Inspired by this idea from Alex Turner’s shortform, I tried to figure out which facts are truth or fiction based on prompting gpt-4 to mess with a Wikipedia article on Developmental Psychology. (First I let GPT-4 munch a big chunk of the article, and then I chose the first chunk I saw that contained lots of concrete claims.)
Crecedences are 0% if claim false, and 100% if the text written by gpt-4 is true/reflects the original article. Outcomes are on the line afterwards. Written more as personal notes (very rough).
Vision is sharper in infants than in older children.
Vision is probably not sharper for infants, but the opposite! (10%)
false
Infant sight tends to remain stable with little improvement over time.
Infant sight should rapidly improve! (at least at some point it has too!) (10%)
false
Color perception is limited in the first year, with infants primarily seeing in shades of gray [79]. Infants only begin to develop adult-like vision at about twelve months.[72]
is no color perception plausible? (70%)
false, In fact they learn it at 4 months!
Hearing is still evolving at the time of birth.
Accidentally skipped this claim
Newborns show no distinct preference for human speech over other sounds, and they can’t distinguish their mother’s voice from others’.
Newborns should probably pay more attention to their mother’s voice! (It seems that this makes more sense if the latter parts are true. Not sure though!) (40%)
false
The belief that these features are learned in the womb has been debunked.
the debunking this seems pretty plausible! (70%) (on reflection that is not super sure that this is how it would be written on wikipedia)
false
By 18 months, infants’ hearing ability is still not on par with adults.
not hearing on par is plausible. On the other hand, the opposite seems more likely to be mentioned? (30%) (seems plausible, at that time some babies start talking right?)
false
Smell and taste are rudimentary, with infants often unable to distinguish between pleasant and unpleasant odors and tastes
The smell seems very implausible to me! Especially for some of the more toxic things, I would expect them to be very ingrained. It seems like valence for a lot of the strongest smells is preprogrammed! (10%) (I give not 5%, because it could be for substances that are not really dangerous? In that case rudimentary would make sense as a description)
false
Newborns do not show a clear preference for the smell of human milk over that of formula.[72]: 150 Older infants, interestingly, do not show a preference for their mother’s scent.[79]
Human milk over formula? Seems like that could go either way with underpowered studies? 55%
true (Huh first positive result … somehow I now want to see how well powered these actually were, or how you detect which smell a baby “likes” at all and whether that’s a strong signal)
Touch and feel, while being one of the first senses to develop in the womb, are not as refined in infants as previously thought.[84] This contradicts the idea of primitive reflexes, which were believed to demonstrate advanced touch capabilities.
This section seemes perhaps a bit weird? Why would primitive reflexes be rather advanced? Is this saying that a baby needs to figure out most motor control and most is not preprogrammed? Seems plausible, I give (40%) that none of the claims above have been altered.
false (In hindsight of course a baby can figure out a lot of motor control before leaving the womb)
Pain perception in infants is believed to be less intense than in older children, indicating that they may not feel pain as acutely.
Not sure how long something is an infant. It seems like a plausible claim if a lot of pain is sorta more of a social thing and babies haven’t developed that so much yet? On the other hand babies seems like they are crying a lot and that they are constantly suffereing. (30%)
false
There is also no substantial evidence that glucose can relieve pain in newborns.[87]
The glucose thing seems like a cointoss? Seems marginally more plausible to be mentioned if true so (45%)
false Wow a lot of these are giving across higher confidence than I would have expected. The sucrose thing is apparently a common thing and the randomized control trial doesn’t seem to have too bad numbers (although I should at some point figure out how to get a useful estimate of the effect-size out of statistics like that). It seems plausible that blinding might be a bit hard.
It also gives me more confidence that Wikipedia is not listing lots of common misconceptions it wants to crush.
Overall, this whole field seems interesting! I think I also underestimated this field because it has psychology in its name (Yeah, I know that sounds dumb). I was not reflecting on my probabilities for long, and now feel like I could have done a lot better if I had (feedback and knowing how wrong my first impressions are is also valuable). Also reminds me of some section of hpmor where harry thinks about how it took a very long time until some human came up with the idea to investigate when children learn what. It also seems like a lot of the problems with testing that you would usually have in psychology studies, especially around surveys and self-report, is that you can’t do that with infants, so you get higher quality data. You also wouldn’t get infants that are trying to figure out what your experimental design is and whether they want to prove you right, wrong etc.
Bonus Challenge
Inspired by this idea from Alex Turner’s shortform, I tried to figure out which facts are truth or fiction based on prompting gpt-4 to mess with a Wikipedia article on Developmental Psychology. (First I let GPT-4 munch a big chunk of the article, and then I chose the first chunk I saw that contained lots of concrete claims.)
Crecedences are 0% if claim false, and 100% if the text written by gpt-4 is true/reflects the original article. Outcomes are on the line afterwards. Written more as personal notes (very rough).
Vision is probably not sharper for infants, but the opposite! (10%)
false
Infant sight should rapidly improve! (at least at some point it has too!) (10%)
false
is no color perception plausible? (70%)
false, In fact they learn it at 4 months!
Accidentally skipped this claim
Newborns should probably pay more attention to their mother’s voice! (It seems that this makes more sense if the latter parts are true. Not sure though!) (40%)
false
the debunking this seems pretty plausible! (70%) (on reflection that is not super sure that this is how it would be written on wikipedia)
false
not hearing on par is plausible. On the other hand, the opposite seems more likely to be mentioned? (30%) (seems plausible, at that time some babies start talking right?)
false
The smell seems very implausible to me! Especially for some of the more toxic things, I would expect them to be very ingrained. It seems like valence for a lot of the strongest smells is preprogrammed! (10%) (I give not 5%, because it could be for substances that are not really dangerous? In that case rudimentary would make sense as a description)
false
true (Huh first positive result … somehow I now want to see how well powered these actually were, or how you detect which smell a baby “likes” at all and whether that’s a strong signal)
This section seemes perhaps a bit weird? Why would primitive reflexes be rather advanced? Is this saying that a baby needs to figure out most motor control and most is not preprogrammed? Seems plausible, I give (40%) that none of the claims above have been altered.
false (In hindsight of course a baby can figure out a lot of motor control before leaving the womb)
Not sure how long something is an infant. It seems like a plausible claim if a lot of pain is sorta more of a social thing and babies haven’t developed that so much yet? On the other hand babies seems like they are crying a lot and that they are constantly suffereing. (30%)
false
The glucose thing seems like a cointoss? Seems marginally more plausible to be mentioned if true so (45%)
false Wow a lot of these are giving across higher confidence than I would have expected. The sucrose thing is apparently a common thing and the randomized control trial doesn’t seem to have too bad numbers (although I should at some point figure out how to get a useful estimate of the effect-size out of statistics like that). It seems plausible that blinding might be a bit hard.
It also gives me more confidence that Wikipedia is not listing lots of common misconceptions it wants to crush.
Overall, this whole field seems interesting! I think I also underestimated this field because it has psychology in its name (Yeah, I know that sounds dumb). I was not reflecting on my probabilities for long, and now feel like I could have done a lot better if I had (feedback and knowing how wrong my first impressions are is also valuable). Also reminds me of some section of hpmor where harry thinks about how it took a very long time until some human came up with the idea to investigate when children learn what. It also seems like a lot of the problems with testing that you would usually have in psychology studies, especially around surveys and self-report, is that you can’t do that with infants, so you get higher quality data. You also wouldn’t get infants that are trying to figure out what your experimental design is and whether they want to prove you right, wrong etc.