I think what TurnTrout wants to say is that things like sexual preferences are actually learned generalizations from very basic hardcoded reward signals that latch onto things like the pheromones of the opposite sex. But I don’t think he’s got it all worked out yet.
What I gather from https://www.lesswrong.com/s/HzcM2dkCq7fwXBej8 is that it’s sort of like what you’re saying but it’s much more about predictions than actual experiences. If the Learning Subsystem is imagining a plan predicted to have high likelihood of smelling sex pheromones, seeing sexy body shapes, experiencing orgasm, etc. then the Steering Subsystem will reward the generation of that plan, basically saying “Yeah, think more thoughts like that!”.
The Learning Subsystem has a bunch of abstract concepts and labels for things the Steering Subsystem doesn’t care about (and can’t even access), but there are certain hardcoded reward channels it can understand. But the important thing is the reward signals can be evaluated for imagined worlds as well as the real immediate world.
I think what TurnTrout wants to say is that things like sexual preferences are actually learned generalizations from very basic hardcoded reward signals that latch onto things like the pheromones of the opposite sex. But I don’t think he’s got it all worked out yet.
What I gather from https://www.lesswrong.com/s/HzcM2dkCq7fwXBej8 is that it’s sort of like what you’re saying but it’s much more about predictions than actual experiences. If the Learning Subsystem is imagining a plan predicted to have high likelihood of smelling sex pheromones, seeing sexy body shapes, experiencing orgasm, etc. then the Steering Subsystem will reward the generation of that plan, basically saying “Yeah, think more thoughts like that!”.
The Learning Subsystem has a bunch of abstract concepts and labels for things the Steering Subsystem doesn’t care about (and can’t even access), but there are certain hardcoded reward channels it can understand. But the important thing is the reward signals can be evaluated for imagined worlds as well as the real immediate world.