So maybe we need to think about systematization happening separately in system 1 and system 2?
I think that’s right. Taking on the natural-abstraction lens, there is a “ground truth” to the “hierarchy of values”. That ground truth can be uncovered either by “manual”/symbolic/System-2 reasoning, or by “automatic”/gradient-descent-like/System-1 updates, and both processes would converge to the same hierarchy. But in the System-2 case, the hierarchy would be clearly visible to the conscious mind, whereas the System-1 route would make it visible only indirectly, by the impulses you feel.
I don’t know about the conflict thing, though. Why do you think System 2 would necessarily oppose System 1′s deepest motivations?
I think that’s right. Taking on the natural-abstraction lens, there is a “ground truth” to the “hierarchy of values”. That ground truth can be uncovered either by “manual”/symbolic/System-2 reasoning, or by “automatic”/gradient-descent-like/System-1 updates, and both processes would converge to the same hierarchy. But in the System-2 case, the hierarchy would be clearly visible to the conscious mind, whereas the System-1 route would make it visible only indirectly, by the impulses you feel.
I don’t know about the conflict thing, though. Why do you think System 2 would necessarily oppose System 1′s deepest motivations?