This is a truly outstanding post, and it feels a bit sad and disappointing that it has not received more engagement.
Beyond the identification of valence as a crucial concept worth taking seriously (as it seems to carve reality at the joints) and the excellent and concrete gears-level explanations of the nature, formation, and adjustment of valence, I want to highlight two portions of Steve’s writing that I consider to be very well-written distillations of pivotal ideas:
Section 2.2 might seem at first glance like just a short application of “the map is not the territory” to the area of human normative evaluations, but what makes it valuable is the way it explains what is wrong with the “essentialist” belief that valence is an intrinsic part of objects or ideas by bringing to light in a clear yet concise manner the contrast between what positive-valence (for example) thoughts feel like from the inside vs what they are actually like from an (objective) outside perspective. The fact that this is not understood and acknowledged by many moral realists is immensely disappointing.
Section 2.7, in my view, represents high-quality meta-ethical reasoning that makes perfect sense, follows directly from the (positive, factual) explanations given before it, and yet calls into question the very foundation of CEV-thought that has become an entrenched part of LW thinking. Highlights of Steve’s writing include crucial observations such as the fact that there is no clean, natural boundary separating “I want to sit down because my legs hurt” and “I want the oppressed masses to obtain justice” (because both of these desires are made of the same brain-stuff and come about from the same structure and reinforcement of innate, base drives through a learning algorithm), that valence is at the root of all normative thinking, and that a human-independent “true morality” would have no reason to emerge as a convergent destination of this process.
This is a truly outstanding post, and it feels a bit sad and disappointing that it has not received more engagement.
Beyond the identification of valence as a crucial concept worth taking seriously (as it seems to carve reality at the joints) and the excellent and concrete gears-level explanations of the nature, formation, and adjustment of valence, I want to highlight two portions of Steve’s writing that I consider to be very well-written distillations of pivotal ideas:
Section 2.2 might seem at first glance like just a short application of “the map is not the territory” to the area of human normative evaluations, but what makes it valuable is the way it explains what is wrong with the “essentialist” belief that valence is an intrinsic part of objects or ideas by bringing to light in a clear yet concise manner the contrast between what positive-valence (for example) thoughts feel like from the inside vs what they are actually like from an (objective) outside perspective. The fact that this is not understood and acknowledged by many moral realists is immensely disappointing.
Section 2.7, in my view, represents high-quality meta-ethical reasoning that makes perfect sense, follows directly from the (positive, factual) explanations given before it, and yet calls into question the very foundation of CEV-thought that has become an entrenched part of LW thinking. Highlights of Steve’s writing include crucial observations such as the fact that there is no clean, natural boundary separating “I want to sit down because my legs hurt” and “I want the oppressed masses to obtain justice” (because both of these desires are made of the same brain-stuff and come about from the same structure and reinforcement of innate, base drives through a learning algorithm), that valence is at the root of all normative thinking, and that a human-independent “true morality” would have no reason to emerge as a convergent destination of this process.
Suggestion: Consider writing a 2023 review of this post (I don’t have enough background reading to write a good one)