When you come to know something because there is a norm to, instead of by happening to be in the appropriate frame of mind to get convinced by arguments, you either broke your existing cognition, or learned to obscure it, perhaps even from yourself when the norm is powerful enough.
This is stated as an absolute, when it is not an absolute. You might want to take a glance at the precursor essay Sapir-Whorf for Rationalists, and separately consider that not everyone’s mind works the way you’re confidently implying All Minds Work.
I want people here to be allowed honesty and integrity, not get glared into cooperative whatever.
You’re strawmanning norms quite explicitly, here, as if “glared into cooperative whatever” is at all a reasonable description of what healthy norms look like. You seem to have an unstated premise of, like, “that whole section where Duncan talked about what a guideline looks like was a lie,” or something.
my point is that cultivation of norms is a step in the wrong direction
I hear that that’s your position, but so far I think you have failed to argue for that position except by strawmanning what I’m saying and rose-ifying what you’re saying.
It’s hard to avoid goodharting and deceptive alignment. Explicitly optimizing for obviously flawed proxies is inherently dangerous.
Agree; I literally created the CFAR class on Goodharting. Explicitly optimizing for obviously flawed proxies is specifically recommended against in this post.
Norms take on a life of their own, telling them to stop when appropriate doesn’t work very well.
This is the closest thing to a-point-I’d-like-to-roll-around-and-discuss-with-you-and-others in your comments above, but I’m going to be loath to enter such a discussion until I feel like my points are not going to be rounded off to the dumbest possible neighbor of what I’m actually trying to say.
This is stated as an absolute, when it is not an absolute. You might want to take a glance at the precursor essay Sapir-Whorf for Rationalists, and separately consider that not everyone’s mind works the way you’re confidently implying All Minds Work.
You’re strawmanning norms quite explicitly, here, as if “glared into cooperative whatever” is at all a reasonable description of what healthy norms look like. You seem to have an unstated premise of, like, “that whole section where Duncan talked about what a guideline looks like was a lie,” or something.
I hear that that’s your position, but so far I think you have failed to argue for that position except by strawmanning what I’m saying and rose-ifying what you’re saying.
Agree; I literally created the CFAR class on Goodharting. Explicitly optimizing for obviously flawed proxies is specifically recommended against in this post.
This is the closest thing to a-point-I’d-like-to-roll-around-and-discuss-with-you-and-others in your comments above, but I’m going to be loath to enter such a discussion until I feel like my points are not going to be rounded off to the dumbest possible neighbor of what I’m actually trying to say.