I don’t recall learning in school that most of “the bad guys” from history (e.g., Communists, Nazis) thought of themselves as “the good guys” fighting for important moral reasons. It seems like teaching that fact, and instilling moral uncertainty in general into children, would prevent a lot of serious man-made problems (including problems we’re seeing play out today). So why hasn’t civilization figured that out already? Or is not teaching moral uncertainty some kind of Chesterton’s Fence, and teaching it widely would make the world even worse off on expectation?
I wonder if anyone has ever written a manifesto for moral uncertainty, maybe something along the lines of:
We hold these truths to be self-evident, that we are very confused about morality. That these confusions should be properly reflected as high degrees of uncertainty in our moral epistemic states. That our moral uncertainties should inform our individual and collective actions, plans, and policies. … That we are also very confused about normativity and meta-ethics and don’t really know what we mean by “should”, including in this document...
Yeah, I realize this would be a hard sell in today’s environment, but what if building Friendly AI requires a civilization sane enough to consider this common sense? I mean, for example, how can it be a good idea to gift a super-powerful “corrigible” or “obedient” AI to a civilization full of people with crazy amounts of moral certainty?
Non-dualist philosophies such as Zen place high value on confusion (they call it “don’t know mind”) and have a sophisticated framework for communicating this idea. Zen is one of the alternative intellectual traditions I alluded to in my controversial post about ethical progress.
The Dao De Jing 道德经, written 2.5 thousand years ago, includes strong warnings against ontological certainty (and, by extension, moral certainty). If we naïvely apply the Lindy Effect then Chinese civilization is likely to continue for thousands more years while Western science annihilates itself after mere centuries. This may not be a coincidence.
So why hasn’t civilization figured that out already?
States evolve to perpetuate themselves. Civilization has figured it out (in the blind idiot god sense of “figured it out”) that moral uncertainty is teachable and decreases trust in the state ideology. You have it backward. The states in existence today promote moral certainty in children for exactly the same reason the Communist and Nazi states did.
Or is not teaching moral uncertainty some kind of Chesterton’s Fence, and teaching it widely would make the world even worse off on expectation?
I expect it is this. General moral uncertainty has all kinds of problems in expectation, like:
It ruins morality as a coordination mechanism among the group.
It weakens moral conviction in the individual, which is super bad from the perspective of people who believe there are direct consequences for a lack of conviction (like Hell).
It creates space for different and possibly weird moralities to arise; I don’t know of any moral systems that think it is a good thing to be a member of a different moral system, so I expect all the current moral systems to agree on this one.
I feel like the first bullet point is the real driving force behind the problems it would prevent, anyhow. Moral uncertainty doesn’t cause people to do good things; it keeps them from doing good things (that are different from other groups’ definitions of good things).
So why hasn’t civilization figured that out already? Or is not teaching moral uncertainty some kind of Chesterton’s Fence, and teaching it widely would make the world even worse off on expectation?
This is sort of a rehash of sibling comments, but I think there are two factors to consider here.
The first is the rules. It is very important that people drive on the correct side of the road, and not have uncertainty about which side of the road is correct, and not very important whether they have a distinction between “correct for <country> in <year>” and “correct everywhere and for all time.”
The second is something like the goal. At one point, people thought it was very important that society have a shared goal, and worked hard to make it expansive; things like “freedom of religion” are the things civilization figured out to both have narrow shared goals (like “keep the peace”) and not expansive shared goals (like “as many get to Catholic Heaven as possible”). It is unclear to me whether we’re better off with moral uncertainty as generator for “narrow shared goals”, whether narrow shared goals is what we should be going for.
Often expressing any understanding towards the motives of a “bad guy” is taken as signaling acceptance for their actions. There was e.g. controversy around the movie Downfall for this:
Downfall was the subject of dispute by critics and audiences in Germany before and after its release, with many concerned of Hitler’s role in the film as a human being with emotions in spite of his actions and ideologies.[40][30][49] The portrayal sparked debate in Germany due to publicity from commentators, film magazines, and newspapers,[25][50] leading the German tabloid Bild to ask the question, “Are we allowed to show the monster as a human being?”.[25]
Did it make you or your classmates doubt your own morality a bit? If not, maybe it needs to be taught along with the outside view and/or the teacher needs to explicitly talk about how the lesson from history is that we shouldn’t be so certain about our morality...
We want to teach children to accept the norms of our society and the narrative we tell about it. A lot of what we teach is essential pro-system propaganda.
Teaching moral uncertainty doesn’t help with that and it also doesn’t help with getting students to score better on standardized tests which was the main goal of educational reforms of the last decades.
Compulsory education is an organ of the state. Nation-states evolve to perpetuate their own existence. Teaching moral uncertainty is counter-productive toward maintaining the norms of a nation-state.
I guess it’s because high-conviction ideologies outperform low-conviction ones, including nationalistic and political ideologies, and religions. Dennett’s Gold Army/Silver Army analogy explains how conviction can build loyatly and strength, but a similar thing is probably true for movement-builders. Also, conviction might make adherents feel better, and therefore simply be more attractive.
If I had to guess, I’d guess the answer is some combination of “most people haven’t realized this” and “of those who have realized it, they don’t want to be seen as sympathetic to the bad guys”.
I don’t recall learning in school that most of “the bad guys” from history (e.g., Communists, Nazis) thought of themselves as “the good guys” fighting for important moral reasons. It seems like teaching that fact, and instilling moral uncertainty in general into children, would prevent a lot of serious man-made problems (including problems we’re seeing play out today). So why hasn’t civilization figured that out already? Or is not teaching moral uncertainty some kind of Chesterton’s Fence, and teaching it widely would make the world even worse off on expectation?
I wonder if anyone has ever written a manifesto for moral uncertainty, maybe something along the lines of:
We hold these truths to be self-evident, that we are very confused about morality. That these confusions should be properly reflected as high degrees of uncertainty in our moral epistemic states. That our moral uncertainties should inform our individual and collective actions, plans, and policies. … That we are also very confused about normativity and meta-ethics and don’t really know what we mean by “should”, including in this document...
Yeah, I realize this would be a hard sell in today’s environment, but what if building Friendly AI requires a civilization sane enough to consider this common sense? I mean, for example, how can it be a good idea to gift a super-powerful “corrigible” or “obedient” AI to a civilization full of people with crazy amounts of moral certainty?
Non-dualist philosophies such as Zen place high value on confusion (they call it “don’t know mind”) and have a sophisticated framework for communicating this idea. Zen is one of the alternative intellectual traditions I alluded to in my controversial post about ethical progress.
The Dao De Jing 道德经, written 2.5 thousand years ago, includes strong warnings against ontological certainty (and, by extension, moral certainty). If we naïvely apply the Lindy Effect then Chinese civilization is likely to continue for thousands more years while Western science annihilates itself after mere centuries. This may not be a coincidence.
Here is the manifesto you are looking for:
Unfortunately, the duality of emptiness and form is difficult to translate into English.
States evolve to perpetuate themselves. Civilization has figured it out (in the blind idiot god sense of “figured it out”) that moral uncertainty is teachable and decreases trust in the state ideology. You have it backward. The states in existence today promote moral certainty in children for exactly the same reason the Communist and Nazi states did.
I expect it is this. General moral uncertainty has all kinds of problems in expectation, like:
It ruins morality as a coordination mechanism among the group.
It weakens moral conviction in the individual, which is super bad from the perspective of people who believe there are direct consequences for a lack of conviction (like Hell).
It creates space for different and possibly weird moralities to arise; I don’t know of any moral systems that think it is a good thing to be a member of a different moral system, so I expect all the current moral systems to agree on this one.
I feel like the first bullet point is the real driving force behind the problems it would prevent, anyhow. Moral uncertainty doesn’t cause people to do good things; it keeps them from doing good things (that are different from other groups’ definitions of good things).
This is sort of a rehash of sibling comments, but I think there are two factors to consider here.
The first is the rules. It is very important that people drive on the correct side of the road, and not have uncertainty about which side of the road is correct, and not very important whether they have a distinction between “correct for <country> in <year>” and “correct everywhere and for all time.”
The second is something like the goal. At one point, people thought it was very important that society have a shared goal, and worked hard to make it expansive; things like “freedom of religion” are the things civilization figured out to both have narrow shared goals (like “keep the peace”) and not expansive shared goals (like “as many get to Catholic Heaven as possible”). It is unclear to me whether we’re better off with moral uncertainty as generator for “narrow shared goals”, whether narrow shared goals is what we should be going for.
I would guess that teaching that fact is not enough to instill moral uncertainty. And that instilling moral uncertainty would be very hard.
Often expressing any understanding towards the motives of a “bad guy” is taken as signaling acceptance for their actions. There was e.g. controversy around the movie Downfall for this:
Wouldn’t more moral uncertainty make people less certain that Communism or Nazism were wrong?
That’s definitely how it was taught in my high school, so it’s not unknown.
Did it make you or your classmates doubt your own morality a bit? If not, maybe it needs to be taught along with the outside view and/or the teacher needs to explicitly talk about how the lesson from history is that we shouldn’t be so certain about our morality...
We want to teach children to accept the norms of our society and the narrative we tell about it. A lot of what we teach is essential pro-system propaganda.
Teaching moral uncertainty doesn’t help with that and it also doesn’t help with getting students to score better on standardized tests which was the main goal of educational reforms of the last decades.
Compulsory education is an organ of the state. Nation-states evolve to perpetuate their own existence. Teaching moral uncertainty is counter-productive toward maintaining the norms of a nation-state.
I guess it’s because high-conviction ideologies outperform low-conviction ones, including nationalistic and political ideologies, and religions. Dennett’s Gold Army/Silver Army analogy explains how conviction can build loyatly and strength, but a similar thing is probably true for movement-builders. Also, conviction might make adherents feel better, and therefore simply be more attractive.
If I had to guess, I’d guess the answer is some combination of “most people haven’t realized this” and “of those who have realized it, they don’t want to be seen as sympathetic to the bad guys”.