Clearly, you and I have different definitions of “easy”.
ChaosMote
Karma: 174
This was a terrific post; insightful and entertaining in excess of what can be conveyed by an upvote. Thank you for making it.
What you’re proposing sounds more like moral relativism than moral nihilism.
Ah, yes. My mistake. I stand corrected. Some cursory googling suggests that you are right. With that said, to me Moral Nihilism seems like a natural consequence of Moral Relativism, but that may be a fact about me and not the universe, so to speak (though I would be grateful if you could point out a way to be morally relativist without morally nihilist).
I think that you’re confusing moral universalism with moral absolutism and value monism.
The last paragraph of my previous post was a claim that unless you an objective way of ordering conflicting preferences (and I don’t see how you can), you are forced to work under value pluralism. I did use this as an argument against moral universalism , though that argument may not be entirely correct. I concede the point.
But there’s that language again that people use when they talk about moral nihilism, where I can’t tell if they’re just using different words, or if they really think that morality can be whatever we want it to be, or that it doesn’t mean anything to say that moral propositions are true or false.
Okay. Correct me if any of this doesn’t sound right. When a person talks about “morality”, you imagine a conceptual framework of some sort—some way of distinguishing what makes actions “good” or “bad”, “right” or “wrong”, etc. Different people will imagine different frameworks, possibly radically so—but there is generally a lot of common ground (or so we hope), which is why you and I can talk about “morality” and more or less understand the gist of each other’s arguments. Now, I would claim that what I mean when I say “morality”, or what you mean, or what a reasonable third party may mean, or any combination thereof—that each of these is entirely unrelated to ground truth.
Basically, moral propositions (e.g. “Murder is Bad”) contain unbound variables (in this case, “Bad”) which are only defined in select subjective frames of reference. “Bad” does not have a universal value in the sense that “Speed of Light” or “Atomic Weight of Hydrogen” or “The top LessWrong contributor as of midnight January 1st, 2015″ do. That is the main thesis of Moral Nihilism as far as I understand it. Does that sound sensible?
I wouldn’t ask people those questions. People can be wrong about what they value. The point of moral philosophy is to know what you should do.
Alright; let me rephrase my point. Let us say that you have access to everything there that can be known about a individual X. Can you explain how you compute their objective contingent morality to an observer who has no concept of morality? You previous statement of “what is moral is what you value” would need to define “what you value” before it would suffice. Note that unless you can do this construction, you don’t actually have something objective.
I think that using this notation is misleading. If I am understanding you correctly, you are saying that given an individual, we can derive their morality from their (real/physically grounded) state, which gives real/physically grounded morality (for that individual). Furthermore, you are using “objective” where I used “real/physically ground”. Unfortunately, one of the common meanings of objective is “ontologically fundamental and not contingent”, so your statement sounds like it is saying something that it isn’t.
On a separate note, I’m not sure why you are casually dismissing moral nihilism as wrong. As far as I am aware, moral nihilism is the position that morality is not ontologically fundamental. Personally, I am a moral nihilist; my experience shows that morality as typically discussed refers to a collection of human intuitions and social constructs—it seems bizarre to believe that to be an ontologically fundamental phenomenon. I think a sizable fraction of LW is of like mind, though I can only speak for myself.
I would even go further and say that I don’t believe in objective contingent morality. Certainly, most people have an individual idea of what they find moral. However, this only establishes that there is an objective contingent response to the question “what do you find moral?” There is similarly an objective contingent response to the related question “what is morality?”, or the question “what is the difference between right and wrong?” Sadly, I expect the responses in each case to differ (due to framing effects, at the very least). To me, this shows that unless you define “morality” quite tightly (which could require some arbitrary decisions on your part), your construction is not well defined.
Note that I expect that last paragraph to be more relativist then most other people here, so I definitely speak only for myself there.
Okay, but at best, this shows that the immediate cause of you being shaken and coming out of it is related to fearful epiphanies. Is it not plausible that the reason that, at a given time, you find particular idea horrific or are able to accept a solution as satisfying depending on your mental state?
Consider this hypothetical narrative. Let Frank (name chosen at random) be a person suffering from occasional bouts of depression. When he is healthy, he notices an enjoys interacting with the world around him. When he is depressed, he instead focuses on real or imagined problems in his life—and in particular, how stressful his work is.
When asked, Frank explains that his depression is caused by problems at work. He explains that when he gets assigned a particularly unpleasant project, his depression flares up. The depression doesn’t clear up until things get easier. Frank explains that once he finishes a project and is assigned something else, his depression clears up (unless the new project is just as bad); or sometimes, through much struggle, he figure out how to make the project bearable, and that resolves the depression as well.
Frank is genuine in expressing his feelings, and correct about work problems being correlated with his depression, but he is wrong about causation between the two.
Do you find this story analogous to your situation? If not, why not?
@gjm:
Just wanted to say that this is well thought out and well written—it is what I would have tried to say (albeit perhaps less eloquently) if it hadn’t been said already. I wish I had more than one up-vote to give.
@Eitan_Zohar:
I would urge you to give the ideas here more thought. Part of the point here is that from you are going to be strongly biased for thinking your explanations are of the first sort and not the second. By virtue of being human, you are almost certainly biased in certain predictable ways, this being one of them. Do you disagree?
Let me ask you this: what would it take to make you change your mind; i.e. that the explanation for this pattern is one of the latter three reasons and not the former three reasons?
I definitely know that my depression is causally tied to my existential pessimism.
Out of curiosity, how do you know that this is the direction of the causal link? The experiences you have mentioned in the thread seem to also be consistent with depression causing you to get hung up on existential pessimism.
Your argument assumes that the algorithm and the prisons have access to the same data. This need not be the case—in particular, if a prison bribes a judge to over-convict, the algorithm will be (incorrectly) relying on said conviction as data, skewing the predicted recidivism measure.
That said, the perverse incentive you mentioned is absolutely in play as well.
Great suggestion! That said, in light of your first paragraph, I’d like to point out a couple of issues. I came up with most of these by asking the questions “What exactly are you trying to encourage? What exactly are you incentivising? What differences are there between the two, and what would make those difference significant?”
You are trying to encourage prisons to rehabilitate their inmates. If, for a given prisoner, we use p to represent their propensity towards recidivism and a to represent their actual recidivism, rehabilitation is represented by p-a. Of course, we can’t actually measure these values, so we use proxies; anticipated recidivism according to your algorithm and re-conviction rate (we’ll call these p’ and a’, respectively).
With this incentive scheme, our prisons have three incentives: increasing p’-p, increasing p-a, and increasing a-a’. The first and last can lead to some problematic incentives.
To increase p’-p, prisons need to incarcerate prisoners which are less prone to recidivism than predicted. Given that past criminality is an excellent predictor of future criminality, this leads to a perverse incentive towards incarcerating those who were unfairly convicted (wrongly convicted innocents or over-convinced lesser offenders). If said prisons can influence the judges supplying their inmates, this may lead to judges being bribed to aggressively convict edge-cases or even outright innocents, and to convict lesser offenses of crimes more correlated with recidivism. (Counterpoint: We already have this problem, so this perverse incentive might not be making things much worse than they already are.)
To increase a-a’, prisons need to reduce the probability of re-conviction relative to recidivism. At the comically amoral end, this can lead to prisons teaching inmates “how not to get caught.” Even if that doesn’t happen, I can see prisons handing out their lawyer’s business cards to released inmates. “We are invested in making you a contributing member of society. If you are ever in trouble, let us know—we might be able to help you get back on track.” (Counterpoint: Some of these tactics are likely to be too expensive to be worthwhile, even ignoring morality issues.)
Also, since you are incentivising improvement but not disincentivizing regression, prisons who are below-average are encouraged to try high-volatility reforms even if they would yield negative expected improvement. For example, if a reform has a 20% chance of making things much better but a 80% chance of making things equally worse, it is still a good business decision (since the latter consequence does not carry any costs).
You are of course entirely correct in saying that this is far too little to retire on. However, it is possible to save without being able to liquidate said saving; for example by paying down debts. The Emergency Fund advice is that you should make a point to have enough liquid savings tucked away to tide you over in a financial emergency before you direct your discretionary income anywhere else.
I’m afraid I don’t know. You might get better luck making this question a top level post.
I am by no means an expert, but here are a couple of options that come to mind. I came up with most of these by thinking “what kind of emergency are you reasonably likely to run into at some point, and what can you do to mitigate them?”
Learn some measure of first aid, or at least the Heimlich maneuver and CPR.
Keep a Seat belt cutter and window breaker in your glove compartment. And on the subject, there are a bunch of other things that you may want to keep in your car as well.
Have an emergency kit at home, and have a plan for dealing with natural disasters (fire, storms, etc). If you live with anyone, make sure that everyone is on the same page about this.
On the financial side, have an emergency fund. This might not impress your friends, but given how likely financial emergencies (e.g. unexpectedly losing a job) are relative to other emergencies, this is a good thing to plan for nonetheless. I think the standard advice is to have something on the order 3-6 months of income tucked away for a rainy day.
You make a good point, and I am very tempted to agree with you. You are certainly correct in that even a completely non-centralized community with no stated goals can be exclusionary. And I can see “community goals” serving a positive role, guiding collective behavior towards communal improvement, whether that comes in the form of non-exclusiveness or other values.
With that said, I find myself strangely disquieted by the idea of Less Wrong being actively directed, especially by a singular individual. I’m not sure what my intuition is stuck on, but I do feel that it might be important. My best interpretation right now is that having an actively directed community may lend itself to catastrophic failure (in the same way that having a dictatorship lends itself to catastrophic failure).
If there is a single person or group of people directing the community, I can imagine them making decisions which anger the rest of the community, making people take sides or split from the group. I’ve seen that happen in forums where the moderators did something controversial, leading to considerable (albeit usually localized) disruption. If the community is directed democratically, I again see people being partisan and taking sides, leading to (potentially vicious) internal politics; and politics is both a mind killer and a major driver of divisiveness (which is typically bad for the community).
Now, to be entirely fair, these are somewhat “worst case” scenarios, and I don’t know how likely they are. However, I am having trouble thinking of any successful online communities which have taken this route. That may just be a failure of imagination, or it could be that something like this hasn’t been tried yet, but it is somewhat alarming. That is largely why I urge caution in the instance.
While your family’s situation is explained by lack of scope insensitivity, I’d like to put forward an alternative. I think the behavior you described also fits with rationalization. If you family had already made up their mind about supporting the Republican party, they could easily justify it to themselves (and to you) by citing a particular close-to-the-heart issue as an iron-clad reason.
Rationalization also explains why “even people who bother thinking for themselves are likely to arrive at the same conclusion as their peers”—it just means that said people are engaging in motivated cognition to come up with reasonable-sounding arguments to support the same conclusions as their peers.
Interesting point! It seems obvious in hindsight that if you reward people for making predictions that correspond to reality, they can benefit both by fitting their predictions to reality or fitting reality to their predictions. Certainly, it is an issue that come up even in real life in the context of sporting betting. That said, this particular spin on things hadn’t occurred to me, so thanks for sharing!
I think the issue you are seeing is that Less Wrong is fundamentally a online community / forum, not a movement or even a self-help group. “Having direction” is not a typical feature of such a medium, nor would I say that it would necessary be a positive feature.
Think about it this way. The majority of the few (N < 10) times I’ve seen explicit criticism of Less Wrong, one of the main points cited was that Less Wrong had a direction, and that said direction was annoying. This usually refereed to Less Wrong focusing on the FAI question and X-risk, though I believe I’ve seen the EA component of Less Wrong challenged as well. By its nature, having direction is exclusionary—people who disagree with you stop feeling welcome in the community.
With that said, I strongly caution about trying to change Less Wrong to import direction to the community as a whole (e.g. by having an official “C.E.O”). With that said, organizing a sub-movement within Less Wrong for that sort of thing carries much less risk of alienating people. I think that would be the most healthy direction to take it, plus it allows you to grow organically (since people can easily join/leave your movement and you don’t need to get the entire community mobilized to get started).
I consider philosophy to be a study of human intuitions. Philosophy examines different ways to think about a variety of deep issues (morality, existence, etc.) and tries to resolve results that “feel wrong”.
On the other hand, I have very rarely heard it phrased this way. Often, philosophy is said to be reasoning directly about said issues (morality, existence, etc.), albeit with the help of human intuitions. This actually seems to be an underlying assumption of most philosophy discussions I’ve heard. I actually find that mildly disconcerting, given that I would expect it to confuse everyone involved with substantial frequency.
If anyone knows of a good argument for the assumption above, I would really like to hear it. I’ve only seen it assumed, never argued.
To address your first question: this has to do with scope insensitivity, hyperbolic discounting, and other related biases. To put it bluntly, most humans are actually pretty bad at maximizing expected utility. For example, when I first head about x-risk, my thought process was definitely not “humanity might be wiped out—that’s IMPORTANT. I need to devote energy to this.” It was more along the lines of “huh; That’s interesting. Tragic, even. Oh well; moving on...”
Basically, we don’t care much about what happens in the distant future, especially if it isn’t guaranteed to happen. We also don’t care much more about humanity than we do about ourselves plus our close ones. Plus we don’t really care about things that don’t feel immediate. And so on. Then end result is that most people’s immediate problems are more important to them then x-risk, even if the latter might be by far the more essential according to utilitarian ethics.
Since the question is about potential dangers, I think it is worth assuming the worst here. Also, realistically, we don’t have a magic want to pop things into existence by fiat so I would guess that by default if such an AI was created it would be created with ML.
So lets say that this is trained largely autonomously with ML. Is there some way that would result in dangers outside the four already-mentioned categories?