In fact, the link takes me to a Google login screen—which makes it look like that phishing scheme Google just warned us about. There is no reason I should have to login just to read a blog posting.
As far as I can tell, it is a real Google login screen, but still …
This comment is racist to the point of being ridiculous. It denigrates humans as petty and subject to being corrupted by power while denying that Clippies have any such negative attributes. Classic racism.
Furthermore, there is an implicit claim that the reason for the moral superiority of Clippies over humans lies in the difference in their origins. Again, classic racism.
Perhaps Clippies use words differently, but the way humans use words, it is not racism to project one’s own race’s characteristics onto another race. It is racist to fail to make that projection.
Can you elaborate?
“World of is” vs “World of is not” is a false dichotomy.
“I lose interest” is possibly the worst and most uncharitable of all forms of philosophical rhetoric. (Hence my ‘turnabout’.)
Luke gave a false impression of what non-naturalistic ethics comprises by providing only a single example of such a position—a position which was absurd because it presumed the existence of a deity.
Even Hume allowed that works of mathematics need not be “committed to the flames”. Mathematics, to my mind, does not deal with the “world of is”, neither does it deal with the “world of is not”. Yet if someone were to provide a non-reductionist, but axiomatic, ‘definition’ of ‘good’ using the methods and practices of mathematical logic, I certainly would not dismiss it as “uninteresting”.
In The Is_Ought Gap, Luke writes
If someone makes a claim of the ‘ought’ type, either they are talking about the world of is, or they are talking about the world of is not. If they are talking about the world of is not, then I quickly lose interest because the world of is not isn’t my subject of interest.
Ironically, this is where I quickly lost interest in this article, because glib word-play isn’t my subject of interest.
This comment might have been more appropriate as a response to this request for expressions of interest or disinterest.
But since it wasn’t made there, lets discuss the issues it raises.
I agree that LW should not be in the business of producing scientific intros. But we do have the precedent of the QM sequence. It was justified as relevant to various “demon-exorcising” (my term) tasks relevant to rationality; I don’t dispute this justification. But surely an intro to systems biology (including topics of the simulation of biological systems) is relevant in a group which frequently discusses the prospects for “uploading” as a path to AI, brain scanning as a path to inferring preferences, and simulation as a technique for cryonic resuscitation.
After reading this posting, I find it very difficult to expect this kind of simulated biology to be possible soon, and I also find it less easy to anticipate that synthetic life research will deliver anything particularly interesting in terms of nanotechnology soon. In other words, my anticipations have been constrained by this excellent article.
As such, we’d be unlikely to get what we really want if the world was re-engineered in accordance with a description of what we want that came from verbal introspective access to our motivations. Less naive proposals would involve probing the neuroscience of motivation at the algorithmic level. (Footnote: Inferring desires from behavior alone probably won’t work, either.)
There is something a bit bizarre about proposing to extract preferences by scanning brains (because raw behavior and reported introspection are not authentic and primitive enough), and then to insist that these fundamental preferences be extrapolated through a process of reflective equilibrium—thereby becoming more refined.
Is there some argument justifying the claim that what I really want is not what I say I want, and not what I do, but rather what the technician running the scanner says I want. By what definition of “really” is this what I really want? By what definition of “want”?
Note: In some ways this echoes steven0461, but I think it makes some additional points.
Where did Sloman claim that AI would be seen as important within a few years?
I inferred that he would characterize it as important in that time frame from:
… within a few years, if there remain any philosophers who are not familiar with some of the main developments in artificial intelligence, it will be fair to accuse them of professional incompetence …
together with a (perhaps unjustified) assumption that philosophers refrain from calling their colleagues “professionally incompetent” unless the stakes are important. And that they generally do what is fair.
Interesting. Though in the scenario I suggested there is no suffering. Only an opportunity to deploy pleasure (ice cream).
I’m curious as to your reasons why you hold the aliens morally responsible for the human clones—I can imagine several reasons, but wonder what yours are. Also, I am curious as to whether you think that the existence of someone with greater moral responsibility than our own acts to decrease or eliminate the small amount of moral responsibility that we Earthlings have in this case.
According to the link:
Aaron Sloman is a philosopher and researcher on artificial intelligence and cognitive science.
So, we have a spectacular mis-estimation of the time frame—claiming 33 years ago that AI would be seen as important “within a few years”. That is off by one order of magnitude (and still counting!) Do we blame his confusion on the fact that he is a philosopher, or was the over-optimism a symptom of his activity as an AI researcher? :)
ETA:
as irresponsible as giving a degree course in physics which includes no quantum theory.
I’m not sure I like the analogy. QM is foundational for physics, while AI merely shares some (as yet unknown) foundation with all those mind-oriented branches of philosophy. A better analogy might be “giving a degree course in biology which includes no exobiology”.
Hmmm. I’m reasonably confident that biology degree programs will not include more than a paragraph on exobiology until we have an actual example of exobiology to talk about. So what is the argument for doing otherwise with regard to AI in philosophy?
Oh, yeah. I remember. Philosophers, unlike biologists, have never shied away from investigating things that are not known to exist.
Right. What you are saying is related to the notion of “credible threats”. If other agents can give you disutility with little disutility for themselves, then they have a credible threat against you. And unless you either change your utility function, or find a way of making it much more difficult and costly for them to harm you, the rational course is to give in to the extortion.
One way to make it costly for others to harm you is to join a large coalition which threatens massive retaliation against anyone practicing extortion against coalition members. But notice that if you join such a coalition, you must be willing to bear your share of the burden should such retaliation be necessary.
The alternative I suggested in the grandparent was to change your utility function so as to make you less vulnerable—only care about things you have control over. Unfortunately, this is advice that may be impossible to carry out. Preferences, as several commentators here have pointed out, tend to be incorrigible.
You seem to be collecting some downvotes that should have gone to me. To even things out, I have upvoted three of your comments. Feel free to downvote three of mine.
I fully agree, by the way, on the distinction between the moral relevance of simulated humans, who have no ability to physically influence our world, and the moral relevance of distant people here on earth, who physically influence us daily (though indirectly through a chain of intermediary agents).
Simulated persons do have the ability to influence us informationally, though, even if they are unaware of our existence and don’t recognize their own status as simulations. I’m not sure what moral status I would assign to a simulated novelist—particularly if I liked his work.
ETA: To Normal_Anomaly: I do not deny people the right to care about simulations in terms of their own terminal values. I only deny them the right to insist that I care about simulations. But I do claim the right to insist that other people care about Chinese, for reasons similar to those Tim has offered.
Is there a justification for not-squaring?
Personally, I am unconvinced that there is any fundamental justification for considering anyone’s utility but one’s own. But, if you have reason to respect the principles of democracy, the median stands out as the unique point acceptable to a majority. That is, if you specify any other point, a majority would vote to replace that point by the median.
What’s the appropriate metric on the space of preferences?
That depends on what kinds of preferences you are comparing. If you are looking at the preferences of a single person, the standard construction of that person’s utility function sets the “metric”. But if you attempt to combine the preferences of two people, you either need to use the Nash Bargaining solution or Harsanyi’s procedure for interpersonal comparison. The first gives a result that is vaguely median-like. The second gives an answer that is suitable for use with the mean.
It’s not clear to me what you’re trying to accomplish by calling people who believe such things (many of whom are very smart and have already seriously reflected on these issues) “crazy” without backing up your claims.
I’m not sure why you think I have called anyone crazy. What I said above is that a particular moral notion is crazy.
Perhaps you instead meant to complain that (in the grandparent) I had referred to the persons in question as “moral idiots”. I’m afraid I must plead guilty to that bit of hyperbole.
Anyway, why don’t you make a post detailing your metaethics, as well as your arguments against “universal unreciprocated altruism”?
I am gradually coming to think that there is little agreement here as to what the word metaethics even means. My current understanding is that metaethics is what you do to prepare the linguistic ground so that people operating under different ethical theories and doctrines can talk to each other. Meta-ethics strives to be neutral and non-normative. There are no meta-ethical facts about the world—only definitions that permit discourse and disputation about the facts.
Given this interpretation of “meta-ethics”, it would seem that what you mean to suggest is that I make a post detailing my normative ethics, which would include an argument against “universal unreciprocated altruism” (which I take to be a competing theory of normative ethics).
Luke and/or Eliezer and/or any trained philosopher here: I would appreciate feedback as to whether I finally have the correct understanding of the scope and purpose of meta-ethics.
Tim, we are completely talking past each other here. To restate my position:
Nash in 1953 characterized rational 2 party bargaining with threats. Part of the solution was to make the quantitative distinction between ‘non-credible’ threats (which should be ignored because they cost the threatener so much to carry out that he would be irrational to do so), and ‘credible’ threats—threats which a threatener might rationally commit to carry out.
Since Nash is modeling the rationality of both parties here, it is irrational to resist a credible threat—in fact, to promise to do so is to make a non-credible threat yourself.
Hence, in Nash’s model, cost-less signaling is pointless if both players are assumed to be rational. Such signaling does not change the dividing line between threats that are credible, and rationally should succeed, and those which are non-rational and should fail.
As for the ‘costly signalling’ that takes place when non-credible threats are resisted—that is already built into the model. And a consequence of the model is that it is a net loss to attempt to resist threats that are credible.
All of this is made very clear in any good textbook on game theory. It would save us all a great deal of time if you keep your amateur political theorizing to yourself until you read those textbooks.
I thought you were claiming that nobody should have that morality.
I do claim that. In two flavors.
Someone operating under that moral maxim will tend to dispense with that maxim as they approach reflective equilibrium.
Someone operating under that ‘moral’ maxim is acting immorally—this operationally means that good people should (i.e. are under a moral obligation to) shun such a moral idiot and make no agreements with him (since he proclaims that he cannot be trusted to keep his commitments).
Part of the confusion between us is that you seem to want the word ‘morality’ to encompass all preferences—whether a preference for chocolate over vanilla, or a preference for telling the truth over lying, or a preference for altruism over selfishness. It is the primary business of metaethics to make the distinction between moral opinions (i.e. opinions about moral issues) and mere personal preferences.
So averaging sucks.
Averaging by taking the mean sucks. Averaging by taking the median sucks less. It is a procedure relatively immune to gaming by would-be utility monsters.
The median is usually the ‘right’ utilitarian algorithm in any case. It minimizes total collective distance from the ‘average’. The mean minimizes total collective distance^2 from the ‘average’. There is no justification for squaring.
Maybe if you provided examples of people seeming to say that “costless signaling is an effective way to influence the behavior of rational agents,” we could ask them what they meant, and they might say something like “no signaling is actually costless”.
Ok. That makes some sense. Though I still don’t have a clue as to why you mention “social costs” or “pseudonymous posting”.
So, for the example of people seeming to say that costless signaling is an effective way to influence the behavior of rational agents, I would direct you to the comment to which I was replying. Tim wrote:
Well you don’t want to signal that you give in to extortion. That would just increase the chances of people attempting extortion against you. Better to signal that you are on a vendetta to stamp out extortion—at your personal expense!!!
I interpreted that as advocating costless signaling as a way of influencing the behavior of would-be extortionists. My response to that advocacy: Announcing that I am on a vendetta is cheap talk, and influences no one. No rational agent will believe such self-serving puffery unless I actually experience a level of personal expense commensurate with what I hope to gain by convincing them. Which makes the signaling not just costly, but irrational.
huh?????
I don’t think anybody is suggesting building an explicit “just say ‘No’ to extortion” heuristic into an AI. (I agree we do not have a good definition of “extortion” so when I use the word I use it in an intuitive sense.) We’re trying to find a general decision theory that naturally ends up saying no to extortion (when it makes sense to).
That is pretty incoherent. If you are trying to come up with a general decision theory that wins and also says no to extortion, then you have overdetermined the problem (or will overdetermine it once you supply your definition). If you are predicting that a decision theory that wins will say no to extortion, then it is a rather pointless claim until you supply a definition. Perhaps what you really intend to do is to define ‘extortion’ as ‘that which a winning decision theory says no to’. In which case, Nash has defined ‘extortion’ for you—as a threat which is not credible, in his technical sense.
ETA: Of course the [informal] concept of “credibility” breaks down a bit when all agents are reasoning this way. Which is why the problem is still unsolved!
Why do you say the problem is still unsolved? What issues do you feel were not addressed by Nash in 1953? Where is the flaw in his argument?
Part of the difficulty of discussing this here is that you have now started to use the word “credible” informally, when it also has a technical meaning in this context.
Not surprisingly, to those who have experience with wedrifid. Merely annoyingly. Though in this case he is making an allusion to a well known trope. Google on the phrase “does not mean what you think it means”. If XiXiDu has referenced that “You’re Entitled …” posting before, for roughly the same debunking purpose, then wedrifid’s terse putdown strikes me as rather clever.