PrawnOfFate

• PrawnOfFate 25 Apr 2013 16:15 UTC

  0 points

  in reply to: MugaSofer’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  I don’t get it: any agent that fooms becomes superintelligent. It’s values don’t necessarily change at all, nor does its connection to its society.

• PrawnOfFate 25 Apr 2013 14:31 UTC

  −6 points

  in reply to: TheOtherDave’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  Asserting that some bases for comparison are “moral values” and others are merely “values” implicitly privileges a moral reference frame.

  I don’t see why. The question of what makes a value a moral value is metaethical, not part of object-level ethics.

  Again: if I decide that this hammer is better than that hammer because it’s blue, is that valid in the sense you mean it?

  It isn’t valid as a moral judgement because “blue” isn’t a moral judgement, so a moral conclusion cannot validly follow from it.

  Beyond that, I don’t see where you are going. The standard accusation of invalidity to judgements of moral progress, is based on circularity or question-begging. The Tribe who Like Blue things are going to judge having all hammers painted blue as moral progress, the Tribe who Like Red Things are going to see it as retrogressive. But both are begging the question—blue is good, because blue is good.

• PrawnOfFate 25 Apr 2013 14:01 UTC

  −4 points

  in reply to: MugaSofer’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  UFAI is about singletons. If you have an AI society whose members compare notes and share information—which ins isntrumentally useful for them anyway—your reduce the probability of singleton fooming.

• PrawnOfFate 25 Apr 2013 13:50 UTC

  −7 points

  in reply to: TheOtherDave’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  The argument against moral progress is that judging one moral reference frame by another is circular and invalid—you need an outside view that doesn’t presuppose the truth of any moral reference frame.

  The argument for is that such outside views are available, because things like (in)coherence aren’t moral values.

• PrawnOfFate 25 Apr 2013 13:38 UTC

  −7 points

  in reply to: ArisKatsaris’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  So much for avoiding the cliche.

• PrawnOfFate 25 Apr 2013 13:09 UTC

  −4 points

  in reply to: TheOtherDave’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  And is that valid or not? If you can validly decide some systems are better than others, you are some of the way to deciding which is best.

  What links here?

  • TheOtherDave's comment on Wel­come to Less Wrong! (5th thread, March 2013) by orthonormal (25 Apr 2013 16:11 UTC; 4 points)

• PrawnOfFate 25 Apr 2013 12:56 UTC

  −6 points

  in reply to: MugaSofer’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  Sorry..did you mean FAI is about societies, or FAI is about singletons?

  But if ethics does emerge as an organisational principle in socieities, that’s all you need for FAI. You don’t even to to worry about one sociopathic AI turning unfriendly, because the majority will be able to restrain it.

• PrawnOfFate 25 Apr 2013 12:44 UTC

  −7 points

  in reply to: MugaSofer’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  If “better” is defined within a reference frame, there is not sensible was of defining moral progress. That is quite a hefty bullet to bite: one can no longer say that South Africa is better society after the fall of Apartheid, and so on.

  But note, that “better” doesn’t have to question-beggingly mean “morally better”. it could mean “more coherent/​objective/​inclusive” etc.

• PrawnOfFate 25 Apr 2013 12:34 UTC

  0 points

  in reply to: Nornagest’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  I’m increasingly baffled as to why AI is always brought in to discussions of metaethics. Societies of rational agents need ethics to regulate their conduct. Out AIs aren’t sophisticated enough to live in their own socieities. A wireheading AI isn’t even going to be able to survive “in the wild”. If you could build an artificial society of AI, then the questions of whether they spontaneously evolved ethics would be a very interesting and relevant datum. But AIs as we know them aren’t good models for the kinds of entities to which morality is relevant. And Clippy is particularly exceptional example of an AI. So why do people keep saying “Ah, but Clippy...”...?

• PrawnOfFate 25 Apr 2013 11:28 UTC

  −4 points

  in reply to: TheOtherDave’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  Isn’t the idea of moral progress based on one reference frame being better than another?

• PrawnOfFate 24 Apr 2013 22:43 UTC

  −8 points

  in reply to: Bugmaster’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  As far as I understand, if anything like objective morality existed, it would be a property of our physical reality, similar to fluid dynamics or the electromagnetic spectrum or the inverse square law that governs many physical interactions. The same laws of physics that will not allow you to fly to Mars on a balloon will not allow you to perform certain immoral actions (at least, not without suffering some severe and mathematically predictable consequences).

  Objective facts, in the sense of objectively true statements, can be derived from other objetive facts. I don’t know why you think some separate ontlogical category is cagtegory is required. I also don’t know why you think the universe has to do the punishing. Morality is only of interest to the kind of agent that has values and lives in societies. Sanctions against moral lapses can be arranged at the social level, along with the inculcation of morality, debate about the subject, and so forth. Moral objectivism only supplies a good, non-arbnitrary epistemic basis for these social institutions. It doesn;t have to throw lightning bolts.

• PrawnOfFate 24 Apr 2013 2:23 UTC

  −4 points

  in reply to: Nornagest’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  At last, an interesting reply!

• PrawnOfFate 24 Apr 2013 1:44 UTC

  −8 points

  in reply to: nshepperd’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  The Futility of Emergence

  “Emergent” in this context means “not explicitly programmed in”. There are robust examples.

  A paperclipper no more has a wall stopping it from updating into morality than my laptop has a wall stopping it from talking to me.

  Your laptop cannot talk to you because the natural language is an unsolved problem.

  Does a stone roll uphill on a whim?

  Not wanting to do something is not the slightest guarantee of not actually doing it.f

  An AI can update its values because value drift is an unsolved problem

  Clippers can’t update their values by definition, but you can’t define anything into existence or statistical significance.

  You do not update into pushing pebbles into prime-numbered heaps because you’re not programmed to do so.

  Not programmed to, or programmed not to? If you can code up a solution to value drift, lets see it. Otherwise, note that Life programmes can update to implement glider generators without being “programmed to”.

• PrawnOfFate 24 Apr 2013 1:24 UTC

  −11 points

  in reply to: Bugmaster’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  1). We lack any capability to actually replace our core values

  ...voluntarily.

  2). We cannot truly imagine what it would be like not to have our core values.

  Which is one of the reasons we cannot keep values stable by predicting the effects of whatever experiences we choose to undergo.How does your current self predict what an updated version would be like? The value stability problem is unsolved in humans and AIs.

• PrawnOfFate 23 Apr 2013 22:51 UTC

  −10 points

  in reply to: Desrtopa’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  but it hasn’t compelled us to try and replace our values.

  The ethical outlook of the Western world has changed greatly in the past 150 years.

• PrawnOfFate 23 Apr 2013 21:38 UTC

  −7 points

  on: Nor­ma­tivity and Meta-Philosophy

  “better according to some shared, motivating standard or procedure of evaluation”,

  I broadly agree. My thinking ties shoulds and musts to rules and payoffs. Wherever you are operating a set of rules (which might be as localised as playing chess), you have certain localised “musts”.

  It seems to me that different people (including different humans) can have different motivating standards and procedures of evaluation, and apparent disagreements about “should’ sentences can arise from having different standards/​procedures, or disagreement about whether something is better according to a shared standard/​procedure.

  I’m very resitant to the idea, promoted by EY in the thread you refenced, that the meaning of should changes. Does he think chess players have a different concept of “rule” to poker players?

• PrawnOfFate 23 Apr 2013 21:25 UTC

  −7 points

  in reply to: Eliezer Yudkowsky’s comment on: A Less Wrong sin­gu­lar­ity ar­ti­cle?

  the criterion by which they choose is simply not that which we name morality.

  Even if “morality” means “criterion for choosing..”? Their criterion might have a different referent, but that does not imply a different sense. cf. “This planet”. Out of the two, sense has more to do with meaning, since it doesn’t change with changes of place and time.

• PrawnOfFate 23 Apr 2013 21:12 UTC

  −9 points

  in reply to: TimS’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  Arbitrary and Bias are not defined properties in formal logic. The bare assertion that they are properties of rationality assumes the conclusion.

  There’s plenty of material on this site and elsewhere advising rationalists to avoid arbitrariness and bias. Arbitrariness and bias are essentially structural/​functional properties, so I do not see why they could not be given formal definitions.

  Sure, but the discussion is partially a search for other criteria to evaluate of the truth of moral propositions. Arbitrary is not such a criteria.

  Arbitrary and biased claims are not candidates for being ethical claims at all.

• PrawnOfFate 23 Apr 2013 19:39 UTC

  −8 points

  in reply to: DaFranker’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  I am not suggesting that human ethics is coincidentally universal ethics. I am suggesting that if neither moral realism nor relativism is initially discarded, one can eventually arrive at a compromise position where rational agents in a particular context arrive at a non arbitrary ethics which is appropriate to that context.

• PrawnOfFate 23 Apr 2013 18:39 UTC

  −6 points

  in reply to: DaFranker’s comment on: Wel­come to Less Wrong! (5th thread, March 2013)

  It’s uncontrovesial that rational agents need to update, and that AIs need to self-modify. The claim that values are in either case insulated from updates is the extraordinary one. The Cipper theory tells you that you could build something like that if you were crazy enough. Since Clippers are contrived, nothing can be inferred from them about typical agents. People are messy, and can accidentally update their values when trying to do something else, For instance, LukeProg updated to “atheist” after studying Christian apologetics for the opposite reason.