Arbitrary and biased are value judgments.
And they’ree built into rationality.
Whether more than one non-contradictory value system exists is the topic of the conversation, isn’t it?
Non contradictoriness probably isn’t a sufficient condition for truth.
Under a totally neutral lens, which implements no values at all, no system of behavior should look any more or less silly than any other?
Including arbitrary, biased or contradictory ones? Are there values built into logic/rationality?
Not very rational for those to adopt a losing strategy (from their point of view), is it? Especially since they shouldn’t reason from a point of “I could be the king”. They aren’t, and they know that. No reason to ignore that information, unless they believe in some universal reincarnation or somesuch.
Someone who adopts the “I don;t like X, but I respect peoples right to do it” approach is sacrificing some of their values to their evaluation of rationality and fairness. They would not do that if their rationality did not outweigh other values, But they are not having all their values maximally satisfied, so in that sense they are losing out.
Yes. Which is why rational agents wouldn’t just go and change/compromise their terminal values, or their ethical judgements (=no convergence).
There’s no evidence of terminal values. Judgements can be updated without changing values.
Starting out with different interests. A strong clippy accomodating a weak beady wouldn’t be in its best self-interest. It could just employ a version of morality which is based on some tweaked axioms, yielding different results.
Not all agents are interested in physics or maths. Doesn’t stop their claims being objetive.
You are the monarch in that society, you do not need to guess which role you’re being born into, you have that information. You don’t need to make all the slaves happy to help your goals, you can just maximize your goals directly. You may choose any moral principle you want to govern your actions. The Categorical Imperative would not give you the best result.
For what value of “best”? If the CI is the correct theory of morality, it will necessarily give your the morally best result. Maybe your complaint is that it wouldn’t maximise your personally utility. But I don’t see why you would expect that. Things like utilitarianism that seek to maximise group utility, don’t promise to make everyone blissfully happy individually. Some will lose out.
A different scenario: Clippy and Anti-Clippy sit in a room. Why can they not agree on epistemic facts about the most accurate laws of physics and other Aumann-mandated agreements, yet then go out and each optimize/reshape the world according to their own goals? Why would that make them not rational?
It would be irrational for Clippy to sing up to an agreement with Beady according to which Beady gets to turn Clippy and all his clips into beads. It is irrational for agents to sign up to anyhting which is not in their interests, and it is not in their interests to have no contract at all. So rational agents, even if they do not converge on all their goals, will negotiate contracts that minimise their disutility Clippy and Beady might take half the universe each.
Lastly, whatever Kant’s justification, why can you not optimize for a different principle—peak happiness versus average happiness, what makes any particular justifying principle correct across all—rational—agents.
If you think RAs can converge on an ultimately correct theory of physics (which we don’t have), what is to stop them converging on the correct theory of morality, which we also don’t have?
How many 5 year olds have the goal of Sitting Down WIth a Nice Cup of Tea?
Nobody cares about clips except clippy. Clips can only seem important because of Clippy’s egotistical bias.
However, both the pebble-sorters and myself share one key weakness: we cannot examine ourselves from the outside; we can’t see our own source code.
Being able to read all you source code could be ultimate in self-reflection (absent Loeb’s theorem), but it doens’t follow that those who can’t read their source-code can;t self reflect at all. It’s just imperfect, like everything else.
I don’t require their values to converge, I require them to accept the truths of certain claims. This happens in real life. People say “I don’t like X, but I respect your right to do it”. The first part says X is a disvalue, the second is an override coming from rationality.
First of all, thanks for the comment. You have really motivated me to read and think about this more
That’s what I like to hear!
If there are no agents to value something, intrinsically or extrinsically, then there is also nothing to act on those values. In the absence of agents to act, values are effectively meaningless. Therefore, I’m not convinced that there is objective truth in intrinsic or moral values.
But there is no need for morality in the absence of agents. When agents are there, values will be there, when agents are not there, the absence of values doesn’t matter.
I think there is a difference between it being objectively true that, in certain circumstances, the values of rational agents converge, and it being objectively true that those values are moral. A rational agent can do really “bad” things if the beliefs and intrinsic values on which it is acting are “bad”. Why else would anyone be scared of AI?
I don’t require their values to converge, I require them to accept the truths of certain claims. This happens in real life. People say “I don’t like X, but I respect your right to do it”. The first part says X is a disvalue, the second is an override coming from rationality.
But is the justification for its global applicability that “if everyone lived by that rule, average happiness would be maximized”?
Well, not, that’s not Kant’s justification!
That (or any other such consideration) itself is not a mandatory goal, but a chosen one.
Why would a rational agent choose unhappiness?
If you find yourself to be the worshipped god-king in some ancient Mesopotanian culture, there may be many more effective ways to make yourself happy, other than the Categorical Imperative.
Yes, but that wouldn’t count as ethics. You wouldn’t want a Universal Law that one guy gets the harem, and everyone else is a slave, because you wouldn’t want to be a slave, and you probably would be. This is brought out in Rawls’ version of Kantian ethics: you pretend to yourself that you are behind a veil that prevents you knowing what role in society you are going to have, and choose rules that you would want to have if you were to enter society at random.
My argument against moral realism and assorted is that if you had an axiomatic system from which it followed that strawberry is the best flavor of ice cream, but other agents which are just as intelligent with just as much optimizing power could use different axiomatic systems leading to different conclusions,
You don’t have object-level stuff like ice cream or paperclips in your axioms (maxims), you have abstract stuff, like the Categorical Imperative. You then arrive at object level ethics by plugging in details of actual circumstances and values. These will vary, but not in an arbitrary way, as is the disadvantage of anything-goes relativism.
how could one such system possibly be taken to be globally correct and compelling-to-adopt across agents with different goals?
The idea is that things like the CI have rational appeal.
Once you’ve taken a rational agent apart and know its goals and, as a component, its ethical subroutines, there is no further “core spark” which really yearns to adopt the Categorical Imperative.
Rational agents will converge on a number of things because they are rational. None of them will think 2+2-=5.
Yea, honestly I’ve never seen the exact distinction between goals which have an ethics-rating, and goals which do not
A number of criteria have been put forward. For instance, do as you would be done by. If you don’t want to be murdered, murder is not an ethical goal.
My problem with “objectively correct ethics for all rational agents” is, you could say, where the compellingness of any particular system comes in. There is reason to believe an agent such as Clippy could exist, and its very existence would contradict some “‘rational’ corresponds to a fixed set of ethics” rule. If someone would say “well, Clippy isn’t really rational then”, that would just be torturously warping the definition of “rational actor” to “must also believe in some specific set of ethical rules”.
The argument is not that rational agents (for some vaue of “rational”) must believe in some rules, it is rather that they must not adopt arbitrary goals. Also, the argument only requires a statistical majority of rational agents to converge, because of the P<1.0 thing.
Should some bionically enhanced human, or an upload on a spacestation which doesn’t even have parents, still share all the same rules for “good” and “bad” as an Amazon tribe living in an enclosed reservation?
Maybe not. The important thing is that variations in ethics should not be arbitrary—they should be systematically related to variations in circumstances.
Spun off from what, and how?
I am not sure I can expalin that succintly at the moment. It is also hard to summarise how you get from counting apples to transfinite numbers.
Why does their being rational demand that they have values in common? Being rational means that they necessarily share a common process, namely rationality, but that process can be used to optimize many different, mutually contradictory things. Why should their values converge?
Rationality is not an automatic process, it is skill that has to be learnt and consciously applied. Individuals will only be rational if their values prompt them to. And rationality itself implies valuing certain things (lack of bias, non arbitrariness).
So what if a paperclipper arrives at “maximize group utility,” and the only relevant member of the group which shares its conception of utility is itself, and its only basis for measuring utility is paperclips? The fact that it shares the principle of maximizing utility doesn’t demand any overlap of end-goal with other utility maximizers.
Utilitarians want to maximise the utiity of their groups, not their own utility. They don;t have to believe the utlity of others is utilitous to them, they just need to feed facts about group utility into an aggregation function. And, using the same facts and same function, different utilitarians will converge. That’s kind of the point.
But, as I’ve pointed out previously, intuitions are often unhelpful, or even actively misleading, with respect to locating the truth.
Compared to what? Remember, I am talking about foundational intuitions, the kind at the bottom of the stack. The empirical method of locating the truth rests on the intuition that the senses reveal a real external world. Which I share. But what proves it? That’s the foundational issue.
Yes, but the fact that the universe itself seems to adhere to the logical systems by which we construct mathematics gives credence to the idea that the logical systems are fundamental, something we’ve discovered rather than producing. We judge claims about nonobserved mathematical constructs like transfinites according to those systems,
But claims about transfinities don’t correspond directly to any object. Maths is “spun off” from other facts, on your view. So, by analogy, moral realism could be “spun off” without needing any Form of the Good to correspond to goodness.
Metaethical systems usually have axioms like “Maximising utility is good”.
But utility is a function of values. A paperclipper will produce utility according to different values than a human.
You seem to be assumig that morality is about individual behaviour. A moral realist system like utiitarianism operates at the group level, and woud take paperclipper values into account along with all others. Utilitarianism doens’t care what values are, it just sums or averages them.
Or perhaps you are making the objection that an entity woud need moral values to care about the preferences of others in the first place. That is addressed by, another kind of realism, the rationality-based kind, which starts from noting that rational agents have to have some value in common, because they are all rational.
Why would most rational minds converge on values?
a) they don’t have to converge on preferences, since thing like utilitariansim are preference-neutral.
b) they already have to some extent because they are rational
Most human minds converge on some values, but we share almost all our evolutionary history and brain structure. The fact that most humans converge on certain values is no more indicative of rational minds in general doing so than the fact that most humans have two hands is indicative of most possible intelligent species converging on having two hands.
I was talking about rational minds converging on the moral claims, not on values.. Rational minds can converge on “maximise group utility” whilst what is utilitous varies considerably.
Philosphers talk about intuitions, because that is the term for something foundational that seems true, but can’t be justified by anything more foundational. LessWrongians don’t like intuitions, but don’t see to be able to explain how to manage without them.
It seems like you’re equating intuitions with axioms here.
Axioms are formal statements, intuitions are gut feelings tha are often used to justify axioms.
We can (and should) recognize that our intuitions are frequently unhelpful at guiding us to he truth, without throwing out all axioms.
There is another sense of “intuition” where someone feels that it’s going to rain tomorrow or something. They’re not the foundational kind.
And philosophers frequently fall into the pattern of believing that other philosophers disagree with each other due to failure to understand the problems they’re dealing with.
So do they call for them to be fired?
Since no claim has a probability of 1.0, I only need to argue that a clear majority of rational minds converge.
The standards by which we judge the truth of mathematical claims are not just inside us.
How do we judge claims about transfinite numbers?
One object plus another object will continue to equal two objects whether or not there are any living beings to make that judgment. Math is not something we’ve created within ourselves, but something we’ve discovered and observed.
If our mathematical models ever stop being able to predict in advance the behavior of the universe, then we will have rather more reason to doubt that the math inside us is different from the math outside of us.
Mathematics isn’t physics. Mathematicians prove theorems from axioms, not from experiments.
Provide evidence that ethics is a whole separate modue, and not part of general reasoning ability.
My assertion is that, if we judge ethics as a rational system, innate values are among the axioms that the system is predicated on.
Not necessarily. Eg, for utilitarians, values are just facts that are plugged into the metaethics to get concrete actions.
You cannot prove the axioms of a system within that system, and an ethical system predicated on premises like “happiness is good” will not itself be able to prove the goodness of happiness.
Metaethical systems usually have axioms like “Maximising utility is good”.
While we could suppose that the axioms which our ethical systems are predicated on are objectively true, we have considerable reason to believe that we would have developed these axioms for adaptive reasons, even if there were no sense in which objective moral axioms exist, and we do not have evidence which suggests that objective, independently existing true moral axioms do exist.
I am not sure what you mean by “exist” here. Claims are objectively true if most rational minds converge on them. That doesn’t require Objective Truth to float about in space here.
Please explain why moral intuitions don’t work that way.
People can be induced to strongly support opposing responses to the same moral dilemma, just by rephrasing it differently to trigger different heuristics. Our moral intuitions are incoherent.
Does that mean we can;t use moral intuitions at all, or that they must be used with caution?
I don’t think I understand this, can you rephrase it?
Philosphers talk about intuitions, because that is the term for something foundational that seems true, but can’t be justified by anything more foundational. LessWrongians don’t like intuitions, but don’t see to be able to explain how to manage without them.
I do not recall any creditable attempts, which places me in a disadvantaged position with respect to locating them.
Did you post any comments explaining to the professional philosophers where they had gone wrong?
Imagine the conversation we’re having now going on for eighty years, and neither of us has changed our minds. If you didn’t find my arguments convincing, and I hadn’t budged in all that time, don’t you’d think you’d start to suspect that I was particularly thick?
I don;’t see the problem. Philosophical competence is largely about understanding the problem.
I haven’t seen anything to say that is for meta discussion, it mostly isn’t de facto, and I haven’t seen a “take it elsewhere” notice anywhere as an aternative to downvote and delete.
All that’s needed to is reject the idea that there are some mysterious properties to sensation which somehow violate basic logic and the principles of information theory.
Blatant strawman.
Banning all meta discussion on LW of any kind seems like an increasingly good idea—in terms of it being healthy for the community, or rather, meta of any kind being unhealth
Have you considered having a separate “place” for it?
Well, that justifies moral realism.
...or its an emergent feature, or they can update into something that works that way. You are tacitly assuming that you clipper is barely an AI at all...that is just has certain functions it performs blindly because its built that way. But a supersmart, uper-rational clipper has to be able to update. By hypothesis, clippers have certain functionalities walled off from update. People are messilly designed and unlikely to work that way. So are likely AIs and aliens.
Only rational agents, not all mindful agents, will have what it takes to derive objective moral truths. They don’t need to converge on all their values to converge on all their moral truths, because ratioanity can tell you that a moral claim is true even if it is not in your (other) interests. Individuals can value rationality, and that valuation can override other valuations.
Only rational agents, not all mindful agents, will have what it takes to derive objective moral truths. The further claim that agents will be motivated to do derive moral truths., and to act on them, requires a further criterion. Morality is about regulating behaviour in a society, So only social rational agents will have motivation to update. Again, they do not have to converge on values beyond the shared value of sociality.