Done. Thank you for running these.
lavalamp
Karma: 2,090
Check out the previous discussion Luke linked to: http://lesswrong.com/lw/c45/almost_every_moral_theory_can_be_represented_by_a/
It seems there’s some question about whether you can phrase deontological rules consequentially—to make this more formal that needs to be settled. My first thought is that the formal version of this would say something along the lines of “you can achieve an outcome that differs by only X%, with a translation function that takes rules and spits out a utility function, which is only polynomially larger.” It’s not clear to me how to define a domain in such a way as to allow you to compute that X%.
...unfortunately, as much as I would like to see people discuss the moral landscape instead of the best way to describe it, I have very little time lately. :/
(Sorry for slow response. Super busy IRL.)
If a consequentialism talks about murder being bad, they mean that it’s bad if anybody does it.
Not necessarily. I’m not saying it makes much sense, but it’s possible to construct a utility function that values agent X not having performed action Y, but doesn’t care if agent Z performs the same action.
It is technically true that all of these ethical systems are equivalent, but saying which ethical system you use nonetheless carries a lot of meaning.
a) After reading Luke’s link below, I’m still not certain if what I’ve said about them being (approximately) isomorphic is correct… b) Assuming my isomorphism claim is true enough, I’d claim that the “meaning” carried by your preferred ethical framework is just framing.
That is, (a) imagine that there’s a fixed moral landscape. (b) Imagine there are three transcriptions of it, one in each framework. (c) Imagine agents would all agree on the moral landscape, but (d) in practice differ on the transcription they prefer. We can then pessimistically ascribe this difference to the agents preferring to make certain classes of moral problems difficult to think about (i.e., shoving them under the rug).
Deontology and virtue ethics don’t care about getting things done.
I maintain that this is incorrect. The framework of virtue ethics could easily have the item “it is virtuous to be the sort of person who gets things done.” And “Make things happen, or else” could be a deontological rule. (Just because most examples of these moral frameworks are lame doesn’t mean that it’s a problem with the framework as opposed to the implementation.)
If indeed the frameworks are isomorphic, then actually this is just another case humans allowing their judgment to be affected by an issue’s framing. Which demonstrates only that there is a bug in human brains.
I think so. I know they’re commonly implemented without that feedback loop, but I don’t see why that would be a necessary “feature”.
Which is why I said “in the limit”. But I think, if it is true that one can make reasonably close approximations in any framework, that’s enough for the point to hold.
Hm, thanks.
Are you saying that some consequentialist systems don’t even have deontological approximations?
It seems like you can have rules of the form “Don’t torture… unless by doing the torture you can prevent an even worse thing” provides a checklist to compare badness …so I’m not convinced?
How does it change the numbers if you condition on the fact that Alcor has already been around for 40 years?
Absolutely, granted. I guess I just found this post to be an extremely convoluted way to make the point of “if you maximize the wrong thing, you’ll get something that you don’t want, and the more effectively you achieve the wrong goal, the more you diverge from the right goal.” I don’t see that the existence of “marketing worlds” makes maximizing the wrong thing more dangerous than it already was.
Additionally, I’m kinda horrified about the class of fixes (of which the proposal is a member) which involve doing the wrong thing less effectively. Not that I have an actual fix in mind. It just sounds like a terrible idea—”we’re pretty sure that our specification is incomplete in an important, unknown way. So we’re going to satisfice instead of maximize when we take over the world.”
90% agree, one other thing you may not know: both dropbox and google drive have options to automatically upload photos from your phone, and you don’t have to sync your desktop with them. So it’s not clear that they merely double the needed space.
I think your expanded point #6 fails to consider alternative pressures for hard drive & flash memory. Consider places like dropbox; they represent a huge demand for cheap storage. People probably (?) won’t want huge(er) drives in their home computers going forward, but they are quite likely to want cloud storage if it comes down another order of magnitude in price. Just because people don’t necessarily directly consume hard drives doesn’t mean there isn’t a large demand.
Consider also that many people have high MP digital cameras, still and video. Those files add up quickly.
It sounds like, “the better you do maximizing your utility function, the more likely you are to get a bad result,” which can’t be true with the ordinary meanings of all those words. The only ways I can see for this to be true is if you aren’t actually maximizing your utility function, or your true utility function is not the same as the one you’re maximizing. But then you’re just plain old maximizing the wrong thing.
Hypocrisy is only a vice for people with correct views. Consistently doing the Wrong Thing is not praiseworthy.
Unfortunately, it’s much easier to demonstrate inconsistency than incorrectness.
Ah, thank you for the explanation. I have complained about the proposed method in another comment. :)
http://lesswrong.com/lw/jao/siren_worlds_and_the_perils_of_overoptimised/aso6
The IC correspond roughly with what we want to value, but differs from it in subtle ways, enough that optimising for one could be disastrous for the other. If we didn’t optimise, this wouldn’t be a problem. Suppose we defined an acceptable world as one that we would judge “yeah, that’s pretty cool” or even “yeah, that’s really great”. Then assume we selected randomly among the acceptable worlds. This would probably result in a world of positive value: siren worlds and marketing worlds are rare, because they fulfil very specific criteria. They triumph because they score so high on the IC scale, but they are outnumbered by the many more worlds that are simply acceptable.
Implication: the higher you set your threshold of acceptability, the more likely you are to get a horrific world. Counter-intuitive to say the least.
TL;DR: Worlds which meet our specified criteria but fail to meet some unspecified but vital criteria outnumber (vastly?) worlds that meet both our specified and unspecified criteria.
Is that an accurate recap? If so, I think there’s two things that need to be proven:
There will with high probability be important unspecified criteria in any given predicate.
The nature of the unspecified criteria is such that it is unfulfilled in a large majority of worlds which fulfill the specified criteria.
(1) is commonly accepted here (rightly so, IMO). But (2) seems to greatly depend on the exact nature of the stuff that you fail to specify and I’m not sure how it can be true in the general case.
EDIT: The more I think about this, the more I’m confused. I don’t see how this adds any substance to the claim that we don’t know how to write down our values.
EDIT2: If we get to the stage where this is feasible, we can measure the size of the problem by only providing half of our actual constraints to the oracle AI and measuring the frequency with which the hidden half happen to get fulfilled.
Possibly of interest: Help Teach 1000 Kids That Death is Wrong. http://www.indiegogo.com/projects/help-teach-1000-kids-that-death-is-wrong
(have not actually looked in detail, have no opinion yet)
I think you’re getting downvoted for your TL;DR, which is extremely difficult to parse. May I suggest:
TL;DR: Treating “computers running minds” as discrete objects might cause a paradox in probability calculations that involve self-location.
An extremely low prior distribution of life is an early great filter.