Caledonian: I can’t think of anyone EVER choosing to interpret statements as stupid rather than sensible to the degree to which you do on this blog. There is usually NO ambiguity and you still get things wrong and then blame them for being stupid.
In all honesty why do you post here? On your own blog you are articulate and intelligent. Why not stick with that and leave commenting to people who want to actually respond to what people say rather than to straw men?
Laura: Should I have explained it in terms of your being Jewish? Geometry is more about logic and symbols, what the WAIT calls “Verbal” intelligence, than it is about visuospatial intelligence (which may be related to why its on the SAT). Does anyone think that its significantly less common for women to be competent sketch artists than for men to be? That honestly never occurred to me. But hey, you are a med student; how good are you at identifying expected drug-ligand interactions based on molecular shapes compared to similarly trained men (preferably gentile men) who are your approximate equals (have you met any?) at anagrams or at writing papers? If the answer is “quite, thank you” then I was wrong in my attribution to you of strong verbal relative to visuospatial skills. You know your abilities better than I do. Even in that case I think my comment above still stands. It just isn’t an insult to claim that two people differ in their relative strengths, at least so long as those strengths are socially held in approximately equal regard. Why should this be different for groups of people?
I definitely do think that it could be an insult to claim that people or groups differ in relative strengths if the strengths in question are NOT held in similar regard, and I am aware that this is a real tendency. It is also a real and unfortunate fact that the same strengths can be socially held in high regard when held by members of one gender (almost always male) and in low regard in the other gender (almost always female). Really though, that is not plausibly what is going on here. Verbal abilities are held in MUCH higher regard in our society than visuospatial abilities. If anything, the visuospatial tasks are included in non-military IQ tests partly as a habitual carry-over from a time when visuospatial abilities were more useful and more highly regarded, partly for the sake of including a wide variety of tasks and partly for the sake of inflating male IQs to equal those of women.
Z.M. Davis: These days I generally try not to identify as a Transhumanist, Humanist, or even atheist except to people who I have reason to believe will react positively. I don’t have the character to identify as Christian to those who will react positively to that, but maybe someday. Honestly, well, on Facebook I’m “Political Views”, “Other”, “Religion” “Philosopher and Bayesian”. As the latter, any group identity that supposedly rests on belief is out, as I explicitly represent, when unpacked precisely, all my beliefs as probabilistic, even the normative ones. Ordinary discussion really doesn’t consistently do this. The closest word, and it isn’t close, is “Skeptic”. There’s no simple translation between “I believe X and I believe that my ignorance can be quantified” and anything thinkable to someone who doesn’t believe that ignorance can be quantified. I really don’t think that I’m engaged in moral intimidation towards Angel at all, but I don’t think you are being petty. As I mentioned, I think that its a serious concern if people are and is worth bringing up if there is a reason to suspect it. There’s a difference between saying “It seems to me that you are probably a pretty bad person so I am not going to trust you” and “you should stop thinking those thoughts because having those thoughts means that you are a bad person”. Ironically, my guess is that with the right audience (probably a VERY small audience, I don’t mean academic feminists in general) some talk about “privilege” and “dominance” would work well for clarifying this point, but with most audiences it would just add to any confusion.
frelkins: thanks for the references.
Minor note: The most appealing characters in The Odyssey are female, namely Athena and Penelope.
Laura: Men and women have the same average IQ and at a best guess the same ‘g’ (itself an astounding scientific finding given how different they are neuroanatomicaly, but well validated), but differences in specific abilities are so large that people with opposite gender typical relative verbal and visuo-spatial abilities are considered learning disabled (Unless they were Jewish, oddly, as massively superior verbal to visuo-spatial abilities are common among Jews and thus not considered abnormal). Is it terrible to belong to groups that have known strengths and weaknesses or for the weaknesses to be invoked as an explanation of an observed error? Why be upset by explaining one observation in terms of another established one unless arguments are soldiers? Since I’m Jewish I sometimes explain my visuo-spatial errors in the same group terms. Likewise, I imagine that you would not feel any shame about not being able to lift something that a man would be able to lift easily, and that in Africa, where women know how to carry things on their heads in a mechanically efficient manner that depends on broader hips, men probably don’t feel shame at not being able to carry something that women can carry easily. I can tell you for sure that in Kazakhstan men aren’t ashamed of lacking the basic skills to take care of themselves. So, are only members of a group ever allowed to bring attention to that group having any weaknesses at all, even if it is generally acknowledged that the group in question also has strengths? One problem with this approach to gender relations is that without the framework of group specific strengths people are liable to interpret failures as individual general weaknesses and to underestimate the general abilities of individual members of the opposite gender in the name of protecting the honor of the opposite gender. That seems worse to me.
Grant: I would second everything in your post except the last paragraph.
Angel: I sincerely do apologize for making you feel bad, and I certainly won’t use your name in the future, as hurting people is bad, hurting people in ways that make them likely to behave worse is worse, and my not trusting the person I harm or not considering them to have good intentions doesn’t make it better.
That said, NO, it is NOT wrongwrongwrong to tell people that its a bad idea to adopt such and such a label because the people who typically use that label are in some respect immoral, crazy, or even simply widely believed to be so. If you call yourself a Communist I will point out that you are choosing a label identified most strongly with mass murderers, and if you call yourself a Christian, one identified with warmongering anti-rationalists. Doing this is beneficial. If you call yourself a ‘communist’, you really will probably end up with an inappropriately positive attitude towards the Soviet Union even as you denounce it as “not really communism”. If you call yourself a Muslim you really will probably end up with a level of sympathy for Islamic terrorists that you lack for terrorists of other types even as you insist that they are misguided and that “Islam is really a religion of peace”.
Labels are in general destructive of rationality. In so far as they are useful, it is because their signaling value exceeds their emotional cost. It is totally appropriate to emphasize the negative messages that a label will send to someone who is considering adopting it, and the main source of such negative messages comes from making claims of similarity to all others who are using that label. In the case of feminism it is totally clear to me that many of the more prominent feminists endorse positions diametrically opposed to equal rights and to rational thought and it sure seemed to me like you did as well based on your posts.
At this point, I still don’t believe with p>50% that you favor rational thought or equal rights, but I’m willing to make that assumption for argument’s sake if you wish to stick around. I don’t really prefer for you to stick around, but I do think that there is a very small chance of my learning something very important by your doing so, so I’ll do what I can not do discourage your continuing presence here. Not in exchange, but rather as a token of your commitment to sincere deliberation I would like to ask you to internally and externally taboo “privilege” and “dominant” while you are here. These concepts seem to me to be too loaded and amorphous to contribute to the dialog.
Angel: I really don’t think that in the modern world we need a label for “someone who believes in equal rights for men and women” any more than we need a label for “someone who thinks that we shouldn’t have slavery” outside of the Muslim world. Abolitionist was once a mark of distinction, now it isn’t. A more useful definition but extremely generous definition of feminism, one that would make me very tentatively feminist, would be something like “someone who believes that many/most cultural institutions and basic assumptions need to be critically reevaluated with attention to the fact that women have historically been denied adequate input into their creation”. A hostile definition of feminism, but one that I think based on your posts seems to describe you, is a person who believes that the historical oppression of women provides a general-purpose excuse for condemning anyone a feminist wishes to condemn and for reaching any conclusion a feminist wishes to reach. It’s a variant of the general problem of knowing about biases and using that knowledge to absolve you from the necessity of engaging with people’s arguments, choosing instead to simply point out that they could be biased. Really, this range of possible interpretations of the word, which only scratches the surface of possibilities, further suggests that the word doesn’t convey enough signal to justify the confusion it causes.
All: I’m really disappointed that no-one else seems to have found my “after the FAI does nothing” frame useful for making sense of this post. Is anyone interested in responding to that version? It seems so much more interesting and complete than the three versions E.C. Hopkins gave.
Dynamically: My “moral philosophy” if you insist on using that term (model of a recipe for generating a utility function considered desirable by certain optimizers in my brain would be a better term) is the main thing that HAS told me to steal, cheat, and murder. Simpler optimization patterns based on herd behavior, operant conditioning, moderately strong typical male primate aversions to violence, projections of parental authority through internalized neural agents etc have told me not to do those things and have won enough attention from the more complex optimizers to convince them (since the complex optimizers can reflect and be convinced of things) not to do so after all, and upon examination those simpler patterns have mostly turned out to be right judged by the standards of the moral philosophy. On a few occasions that I am aware of my conditioned etc morality was very wrong (judged reflectively), and possibly on a few other occasions, but they were much much less wrong than the occasions on which they were right and casual examination of my reflective self was in doubt.
The way I frame this question is “what if I executed my personal volition extrapolating FAI, it ran, created a pretty light show, and then did nothing, and I checked over the code many times with many people who also knew the theory and we all agreed that it should have worked, then tried again with completely different code many (maybe 100 or 1000 or millions) times, sometimes extrapolating somewhat different volitions with somewhat different dynamics and each time it produced the same pretty light show and then did nothing. Lets say I have spend a few thousand years on this while running as an upload. Now what?”
In this scenario there’s no optimization reason I shouldn’t just execute cached thoughts. In fact, that’s pretty much what anything I do in this scenario amounts to doing. Executing cached thoughts does, of course, happen lawfully, so there is a reason to dress in black etc in that sense. I used to be pretty good at writing some sad but mostly non-gloomy poetry and denouncing people as fools. Might be even more fun to do that with other modified upload copies of myself. When that got old, maybe use my knowledge of FAI theory to build myself a philosophy of math oracle neural module. Hard to guess how my actions would differ once it was brought on-line. It seems to me that it might add up to normality because there might be an irreducible difference between utility for me and utility for an external AGI even if it was an extrapolation of my volition, but for now I’m a blind man speculating on the relative merits of Picasso and Van Gogh.
Honestly I’m much less concerned about this scenario than I once was. Pretty convinced that there are ways to extrapolate me that do something even if they discover infinite computing power.
Dynamically linked: No-one but nerds and children care what moral philosophies say anyway, at least, not in a way that effects their actions. You, TGGP and Unknown are very atypical. Poke is much closer to correct. If anything, when the dust settled the world would be more peaceful if most people understood the proof.
Eric Mesoy: If utilities = 0 then dying from malnourishment isn’t horrible.
Andy M: Your answer sounds more appropriate for someone fairly shallow and 20 years old who discovers that the world or his life will end in 6 months than for someone for whom utilities are set to zero or morality is lost.
Constant Pablo and especially Sebastian: Clearly thought! I should probably start reading your comments more carefully in the future.
Laura: Why unsympathetic? My guess is that you still confuse my and Eliezer’s aspirations with some puerile Nietzschean ambition. I like who I am now too thank you very much, and if my extrapolated volition does want to replace who I am it is for reasons that I would approve of if I knew them, e.g. what it will replace me with is not “completely different, incomprehensible, and unsympathetic”. That’s the difference between a positive and a negative singularity. Death isn’t abhorrent, life/experience/growth/joy/flourishing/fulfillment, rather, is good, and a universe more full of them more good than one less full, whether viewed from inside or from outside. Math is full of both death and flourishing and is not lessened by the former.
Phil: Very entertaining and thoughtful post.
Laura: It seems to me that when you apply the feminist label you might want to think about the risk of being understood to mean “like Angel”. If that prospect doesn’t appall you you may want to read her posts more carefully. It’s not just normal academic dishonesty. Rather, its a very distinctive establishment of moral asymmetries mixed with postmodernism that characterizes major strains of modern (second and third wave?) feminism. Maybe you want the term “first wave feminist” but that’s too pretentious.
Bambi: The particular reason for blogging rather than rum is that the math says he blogs here and now. The future isn’t immune to our actions, it is what it is as the result of our actions, which likewise are what they are. We cause it to be in the same manner that the earlier states of a Turing Machine cause the later states to be.
Eliezer: I really like this post, but it seems to me that empirically it was substantially a cultural practice in philosophy, including Kant etc, that enabled those early 20th century Germans (and only those people, in that particular culture, with that particular philosophical tradition) to seem, vaguely a significant subset of those assumptions that they did know existed but that other philosophers and lay people didn’t know existed. That philosophy also lead them down some wrong roads, such as towards thinking mind was fundamental rather than emergent, and it certainly didn’t enable them to see all of the assumptions that they didn’t know existed, but there seems to be a known partial reason that the quantum revolution was so local, one credited by some of the physicists in question.
For what it’s worth, I have only a lay understanding of quantum physics, still don’t really know what you mean by configurations and amplitudes, and was able to see, fairly easily, the assumption that “Bob” in your assumption didn’t see, basically about particles being “things” with “properties” attached to them, (an assumption that Chalmers, in “The Conscious Mind” seems to know is rejected by physics but to find it impossible to reject, leading him to mention his disturbance in a physical view of particles as “pure causal flux”, which I would call “pure relationship” and which at least a few philosophers surely mean by “radical emergence”) although I would have described it somewhat differently, e.g. not by explicit reference to technical information that I didn’t have.
I don’t think that the problem is that it is impossible with effort and training to learn to recognize one’s blind spots a-priori. Rather, I think that philosophy attracts many kinds of people, only one of which is the type of person who has a talent that he wants to develop in recognizing his blind spots. Philosophy then provides, to different extents in different places and times, some training in this skill and some reward of status for the development of it. Currently, it seems to me that neither Analytic nor Continental philosophy provides significant training or status relating to this as opposed to other skills. More particularly, both seem to provide far less such training or reward in status than contemporary theoretical physics, theoretical computer science and probably some parts of math.
The main problem, it seems to me, relates to this issue of rewarding with status. In physics, ultimately status goes to those who make the correct predictions enabling correct beliefs to actually attain dominance in the field even if they are counter-intuitive (or too intuitive to qualify as ‘deep’), while in philosophy, without experiments, correct beliefs always exist at a very low incidence at equilibrium, far less popular or ‘official’ than clever descriptions of those cognitive illusions such as empty labels http://lesswrong.com/lw/ns/empty_labels/ (in this case, the particle without the mathematical relationships it participates in) that act as attractors to human naive ontology. As a result, the average physicist is better at this type of philosophy than the average philosopher is, while the average highly esteemed physicist is astronomically better at it than the average highly esteemed philosopher.
BTW I’m not really convinced that “Bob” would be correct in “any classical universe”, or even that classical universes are conceivable rather than apparently conceivable.
Poke: Are you sure about mineralogy and physics as foundations of modern geology?
Patrick:
I agree that something roughly along the lines of what you are discussing can be done and is unavoidable. I am primarily attempting to refute the proposal that it is or can be corrected to become Bayesian, and hence the proposal that the process that we use to do things like this stands with the same sort of logical foundations as Bayesian reasoning does. It definitely seems to me that strictly speaking, once you remove logical omniscience, unless you replace it with some very specific abstraction (most of which have their own problems) you need to assign probabilities to “The RH can be proved”, “The RH can be disproved”, “The RH is undecidable from ZFC”, “The RH can be proved AND the RH can be disproved”, “The RH can be proved AND the RH can be disproved AND The RH is undecidable from ZFC” “The RH can be disproved AND The RH is undecidable from ZFC” etc. In practice we can apportion zero probability to the latter cases, but only conditional upon the quality of our reflection being perfect, which we know to be false, and only after SOME reflection. It seems to me that as we assign probabilities we have to do reflection that moves our estimates continually.
I second tabooing probability, but I think that we need more than two words to replace it. Casually, I think that we need, at the least, ‘quantum measure’, ‘calibrated confidence’, and ‘justified confidence’. Typically we have been in the habit of calling both “Bayesian”, but they are very different. Actual humans can try to be better approximations of Bayesians, but we can’t be very close. Since we can’t be Bayesian, due to our lack of logical omniscience, we can’t avoid making stupid bets and being Dutch Booked by smarter minds. It’s therefore disingenuous to claim that vulnerability to Dutch Books is a decisive argument against a behavioral strategy. Calibrated confidence is the strategy that we can try to use to minimize our vulnerability to being Dutch Booked by people who aren’t smarter than we are but who know exploits in our heuristics. They tend to be much much closer to 50% than Bayesian confidences, and are pretty much unavoidably subject to some framing based biases as a result.
To make the simulation really compelling it has to include some sort of assortative mating.
No Mike, your intuition for really large numbers is non-baffling, probably typical, but clearly wrong, as judged by another non-Utilitarian consequentialist (this item is clear even to egoists).
Personally I’d take the torture over the dust specks even if the number was just an ordinary incomprehensible number like say the number of biological humans who could live in artificial environments that could be built in one galaxy. (about 10^46th given a 100 year life span and a 300W (of terminal entropy dump into a 3K background from 300K, that’s a large budget) energy budget for each of them). It’s totally clear to me that a second of torture isn’t a billion billion billion times worse than getting a dust speck in my eye, and that there are only about 1.5 billion seconds in a 50 year period. That leaves about a 10^10 : 1 preference for the torture.
The only considerations that dull my certainty here is that I’m not convinced that my utility function can even encompass these sorts of ordinary incomprehensible numbers, but it seems to me that there is at least a one-in-a-billion chance that it can.
I think that Robin’s point solves this problem, but doesn’t solve the more general problem of an AGI’s reaction to low probability high utility possibilities and the attendant problems of non-convergence.
The guy with the button could threaten to make an extra-planar factory farm containing 3^^^^^3 pigs instead of killing 3^^^^3 humans. If utilities are additive, that would be worse.
Robin: Great point about states with many people having low correlations with what one random person can effect. This is fairly trivially provable.
Utilitarian: Equal priors due to complexity, equal posteriors due to lack of entanglement between claims and facts.
Wei Dai, Eliezer, Stephen, g: This is a great thread, but it’s getting very long, so it seems likely to be lost to posterity in practice. Why don’t the three of you read the paper Neel Krishnaswami referenced, have a chat, and post it on the blog, possibly edited, as a main post?
“The paper I referenced:
Vann McGee (1999)
An airtight Dutch book
Analysis 59 (264), 257â265.
Posted by: Neel Krishnaswami | October 20, 2007 at 06:29 PM”
Rob: I think that some psychologists might say something like the following. Confirmation bias causes new evidence to accumulate in favor of existing beliefs. Subsequent to the accumulation of such evidence, the refutation of the original evidence for a belief will not eliminate the belief. When the beliefs in question are normative evaluations, this is called the “halo effect”.
Initial evidence that something, for instance ethical behavior, is good because it leads to Santa giving presents could lead to a bias in favor of noticing other evidence that ethical behavior is good, such as it eliminates some of the cognitive burden of lying, and it makes other people happy thus promoting vicarious pleasure. Sensitivity to these possibilities may lead to their better exploitation, and thus to a history of Pavlovian reinforcement from good behaviors.
That being said, much of Jewish and to a lesser degree Catholic law consists of the justification of reversion to preferred morality where it differs (usually in the sense of being much more ethical as modern seculars would understand the term) than the morality promoted by their theoretical authority figure, implying a (fortunate) failure of internalization of religious morality. More speculatively, some scholars have blamed the barbarities of the Soviets and NAZIs on (among other things) the reversion to a tribal/nationalist preferred morality which Christianity had advocated replacing, or simply to conformity or sadism in response to a moral vacuum. Unfortunately for this thesis, the Japanese, with a presumably more resilient internalized morality, committed comparable though somewhat lesser atrocities, suggesting, at best, that reverting to tribalism/nationalism is not much worse for outsiders than never having abandoned it. Unfortunately for this discussion, there are no obvious examples of maturely secular nations with universalist ethical axioms fighting major wars, but that fact could also be seen as tending to confirm the above speculation.
TGGP: I don’t think that what you are describing would be considered by most people to constitute a disbelief in morality. Instead, I think it would be considered an atypically reflective and possibly slightly atypically targeted belief about what being means.
Rob Spear: “good old fashioned Pavlovian conditioning” requires that rewards and punishments (reinforcers) be associated very closely in time with the behaviors being reinforced. Santa doesn’t do this. Neither does god. This is NOT a minor quibble, but rather is a critically important distinction. Evolutionary psychologists have discovered a few instances of non-old-fashioned Pseudo-Pavolvian conditioning, such as the development of aversions to foods as a result of sickness hours after eating (still not months or years after) and immediate acquisition of certain phobias (for instance, fear of snakes) without the need for repetition, but these are due to very distinctive context-specific mechanisms. There is no evidence that any analogous distinctive Pseudo-Pavolvian mechanism is at work in generating “Santa efficacy”.
Rob: In practice there is a HUGE difference. If you behave morally when no0one is watching you, new information doesn’t threaten your moral foundations, as your morality is grounded in your preferences. If you believe that you are always being watched then your moral behavior will be grounded in supposed facts about the world. In this case, evidence that undermines your belief in those facts undermines your morality, leading in the direction of Nietzsche’s “total eclipse of all values” as the inevitable consequence of the “twilight of the idols”.
There are also important practical consequences to following orders as opposed to acting from one’s own initiative. Psychologically, the former will create resentment and dissatisfaction while the latter will not. The latter will favor initiative, and active pursuit of the best satisfaction of one’s (hopefully ethical) desires, while the former will focus only on satisfying some perceived standard of acceptability, which implies, among other things, not seeking out better third alternatives when presented with a false dilemma and evidence that one flawed alternative is considered to be permissible. Actions “above the call of duty” depend entirely upon self-driven internalized ethics.
BTW, I have significant personal experience with one of “the smartest guys in the room” and yes, they were (or at least he is) VERY smart by any normal business world standards. He’s particularly great at giving obvious in retrospect answers to marketing type problems. It would be a somewhat unusual room full of business people or other social elites where the guy I’m thinking of isn’t the smartest guy.