Zetetic

Karma: 694

Zetetic Mar 2, 2012, 2:36 AM
1 point
on: Troubles With CEV Part1 - CEV Sequence

There is little in common between Eliezer, Me and Al Qaeda terrorists, and most of it is in the so called reptilian brain. We may end up with a set of goals and desires that are nothing more than “Eat Survive Reproduce,” which would qualify as a major loss in the scheme of things.

I think you may possibly be committing the fundamental attribution error. It’s my understanding that Al Qaeda terrorists are often people who were in a set of circumstances that made them highly succeptible to propaganda—often illiterate, living in poverty and with few, if any, prospects for advancment. It is easy to manipulate the ignorant and disenfranchised. If they knew more, saw the possibilities and understood more about the world I would be surprised if they would choose a path that diverges so greatly with your own that CEV would have to resort to looking at the reptilian brain.

In this specific case, what ends up dominating CEV is what evolution wants, not what we want. Instead of creating a dynamic with a chance of creating the landscape of a Nice Place to Live, we end up with some exotic extrapolation of simple evolutionary drives.

“What evolution wants” doesn’t seem like a clear concept—at least I’m having trouble making concrete sense of it. I think that you’re conflating “evolution” with “more ancient drives”—the described extrapolation is an extrapolation with respect to evolutionarily ancient drives.

In particular, you seem to be suggesting that a CEV including only humans will coincide with a CEV including all vertibrates possessing a reptillian brain on the basis that our current goals seem wildly incompatible. However, as I understand it, CEV asks what we would want if we “knew more, grew up further together” etc.

Zetetic Feb 18, 2012, 9:07 AM
0 points
in reply to: Nisan’s comment on: Formulas of arithmetic that behave like decision agents
Well the agent definition contains a series of conditionals. You have as the last three lines: if “cooperating is provably better than defecting”, then cooperate; else, if “defecting is provably better than cooperating” then defect; else defect. Intuitively, assuming the agent’s utility function is consistent, only one antecedent clause will evaluate to true. In the case that the first one does, the agent will output C. Otherwise, it will move through to the next part of the conditional and if that evaluates to true the agent will output D. If not, then it will output D anyway. Because of this, I would go for proving that line three and 4 can’t both obtain.

Or you could prove that line 3 and line 4 can’t both obtain; I haven’t figured out exactly how to do this yet.

How about this? Suppose condition 3 and 4 both obtain. Then there exists and a and b such that U(C) = #a and U(D) = #b (switching out the underline for ‘#’). Also, there exists a p and q such that p > q and U(D) = #p and U(C) = #q. Now U(C) = #a > #b = U(D) = #c > #q = U(C) so U(C) > U(C). I may actually be confused on some details, since you indicate that a and b are numbers rather than variables (for instance in proposition 2 you select a and b as rational numbers), yet you further denote numerals for them, but I’m assuming that a >b iff #a > #b. Is this a valid assumption? I feel as though I’m missing some details here, and I’m not 100% sure how to fill them in. If my assumption isn’t valid I have a couple of other ideas. I’m not sure if I should be thinking of #a as ‘a’ under some interpretation.

Going a little bit further, it looks like your lemma 1 only invokes the first two lines and can easily be extended for use in conjecture 4 - look at parts 1 and 3. Part 1 is one step away from being identical to part 3 with C instead of D.
~Con(PA) → ~ Con(PA + anything), just plug in Con(PA) in there. Adding axioms can’t make an inconsistent theory consistent. Getting the analogous lemma to use in conjecture 4 looks pretty straightforward—switching out C with D and vice versa in each of the parts yields another true four part lemma, identical in form but for the switched D’s and C’s so you can use them analogously to how you used the original lemma 1 in the proof of proposition 3.

Zetetic Feb 17, 2012, 1:06 AM
0 points
in reply to: endoself’s comment on: Rational philosophies

That would be very impressive, but I don’t see that in any of the stuff on his semiotics on Wikipedia.

A caveat: I’m not at all sure how much I’m projecting on Peirce as far as this point goes. I personally think that his writings clarified my views on the scientific method (at the time I originally read them, which was a good while back) and I was concurrently thinking about machine learning—so I might just be having a case of cached apophenia.

However; if you want a condensed version of his semiotic look over this. You might actually need to read some of the rest of that article (which, I admit, is a bit long) to put it in more context. Also, this wikipedia page looks pretty comprehensive. I’m pretty confident that they’re leaving a bit out that might be clearer if you read Peirce, but I’m not sure of how much instrumental value that would be to you. The issue with reading Peirce is that he was a crazy hermit with thousands of unpublished notes who continuously updated his views in significant ways (another point for him, he continued to reconsider/shift his views in a systematic way until he died), so lots of what you read about/by Peirce is compiled from a vast repository of his notes collected from his workspace after his death.

Also, something neat I found: Peirce’s three valued logic predating Post. That was among his tens of thousands of unpublished pages of notes. Going further in this direction, I found an interesting article on Peirce’s logic. There is some interesting discussion there about his influence on modern logic.

Points for coolness—Simon Newcomb was quite possibly his evil arch-nemesis.

Anyway—I think what attracts me to Peirce the most is his seemingly endless ability to carve reality at the joints in novel (at the time at least) ways, coupled with his nearly superhuman productivity levels—I mean he was highly influential in the realm of statistical theory, his influence on experimental design was impressive, he invented an axiomatization for arithmetic before Peano, he invented a modern characterization of first-order logic on par with Frege’s (but arguably with a more algebraic/model theoretic than syntactic approach), he was a skilled expositor and clear writer, he invented pragmaticism, he had a lifetime of smaller results in logic, earth sciences and mathematics that anyone would be proud of—what more could you want from a single person before you can understand why people admire them?

Oh, he seems to have disobeyed endoself’s first law of philosophy: “Have as little to do with Hegel as possible.”

I think it’s significantly more forgivable for a contemporary of Hegel’s to be influenced by him than someone today being influenced by him. Further, when I say “influenced”—he scavenged a set of ideas that he seems to have reinterpreted in terms of his own philosophy because he saw that they could round out his ideas in a variety of ways—he was still pretty critical of Hegel of some major points. I think just browsing through these excerpts reveals the lines of influence a bit.

Zetetic Feb 15, 2012, 9:34 PM
5 points
in reply to: Solvent’s comment on: Hard philosophy problems to test people’s intelligence?
I think there’s a problem with your thinking on this—people can spot patterns of good and bad reasoning. Depending on the argument, they may or may not notice a flaw in the reasoning for a wide variety of reasons. Someone who is pretty smart probably notices the most common fallacies naturally—they could probably spot at least a few while watching the news or listening to talk shows.

People who study philosophy are going to have been exposed to many more diverse examples of poor reasoning, and will have had practice identifying weak points and exploiting them to attack an argument. This increases your overall ability to dissolve or decompose arguments by increasing your exposure and by equipping you with a trick bag of heuristics. People who argue on well moderated forums or take part in discussions on a regular basis will likely also pick up some tricks of this sort.

However, there are going to be people who can dissolve one problem, but not another because they have been exposed to something sufficiently similar to one (and are thus probably have some cached details relevant to solving it) but not so for the other:

E.g. a student of logic will probably make the correct choice in the Wason Selection Task and may be able to avoid making the conjunction fallacy, but they may not two box because they fall into the CDT reasoning trap. However, a student of the sciences or statistics may slip up in the selection task but one box, by following the EDT logic.

So if you’re using this approach as an intelligence test, I’d worry about committing the fundamental attribution error pretty often. However, I doubt you’re carrying out this test in isolation. In practice, it probably is reasonable to engage people you know or meet in challenging discussions if you’re looking for people that are sharp and enjoy that sort of thing. I do it every time I meet someone who seems like they might have some inclination toward that sort of thing.

It might help if you provide some context though—who are you asking and how do you know them? Are you accosting strangers with tricky problems or are you probing acquaintances and friends?

Zetetic Feb 15, 2012, 1:01 AM
2 points
in reply to: endoself’s comment on: Rational philosophies
Well I can relay my impressions on Peirce and why people seem to be interested in him (and why I am):

I think that the respect for Peirce comes largely from his “Illustrations in the Logic of Science” series for Scientific American. Particularly “The Fixation of Belief” and “How to Make Our Ideas Clear”.

When it comes to Tychism, it’s kind of silly to take it in a vacuum, especially given that the notion of statistics being fundamental to science was new, and Newtonian determinism was the de facto philosophical stance of his day. He was standing in direct conflict to the then popular (but false) view of Newtonian determinism. Observe the following excerpt from the beginning of one of his papers on the subject:

“I propose here to examine the common belief that every single fact in the universe is precisely determined by law. ”

Going further

“The proposition in question is that the state of things existing at any time, together with certain immutable laws, completely determine the state of things at every other time (for a limitation to future time is indefensible). Thus, given the state of the universe in the original nebula, and given the laws of mechanics, a sufficiently powerful mind could deduce from these data the precise form of every curlicue of every letter I am now writing.”

Given the context, although he might have been guilty of the mind projection fallacy (he was a realist when it comes to probabilities), and was pretty much a frequentist, I don’t think it’s reasonable to criticize him very harshly for either position—he was a very early pioneer in statistics (just look over the second paragraph). His embrace of statistical inference was, as far as I can tell, somewhat unusual for the day, and he made several contributions to the use of statistics in psychology and psycho physics (including the use of double blind studies to re-examine previous findings). This is in addition to his contributions to logic, mathematics and geology—so if his logic is on par with Frege (just look at his contributions to mathematics and logic).

His semiotics is interesting as well as it seems to yield an early attempt to look at science as a process of improving statistical models. Pierce’s semiotic reflects this, and he uses it in his phenomenological characterizations of the scientific method. I think that this allowed him to view human scientists as statistical learners. When I read some of this it certainly invoked a more machine learning/information theoretic picture of scientific discovery than any other (non-modern) philosophers of science had managed to touch on.

As far as the “weird numerology”—Peirce’s fixation on the number three seems to be mainly a side effect of Hegel’s influence—Hegel focused on a tertiary relationship between ideas—thesis, antithesis, synthesis. We have an idea, a conflict is presented, and we synthesize the two into something less wrong. I think that a number of odd ideas held by Peirce were influenced by Hegel (such as his view of continuity which seems borderline incoherent to me). I’ve found that sometimes this tertiary form yields something nice, but it often seems to result in something strained. I’m not sure why this became so pervasive in Peirce’s writings.

That’s what I can think of off the top of my head, but if something else occurs to me I’ll add it.

Zetetic Feb 14, 2012, 9:49 PM
1 point
in reply to: AlexMennen’s comment on: Formulas of arithmetic that behave like decision agents
Yes, you’re right. Looking at the agent function, the relevant rule seems to be defined for the sole purpose of allowing the agent to cooperate in the even that cooperation is provably better than than defecting. Taking this out of context, it allows the agent to choose one of the actions it can take if it is provably better than the other. It seems like the simple fix is just to add this:

=D\to\pi_{\underline{i}}\chi()=\underline{a})\\\mbox{\quad\quad\quad\quad}\wedge(\psi(\ulcorner%20U%20\urcorner,\underline{i})=C\to\pi_{\underline{i}}\chi()=\underline{b})\biggr)\wedge%20a%20%3E%20b)

If you alter the agent definition by replacing the third line with this and the fourth with C you have an agent that is like the original one but can be used to prove conjecture 4 but not prop 3, right? So if you instead add this line to the current agent definition and simply leave D as the last line, then if neither the old third line nor this new one hold we simply pick D, which is neither provably better nor provably worse than C, so that’s fine. It seems that we can also prove conjecture 4 by using the new line in lieu of the old one, which should allow us to use a proof that is essentially symmetric to the proof of proposition 3.

Does that seem like a reasonable fix?

Zetetic Feb 13, 2012, 2:35 AM
2 points
in reply to: Thomas’s comment on: Rational philosophies
The utilitarian case is interesting because both Mill and Bentham seemed to espouse a multidimensional utility vector rather than a uni-dimensional metric. There is an interesting paper I’ve been considering summarizing that takes a look at this position in the context of neuroeconomics and the neuroscience of desire.

Of interest from the paper: They argue that “pleasure” (liking), though it comes from diverse sources, is evaluated/consolidated at the neurological level as a single sort of thing (allowing a uni-dimensional representation as is common in economics), but show that when it comes to anticipation of pleasure (and the heuristics for evaluating the future rewards an action might yield) we can be quite inaccurate. So effectively, the Bentham/Mill model of utilitarianism is predicated on the perfect or near perfect coherence of “wanting” and “liking” (which does not really exist) and incorrectly prescribes a multidimensional utility measure because of this.

However, Bentham does appear to be unusually rational for his day. Look at his highly unpopular progressive stances and this becomes obvious—they are typically stances that we would consider sane and moral today, but which were highly taboo at the time.

Personally, when it comes to clarity of thought, I’ve never found anyone who tops C.S. Peirce. He was unbelievably rational in his view of the world and of science and mathematics—he was scary. This is someone who recognized the fundamentally statistical nature of the sciences in the 1800′s. He was able to see the broad philosophical implications of evolution in nearly as much clarity as many of your so-called “evolutionist” thinkers (can provide citation if requested).

C.S. Peirce is by far my greatest inspiration as an epistemic rationalist. Just look at what he did. The clarity and breadth of his thought and philosophy is incredible to me.

Zetetic Feb 4, 2012, 2:16 AM
1 point
in reply to: Nisan’s comment on: Formulas of arithmetic that behave like decision agents
I only looked at this for a bit so I could be totally mistaken, but I’ll look at it closely soon, it’s a nice write up!

My thoughts:

A change of variables/values in your proof of proposition 3 definitely doesn’t yield conjecture 4? At first glance it looks like you could just change the variables and flip the indicies for the projections (use pi_1 instead of pi_0) and in the functions A[U,i]. If you look at the U() defined for conjecture 4, it’s exactly the one in proposition 3 with the indices i flipped and C and D flipped, so it’s surprising to me if this doesn’t work or if there isn’t some other minor transformation of the the first proof that yields a proof of conjecture 4.

Zetetic Jan 30, 2012, 11:08 PM
1 point
in reply to: Nisan’s comment on: So You Want to Save the World
Ha! Yeah, it seems that his name is pretty ubiquitous in mathematical logic, and he wrote or contributed to quite a number of publications. I had a professor for a sequence in mathematical logic who had Barwise as his thesis adviser. The professor obtained his doctoral degree from UW Madison when it still had Barwise, Kleene and Keisler so he would tell stories about some class he had with one or the other of them.

Barwise seems to have had quite a few interesting/powerful ideas. I’ve been wanting to read Vicious Circles for a while now, though I haven’t gotten around to it.

Zetetic Jan 17, 2012, 6:12 AM
2 points
in reply to: CarlShulman’s comment on: Q&A with Richard Carrier on risks from AI
Is there anything, in particular, you do consider a reasonably tight lower bound for a man-made extinction event? If so, would you be willing to explain your reasoning?

Zetetic Jan 16, 2012, 8:13 AM
3 points
on: [LINK]The Edge’s yearly question
I’ll start off by saying that I like Linde’s writing style, in particular I like his seamless segue from an easy explanation into a challenge to explore further:

One could always respond that God created the universe and made it simple enough so that we can comprehend it. This would match the words about a miracle and an undeserved gift. But shall we give up so easily?

In addition I thought that Myhrvold’s post was pretty fantastic. I also liked Stewart Brand. Iacoboni is good as well. That’s out of the quarter of them I read.

Zetetic Jan 13, 2012, 5:27 AM
3 points
on: Knowledge vs Technology

But at the same time if we would collect all the uncertainties in our models/simulations/calculations etc in a variable u. What happens as u->0 ? Shouldn’t the gap between 2 and 3 go to zero? That would mean that enough knowledge about a technology is equivalent to actually possessing the technology.

All the uncertainties? If we had infinite evidence that the implementation will work the way you want it to then you could do this. So in this case “enough knowledge” effectively means infinite knowledge—everything about the construction process down to the quantum level and all of the (infinitely many) logical implications of your evidence. Actually, you arguably need more than that to make u go to 0.

So the conclusion ends up being that having infinite knowledge about everything that might affect the implementation is equivalent to having the technology. That doesn’t hold at all. If you knew that much about the implementation and all the related issues you would have vastly more power at your disposal than if you just had access to a sensible plan that happens to work (even though you’re not 100% certain about its soundness).

Zetetic Jan 5, 2012, 10:50 PM
0 points
in reply to: TheOtherDave’s comment on: [META] ‘Rational’ vs ‘Optimized’
I was thinking about whether telling someone I’m an aspiring optimizer is going to result in less confusion than telling them that I’m an aspiring rationalist. I think that the term ‘optimizer’ needs a little more specification to work; how about Decision Optimization? If I tell someone I’m working on decision optimization, I pretty effectively convey what I’m doing—learning and practicing heuristics in order to make better decisions.

Zetetic Jan 5, 2012, 11:22 AM
6 points
in reply to: taelor’s comment on: The Parable of the Dagger
The first inscription says that the inscriptions have the same truth value. If the second one is false then the first one implies that it is false which, in turn, implies that the first one is true. Contradiction. So the premise that “the second inscription is false” is false. So the second inscription is true.

The Jester’s logical inference is right. The point isn’t that the Jester’s logic was wrong—it wasn’t. It’s that the Jester assumed that the locations of the key and the dagger would follow the logic when there really was no good reason to assume so. This is meant to illustrate that making unwarranted assumptions about reality isn’t a good idea.

Zetetic Jan 5, 2012, 10:13 AM
4 points
in reply to: wedrifid’s comment on: [META] ‘Rational’ vs ‘Optimized’

One s. One o.

Opss. Fixed.

Zetetic Jan 5, 2012, 4:07 AM
3 points
in reply to: Dwelle’s comment on: So You Want to Save the World
You need to take desire out of the equation. The way you program the utility function fully determines the volition of the machine. It is the volition of the machine. Postulating that a machine can desire something that it’s utility function doesn’t define or include is roughly equivalent to postulating that 1 = 0. I think you might benefit from reading this actual SIAI article by Eliezer. It specifically address your concern.

There is one valid point—closely related to what you’re saying here:

The AI would have access to every information humans ever conceived, including the discussions, disputes and research put into programming this AI’s goals and nature. It may then adopt new goals based on the information gathered, realizing its former ones are no longer desirable.

But you’re thinking about it the wrong way. The issue that the machine “realizes” that something is “no longer desirable” doesn’t actually make a lot of sense because the AI is its programing and it can only “realize” things that its programing allows for (of course, since an AGI is so complicated, a simple utility function could result in a situation similar to presenting a Djinn (genie) an ill-specified request i.e. a be-careful-what-you-wish-for scenario).

A variant that does make sense and is a real concern is that as the AGI learns, it could change its definitions in unpredictable ways. Peter De Blanc talks about this here. This could lead to part of the utility function becoming undefined or to the machine valuing things that we never intended it to value—basically it makes the utility function unstable under the conditions you describe. The intuition is roughly that if you define a human in one way, according to what we currently know about physics, some new discovery made available to the AI might result in it redefining humans in new terms and no longer having them as a part of its utility function. Whatever the utility function describes is now separate from how humans appear to it.

Zetetic Jan 5, 2012, 3:45 AM
−2 points
in reply to: TheOtherDave’s comment on: [META] ‘Rational’ vs ‘Optimized’
I didn’t down-vote, but my two cents:

I think you’re misunderstanding Nesov’s comment. Becoming a better optimizer loses a useful distinction- to see this you need to take an outside view of optimization in general. Rationalists want to optimize their behaviors relative to their values/goals—which includes only a very narrow slice of things to optimize for (generally not the number of paperclips in the universe any other process that might be encoded in a utility function) and a specific set of heuristics to master. Hence the claim that rationality isn’t that great for many purposes- there are only relatively few purposes we wish to actually pursue currently.

Even though becoming a sufficiently stronger optimizer-in-general will help you achieve your narrow range of goals, unless you specifically work towards optimizing for your value set, it’s not optimal to do so relative to your actual utility function. An optimizer-in-general, strictly speaking, will on average be just as good at optimizing for the number of paperclips in the universe as you will at managing your relationships. The useful distinction is lost here.

Zetetic Jan 5, 2012, 12:28 AM
5 points
in reply to: Peacewise’s comment on: Doublethink (Choosing to be Biased)

I’m still struggling to understand when strict argument and/or more conversational discussion are appropriate on this website. Kind of amusing, in a frustrating way. To present historical evidence in support of a reasonable point about psychology, have it trolled (imo), yet decide to give the benefit of the doubt and present more detail, then move on and 2 months later be hit with counterargument for moving past the troll bait and staying on the original topic.

I’ll try to help you out.

I think the standards of evidence are the highest here of any place on the internet save for maybe some professional groups, and I for one would like to keep it that way. As far as the “generalization from fictional evidence”—that wasn’t because the evidence you presented was fictional, it was because you said that it didn’t matter whether the event actually happened—the concept is sufficient to provide evidence of the more general point. That is, per definition, an endorsement of generalization from fictional evidence.

As far as the evidence you did provide:

To be as specific as I can, there are a lot of issues with the sites you linked to—I tend to expect some sort of peer reviewed meta-analysis of the existing evidence that is well organized and hopefully somewhat up to date, and none of the sources really meet muster. The first one has no bibliography at all. The second is largely a book review with a couple of claims about new evidence, a particularly relevant quote from the author of said book (in the link):

“Hitler himself claimed that the war ended for him when he had to spend weeks in an army hospital after having been blinded by mustard gas. Circumstantial evidence and hearsay, however, have led to the suggestion that Hitler was, in fact, suffering from and treated for psychosomatic blindness. This hypothesis could never be conclusively tested, as Hitler had his medical file destroyed and had his henchmen kill those people with knowledge of the file,” said Dr Weber

That isn’t something to use in support of your argument. It’s not very good supporting evidence. The last source might be a good first person account or it might be a terrible exaggeration. I would much prefer an analysis from an actual historian with less personal bias because my background knowledge is insufficient for assessing the credibility of the source and what is being said.

Now, if you want to support ata’s point, that:

I’d estimate that it would be pretty dangerous to grant yourself permission to decide what delusions to instill in other people for their own good.

Here’s some of what I would like to see:

First, find some articles establishing that you can create such an elaborate delusion in a clinical setting—preferably using techniques that would have been available during the time period. To that effect I found a few articles, unfortunately behind paywalls, but the abstracts look promising in that they indicate the feasibility of imparting specific delusion. Unfortunately they’re all pretty modern results and the “imparted delusions” (which as far as I can tell aren’t totally established as true delusions, though they may be) all mirror actual delusions that might be normally encountered. The delusion supposedly imparted to Hitler was fairly detailed and unusual—I don’t know if this is a feasible to impart (if it is feasible at all) as compared to these more mundane cases.

The further claim that Hitler’s doctor imparted a delusion lasting for decades that was fairly intricate and cured his Conversion disorder (hysterical blindness) requires an awful lot of evidence—mostly because, beyond the lack of historical evidence, the physical possibility of this is in question. It doesn’t look to me like there is much (if any) evidence supporting the feasibility of such a feat of hypnotic suggestion and the links you provided do nothing in the way of establishing otherwise.

If you or anyone else has some strong evidence of the feasibility of imparting a robust, long-term delusion using hypnosis I’d be glad to consider it, but I don’t see why I should accept the possibility given that I can’t seem to find any evidence for it outside this (possibly false) Hitler anecdote.

I do hope this has been helpful for you, this site still has a very steep learning curve (although I think it has loosened up a bit lately) and community expectations aren’t immediately obvious to newcomers. We aren’t (as far as I can tell) trying to troll you - we just hold very high expectations for a post that makes a (highly controversial) factual claim. Of course, you might not view the claim as highly controversial, but unless you have some further evidence of the physical possibility of this sort of intricate, long-term hypnosis, it seems like this community might have a somewhat more stringent standard of evidence than you’re used to.

Zetetic 4 Jan 2012 10:21 UTC
4 points
in reply to: Peacewise’s comment on: Doublethink (Choosing to be Biased)

Anyways, regardless of the truth of Hitler’s hysterical blindness and his treatment by his Psychiatrist of instilling a delusion, the concept is supportive of ata’s point that… “I’d estimate that it would be pretty dangerous to grant yourself permission to decide what delusions to instill in other people for their own good.”

You’re aware that this basically endorses generalization from fictional evidence?

Zetetic 4 Jan 2012 2:38 UTC
4 points
on: [link] Anger as antidote to Confirmation Bias
Is this really an antidote or more of an inversion?