Jonii

Karma: 513

Jonii Sep 16, 2013, 8:41 PM
0 points
in reply to: cousin_it’s comment on: Notes on logical priors from the MIRI workshop
Try as I might, I cannot find any reference to what’s canonical way of building such counterfactual scenarios. Closest I could get was in http://lesswrong.com/lw/179/counterfactual_mugging_and_logical_uncertainty/ , where Vladimir Nesov seems to simply reduce logical uncertainty to ordinary uncertainty, but this does not seem to have anything to do with building formal theories and proving actions or any such thing.

To me, it seems largely arbitrary how agent should do when faced with such a dilemma, all dependent on actually specifying what it means to test a logical counterfactual. If you don’t specify what it means, whatever could happen as a result.

Jonii Sep 16, 2013, 8:08 PM
0 points
in reply to: cousin_it’s comment on: Notes on logical priors from the MIRI workshop
I asked about these differences in my second post in this post tree, where I explained how I understood these counterfactuals to work. I explained as clearly as I could that, for example, calculators should work as they do in real world. I did this explaining in hopes of someone voicing disagreement if I had misunderstood how these logical counterfactuals work.

However, modifying any calculator would mean that there can not be, in principle, any “smart” enough ai or agent that could detect it was in counterfactual. Our mental hardware that checks if logical coin should’ve been heads or tails is a calculator the same as any computer, and again, there does not seem to be any reason to assume Omega leaves some calculators unchanged while changes results of others.

Unless, this thing is just assumed to happen, with some silently assumed cutaway point where calculators become so internal they are left unmodified.

Jonii Sep 16, 2013, 7:45 PM
0 points
in reply to: cousin_it’s comment on: Notes on logical priors from the MIRI workshop
Well, to be exact, your formulation of this problem has pretty much left this counterfactual entirely undefined. Naive approximation, that the world is just like ours, and Omega just lies in counterfactual, would not contain such weird calculators which give you wrong answers. If you want to complicate problem by saying that some specific class of agents have a special class of calculators that one would usually think to work in certain way, but actually they work in a different way, well, so be it. That’s however just a free-floating parameter you have left unspecified and that, unless stated otherwise, should be assumed not to be the case.

Jonii Sep 16, 2013, 7:10 PM
0 points
in reply to: cousin_it’s comment on: Notes on logical priors from the MIRI workshop
Yes, those agents you termed “stupid” in your post, right?

Jonii Sep 16, 2013, 6:15 PM
2 points
in reply to: cousin_it’s comment on: Notes on logical priors from the MIRI workshop
After asking about this on #LW irc channel, I take back my initial objection, but I still find this entire concept of logical uncertainty kinda suspicious.

Basically, if I’m understanding this correctly, Omega is simulating an alternate reality which is exactly like ours, and where the only difference is that Omega says something like “I just checked if 0=0, and turns out it’s not. If it was, I would’ve given you moneyzzz(iff you would give me moneyzzz in this kind of situation), but now that 0!=0, I must ask you for $100.” Then the agent notices, in that hypothetical situation, that actually 0=0, so actually Omega is lying, so he is in hypothetical, and thus he can freely give moneyzzz away to help to real you. Then, because some agents can’t tell for all possible logical coins if they are lied to or not, they might have to pay real moneyzzz, while sufficiently intelligent agents might be able to cheat the system if they are able to notice if they are lied to about the state of the logical coin.

I still don’t understand why a stupid agent would want to make a smart AI that did pay. Also, there are many complications that restrict decisions of both smart and stupid agents, given argument I’ve given here, stupid agents still might prefer not paying, and smart agents might prefer paying, if they gain some kind of insght to how Omega chose these logical coins. Also, this logical coin problemacy seems to me like a not-too-special special class of Omega problems where some group of agents is able to detect if they are in counterfactuals

Jonii Sep 16, 2013, 8:02 AM
1 point
on: Notes on logical priors from the MIRI workshop
You lost me at part

In Counterfactual Mugging with a logical coin, a “stupid” agent that can’t compute the outcome of the coinflip should agree to pay, and a “smart” agent that considers the coinflip as obvious as 1=1 should refuse to pay.

The problem is that, I see no reason why smart agent should refuse to pay. Both stupid and smart agent know it as logical certainty that they just lost. There’s no meaningful difference between being smart and stupid in this case, that I can see. Both however like to be offered such bets, where logical coin is flipped, so they pay.

I mean, we all agree that a “smart” agent, that refused to pay here, would receive $0 if Omega flipped logical coin of asking if 1st digit of pi was an odd number, while “stupid” agent would get $1,000,000.

Jonii Jun 17, 2013, 7:05 AM
3 points
in reply to: Michelle_Z’s comment on: On manipulating others
This actually was one of the things inspiring me to write this post. I was wondering if I could make use of LW community to run such tests, because it would be interesting to get to practice these skills with consent, but trying to devise such tests stumped me. It’s actually pretty difficult to come up with a goal that’s actually difficult to achieve in any not-overtly-hostile social context. Laborious, maybe, but that’s not the same thing. I just kinda generalized from this, that it should actually be pretty easy to run with any consciously named goal and achieve it, but there must be some social inhibition.

The set of things that inspired me was wide and varying. It just may be reflected in how the essay was… Not as coherent as I’d have hoped.

Jonii Jun 17, 2013, 12:38 AM
−3 points
in reply to: Xachariah’s comment on: On manipulating others
That’s a nice heuristic, but unfortunately, it’s easy to come up with cases where this heuristic is wrong. Say, people want to play a game, I’ll use chess for availability, not because it best exemplifies this problem. If you want to have a fun game of chess, ideally you’d hope you did have roughly equal matches. If 9 out of 10 players are pretty weak, just learning the rules, and want to play and have fun with it, you, the 10th player, a strong club player, being an outlier, cannot partake because you are too good(with chess, you could maybe try giving your queen to handicap yourself, or take time handicap, to make games more interesting, but generally I feel that sorta of tricks still make it less for fun for all parties)

While there might be obvious reasons to suspect bias being at play, unless you want to ban ever discussing topics that might involve bias, the best way around it, that I know of, is to actually focus on the topic. Just stating “woah, you probably are biased if you think thoughts like this” is something I did take into consideration. I was still curious to hear LW thoughts on this topic. The actual topic, not on whether LW thinks it’s a bias-inducing topic or not. If you want me to add some disclaimer for other people, I’m open to suggestions. I was going to include one myself, that was basically saying “Failing socially in a way described here would at best be very very weak evidence of you being socially gifted, intelligent, or whatever. Reasoning presented here is not peer-reviewed, and might as well contain errors”. I did not, because I didn’t want to add yet another shiny distraction from the actual point presented. I didn’t think it would be needed, either.

Jonii Jun 16, 2013, 10:32 PM
2 points
in reply to: tim’s comment on: On manipulating others
Oh, yes, that is basically my understanding: We do social manipulation to the extent it is deemed “fair”, that is, to the point it doesn’t result in retaliation. But at some point it starts to result in such retaliation, and we have this “fairness”-sensor that tells us when to retaliate or watch out for retaliation.

I don’t particularly care about manipulation that results in obtaining salt shaker or a tennis partner. What I’m interested in is manipulation you can use to form alliances, make someone liable to help you with stuff you want, make them like you, make them think of you as their friend or “senpai” for the lack of better term, or make them fall in love with you. What also works is getting them to have sex with you, to reveal something embarrassing about themselves, or otherwise become part of something they hold sacred. Pretending to be a god would fall into this category. I’m struggling to explain why I think manipulation on those cases is iffy, I think it has to do with that kind of interaction kinda assuming that there are processes involved beyond self-regulation. With manipulation, you could bypass that and in effect you would lie about your alliance.

It is true many social interactions are not about anything deeper than getting the salt shaker. I kind of just didn’t think of them while writing this post. I might need to clarify that point.

Jonii Jun 16, 2013, 7:27 PM
−2 points
in reply to: Kawoomba’s comment on: On manipulating others
This I agree with completely. However, it sounding like power fantasy doesn’t mean it’s wrong or mistaken.

Jonii Jun 16, 2013, 6:31 PM
−1 points
in reply to: Shmi’s comment on: On manipulating others
True. However, it’s difficult to construct culturally neutral examples that are not obvious. The ones that pop to my mind are the kind of “it’s wrong to be nice to an old, really simple-minded lady because that way you can make her rewrite her will to your benefit”, or “It’s allright to try to make your roommate do the dishes as many times as you possibly can, as long as you’re both on equal footing on this “competition” of “who can do the least dishes”″.

I’m not sure how helpful that kind of examples are.

Jonii May 30, 2013, 10:30 PM
1 point
in reply to: OrphanWilde’s comment on: Open Thread, April 15-30, 2013
This strikes to me as massively confused.

Keeping track of cancelled values is not required as long as you’re working with a group, that is, a set(like reals), and an operation(like addition) that follows the kinda rules addition with integers and multiplication with non-zero real values do. If you are working with a group, there’s no sense in which those canceled out values are left dangling. Once you cancel them out, they are gone.

http://en.wikipedia.org/wiki/Group_%28mathematics%29 ← you can check group axioms here, I won’t list them here.

Then again, canceling out, as it is procedurally done in math classes, requires each and every group axiom. That basically means it’s nonsense to speak of canceling out with structures that aren’t groups. If you tried to cancel out stuff with non-group, that’d be basically assuming stuff you know ain’t true.

Which begs a question: What are these structures in advanced maths that you speak of?

Jonii May 15, 2013, 2:11 PM
3 points
in reply to: Scott Alexander’s comment on: Pascal’s Muggle: Infinitesimal Priors and Strong Evidence
Are you sure it wouldn’t be rational to pay up? I mean, if the guy looks like he could do that for $5, I’d rather not take chances. If you pay, and it turns out he didn’t have all that equipment for torture, you could just sue him and get that $5 back, since he defrauded you. If he starts making up rules about how you can never ever tell anyone else about this, or later check validity of his claim or he’ll kidnap you, you should, for game-theoretical reasons not abide, since being the kinda agent that accepts those terms makes you valid target for such frauds. Reasons for not abiding being the same as for single-boxing.

Jonii May 15, 2013, 1:56 PM
2 points
in reply to: Benya’s comment on: Pascal’s Muggle: Infinitesimal Priors and Strong Evidence
Actually, there is such a law. You cannot reasonably start, when you are born into this world, naked, without any sensory experiences, expecting that the next bit you experience is much more likely to be 1 rather than 0. If you encounter one hundred zillion bits and they all are 1, you still wouldn’t assign 1/3^^^3 probability to next bit you see being 0, if you’re rational enough.

Of course, this is mudded by the fact that you’re not born into this world without priors and all kinds of stuff that weights on your shoulders. Evolution has done billions of years worth of R&D on your priors, to get them straight. However, the gap these evolution-set priors would have to cross to get even close to that absurd 1/3^^^3… It’s a theoretical possibility that’s by no stretch a realistic one.

Jonii Apr 25, 2013, 8:08 PM
2 points
in reply to: Peter Wildeford’s comment on: Open Thread, April 15-30, 2013
I don’t think you need to change the domain name. For marketability, you might wanna have the parts named so that stuff within your site becomes brand in itself, so greatplay.net becomes associated with ” utilitarianism”, ” design” etc. Say, I read a blog by a chemist who has series of blog posts titled “stuff i won’t work with: ”. I can’t remember the domain name, but I know that whenever I want to read about nasty chemical, i google that phrase.

Jonii Apr 19, 2013, 11:39 AM
1 point
in reply to: Watercressed’s comment on: Open Thread, April 15-30, 2013
yes. yes. i remember thinking “x + 0 =”. after that it gets a bit fuzzy.

Jonii Apr 18, 2013, 8:06 PM
7 points
in reply to: [deleted]’s comment on: Open Thread, April 15-30, 2013
Qiaochu_Yuan already answered your question, but because he was pretty technical with his answer, I thought I should try to simplify the point here a bit. The problem with division by zero is that division is essentially defined through multiplication and existence of certain inverse elements. It’s an axiom in itself in group theory that there are inverse elements, that is, for each a, there is x such that ax = 1. Our notation for x here would be 1/a, and it’s easy to see why a 1/a = 1. Division is defined by these inverse elements: a/b is calculated by a * (1/b), where (1/b) is the inverse of b.

But, if you have both multiplication and addition, there is one interesting thing. If we assume addition is the group operation for all numbers(and we use “0” to signify additive neutral element you get from adding together an element and its additive inverse, that is, “a + (-a) = 0″), and we want multiplication to work the way we like it to work(so that a(x + y) = (ax) + (a*y), that is, distributivity hold, something interesting happens.

Now, neutral element 0 is such that x + 0 = x, this is by definition of neutral element. Now watch the magic happen: 0x = (0 + 0)x
= 0x + 0x So 0x = 0x + 0x.

We subtract 0x from both sides, leaving us with 0x = 0.

Doesn’t matter what you are multiplying 0 with, you always end up with zero. So, assuming 1 and 0 are not the same number(in zero ring, that’s the case, also, 0 = 1 is the only number in the entire zero ring), you can’t get a number such that 0*x = 1. Lacking inverse elements, there’s no obvious way to define what it would mean to divide by zero. There are special situations where there is a natural way to interpret what it means to divide by zero, in which cases, go for it. However, it’s separate from the division defined for other numbers.

And, if you end up dividing by zero because you somewhere assumed that there actually was such a number x that 0*x = 1, well, that’s just your own clumsiness.

Also, you can prove 1=2 if you multiply both sides by zero. 1 = 2. Proof: 10 = 20 ⇒ 0 = 0. Division and multiplication work in opposite directions, multiplication gets you from not equals to equals, division gets you from equals to not equals.

Jonii Feb 13, 2013, 11:53 PM
7 points
on: A brief history of ethically concerned scientists
My friend told me he wanted to see http://en.wikipedia.org/wiki/Andrei_Sakharov on this list. I must say that I don’t know the guy, but based on the Wikipedia article, he was a brilliant Soviet nuclear physicist behind few of the largest man-made explosions ever to happen, and somewhere around 1960′s he turned to political activism regarding dangers posed by nuclear arms race. In the political climate of 1960 Soviet Union, that was a brave move, too, and the powers that be made him lose much because of that choice.

Jonii Aug 4, 2012, 4:10 PM
2 points
in reply to: David_Gerard’s comment on: “Epiphany addiction”
Sequences contain a rational world view. Not a comprehensive one, but still, it gives some idea about how to avoid thinking stupid and how to communicate with other people that are also trying to find out what’s true and what’s not. It gives you words by which you can refer to problems in your world view, meta-standards to evaluate whether whatever you’re doing is working, etc. I think of it as an unofficial manual to my brain and the world that surrounds me. You can just go ahead and figure out yourself what works, without reading manuals, but reading a manual before you go makes you better prepared.

Jonii May 23, 2012, 9:32 PM
1 point
on: Problematic Problems for TDT
Interaction of this simulated TDT and you is so complicated I don’t think many of commenters here actually did the math to see how should they expect the simulated TDT agent to react in these situations. I know I didn’t. I tried, and failed.