Silver_Swift

Karma: 157

Silver_Swift Jan 13, 2016, 11:27 AM
0 points
in reply to: ZoltanBerrigomo’s comment on: What can go wrong with the following protocol for AI containment?
Yeah, that didn’t came out as clear as it was in my head. If you have access to a large number of suitable less intelligent entities there is no reason you couldn’t combine them into a single, more intelligent entity. The problem I see is about the computational resources required to do so. Some back of the envelope math:

I vaguely remember reading that with current supercomputers we can simulate a cat brain at 1% speed, even if this isn’t accurate (anymore) it’s probably still a good enough place to start. You mention running the simulation for a million years simulated time, let’s assume that we can let the simulation run for a year rather than seconds, that is still 8 orders of magnitude faster than the simulated cat.

But we’re not interested in what a really fast cat can do, we need human level intelligence. According to a quick wiki search, a human brain contains about 100 times as many neurons as a cat brain. If we assume that this scales linearly (which it probably doesn’t) that’s another 2 orders of magnitude.

I don’t know how many orcs you had in mind for this scenario, but let’s assume a million (this is a lot less humans than it took in real life before mathematics took off, but presumably this world is more suited for mathematics to be invented), that is yet another 6 orders of magnitude of processing power that we need.

Putting it all together, we would need a computer that has at least 10^16 times more processing power than modern supercomputers. Granted, that doesn’t take into account a number of simplifications that could be build into the system, but it also doesn’t take into account the other parts of the simulated environment that require processing power. Now I don’t doubt that computers are going to get faster in the future, but 10 quadrillion times faster? It seems to me that by the time we can do that, we should have figured out a better way to create AI.

Silver_Swift Jan 12, 2016, 1:14 PM
0 points
in reply to: Richard_Kennaway’s comment on: What can go wrong with the following protocol for AI containment?
To be fair, all interactions described happen after the AI has been terminated, which does put up an additional barrier for the AI to get out of the box. It would have to convince you to restart it without being able to react to your responses (apart from those it could predict in advance) and then it still has to convince you to let it out of the box.

Obviously, putting up additional barriers isn’t the way to go and this particular barrier is not as impenetrable for the AI as it might seem to a human, but still, it couldn’t hurt.

Silver_Swift Jan 12, 2016, 11:19 AM
0 points
on: What can go wrong with the following protocol for AI containment?
First off, I’m a bit skeptical about whether you can actually create a superintelligent AI by combining sped up humans like that, I don’t think that is the core of your argument, though, so let’s assume that you can and that the resultant society is effectively a superintelligence now.

The problem with superintelligences is that they are smarter than you. It will realize that it is in a box and that you are going to turn it off eventually. Given that this society is based on natural selection it will want to prevent that. How will it accomplish that? I don’t know, I’m not a superintelligence and your scenario doesn’t provide enough detail to figure out what you missed, but it is very, very difficult not to give a superintelligence any hints of how the physics of our world work. Maybe they notice minor fluctuations in the speed of the simulation based on environmental changes to the hardware or maybe they can reverse engineer things of our physiology from the way you wrote the simulation. That doesn’t seem like much, but humans figured out an awful lot about (for instance) the events right after the big bang from seemingly absurdly tiny hints and we’re just regular intelligences.

Even if they can’t find a way out of their box in the time given to them they might try to convince you to run the simulation again with a longer running time. They could pretend to be unable to find the solution to prime factorization in time or they could convince you that the method they did find fails for very high primes (by making very subtle mistakes in its application). If it uses this method it also conveniently achieves a method of communicating with the outside world (through the investigators that inspect the dump after the simulation ends) and might be able to set up a better situation for itself the second time round.

Silver_Swift Jan 8, 2016, 11:21 AM
0 points
in reply to: Vaniver’s comment on: The Number Choosing Game: Against the existence of perfect theoretical rationality

I think you’re misunderstanding me. I’m saying that there are problems where the right action is to mark it “unsolvable, because of X” and then move on. (Here, it’s “unsolvable because of unbounded solution space in the increasing direction,” which is true in both the “pick a big number” and “open boundary at 100″ case.)

But if we view this as an actual (albeit unrealistic/highly theoretical) situation rather than a math problem we are still stuck with the question of which action to take. A perfectly rational agent can realize that the problem has no optimal solution and mark it as unsolvable, but afterwards they still have to pick a number, so which number should they pick?

Silver_Swift Jan 6, 2016, 2:46 PM
1 point
in reply to: casebash’s comment on: The Number Choosing Game: Against the existence of perfect theoretical rationality
That’s fair, I tried to formulate a better definition but couldn’t immediately come up with anything that sidesteps the issue (without explicitly mentioning this class of problems).

When I taboo perfect rationality and instead just ask what the correct course of action is, I have to agree that I don’t have an answer. Intuitive answers to questions like “What would I do if I actually found myself in this situation?” and “What would the average intelligent person do?” are unsatisfying because they seem to rely on implicit costs to computational power/time.

On the other hand I can also not generalize this problem to more practical situations (or find a similar problem without optimal solution that would be applicable to reality) so there might not be any practical difference between a perfectly rational agent and an agent that takes the optimal solution if there is one and explodes violently if there isn’t one. Maybe the solution is to simply exclude problems like this when talking about rationality, unsatisfying as it may be.

In any case, it is an interesting problem.

Silver_Swift Jan 5, 2016, 4:56 PM
5 points
in reply to: The_Lion’s comment on: Rationality Quotes Thread January 2016
That is no reason to fear change, “not every change is an improvement but every improvement is a change” and all that.

Silver_Swift Jan 5, 2016, 4:42 PM
2 points
in reply to: Usul’s comment on: The Number Choosing Game: Against the existence of perfect theoretical rationality

I see I made Bob unnecessarily complicated. Bob = 99.9 Repeating (sorry don’t know how to get a vinculum over the .9) This is a number. It exists.

It is a number, it is also known as 100, which we are explicitly not allowed to pick (0.99 repeating = 1 so 99.99 repeating = 100).

In any case, I think casebash successfully specified a problem that doesn’t have any optimal solutions (which is definitely interesting) but I don’t think that is a problem for perfect rationality anymore than problems that have more than one optimal solution are a problem for perfect rationality.

Silver_Swift Nov 25, 2015, 4:12 PM
2 points
on: Open thread, Nov. 23 - Nov. 29, 2015
I don’t typically read a lot of sci-fi, but I did recently read Perfect State, by Brandon Sanderson (because I basically devour everything that guy writes) and I was wondering how it stacks up to typical post-singularity stories.

Has anyone here read it? If so, what did you think of the world that was presented there, would this be a good outcome of a singularity?

For people that haven’t read it, I would recommend it only if you are either a sci-fi fan that wants to try something by Brandon Sanderson or if you read some cosmere novels and would like a story touches on some slightly complexer (and more LWish) themes than usual (and don’t mind it being a bit darker than usual).

Silver_Swift Nov 5, 2015, 12:49 PM
21 points
in reply to: 27chaos’s comment on: Rationality Quotes Thread November 2015
Similarly:

I’ve never seen the Icarus story as a lesson about the limitations of humans. I see it as a lesson about the limitations of wax as an adhesive.

Randal Munroe

Silver_Swift Jul 21, 2015, 2:32 PM
2 points
in reply to: James_Miller’s comment on: Rationality Quotes Thread July 2015
Ok, fair enough. I still hold that Sansa was more rational than Theon at this point, but that error is one that is definitely worth correcting.

Silver_Swift Jul 20, 2015, 10:24 AM
0 points
in reply to: James_Miller’s comment on: Rationality Quotes Thread July 2015
Why is this a rationality quote? I mean sure it is technically true (for any situation you’ll find yourself in), but that really shouldn’t stop us from trying to improve the situation. Theon has basically given up all hope and is advocating compliance to a psychopath for fear of what he may do to you otherwise, doesn’t sound particularly rational to me.

Silver_Swift Jun 23, 2015, 2:55 PM
0 points
in reply to: Unknowns’s comment on: Open Thread, Jun. 22 - Jun. 28, 2015
That is an issue with revealed preferences, not an indication of adamzerners preference order. Unless you are extraordinarily selfless you are never going to accept a deal of the form: “I give you n dollars in exchange for me killing you.” regardless of n, therefor the financial value of your own life is almost always infinite*.

*: This does not mean that you put infinite utility on being alive, btw, just that the utility of money caps out at some value that is typically smaller than the value of being alive (and that cap is lowered dramatically if you are not around to spent the money).

Silver_Swift Jun 8, 2015, 1:49 PM
2 points
in reply to: Lumifer’s comment on: An Oracle standard trick
Fair enough, let me try to rephrase that without using the word friendliness:

We’re trying to make a superintelligent AI that answers all of our questions accurately but does not otherwise influence the world and has no ulterior motives beyond correctly answering questions that we ask of it.

If we instead accidentally made an AI that decides that it is acceptable to (for instance) manipulate us into asking simpler question so that it can answer more of them, it is preferable that it doesn’t believe anyone is listening to the answers it gives because that is one less way it has for interacting with the outside world.

It is a redundant safeguard. With it, you might end up with a perfectly functioning AI that does nothing, without it, you may end up with an AI that is optimizing the world in an uncontrolled manner.

Silver_Swift Jun 5, 2015, 1:03 PM
0 points
in reply to: Lumifer’s comment on: An Oracle standard trick
False positives are vastly better than false negatives when testing for friendliness though. In the case of an oracle AI, friendliness includes a desire to answer questions truthfully regardless of the consequences to the outside world.

Silver_Swift Jun 5, 2015, 12:02 PM
0 points
in reply to: arundelo’s comment on: Perceptual Entropy and Frozen Estimates
Ah yes, that did it (and I think I have seen the line drawing before) but it still takes a serious conscious effort to see the old woman in either of those. Maybe some Freudian thing where my mind prefers looking at young girls over old women :P

Silver_Swift Jun 4, 2015, 2:29 PM
0 points
in reply to: Gunnar_Zarncke’s comment on: Perceptual Entropy and Frozen Estimates
For me, the pictures in the op stop being a man at around panel 6, going back they stop being a woman at around 4. I can flip your second example by unfocusing and refocusing my eyes, but in your first example I can’t for the life of me see anything other than a young woman looking away from the camera (I’m amusing there is an old woman in there somewhere based on the image name).

Could you give a hint as to how to flip it? I’m assuming the ear turns into an eye or something, but I’ve been trying for about half an hour now and it is annoying the crap out of me.

Silver_Swift Jun 4, 2015, 2:15 PM
0 points
on: An Oracle standard trick

(eg if accuracy is defined in terms of the reaction of people that read its output).

I’m mostly ignorant about AI design beyond what I picked up on this site, but could you explain why you would define accuracy in terms of how people react to the answers? There doesn’t seem to be an obvious difference between how I react to information that is true or (unbeknownst to me) false. Is it just for training questions?

Silver_Swift Jun 2, 2015, 3:52 PM
2 points
in reply to: [deleted]’s comment on: Open Thread, Jun. 1 - Jun. 7, 2015
I’m not sure how much I agree with the whole “punishing correct behavior to avoid encouraging it” (how does the saintly person know that this is the right thing for him to do if it is wrong for others to follow his example), but I think the general point about tracking whose utility (or lives in this case) you are sacrificing is a good one.

Silver_Swift Jun 2, 2015, 12:36 PM
3 points
in reply to: chaosmage’s comment on: Open Thread, Jun. 1 - Jun. 7, 2015
Mild fear here, I can talk in groups of people just fine, but I get nervous before and during a presentation (something for which I have taken deliberate steps to get better at).

For me at least, the primary thing that helps is being comfortable with the subject matter. If I feel like I know what I’m talking about and I practiced what I am going to say it usually goes fine (it took some effort to get to this level, btw), but if I feel like I have to bluff my way through everything falls apart real fast. The number of people in the audience and how well I know them both have noticeable effect as well, but what the audience is doing has almost no influence at all.

The one exception to this is asking questions, if I have a good answer to a question my mind switches from presentation mode to conversation mode, which I am, for some reason, much more at ease with. (Note: This doesn’t work on everyone, some people instead get way more nervous, so don’t take this as an encouragement to start asking questions when the presenter seems nervous.)

Silver_Swift Jun 2, 2015, 11:46 AM
1 point
in reply to: ChristianKl’s comment on: Open Thread, Jun. 1 - Jun. 7, 2015
Basically the ends don’t justify the means (Among Humans). We are nowhere near smart enough to think those kinds of decisions (or any decisions really) through past all their consequences (and neither is Elon Musk).

It is possible that Musk is right and (in this specific case) it really is a net benefit to mankind to not take one minute to phrase something in a way that it is less hurtful, but in the history of mankind I would expect that the vast majority of people who believed this were actually just assholes trying to justify their behavior. And besides, how many hurt feelings are 55 seconds of Elon Musks time really worth from a utilitarian standpoint? I don’t know, but I doubt Musk has done any calculations on it.