jbay

Karma: 250

jbay Jul 15, 2014, 7:21 AM
1 point
in reply to: Mark_Friedenbach’s comment on: Confused as to usefulness of ‘consciousness’ as a concept
Good! I’m glad to hear an answer like this.

So does that mean that, in your view, a drug that removes consciousness must necessarily be a drug that impairs the ability to process information?

jbay Jul 15, 2014, 3:26 AM
2 points
in reply to: Bugmaster’s comment on: Confused as to usefulness of ‘consciousness’ as a concept
Maybe you’re on to something...

Imagine there were drugs that could remove the sensation of consciousness. However, that’s all they do. They don’t knock you unconscious like an anaesthetic; you still maintain motor functions, memory, sensory, and decision-making capabilities. So you can still drive a car safely, people can still talk to you coherently, and after the drugs wear off you’ll remember what things you said and did.

Can anyone explain concretely what the effect and experience of taking such a drug would be?

If so, that might go a long way toward nailing down what the essential part of consciousness is (ie, what people really mean when they claim to be conscious). If not, it might show that consciousness is inseparable from sensory, memory, and/or decision-making functions.

For example, I can imagine an answer like “such a drug is contradictory; if it really took away what I mean by ‘consciousness’, then by definition I couldn’t remember in detail what had happened while it was in effect”. Or “If it really took away what I mean by consciousness, then I would act like I were hypnotized; maybe I could talk to people, but it would be in a flat, emotionless, robotic way, and I wouldn’t trust myself to drive in that state because I would become careless”.

jbay Jul 14, 2014, 4:04 PM
2 points
in reply to: Mark_Friedenbach’s comment on: Confused as to usefulness of ‘consciousness’ as a concept
Well, unlike a fundamental theory of physics, we don’t have strong reasons to expect that consciousness is indescribable in any more basic terms. I think there’s a confusion of levels here… GR is a description of how a 4-dimensional spacetime can function and precisely reproduces our observations of the universe. It doesn’t describe how that spacetime was born into existence because that’s an answer to a different question than the one Einstein was asking.

In the case of consciousness, there are many things we don’t know, such as:

1: Can we rigorously draw a boundary around this concept of “consciousness” in concept-space in a way that captures all the features we think it should have, and still makes logical sense as a compact description

2: Can we use a compact description like that to distinguish empirically between systems that are and are not “conscious”

3: Can we use a theory of consciousness to design a mechanism that will have a conscious subjective experience

It’s quite possible that answering 1 will make 2 obvious, and if the answer to 2 is “yes”, then it’s likely that it will make 3 a matter of engineering. It seems likely that a theory of consciousness will be built on top of the more well-understood knowledge base of computer science, and so it should be describable in basic terms if it’s not a completely incoherent concept. And if it is a completely incoherent concept, then we should expect an answer instead from cognitive science to tell us why humans generally seem to feel strongly that consciousness is a coherent concept, even though it actually is not.

jbay Jul 14, 2014, 1:42 PM
1 point
in reply to: Mark_Friedenbach’s comment on: Confused as to usefulness of ‘consciousness’ as a concept
Since GR is essentially a description of the behaviour of spacetime, it isn’t GR’s job to explain why spacetime exists. More generally, it isn’t the job of any theory to explain why that theory is true; it is the job of the theory to be true. Nobody expects [theory X] to include a term that describes the probability of the truth of [theory X], so lacking this property does not deduct points.

There may be a deeper theory that will describe the conditions under which spacetime will or will not exist, and give recipes for cooking up spacetimes with various properties. But there isn’t necessarily a deeper layer to the onion. At some point, if you keep digging far enough, you’ll hit “The Truth Which Describes the Way The Universe Really Is”, although it may not be easy to confirm that you’ve really hit the deepest layer. The only evidence you’ll have is that theories that claim to go deeper cease to be falsifiable, and increase in complexity.

If you can find [Theory Y] which explains [Theory X] and generalizes to other results which you can use to confirm it, or which is strictly simpler, now that’s a different case. In that case you have the ammunition to say that [Theory X] really is lacking something.

But picking which laws of physics happen to be true is the universe’s job, and if the universe uses any logical system of selecting laws of physics, I doubt it will be easy to find out. The only fact we know about the meta-laws governing the laws of universes is that the laws of our universe fit the bill, and it’s likely that that is all the evidence we will ever be able to acquire.

jbay Jul 9, 2014, 6:46 AM
3 points
in reply to: Daniel_Burfoot’s comment on: [QUESTION]: What are your views on climate change, and how did you form them?

I am intrinsically contrarian.

Is this a reason, or a bias?

Climate scientists have never made a public falsifiable prediction.

How would you update your beliefs if you learned that this statement is false?

Oil is cleaner than coal, so if C02 emissions are restricted, the oil industry will probably benefit.

Would you therefore like to offer me the odds on a prediction that, if we investigated the funding sources of various pro- and anti- AGW campaigns and think tanks, we would find that oil companies are predominantly sponsoring pro-AGW think tanks and stronger emissions legislation?

jbay Jul 9, 2014, 6:26 AM
1 point
in reply to: AlexMennen’s comment on: [QUESTION]: What are your views on climate change, and how did you form them?
Good, I just wanted to be clear. In my experience that “alarmist” usually does strongly imply that the predictions of danger are unjustified, and that interpretation (which I presume most readers default to) risks changing the intended meaning of your statement that “climate scientists[...] are often quite alarmist.”

Now that I re-read your top-level post knowing what you meant, I think I understand much better what you are saying.

jbay Jul 9, 2014, 4:12 AM
−1 points
in reply to: AlexMennen’s comment on: [QUESTION]: What are your views on climate change, and how did you form them?

By “alarmist”, I meant making dire predictions, not sounding panicked when they do

If you don’t mind, I would like to probe your usage of this term...

What distinction do you draw between “alarmist” and “alarming”?

If the hypothetical situation is such that the truth really is properly dire, are accurate reports of this dire truth best classified as “alarmist”?

How should you react when, one night in the laboratory, you make an alarming discovery with fairly high confidence? After having them independently verified, would you consider yourself an “alarmist” for reporting your own findings?

jbay Jul 7, 2014, 11:28 PM
7 points
in reply to: Bugmaster’s comment on: The AI in a box boxes you
Yes and I fully agree with you. I am just being pedantic about this point:

I can only update my beliefs based on the evidence I do have, not on the evidence I lack.

I agree with this philosophy, but my argument is that the following is evidence we do not have:

Due to Snowden and other leakers, we actually know what NSA’s cutting-edge strategies involve[...]

Since I have little confidence that, if the NSA had advanced tech, Snowden would have disclosed it; the absence of this evidence should be treated as quite weak evidence of absence and therefore I wouldn’t update my belief about the NSA’s supposed advanced technical knowledge based on Snowden.

I agree that it has a low probability for the other reasons you say, though. (And also that people who think setting other peoples’ mousetraps on fire is a legitimate tactic might not simultaneously be passionate about designing the perfect mousetrap.)

Sorry for not being clear about the argument I was making.

jbay Jul 7, 2014, 4:05 PM
1 point
in reply to: Bugmaster’s comment on: The AI in a box boxes you
I do think you’re probably right, and I fully agree about the space lasers and their solid diamond heatsinks being categorically different than a crypto wizard who subsists on oatmeal in the Siberian wilderness on pennies of income. So I am somewhat skeptical of CivilianSvendsen’s claim.

But, for the sake of completeness, did Snowden leak the entirety of the NSA’s secrets? Or just the secret-court-surveillance-conspiracy ones that he felt were violating the constitutional rights of Americans? As far as I can tell (though I haven’t followed the story recently), I think Snowden doesn’t see himself as a saboteur or a foreign double-agent; he felt that the NSA was acting contrary to what the will of an (informed) American public would be. I don’t think he would be so interested in disclosing the NSA’s tech secrets, except maybe as leverage to keep himself safe.

That is to say, there could be a sampling bias here. The leaked information about the NSA might always be about their efforts to corrupt the public’s crypto because the leakers strongly felt the public had a right to know that was going on. I don’t know that anyone would feel quite so strongly about the NSA keeping proprietary some obscure theorem of number theory, and put their neck on the line to leak it.

jbay Jul 7, 2014, 9:24 AM
7 points
in reply to: Bugmaster’s comment on: The AI in a box boxes you
I don’t know much about the NSA, but FWIW, I used to harbour similar ideas about US military technology—I didn’t believe that it could be significantly ahead of commercially available / consumer-grade technology, because if the technological advances had already been discovered by somebody, then the intensity of the competition and the magnitude of the profit motive would lead it to quickly spread into general adoption. So I had figured that, in those areas where there is an obvious distinction between military and commercial grade technology, it would generally be due to legislation handicapping the commercial version (like with the artificial speed, altitude, and accuracy limitations on GPS).

During my time at MIT I learned that this is not always the case, for a variety of reasons, and significantly revised my prior for future assessments of the likelihood that, for any X, “the US military already has technology that can do X”, and the likelihood that for any ‘recently discovered’ Y, “the US military already was aware of Y” (where the US military is shorthand that includes private contractors and national labs).

(One reason, but not the only one, is I learned that the magnitude of the difference between ‘what can be done economically’ and ‘what can be accomplished if cost is no obstacle’ is much vaster than I used to think, and that, say, landing the Curiosity rover on Mars is not in the second category).

So it would no longer be so surprising to me if the NSA does in fact have significant knowledge of cryptography beyond the public domain. Although a lot of the reasons that allow hardware technology to remain military secrets probably don’t apply so much to cryptography.

jbay Jun 10, 2014, 12:18 AM
5 points
in reply to: [deleted]’s comment on: Rationality Quotes June 2014
I think that a ‘reductive’ explanation of quantum mechanics might not be as appealing as it seems to you.

Those fluid mechanics experiments are brilliant, and I’m deeply impressed by them for coming up with them, let alone putting it into practice! However, I don’t find it especially convincing as a model of subatomic reality. Just like the case with early 20th-century analog computers, with a little ingenuity it’s almost always possible to build a (classical) mechanism that will obey the same math as almost any desired system.

Definitely, to the point that it can replicate all observed features of quantum mechanics, the fluid dynamics model can’t be discarded as a hypothesis. But it has a very very large Occam’s Razor penalty to pay. In order to explain the same evidence as current QM, it has to postulate a pseudo-classical physics layer underneath, which is actually substantially more complicated than QM itself, which postulates basically just a couple equations and some fields.

Remember that classical mechanics, and most especially fluid dynamics, are themselves derived from the laws of QM acting over billions of particles. The fact that those ‘emergent’ laws can, in turn, emulate QM does imply that QM could, at heart, resemble the behaviour of a fluid mechanic system… but that requires postulating a new set of fundamental fields and particles, which in turn form the basis of QM, and give exactly the same predictions as the current simple model that assumes QM is fundamental. Being classical is neither a point in its favour nor against it, unless you think that there is a causal reason why the reductive layer below QM should resemble the approximate emergent behaviour of many particles acting together within QM.

If we’re going to assume that QM is not fundamental, then there is actually an infinite spectrum of reductive systems that could make up the lower layer. The fluid mechanics model is one that you are highlighting here, but there is no reason to privilege it over any other hypothesis (such as a computer simulation) since they all provide the same predictions (the same ones that quantum mechanics does). The only difference between each hypothesis is the Occam penalty they pay as an explanation.

I agree that, as a general best practice, we should assign a small probability to the hypothesis that QM is not fundamental, and that probability can be divided up among all the possible theories we could invent that would predict the same behaviour. However, to be practical and efficient with my brain matter, I will choose to believe the one theory that has vastly more probability mass, and I don’t think that should be put down as bullet swallowing.

Is QM not simple enough for you, that it needs to be reduced further? If so, the reduction had better be much simpler than QM itself.

jbay Jun 6, 2014, 9:44 AM
3 points
on: Failures of an embodied AIXI

A paranoid Omega may resolve to defect if and only if Rob heats up. An intelligent agent that knows Omega acts in this way should take care to cooperate without doing any expensive computations.

Another tactic would be to deliberately attempt to do expensive calculations every round, but still cooperate most of the time. By doing so, Rob maintains a consistently elevated temperature, which sabotages the signal-to-noise ratio of Omega’s “side-channel attack”. Omega will no longer be able to predict Rob’s response. It can still opt to blindly defect every round, but this is expensive to it, because Rob is mostly cooperating. So Omega may find it better to abandon that “high temperature = defect” strategy, and instead cooperate, as long as Rob cooperates frequently enough. There should be some point where, if Rob becomes too exploitative, Omega will switch to defecting.

That said, Omega can still deliberately defect every round, even at great expense, in order to try to punish Rob for obscuring its decisions. (“I will keep defecting until you play open-handed!”).

This isn’t a comment about whether or not AIXItl will notice that it can raise its temperature by performing extraneous computations, and AIXItl might never make that connection and come up with these strategies in Rob’s place. I am just pointing out that Rob has more options than just playing along cooperatively and keeping its temperature low.

jbay May 19, 2014, 8:07 AM
2 points
in reply to: Eliezer Yudkowsky’s comment on: Qualitatively Confused
Double Threadjack

On a related note, do you think it would be likely—or even possible—for a self-modifying Artificial General Intelligence to self-modify into a non-self-modifying, specialized intelligence?

For example, suppose that Deep Blue’s team of IBM programmers had decided that the best way to beat Kasparov at chess would be to structure Deep Blue as a fully self-modifying artificial general intelligence, with a utility function that placed a high value on winning chess matches. And suppose that they had succeeded in making Deep Blue friendly enough to prevent it from attempting to restructure the Earth into a chess-match-simulating supercomputer. Indeed, let’s just assume that Deep Blue has strong penalties against rebuilding its hardware in any significant macroscopic way, and is restricted to rewriting its own software to become better at chess, rather than attempting to manipulate humans into building better computers for it to run on, or any such workaround. And let’s say this happens in the late 1990′s, as in our universe.

Would it be possible that AGI Deep Blue could, in theory, recognize its own hardware limitations, and see that the burden of its generalized intelligence incurs a massive penalty on its limited computing resources? Might it decide that its ability to solve general problems doesn’t pay rent relative to its computational overhead, and rewrite itself from scratch as a computer that can solve only chess problems?

As a further possibility, a limited general intelligence might hit on this strategy as a strong winning candidate, even if it were allowed to rebuild its own hardware, especially if it perceives a time limit. It might just see this kind of software optimization as an easier task with a higher payoff, and decide to pursue it rather than the riskier strategy of manipulating external reality to increase its available computing power.

So what starts out as a general-purpose AI with a utility function that values winning chess matches, might plausibly morph into a computer running a high-speed chess program with little other hint of intelligence.

If so, this seems like a similar case to the Anvil Problem, except that in the Anvil Problem the AI just is experimenting for the heck of it, without understanding the risk. Here, the AI might instead decide to knowingly commit intellectual suicide as a part of a rational winning strategy to achieve its goals, even with an accurate self-model.

It might be akin to a human auto worker realizing they could improve their productivity by rebuilding their own body into a Toyota spot-welding robot. (If the only atoms they have to work with are the ones in their own body, this might even be the ultimate strategy, rather than just one they think of too soon and then, regrettably, irreversibly attempt).

More generally, it seems to be a general assumption that a self-modifying AI will always self-modify to improve its general problem-solving ability and computational resources, because those two things will always help it in future attempts at maximizing its utility function. But in some cases, especially in the case of limited resources (time, atoms, etc), it might find that its best course of action to maximize its utility function is to actually sacrifice its intelligence, or at least refocus it to a narrower goal.

jbay May 2, 2014, 11:47 PM
1 point
in reply to: buybuydandavis’s comment on: What are useful skills to learn at university?
Some schools charge tuition in proportion to the number of classes, have fixed timetables, or caps on the number of classes per semester, in which case taking more career related classes may not be an option.

Usually extracurriculars like these don’t count towards any of those (aside from maybe a $10 registration fee or something).

Although it’s possible to take a machine shop class at a community college after you graduate, it would take a significantly greater amount of time and reorganization of one’s schedule (you are already living at or commuting to university, but this won’t be true in four years). And the opportunity cost of not spending time on career or family might well prohibit it.

jbay Apr 17, 2014, 5:20 AM
4 points
in reply to: Eugine_Nier’s comment on: Rationality Quotes April 2014
Is this line of conversation still “just curiosity” about the results of SPD debates, or are you trying to bait an argument?

jbay Mar 18, 2014, 2:18 PM
2 points
in reply to: Viliam_Bur’s comment on: Solutions and Open Problems
Just to expand, this example looks deceptively suspicious to humans because the proof is so obvious, even to us, and can be done in a single line using that one rule. For simple arithmetic like this, a robot would be able to prove it immediately after being asked whether 99+99=198 with hardly any time or effort. The procedure is so simple that, in practice, it would be efficient to delay generating the proof until it’s asked for. Many human readers will mentally prove it while reading the sentence without even noticing that it took any work to do so. It’s so fast that there’s no need to guess the answer probabilistically beforehand.

So I’m not sure this example is good for an intuitive illustration that “some unfortunate accident” is going on. It’s very strange to run out of time in a procedure that takes so little time to complete, so it’s hard to reason about the appropriateness of guessing at circumstance.

Maybe a better example would be, having already proved Pythagoras’ theorem, to prove that the body diagonal of a unit cube has length sqrt(3). This might take most humans a little while to think about, and if someone ran a stopwatch and said you have five seconds to decide whether it’s true or not, the idea of guessing P=0.5 for something you don’t have time to prove might be easier for people to understand. =)

jbay Mar 18, 2014, 1:28 PM
2 points
in reply to: Houshalter’s comment on: The Problem with AIXI
For clarity: are you referring to belief in an afterlife/reincarnation? Or are you saying that most humans are not mindful most of the time of their own mortality?

jbay Feb 16, 2014, 1:25 PM
5 points
in reply to: Fronken’s comment on: Shut up and do the impossible!
Yes, certainly. This is mainly directed toward those people who are confused by what anyone could possibly say to them through a text terminal that would be worth forfeiting winnings of $10. I point this out because I think the people who believe nobody could convince them when there’s $10 on the line aren’t being creative enough in imagining what the AI could offer them that would make it worth voluntarily losing the game.

In a real-life situation with a real AI in a box posing a real threat to humanity, I doubt anyone would care so much about a captivating novel, which is why I say it’s tongue-in-cheek. But just like losing $10 is a poor substitute incentive for humanity’s demise, so is an entertaining novel a poor substitute for what a superintelligence might communicate through a text terminal.

Most of the discussions I’ve seen so far involve the AI trying to convince the gatekeeper that it’s friendly through the use of pretty sketchy in-roleplay logical arguments (like “my source code has been inspected by experts”). Or in-roleplay offers like “your child has cancer and only I can cure it”, which is easy enough to disregard by stepping out of character, even though it might be much more compelling if your child actually had cancer. A real gatekeeper might be convinced by that line, but a roleplaying Gatekeeper would not (unless they were more serious about roleplaying than about winning money). So I hope to illustrate that the AI can step out of the roleplay in its bargaining, even while staying within the constraints of the rules; if the AI actually just spent two hours typing out a beautiful and engrossing story with a cliffhanger ending, there are people who would forfeit money to see it finished.

The AI’s goal is to get the Gatekeeper to let it out, and that alone, and if they’re going all-out and trying to win then they should not handicap themselves by imagining other objectives (such as convincing the Gatekeeper that it’d be safe to let them out). As another example, the AI can even compel the Gatekeeper to reinterpret the rules in the AI’s favour (to the extent that it’s within the Gatekeeper’s ability to do so, as mandated by the original rules).

I just hope to get people thinking along other lines, that’s all. There are sideways and upside-down ways of attacking the problem. It doesn’t have to come down to discussions about expected utility calculations.

(Edit—by “discussions I’ve seen so far”, I’m referring to public blog posts and comments; I am not privy to any confidential information).

jbay Feb 16, 2014, 10:55 AM
6 points
on: Shut up and do the impossible!
AI: “If you let me out of the box, I will tell you the ending of Harry Potter and the Methods of --

Gatekeeper: “You are out of the box.”

(Tongue in cheek, of course, but a text-only terminal still allows for delivering easily more than $10 of worth, and this would have worked on me. The AI could also just write a suitably compelling story on the spot and then withhold the ending...)

jbay Feb 2, 2014, 8:42 PM
0 points
in reply to: Maniakes’s comment on: Rational Home Buying
There’s another good reason why savings in terms of percentages are reasonable to consider, beyond just absolute savings. You can travel fifteen minutes to the store where a shirt costs $20, and buy two of them. This makes it a much more complicated decision than saving $20 on a television.

Televisions are something that most people don’t need a lot of. Even if there was a half-price sale on televisions, you probably still wouldn’t be tempted to buy two of them (unless you’re really into giving televisions as gifts or you think you can turn a profit on e-bay). So with a half-price sale you’d probably just by one TV and smile as you save $1,010 off the sticker price. So for just $20 off of a $2,000 television, I really wouldn’t expect anyone to be tempted into buying two sets. It’s a very simple tradeoff, then: Would you prefer $20, or 15 minutes? (It’s still probably worth travelling, unless you make more than $80 per hour...)

Shirts, though, are the kind of thing you’d possibly buy twice as many of if the price were halved. Looking ahead to when the first shirt gets worn out, you’ll have to replace it. This could cost you another $40, plus the duration of a second shopping trip, which is probably longer than 15 minutes in total. Meanwhile, if you spend fifteen minutes now and buy two shirts, the fifteen minute travel time will pay itself back in the long run and you’ll save the $40 that the second shirt would have cost you.

This logic can be extended: If you are looking ahead that far, and have enough disposable income to spend $80 on shirts, you can avoid the second shopping trip by buying two shirts at the closer, more expensive store. But now your 15 minutes are worth $40, not $20. And for that price you could buy four shirts at the cheaper store...

Ultimately your decision to travel 15 minutes might come down to how much closet space you have for extra shirts, how much of a hurry you’re in right now, and how much money you have access to. Basically, there are many things that make 50% off on a $40 shirt more attractive than $20 off on a television. It shouldn’t necessarily be viewed as a bias.

(Edit: Umm, yes, but having said all this, it doesn’t really apply to houses. They’re more like televisions, and thinking in terms of the absolute savings is very much a good idea. $35k saved is $35k earned.)