scav

Karma: 712

scav Nov 7, 2013, 2:51 PM
6 points
on: Yes, Virginia, You Can Be 99.99% (Or More!) Certain That 53 Is Prime
I’ve never been completely happy with the “I could make 1M similar statements and be wrong once” test. It seems, I dunno, kind of a frequentist way of thinking about the probability that I’m wrong. I can’t imagine making a million statements and have no way of knowing what it’s like to feel confidence about a statement to an accuracy of one part per million.

Other ways to think of tiny probabilities:

(1) If probability theory tells me there’s a 1 in a billion chance of X happening, then P(X) is somewhere between 1 in a billion and P(I calculated wrong), the latter being much higher.

If I were running on hardware that was better at arithmetic, P(I calculated wrong) could be got down way below 1 in a billion. After all, even today’s computers do billions of arithmetic operations per second. If they had anything like a one-in-a-billion failure rate per operation, we’d find them much less useful.

(2) Think of statements like P(7 is prime) = 1 as useful simplifications. If I am examining whether 7 is prime, I wouldn’t start with a prior of 1. But if I’m testing a hypothesis about something else and it depends on (among other things) whether 7 is prime, I wouldn’t assign P(7 is prime) some ridiculously specific just-under-1 probability; I’d call it 1 and simplify the causal network accordingly.

scav Nov 5, 2013, 5:39 PM
3 points
in reply to: A1987dM’s comment on: Rationality Quotes November 2013
From the point of view of your genes, likely to reproduce and beneficial are exactly the same thing. That’s trivially true.

Also not particularly interesting even if true: crazy beliefs that get you killed or prevent you from breeding have to spread non-parentally. They don’t have to be particularly persuasive or virulent, there just has to be some other mechanism (e.g. state control of education, military discipline, enjoyable but idiotic forms of mass entertainment) to spread them.

The prevalence of these means doesn’t even have to depend on the ones spreading the memes adopting them for themselves: there can be an economic motive for those who promote them for others to believe crazy things; like there is for drug dealers to sell rather than use heroin.

scav Nov 5, 2013, 5:28 PM
6 points
in reply to: Eugine_Nier’s comment on: Rationality Quotes November 2013
I don’t think there’s any way to discriminate between crazy things your mum believes and crazy things the man on the street corner believes.

I also think the virulence of a meme complex, like the virulence of a virus, is very dependent on the context i.e. the population it is introduced to and the other memes it competes with in that population.

“what do you think you know, and how do you think you know it?” is snappy enough to be “virulent” and, I think, not too harmful to the individual host.

scav Oct 20, 2013, 11:10 AM
1 point
in reply to: bokov’s comment on: Looking for opinions of people like Nick Bostrom or Anders Sandberg on current cryo techniques
Oh, OK. I get you. I don’t describe myself as a patternist, and I might not be what you mean by it. In any case I am not making the first of those claims.

However, it seems possible to me that a sufficiently close copy of me would think it was me, experience being me, and would maybe even be more similar to me as a person than biological me of five years ago or five years hence.

I do claim that it is theoretically possible to construct such a copy, but I don’t think it is at all probable that signing up for cryonics will result in such a copy ever being made.

If I had to give a reason for thinking it’s possible in principle, I’d have to say: I am deeply sceptical that there is any need for a “self” to be made of anything other than classical physical processes. I don’t think our brains, however complex, require in their physical construction, anything more mysterious than room-temperature chemistry.

The amazing mystery of the informational complexity of our brains is undiminished by believing it to be physically prosaic when you reduce it to its individual components, so it’s not like I’m trying to disappear a problem I don’t understand by pretending that just saying “chemistry” explains it.

I stand by my scepticism of the self as a single indivisible entity with special properties that are posited only to make it agreeable to someone’s intuition, rather than because it best fits the results of experiment. That’s really all my post was about: impatience with argument from intuition and argument by hand-waving.

I’ll continue to doubt the practicality of cryonics until they freeze a rat and restore it 5 years later to a state where they can tell that it remembers stimuli it was taught before freezing. If that state is a virtual rat running on silicon, that will be interesting too.

scav Oct 18, 2013, 7:20 PM
1 point
in reply to: bokov’s comment on: Looking for opinions of people like Nick Bostrom or Anders Sandberg on current cryo techniques
Just for exercise, let’s estimate the probability of the conjunction of my claims.

claim A: I think the idea of a single ‘self’ in the brain is provably untrue according to currently understood neuroscience. I do honestly think so, therefore P(A) as close to 1.0 as makes no difference. Whether I’m right is another matter.

claim B: I think a wildly speculative vague idea thrown into a discussion and then repeatedly disclaimed does little to clarify anything. P(B) approx 0.998 - I might change my mind before the day is out.

claim C: The thing I claim to think in claim B is in fact “usually” true. P(C) maybe 0.97 because I haven’t really thought it through but I reckon a random sample of 20 instances of such would be unlikely to reveal 10 exceptions, defeating the “usually”.

claim D: A running virtual machine is a physical process happening in a physical object. P(D) very close to 1, because I have no evidence of non-physical processes, and sticking close to the usual definition of a virtual machine, we definitely have never built and run a non-physical one.

claim E: You too are a physical process happening in a physical object. P(E) also close to 1. Never seen a non-physical person either, and if they exist, how do they type comments on lesswrong?

claim F: Nobody knows enough about the reality of consciousness to make legitimate claims that human minds are not information-processing physical processes. P(F) = 0.99. I’m pretty sure I’d have heard something if that problem had been so conclusively solved, but maybe they were disappeared by the CIA or it was announced last week and I’ve been busy or something.

P( A B C D E F) is approx 0.96.

The amount of money I’d bet would depend on the odds on offer.

I fear I may be being rude by actually answering the question you put to me instead of engaging with your intended point, whatever it was. Sorry if so.

scav Oct 18, 2013, 4:05 PM
0 points
in reply to: Mitchell_Porter’s comment on: Looking for opinions of people like Nick Bostrom or Anders Sandberg on current cryo techniques

My best bet is that the self is a single physical thing, a specific physical phenomenon, which forms at a definite moment in the life of the organism, persists through time even during unconsciousness, and ceases to exist when its biological matrix becomes inhospitable.

How much do you want to bet on the conjunction of all those claims? (hint: I think at least one of them is provably untrue even according to current knowledge)

That is just a wild speculation, made for the sake of concreteness.

I don’t think it supplied the necessary amount of concreteness to be useful; this is usual for wild speculation. ;)

The physical object is the reality, the virtual machine is just a concept.

A running virtual machine is a physical process happening in a physical object. So are you.

This is false to the robust reality of consciousness

Well, nobody actually knows enough about the reality of consciousness to make that claim. It may be that it is incompatible with your intuitions about consciousness. Mine too, so I haven’t any alternative claims to make in response.

scav Oct 11, 2013, 11:42 AM
8 points
in reply to: ChrisHallquist’s comment on: A Voting Puzzle, Some Political Science, and a Nerd Failure Mode
I find the conclusion that the US would be better off with some form of proportional representation pretty compelling actually, and I don’t think it’s so implausible that it would make a positive difference.

The difference it makes in Europe (compared to the UK for example) seems to be that the smaller parties with agendas the median voter doesn’t care much about still get a voice in parliament. It’s worth it for the Greens or the Pirate party to campaign for another 1% of the vote, because they get another 1% of the seats, instead of nothing.

It should be a better marketplace of ideas; although a few major parties still keep most of the power, they have more incentive to accommodate or adopt new ideas. I suppose the presence of the minor parties increases the visibility of multiple policy axes, forcing the major parties to compete for the median voter along each axis.

Having said that, it still isn’t very relevant to the thrust of the post, so the decision to footnote it was probably correct.

scav Oct 10, 2013, 10:44 AM
2 points
on: A Voting Puzzle, Some Political Science, and a Nerd Failure Mode
Thanks for identifying Duverger’s Law. I had never heard of it, but I had informally grasped its application in UK politics.

scav Oct 8, 2013, 8:07 AM
3 points
in reply to: Lumifer’s comment on: Rationality Quotes October 2013
Depressing but plausible :(

I suspect “the way they are presented in the popular media” is crafted with that in mind.

scav Oct 7, 2013, 11:39 AM
9 points
in reply to: James_Miller’s comment on: Rationality Quotes October 2013
Corollary: all organisations eventually contain sub-competent people. Design protocols accordingly.

scav Oct 7, 2013, 11:36 AM
0 points
in reply to: Lumifer’s comment on: Rationality Quotes October 2013
Citation, or at least a clear example, needed. I can probably construct two policy alternatives, and predict which will be attractive to people who identify with a given political tribe. Then I suppose I get to call one of those options the “stupid” one based on my own value system.

Please tell me that isn’t the sort of thing you mean.

I have met people with what I consider to be very irrational political views (in that they are little more than clusters of rote debating points never subjected to analysis). Outside of the well-worn habitual responses their politics would dictate they regurgitate, I have no idea how they would choose on an issue they had never encountered before.

Maybe stupidly (because they aren’t in the habit of reflective thought), but maybe less so (because without a knee-jerk political reaction ready to hand, they might take a few seconds to think).

I will go so far as to agree that in too many cases, simple answers will be favoured over complex questions, and instant gratification will be favoured over longer-term advantage.

scav Sep 9, 2013, 7:38 PM
1 point
in reply to: pengvado’s comment on: The genie knows, but doesn’t care
It’s still probably premature to guess whether friendliness is provable when we don’t have any idea what it is. My worry is not that it wouldn’t be possible or provable, but that it might not be a meaningful term at all.

But I also suspect friendliness, if it does mean anything, is in general going to be so complex that “only [needing] to find a single program that provably has behaviour X” may be beyond us. There are lots of mathematical conjectures we can’t prove, even without invoking the halting problem.

One terrible trap might be the temptation to make simplifications in the model to make the problem provable, but end up proving the wrong thing. Maybe you can prove that a set of friendliness criteria are stable under self-modification, but I don’t see any way to prove those starting criteria don’t have terrible unintended consequences. Those are contingent on too many real-world circumstances and unknown unknowns. How do you even model that?

scav Sep 9, 2013, 7:15 PM
0 points
in reply to: John_Maxwell’s comment on: The genie knows, but doesn’t care
That first one would be worth doing even if we didn’t dare hand the AI the keys to go and make changes. To study a non-human-created ontology would be fascinating and maybe really useful.

scav Sep 9, 2013, 1:47 PM
4 points
in reply to: XiXiDu’s comment on: The genie knows, but doesn’t care
First list:

1) Poorly defined terms “human intention” and “sufficient”.

2) Possibly under any circumstances whatsoever, if it’s anything like other non-trivial software, which always has some bugs.

3) Anything from “you may not notice” to “catastrophic failure resulting in deaths”. Claim that failure of software to work as humans intend will “generally fail in a way that is harmful to it’s own functioning” is unsupported. E.g. a spreadsheet works fine if the floating point math is off in the 20th bit of the mantissa. The answers will be wrong, but there is nothing about that that the spreadsheet could be expected to care about,

4) Not necessarily. GAI may continue to try to do what it was programmed to do, and only unintentionally destroy a small city in the process :)

Second list:

1) Wrong. The abilities of sufficiently complex systems are a huge space of events humans haven’t thought about yet, and so do not yet have preferences about. There is no way to know what their preferences would or should be for many many outcomes.

2) Error as failure to perform the requested action may take precedence over error as failure to anticipate hypothetical objections from some humans to something they hadn’t expected. For one thing, it is more clearly defined. We already know human-level intelligences act this way.

3) Asteroids and supervolcanoes are not better than humans at preventing errors. It is perfectly possible for something stupid to be able to kill you. Therefore something with greater cognitive and material resources than you, but still with the capacity to make mistakes can certainly kill you. For example, a government.

4) It is already possible for a very fallible human to make something that is better than humans at detecting certain kinds of errors.

5) No. Unless by dramatic you mean “impossibly perfect, magical and universal”.

scav Sep 4, 2013, 9:33 PM
3 points
in reply to: DanArmak’s comment on: Rationality Quotes September 2013
A little cynical maybe? Politicians don’t spend 100% of the time making decisions for purely political reasons. Sometimes they are trying to achieve something, even if broadly speaking the purposes of politics are as you imply.

But of course, most of the people we would prefer to be more rational don’t know that’s what politics is for, so they aren’t hampered by that particular excuse to give up on it. Anyway, they could quite reasonably expect more rational decision making from co-workers, doctors, teachers and others.

I don’t think the people making decisions to optimise an outcome are well exemplified by bureaucrats. Try engineers.

Knowing that politics is part of what people do, and that destroying it is impossible, yes I would be trying to improve it, and hope for a more-rational population of participants to reform it. I would treat a claim that the way it is now is eternal and unchangeable as an extraordinary one that’s never been true so far. So, good luck with that :)

You aren’t seriously suggesting the mean of the sanity distribution hasn’t moved a huge amount since the Bible was written? Or even in the last 100 years? I know I’m referring to a “sanity distribution” in an unquantifiable hand-wavy way, but do you doubt that those people who believe in a literalist interpretation of the Bible are now outliers, rather that the huge majority they used to be?

scav Sep 4, 2013, 11:00 AM
3 points
in reply to: DanArmak’s comment on: Rationality Quotes September 2013
Yet.

And you don’t even need a majority of rationalists by headcount. You just need to find and hack the vulnerable parts of your culture and politics where you have a chance of raising people’s expectations for rational decision making. Actual widespread ability in rationality skills comes later.

Whenever you feel pessimistic about moving the mean of the sanity distribution, try reading the Bible or the Iliad and see how far we’ve come already.

scav Sep 4, 2013, 10:46 AM
2 points
in reply to: johnswentworth’s comment on: Rationality Quotes September 2013
That is incoherent at best. Is there any context to the quote that might explain why it is here?

scav Aug 13, 2013, 12:27 PM
1 point
in reply to: ESRogs’s comment on: Post ridiculous munchkin ideas!
Some other reason: I just don’t know how EY pronounces “Yudkowsky” -- [jʊd’kaʊski] or [ju:d’kɔvski] or otherwise.

But there is a significant overlap between great names for scientists and words that would be worth a lot in Scrabble if proper nouns were allowed.

scav Jun 5, 2013, 2:07 PM
0 points
in reply to: sediment’s comment on: Rationality Quotes June 2013
Of course, when you are trying to get more of “them” to be “us”, it’s worth pointing out what “they” are doing wrong. It’s not like anyone without brain damage is born and destined to be an “unscientific man” for life.

scav May 31, 2013, 1:02 PM
0 points
in reply to: William_Quixote’s comment on: The Centre for Applied Rationality: a year later from a (somewhat) outside perspective
Yeah, I can see how hierarchical organisations benefit certain goals and activities. I was speaking specifically about the goal of teaching rationality, in case that wasn’t clear from context. You don’t need a central authority to control what is being taught so much unless you are teaching irrationality (c.f. Scientology, Roman Catholicism or any political organisation).

You could probably run a million rationality courses a year using just a wiki and a smartphone app. (Left as an exercise for the reader)