alex_zag_al

• alex_zag_alNov 5, 2017, 11:13 PM

  1 point

  in reply to: Zuckaschnegge’s comment on: Belief in Self-Deception

  I think that someone who merely believed they were happy, and then experienced real happiness, would not want to go back.

• alex_zag_alFeb 24, 2017, 4:32 PM

  1 point

  on: The cor­rect re­sponse to un­cer­tainty is *not* half-speed

  There’s an important category of choices: the ones where any good choice is “acting as if” something is true.

  That is, there are two possible worlds. And there’s one choice best if you knew you were in world 1, and another choice best if you knew you were in world 2. And, in addition, under any probabilistic mixture of the two worlds, one of those two choices is still optimal.

  The hotel example falls into this category. So, one of the important reasons to recognize this category is to avoid a half-speed response to uncertainty.

  Many choices don’t fall into this category. You can tell because in many decision-making problems, gathering more information is a good decision. But, this is never acting as if you knew one of the possibilities for certain.

  Arguably in your example, information-seeking actually was the best solution: pull over and take out a map or use a GPS.

  It seems like another important category of choices is those where the best option is trying the world 1 choice for a specified amount of time and then trying the world 2 choice. Perhaps these are the choices where the best source of information is observing whether something works? Reminds me of two-armed bandit problems, where acting-as-if and investigating manifest in the same kind of choice (pulling a lever).

• alex_zag_alFeb 24, 2017, 2:52 PM

  0 points

  in reply to: James_Miller’s comment on: The price you pay for ar­riv­ing to class on time

  Yeah. I mean, I’m not saying you should arrive late to class.

  The way to work what you’re saying into the framework is:

  • The cost of consistently arriving late is high

  • The cost (in minutes spent waiting for the class to start) of avoiding consistent lateness is less high

  • Therefore, you should pay this cost in minutes spent waiting

  The point is to quantify the price, not to say you shouldn’t pay it.

The price you pay for ar­riv­ing to class on time

• alex_zag_alFeb 23, 2017, 7:39 AM

  1 point

  in reply to: Lumifer’s comment on: Fact Posts: How and Why

  the soft sciences have to deal with situations which never exactly repeat

  This is also true of evolutionary biology—I think it’s not widely recognized that evolutionary biology is like the soft sciences in this way.

• alex_zag_alFeb 23, 2017, 6:09 AM

  0 points

  on: Fur­ther dis­cus­sion of CFAR’s fo­cus on AI safety, and the good things folks wanted from “cause neu­tral­ity”

  iii. Emphasize all rationality use cases evenly. Cause all people to be evenly targeted by CFAR workshops.

  We can’t do this one either; we are too small to pursue all opportunities without horrible dilution and failure to capitalize on the most useful opportunities.

  This surprised me, since I think of rationality as the general principles of truth-finding.

  What have you found about the degree to which rationality instruction needs to be tailored to a use-case?

• alex_zag_alFeb 23, 2017, 6:09 AM

  0 points

  on: Fur­ther dis­cus­sion of CFAR’s fo­cus on AI safety, and the good things folks wanted from “cause neu­tral­ity”

  Several of these had the form “I, too, think that AI safety is incredibly important — and that is why I think CFAR should remain cause-neutral, so it can bring in more varied participants who might be made wary by an explicit focus on AI.”

  I don’t think that AI safety is important, which I guess makes me one of the “more varied participants made wary by an explicit focus on AI.” Happy you’re being explicit about your goals but I don’t like them.

• alex_zag_alFeb 23, 2017, 6:07 AM

  2 points

  in reply to: Vaniver’s comment on: Fur­ther dis­cus­sion of CFAR’s fo­cus on AI safety, and the good things folks wanted from “cause neu­tral­ity”

  Wow, I’ve read the story but I didn’t quite realize the irony of it being a textbook (not a curriuculum, a textbook, right?) about judgment and decision making.

• alex_zag_alMay 1, 2016, 1:45 AM

  0 points

  on: When (Not) To Use Probabilities

  The alternative I would propose, in this particular case, is to debate the general rule of banning physics experiments because you cannot be absolutely certain of the arguments that say they are safe.

  Giving up on debating the probability of a particular proposition, and shifting to debating the merits of a particular rule, is I feel one of the ideas behind frequentist statistics. Like, I’m not going to say anything about whether the true mean is in my confidence interval in this particular case. But note that using this confidence interval formula works pretty well on average.

• alex_zag_alFeb 23, 2016, 9:56 PM

  0 points

  on: Willing gam­blers, spher­i­cal cows, and AIs

  I don’t know about the role of this assumption in AI, which is what you seem to care most about. But I think I can answer about its role in philosophy.

  One thing I want from epistemology is a model of ideally rational reasoning, under uncertainty. One way to eliminate a lot of candidates for such a model is to show that they make some kind of obvious mistake. In this case, the mistake is judging something as a good bet when really it is guaranteed to lose money.

• alex_zag_alFeb 14, 2016, 2:45 PM

  2 points

  on: Ex­am­ples of Ra­tion­al­ity Tech­niques adopted by the Masses

  Inquiring after the falsifiability of a theory?

  Not perfect but very good, and pretty popular.

• alex_zag_alNov 14, 2015, 11:13 AM

  2 points

  on: When Science Can’t Help

  After a few years in grad school, I think the principles of science are different from what you’ve picked up from your own sources.

  In particular, this stands out to me as incorrect:

  (1) I had carefully followed everything I’d been told was Traditionally Rational, in the course of going astray. For example, I’d been careful to only believe in stupid theories that made novel experimental predictions, e.g., that neuronal microtubules would be found to support coherent quantum states.

  My training in writing grant applications contradicts this depiction of science. A grant has an introduction that reviews the facts of the field. It is followed by your hypothesis, and the mark of a promising grant is that the hypothesis looks obvious given your depiction of the facts. In fact, it is best if your introduction causes the reader to think of the hypothesis themselves, and anticipate its introduction.

  This key feature of a good hypothesis is totally separate from its falsifiability (important later in the application). And remember, the hypothesis has to appear obvious in the eyes of a senior in the field, since that’s who judges your proposal. Can you say this for your stupid theory?

  (2) Science would have been perfectly fine with my spending ten years trying to test my stupid theory, only to get a negative experimental result, so long as I then said, “Oh, well, I guess my theory was wrong.”

  Given the above, the social practice of science would not have funded you to work for ten years on this theory. And this reflects the social practice’s implementation of the ideals of Science. The ideals say your hypothesis, while testable, is stupid.

  I think you have a misconception about how science handles stupid testable ideas. However, I can’t think of a way that this undermines this sequence, which is about how science handles rational untestable ideas.

  EDIT: it seems poke said all this years ago.

• alex_zag_alNov 14, 2015, 9:39 AM

  3 points

  in reply to: johnswentworth’s comment on: The Power of Noise

  Bayesian adaptive clinical trial designs place subjects in treatment groups based on a posterior distribution. (Clinical trials accrue patients gradually, so you don’t have to assign the patients using the prior: you assign new patients using the posterior conditioned on observations of the current patients.)

  These adaptive trials are, as you conjecture, much more efficient than traditional randomized trials.

  Example: I-SPY 2. Assigns patients to treatments based on their “biomarkers” (biological measurements made on the patients) and the posterior derived from previous patients.

  When I heard one of the authors explain adaptive trials in a talk, he said they were based on multi-armed bandit theory, with a utility function that combines accuracy of results with welfare of the patients in the trial.

  However, unlike in conventional multi-armed bandit theory, the trial design still makes random decisions! The trials are still sort of randomized: “adaptively randomized,” with patients having a higher chance of being assigned to certain groups than others, based on the current posterior distribution.

• alex_zag_alNov 14, 2015, 9:14 AM

  0 points

  on: Per­mit­ted Pos­si­bil­ities, & Locality

  Here are some things that shouldn’t happen, on my analysis: An ad-hoc self-modifying AI as in (1) undergoes a cycle of self-improvement, starting from stupidity, that carries it up to the level of a very smart human—and then stops, unable to progress any further.

  I’m sure this has been discussed elsewhere, but to me it seems possible that progress may stop when the mind becomes too complex to make working changes to.

  I used to think that a self-improving AI would foom because as it gets smarter, it gets easier for it to improve itself. But it may get harder for it to improve itself, because as it self-improves it may turn itself into more and more of an unmaintainable mess.

  What if creating unmaintainable messes is the only way that intelligences up to very-smart-human-level know how to create intelligences up to very-smart-human level? That would make that level a hard upper limit on a self-improving AI.

• alex_zag_alNov 1, 2015, 6:06 PM

  0 points

  on: Against the Bot­tom Line

  As I understand the post, its idea is that a rationalist should never “start with a bottom line and then fill out the arguments”.

  I disagree. The idea, rather, is that your beliefs are as good as the algorithm that fills out the bottom line. Doesn’t mean you shouldn’t start by filling out the bottom line; just that you shouldn’t do it by thinking of what feels good or what will win you an argument or by any other algorithm only weakly correlated with truth.

  Also, note that if what you write above the bottom line can change the bottom line, that’s part of the algorithm too. So, actually, I do agree that a rationalist should not write the bottom line, look for a chain of reasoning that supports it, and refuse to change the bottom line if the reasoning doesn’t.

• alex_zag_alJul 30, 2015, 1:36 PM

  0 points

  on: Don’t You Care If It Works? - Part 1

  By trusting Eliezer on MWI, aren’t you trusting both his epistemology and his mathematical intuition?

  Eliezer believes that the MWI interpretation allows you to derive quantum physics without any additional hypotheses that add complexity, such as collapse or the laws of movement for Bohm’s particles. But this belief is based on mathematical intuition, according to the article on the Born probabilities. Nobody knows how to derive the observations without additional hypotheses, but a lot of people such as Eliezer conjecture it’s possible. Right?

  I feel like this point must have been made many times before, as Eliezer’s quantum sequence has been widely discussed, so maybe instead of a response I need a link to a previous conversation or a summary of previous conclusions.

  But relating it to the point of your article… If Eliezer is wrong about quantum mechanics, should that lower my probability that his other epistemological views are correct? This is important because it affects whether or not I bother learning those views. The answer is “yes but not extremely”, because I think if there’s an error, it may be in the mathematical intuition.

  To generalize a bit, it’s hard to find pure tests of a single ability. Though your example of stopping rules is actually a pretty good one, for understanding the meaning of all the probability symbols. But usually we should not be especially confused when someone with expertise is wrong about a single thing, since that single thing is probably not a pure test of that expertise. However we should be confused if on average they are wrong as many times as people without the expertise. Then we have to doubt the expertise or our own judgments of the right and wrong answers.

• alex_zag_alJan 8, 2015, 11:13 PM

  0 points

  on: Train­ing Reflec­tive Attention

  This reminds me of hitting Ctrl+C, but on a thought process or object of focus instead of a program. After reading, I do it when i suspect I’m about to voluntarily do something I’m going to regret.

  EDIT: At least, I think I’m doing it… I haven’t done any training approaching the amount of time the training in your post takes.

• alex_zag_alNov 22, 2014, 7:37 PM

  0 points

  in reply to: christopherj’s comment on: In­for­ma­tion the­ory and the sym­me­try of up­dat­ing beliefs

  Yes… if a theory adds to the surprisal of an experimental result, then the experimental result adds precisely the same amount of the surprisal of the theory. That’s interesting.

• alex_zag_alNov 22, 2014, 5:32 AM

  2 points

  in reply to: thomblake’s comment on: In­for­ma­tion the­ory and the sym­me­try of up­dat­ing beliefs

  Much like how inches and centimeters are off by a constant factor. Different log bases are analogous to different units.

• alex_zag_alNov 16, 2014, 2:00 AM

  2 points

  on: Un­friendly Nat­u­ral Intelligence

  The fear of cults, and the related fear of cults of personality, are antimemes against excessive awe of persons.