endoself

• endoselfJun 13, 2017, 3:41 AM

  2 points

  in reply to: cousin_it’s comment on: We are the Athe­ni­ans, not the Spartans

  Note that I … wrote the only comment on the IAF post you linked

  Yes, I replied to it :)

  Unfortunately, I don’t expect to have more Eliezer-level explanations of these specific lines of work any time soon. Eliezer has a fairly large amount of content on Arbital that hasn’t seen LW levels of engagement either, though I know some people who are reading it and benefiting from it. I’m not sure how LW 2.0 is coming along, but it might be good to have a subreddit for content similar to your recent post on betting. There is an audience for it, as that post demonstrated.

• endoselfJun 12, 2017, 9:49 PM

  2 points

  in reply to: whpearson’s comment on: We are the Athe­ni­ans, not the Spartans

  Maybe you’ve heard this before, but the usual story is that the goal is to clarify conceptual questions that exist in both the abstract and more practical settings. We are moving towards considering such things though—the point of the post I linked was to reexamine old philosophical questions using logical inductors, which are computable.

  Further, my intuition from studying logical induction is that practical systems will be “close enough” to satisfying the logical induction critereon that many things will carry over (much of this is just intuitions one could also get from online learning theory). E.g. in the logical induction decision theory post, I expect the individual points made using logical inductors to mostly or all apply to practical systems, and you can use the fact that logical inductors are well-defined to test further ideas building on these.

• endoselfJun 11, 2017, 11:42 PM

  3 points

  in reply to: cousin_it’s comment on: We are the Athe­ni­ans, not the Spartans

  Scott Garrabrant and I would be happy to see more engagement with the content on Agent Foundations (IAF). I guess you’re right that the math is a barrier. My own recent experiment of linking to Two Major Obstacles for Logical Inductor Decision Theory on IAF was much less successful than your post about betting, but I think that there’s something inessential about the inaccessiblity.

  In that post, for example, I think the math used is mostly within reach for a technical lay audience, except that an understanding of logical induction is assumed, though I may have missed some complexity in looking it over just now. Even for that, it should be possible to explain enough about logical inductors briefly and accessibly enough to let someone understand a version of that post, though I’m not sure if that has been done. People recommend this talk as the best existing introduction.

Two Ma­jor Ob­sta­cles for Log­i­cal In­duc­tor De­ci­sion Theory

• endoselfNov 30, 2015, 7:06 AM

  3 points

  in reply to: diegocaleiro’s comment on: The Growth of My Pes­simism: Tran­shu­man­ism, Im­mor­tal­ism, Effec­tive Altru­ism.

  I model probabilistic thinking as something you build on top of all this. First you learn to model the world at all (your steps 3-8), then you learn the mathematical description of part of what your brain is doing when it does all this. There are many aspects of normative cognition that Bayes doesn’t have anything to say about, but there are also places where you come to understand what your thinking is aiming at. It’s a gears model of cognition rather than the object-level phenomenon.

  If you don’t have gears models at all, then yes, it’s just another way to spout nonsense. This isn’t because it’s useless, it’s because people cargo-cult it. Why do people cargo-cult Bayesianism so much? It’s not the only thing in the sequences. The first post, The Simple Truth, big parts of Mysterious Answers to Mysterious Questions, and basically all of Reductionism are about the gears-model skill. Even the name rationalism evokes Descartes and Leibniz, who were all about this skill. My own guess is that Eliezer argued more forcefully for Bayesianism than for gears models in the sequences because, of the two, it is the skill that came less naturally to him, and that stuck.

  What would cargo-cult gears models look like? Presumably, scientism, physics envy, building big complicated models with no grounding in reality. This too is a failure mode visible in our community.

• endoselfJul 27, 2015, 9:48 PM

  5 points

  in reply to: Yaacov’s comment on: Wel­come to Less Wrong! (8th thread, July 2015)

  Hi Yaacov!

  The most active MIRIx group is at UCLA. Scott Garrabrant would be happy to talk to you if you are considering research aimed at reducing x-risk. Alternatively, some generic advice for improving your future abilities is to talk to interesting people, try to do hard things, and learn about things that people with similar goals do not know about.

• endoselfApr 9, 2015, 12:38 AM

  2 points

  in reply to: Mark_Friedenbach’s comment on: Even when con­trar­i­ans win, they lose: Jeff Hawkins

  As far as I can tell, you’ve misunderstood what I was trying to do with this post. I’m not claiming that Hawkins’ work is worth pursuing further; passive_fist’s analysis seems pretty plausible to me. I was just trying to give people some information that they may not have on how some ideas developed, to help them build a better model of such things.

  (I did not downvote you. If you thought that I was arguing for further work towards Hawkins’ progam, then your comment would be justified, and in any case this is a worthwhile thing for me to explicitly disclaim.)

• endoselfApr 9, 2015, 12:24 AM

  3 points

  in reply to: joaolkf’s comment on: Even when con­trar­i­ans win, they lose: Jeff Hawkins

  Yeah, I didn’t mean to contradict any of this. I wonder how much a role previous arguments from MIRI and FHI played in changing the zeitgeist and contributing to the way Superintelligence was received. There was a slow increase in uninformed fear-of-AI sentiments over the preceding years, which may have put people in more of a position to consider the arguments in Superintelligence. I think that much of this ultimately traces back to MIRI and FHI; for example many anonymous internet commenters refer to them or use phrasing inspired by them, though many others don’t. I’m more sceptical that this change in zeitgeist was helpful though.

  Of course specific people who interacted with MIRI/​FHI more strongly, such as Jaan Tallinn and Peter Thiel, were helpful in bring the discourse to where it is today.

• endoselfApr 8, 2015, 8:36 AM

  2 points

  in reply to: IlyaShpitser’s comment on: Even when con­trar­i­ans win, they lose: Jeff Hawkins

  The quote from Ng is

  The big AI dreams of making machines that could someday evolve to do intelligent things like humans could, I was turned off by that. I didn’t really think that was feasible, when I first joined Stanford. It was seeing the evidence that a lot of human intelligence might be due to one learning algorithm that I thought maybe we could mimic the human brain and build intelligence that’s a bit more like the human brain and make rapid progress. That particular set of ideas has been around for a long time, but [AI expert and Numenta cofounder] Jeff Hawkins helped popularize it.

  I think it’s pretty clear that he would have worked on different things if not for Hawkins. He’s done a lot of work in robotics, for example, so he could have continued working on robotics if he didn’t get interested in general AI. Maybe he would have moved into deep learning later in his career, as it started to show big results.

Even when con­trar­i­ans win, they lose: Jeff Hawkins

• endoselfFeb 18, 2015, 10:09 PM

  14 points

  in reply to: Sherincall’s comment on: Open thread, Feb. 16 - Feb. 22, 2015

  GiveWell, GiveDirectly, Evidence Action/​Deworm the World. You can vote for multiple charities.

• endoselfOct 24, 2014, 4:23 AM

  63 points

  on: 2014 Less Wrong Cen­sus/​Survey

  I took the survey.

• endoselfJul 22, 2014, 6:53 AM

  3 points

  in reply to: Wei Dai’s comment on: Knigh­tian Uncer­tainty and Am­bi­guity Aver­sion: Motivation

  I can’t see how this would work. Wouldn’t the UDT-ish approach be to ask an MMEU agent to pick a strategy once, before making any updates? The MMEU agent would choose a strategy that makes it equivalent to a Bayesian agent, as I describe. The characteristic ambiguity-averse behaviour only appears if the agent is allowed to update.

  Given a Cartesian boundary between agent and environment, you could make an agent that prefers to have its future actions be those that are prescribed by MMEU, and you’d then get MMEU-like behaviour persisting upon reflection, but I assume this isn’t what you mean since it isn’t UDT-ish at all.

• endoselfJul 22, 2014, 5:01 AM

  12 points

  0

  on: Knigh­tian Uncer­tainty and Am­bi­guity Aver­sion: Motivation

  MMEU isn’t stable upon reflection. Suppose that in addition to the mysterious [0.4, 0.6] coin, you had a fair coin, and I tell you that all offer bet 1 (“pay 50¢ to be payed $1.10 if the coin came up heads”) if the fair coin comes up heads and bet 2 if the fair coin comes up tails, but you have to choose whether to accept or reject before flipping the fair coin to decide which bet will be chosen. In this case, the Knighian uncertainty cancels out, and your expected winnings are +5¢ no matter which value is [0.4, 0.6] is taken to be the true probabilty of the mysterious coin, so you would take this bet on MMEU.

  Upon seeing how the fair coin turns out, however, MMEU would tell you to reject whichever of bets 1 and 2 is offered. Thus, if I offer to let you see the result of the fair coin before deciding whether to accept the bet, you will actually prefer not to see the coin, for an expected outcome of +5¢, rather than see the coin, reject the bet, and win nothing with certainty. Alternatively, if given the chance, you would prefer to self-modify so as to not exhibit ambiguity aversion in this scenario.

  In general, any agent using a decision rule that is not generalized Bayesian performs strictly worse than some generalized Bayes decision rule. Note, though, that this does not mean that such an agent is forced to accept at least one of bets 1 and 2, since rejecting whichever of them is offered is a Bayes rule; for example, a Bayesian agent who believes that the bookie knows something that they don’t will behave in this way. It does mean, though, that there are many situations where MMEU cannot work, such as in my example above, since in such scenarios it is not equivalent to any Bayes rule.

• endoselfJul 22, 2014, 4:57 AM

  10 points

  in reply to: Manfred’s comment on: Knigh­tian Uncer­tainty and Am­bi­guity Aver­sion: Motivation

  This is a very general point. Most of the uncertainty people face is of the sort that they would naively classify as Knighian, so if people actually behaved according to MMEU, then they would essentially be playing minimax against the world.

• endoselfNov 24, 2013, 6:31 AM

  30 points

  on: 2013 Less Wrong Cen­sus/​Survey

  I took the census. My answers for MWI and Ailens were conditional on ¬Simulation, since if we are in a simulation where MWI doesn’t hold, the simulation is probably intended to provide information about a universe in which MWI does hold.

• endoselfNov 18, 2013, 7:06 AM

  15 points

  on: Quan­tum ver­sus log­i­cal bombs

  I’m not sure what quantum mechanics has to do with this. Say humanity is spread over 10 planets. Would you rather take a logical ⁹⁄₁₀ chance of wiping out humanity, or destroy 9 of the planets with certainty (and also destroy 90% of uninhabited planets to reduce the potential for future growth by the same degree)? Is there any ethically relevant difference between these scenarios?

• endoselfNov 18, 2013, 4:07 AM

  0 points

  in reply to: Stuart_Armstrong’s comment on: Re­duced im­pact AI: no back channels

  even if P is omniscient, P’ still has to estimate it’s expected output from its own limited perspective. As long as this estimate is reasonable, the omniscience of P doesn’t cause a problem (and remember that P is fed noisy data).

  Don’t you have to get the exact level of noise that will prevent the AI from hiding from P without letting P reconstruct the AI’s actions if it does allow itself to be destroyed? An error in either direction can be catastrophic. If the noise is to high, the AI takes over the world. If the noise is to low, E’(P(Sᵃ|X,Oᵃ,B)/​P(Sᵃ|¬X,Õᵃ,B) | a) is going to be very far from 1 no matter what, so there is no reason to expect that optimizing it is still equivalent to reducing impact.

• endoselfNov 2, 2013, 8:25 PM

  8 points

  in reply to: EHeller’s comment on: Lone Ge­nius Bias and Re­turns on Ad­di­tional Researchers

  He’s talking specifically about people donating to AMF. There are more things people can do than donate to AMF and donate to one of MIRI, FHI, CEA, and CFAR.

• endoselfOct 18, 2013, 11:16 PM

  1 point

  in reply to: Eliezer Yudkowsky’s comment on: MIRI’s 2013 Sum­mer Match­ing Challenge

  Yes, you can take the probability that they will halt given a random input. This is analogous to the case of a universal Turing machine, since the way we ask it to simulate a random Turing machine is by giving it a random input string.