Louie

• LouieJan 10, 2013, 6:14 AM

  12 points

  in reply to: Qiaochu_Yuan’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  Yep, SI has summer internships. You’re already in Berkeley, right?

  Drop me an email with the dates you’re available and what you’d want out of an internship. My email and Malo’s are both on our internship page:

  http://​​singularity.org/​​interns/​​

  Look forward to hearing from you.

• LouieJan 9, 2013, 7:23 PM

  8 points

  in reply to: jimrandomh’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  Well, I figure I don’t really want to recommend a ton of programming courses anyway. I’m already recommending what I presume is more than a bachelor’s degree worth of course when pre-reqs and outside requirements at these universities are taken into account.

  So if someone takes one course, they can learn so much more that helps them later in this curriculum from the applied, function programming course than its imperative counterpart. And the normal number of functional programming courses that people take in a traditional math or CS program is 0. So I have to make a positive recommendation here to correct this. I couldn’t make people avoid imperative programming courses anyway, even if I tried. So people will oversample them (and follow your implied recommendation) relative to my core recommendations anyway.

  So in practice, most people will follow your advice, by following mine and actually studying some functional programming instead of none and then study a ton of imperative programming no matter what anyone says.

• LouieJan 9, 2013, 6:40 PM

  12 points

  in reply to: NancyLebovitz’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  Ahh. Yeah, I’d expect that kind of content is way too specific to be built into initial FAI designs. There are multiple reasons for this, but off the top of my head,

  • I expect design considerations for Seed AI to favor smaller designs that only emphasize essential components for both superior ability to show desirable provability criteria, as well as improving design timelines.

  • All else equal, I expect that the less arbitrary decisions or content the human programmers provide to influence the initial dynamic of FAI, the better.

  • And my broadest answer is it’s not a core-Friendliness problem, so it’s not on the critical path to solving FAI. Even if an initial FAI design did need medical content or other things along those lines, this would be something that we could hire an expert to create towards the end of solving the more fundamental Friendliness and AI portions of FAI.

• LouieJan 9, 2013, 6:04 PM

  7 points

  in reply to: NancyLebovitz’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  I don’t think those courses would impoverish anyones’ minds. I expect people to take courses that aren’t on this list without me having to tell people to do that. But I wouldn’t expect courses drawn from these subjects to be mainstream recommendations for Friendliness researchers who were doing things like formalizing and solving problems relating to self-referencing mathematical structures and things along those lines.

• LouieJan 9, 2013, 6:00 PM

  6 points

  in reply to: gwern’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  Good question. If I remember correctly, Berkeley teaches from it and one person I respect agreed it was good. I think the impenetrability was consider more of a feature than a bug by the person doing the recommending. IOW, he was assuming that people taking my recommendations would be geniuses by-and-large and that the harder book would be better in the long-run for the brightest people who studied from it.

  Part of my motivation for posting this here was to improve my recommendations. So I’m happy to change the rec to something more accessible if we can crowd-source something like a consensus best choice here on LW that’s still good for the smartest readers.

  What links here?

  • Book Re­view: Math­e­mat­ics for Com­puter Science (Sugges­tion for MIRI Re­search Guide) by richard_reitz (Jul 22, 2017, 7:26 PM; 22 points)

• LouieJan 9, 2013, 4:41 PM

  2 points

  in reply to: bgaesop’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  Fixed. Thanks.

• LouieJan 9, 2013, 4:37 PM

  10 points

  0

  in reply to: jimrandomh’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  The functional/​imperative distinction is not a real one

  How is the distinction between functional and imperative programming languages “not a real one”? I suppose you mean that there’s a continuum of language designs between purely functional and purely imperative. And I’ve seen people argue that you can program functionally in python or emulate imperative programming in Haskell. Sure. That’s all true. It doesn’t change the fact that functional-style programming is manifestly more machine checkable in the average (and best) case.

  it’s less important to provability than languages’ complexity, the quality of their type systems and the amount of stupid lurking in their dark corners.

  Agreed. The most poorly programmed functional programs will be harder to machine check than the mostly skillfully designed imperative programs. But I think for typical programming scenarios or best case scenarios, functional-style programming makes it hands-down more natural to write the correct kind of structures that can be reliably machine checked and imperative programming languages just don’t.

  The entry level functional programming course is going to focus on all the right things: type theory, model theory, deep structure. The first imperative programming course at most universities is going to teach you how to leverage side-effects, leverage side-effects more, and generally design your code in a way that makes it less tractable for verification and validation later on.

• LouieJan 9, 2013, 4:21 PM

  16 points

  in reply to: Mitchell_Porter’s comment on: Course recom­men­da­tions for Friendli­ness researchers

  But I’m not sure where that is best covered.

  Yeah, universities don’t reliably teach a lot of things that I’d want people to learn to be Friendliness researchers. Heuristics and Biases is about the closest most universities get to the kind of course you recommend… and most barely have a course on even that.

  I’d obviously be recommending lots of Philosophy and Psychology courses as well if most of those courses weren’t so horribly wrong. I looked through the course handbooks and scoured them for courses I could recommend in this area that wouldn’t steer people too wrong. As Luke has mentioned (partially from being part of this search with me), you can still profitably take a minority of philosophy courses at CMU without destroying your mind, a few at MIT, and maybe two or three at Oxford. And there are no respectable, mainstream textbooks to recommend yet.

  Believe me, Luke and I are sad beyond words every day of our lives that we have to continue recommending people read a blog to learn philosophy and a ton of other things that colleges don’t know how to teach yet. We don’t particularly enjoy looking crazy to everyone outside of the LW bubble.

• LouieJan 9, 2013, 2:39 PM

  2 points

  on: Course recom­men­da­tions for Friendli­ness researchers

  PS—I had some initial trouble formatting my table’s appearance. It seems to be mostly fixed now. But if an admin wants to tweak it somehow so the text isn’t justified or it’s otherwise more readable, I won’t complain! :)

Course recom­men­da­tions for Friendli­ness researchers

• LouieDec 19, 2012, 2:13 PM

  8 points

  in reply to: John_Maxwell’s comment on: Bound­ing the im­pact of AGI

  I believe Coq is already short and proven using other proving programs that are also short and validated. So I believe the tower of formal validation that exists for these techniques is pretty well secured. I could be wrong about that though… would be curious to know the answer to that.

  Relatedly, there are a lot of levels you can go with this. For instance, I wish someone would create other programming languages like CompCert for programming formally validated programs.

• LouieDec 19, 2012, 2:55 AM

  0 points

  in reply to: Louie’s comment on: Bound­ing the im­pact of AGI

  Martel (1997) estimates a considerably higher annualized death rate of 3,500 from meteorite impacts alone (she doesn’t consider continental drift or gamma-ray bursts), but the internal logic of safety engineering demands we seek a lower bound, one that we must put up with no matter what strides we make in redistribution of food, global peace, or healthcare.

  Is this correct? I’d expect that this lower-bound was superior to the above (10 deaths /​ year) for the purpose of calculating our present safety factor… unless we’re currently able to destroy earth-threatening meteorites and no one told me.

• LouieDec 19, 2012, 2:26 AM

  8 points

  in reply to: gwern’s comment on: Bound­ing the im­pact of AGI

  http://​​kornai.com/​​Drafts/​​agi12.pdf

• LouieDec 19, 2012, 2:20 AM

  6 points

  on: Bound­ing the im­pact of AGI

  To paraphrase Kornai’s best idea (which he’s importing from outside the field):

  A reasonable guideline is limiting the human caused xrisk to several orders of magnitude below the natural background xrisk level, so that human-caused dangers are lost in the noise compared to the pre-existing threat we must live with anyway.

  I like this idea (as opposed to foolish proposals like driving risks from human made tech down to zero), but I expect someone here could sharpen the xrisk level that Kornai suggests. Here’s a disturbing note from the appendix where he does his calculation:

  Here we take the “big five” extinction events that occurred within the past half billion years as background. Assuming a mean time of 10^8 years between mass extinctions and 10^9 victims in the next one yields an annualized death rate of 10, comparing quite favorably to the reported global death rate of ~500 for contact with hornets, wasps, and bees (ICD-9-CM E905.3). [emphasis added]

  Obviously, this is a gross mis-understanding of xrisks and why they matter. No one values human lives linearly straight down to 0 or assumes no expansion factors for future generations.

  A motivated internet researcher could probably just look up the proper citations from Bostrom’s “Global Catastrophic Risks” and create a decomposed model that estimated the background xrisk level from only nature (and then only nature + human risks w/​o AI), and develop a better safety margin that would be lower than the one in this paper (implying that AGI could afford to be a few orders of magnitude riskier than Kornai’s rough estimates).

Bound­ing the im­pact of AGI

LessWrong podcasts

New WBE implementation

• LouieNov 18, 2012, 10:04 AM

  13 points

  in reply to: Steve_Rayhawk’s comment on: Thoughts on the Sin­gu­lar­ity In­sti­tute (SI)

  Note that this was most of the purpose of the Fellows program in the first place -- [was] to help sort/​develop those people into useful roles, including replacing existing management

  FWIW, I never knew the purpose of the VF program was to replace existing SI management. And I somewhat doubt that you knew this at the time, either. I think you’re just imagining this retroactively given that that’s what ended up happening. For instance, the internal point system used to score people in the VFs program had no points for correctly identifying organizational improvements and implementing them. It had no points for doing administrative work (besides cleaning up the physical house or giving others car rides). And it had no points for rising to management roles. It was all about getting karma on LW or writing conference papers. When I first offered to help with the organization directly, I was told I was “too competent” and that I should go do something more useful with my talent, like start another business… not “waste my time working directly at SI.”

• LouieNov 4, 2012, 3:58 AM

  0 points

  in reply to: Kevin’s comment on: Pop­u­lar me­dia cov­er­age of Sin­gu­lar­ity Sum­mit -the Verge [link]

  I didn’t notice any factual inaccuracies

  Although, multiple quotes were manufactured and misattributed.

Sign up to be no­tified about new LW mee­tups in your area