robo

• robo 19 May 2024 6:05 UTC

  1 point

  0

  in reply to: TsviBT’s comment on: robo’s Shortform

  I think this is the sort of conversation we should be having! [Side note: I think restricting compute is more effective than restricting research because you don’t need 100% buy in.

  1. it’s easier to prevent people from manufacturing semiconductors than to keep people from learning ideas that fit on a napkin

  2. It’s easier to prevent scientists in Eaccistan from having GPUs than to prevent scientists in Eaccistan from thinking.

  The analogy to nuclear weapons is, I think, a good one. The science behind nuclear weapons is well known—what keeps them from being built is access to nuclear materials.

  (Restricting compute also seriously restricts research. Research speed on neural nets is in large part bounded by how many experiments you run rather than ideas you have.)]

• robo 18 May 2024 22:10 UTC

  1 point

  0

  on: To Limit Im­pact, Limit KL-Divergence

  I think the weakness with KL divergence is that the potentially harmful model can do things the safe model would be exponentially unlikely to do. Even if the safe model has a 1 in 1 trillion chance of stabbing me in the face, the KL penalty to stabbing me in the face is log(1 trillion) (and logs make even huge numbers small).

  What about limiting the unknown model to chose one of the cumulative 98% most likely actions for the safe model to take? If the safe model never has more than a 1% chance of taking an action that will kill you, then the unknown model won’t be able to take an action that kills you. This isn’t terribly different from the Top-K sampling many language models use in practice.

• robo 18 May 2024 21:11 UTC

  33 points

  17

  on: robo’s Shortform

  Our current big stupid: not preparing for 40% agreement

  Epistemic status: lukewarm take from the gut (not brain) that feels rightish

  The “Big Stupid” of the AI doomers 2013-2023 was AI nerds’ solution to the problem “How do we stop people from building dangerous AIs?” was “research how to build AIs”. Methods normal people would consider to stop people from building dangerous AIs, like asking governments to make it illegal to build dangerous AIs, were considered gauche. When the public turned out to be somewhat receptive to the idea of regulating AIs, doomers were unprepared.

  Take: The “Big Stupid” of right now is still the same thing. (We’ve not corrected enough). Between now and transformative AGI we are likely to encounter a moment where 40% of people realize AIs really could take over (say if every month another 1% of the population loses their job). If 40% of the world were as scared of AI loss-of-control as you, what could the world do? I think a lot! Do we have a plan for then?

  Almost every LessWrong post on AIs are about analyzing AIs. Almost none are about how, given widespread public support, people/​governments could stop bad AIs from being built.

  [Example: if 40% of people were as worried about AI as I was, the US would treat GPU manufacture like uranium enrichment. And fortunately GPU manufacture is hundreds of time harder than uranium enrichment! We should be nerding out researching integrated circuit supply chains, choke points, foundry logistics in jurisdictions the US can’t unilaterally sanction, that sort of thing.]

  TLDR, stopping deadly AIs from being built needs less research on AIs and more research on how to stop AIs from being built.
  
  *My research included 😬

• robo 16 May 2024 14:43 UTC

  3 points

  0

  in reply to: cousin_it’s comment on: Why you should learn a mu­si­cal instrument

  How did training aural imagination go for you, 15 years later?

• robo 15 May 2024 7:28 UTC

  4 points

  −3

  in reply to: Alex_Altair’s comment on: Ilya Sutskever and Jan Leike re­sign from OpenAI

  My comment here is not cosmically important and I may delete it if it derails the conversation.
  
  There are times when I would really want a friend to tap me on the shoulder and say “hey, from the outside the way you talk about <X> seems way worse than normal. Are you hungry/​tired/​too emotionally close?”. They may be wrong, but often they’re right.
  If you (general reader you) would deeply want someone to tap you on the shoulder, read on, otherwise this comment isn’t for you.

  If you burn at NYT/​Cade Metz intolerable hostile garbage, are you have not taken into account how defensive tribal instincts can cloud judgements, then, um <tap tap>?

• robo 10 May 2024 22:57 UTC

  7 points

  1

  in reply to: Daniel Kokotajlo’s comment on: simeon_c’s Shortform

  I appreciate that you are not speaking loudly if you don’t yet have anything loud to say.

• robo 10 May 2024 18:36 UTC

  33 points

  0

  in reply to: Daniel Kokotajlo’s comment on: simeon_c’s Shortform

  Is that your family’s net worth is $100 and you gave up $85? Or your family’s net worth is $15 and you gave up $85?
  
  Either way, hats off!

• robo 21 Apr 2024 7:26 UTC

  3 points

  0

  in reply to: niplav’s comment on: ni­plav’s Shortform

  How close would this rank a program p with a universal Turing machine simulating p? My sense is not very close because the “same” computation steps on each program don’t align.

  My “most naïve formula for logical correlation” would be something like put a probability distribution on binary string inputs, treat $p_1$ and $p_2$ as random variables $\{0,1{\}}^{\ast }\to \{0,1{\}}^{\ast }\cup \left\{\perp \right\}$, and compute their mutual information.

• robo 14 Apr 2024 17:02 UTC

  1 point

  0

  in reply to: Tamsin Leake’s comment on: carado’s Shortform

  Interesting idea.
  I don’t think using a classical Turing machine in this way would be the right prior for the multiverse. Classical Turing machines are a way for ape brains to think about computation using the circuitry we have available (“imagine other apes following these social contentions about marking long tapes of paper”). They aren’t the cosmically simplest form of computation. For example, the (microscopic non-course-grained) laws of physics are deeply time reversible, where Turing machines are not.
  I suspect this computation speed prior would lead to Boltzmann-brain problems. Your brain at this moment might be computed at high fidelity, but everything else in the universe would be approximated for the computational speed-up.

• robo 8 Apr 2024 13:03 UTC

  4 points

  4

  on: A Dozen Ways to Get More Dakka

  Counterpoint worth considering:

  It’s hard to get enough of something that almost works.

  (Vincent Felitti, as quoted from In the Realm of Hungry Ghosts: Close Encounters with Addiction)

• robo 27 Mar 2024 9:52 UTC

  2 points

  1

  on: robo’s Shortform

  “When you talk about the New York Times, rational thought does not appear to be holding the mic”

  --Me, mentally, to many people in the rationalist/​tech sphere since Scott took SlateStarCodex down.

robo’s Shortform

• robo 5 Mar 2024 13:51 UTC

  1 point

  0

  in reply to: Ape in the coat’s comment on: The Solu­tion to Sleep­ing Beauty

  Right, I read all that. I still don’t understand what it means to append two things to the list.

  Here’s how I understand modelLewis, modelElga, etc.

  “This model represent the world as a probability distribution. To get a more concrete sense of the world model, here’s a function which generates a sample from that probability distribution”

  Here’s how I understand your model.

  “This model represents the world as a ????, which like a probability distribution but different. To get a concrete sense of the world model, here’s a function which generates a sample from that probability distribution JUST KIDDING here’s TWO samples”.

  Why can you generate two samples at once? What does that even mean?? The world model isn’t quite just a stationary probability distribution, fine, what is it then? Your model isn’t structured like other models, fine, but how is it structured? I’m drowning in type errors.

  EDIT and I’m suggesting be really concrete, if you can, if that will help. Like come up with some concrete situation where Beauty makes a bet, or says a thing, (“Beauty woke up on Monday and said ‘I think there’s a 50% chance the coin came up on heads, and refuse to say there’s a state of affairs about what day it presently is’”) and explain what in her model made her make that bet or say that thing. Or maybe draw a picture which what her brain looks like under that circumstance compared to other circumstances.

• robo 5 Mar 2024 11:40 UTC

  1 point

  0

  in reply to: Ape in the coat’s comment on: The Solu­tion to Sleep­ing Beauty

  You don’t have to reply, but FYI I don’t understand what ListC represents (a total ordering of events defined by a logical clock? A logical clock ordering Beauty’s thoughts, or a logical clock ordering what causally can affect what, or logically affect what allowing for Newcomb-like situations? Why is there a clock at all?), how ListC is used, what concatenating multiple entries to ListC means in terms of beliefs, etc. If it’s important for readers to understand this you might have to step us through (or point us to an earlier article where you stepped us through).

• robo 4 Mar 2024 15:21 UTC

  3 points

  1

  on: The Solu­tion to Sleep­ing Beauty

  I don’t understand what return ['Tails&Monday','Tails&Tuesday'] and ListC += outcome mean. Can you explain it more? Perhaps operationalize it into some specific way Sleeping Beauty should act in some situation?

  For example, if Sleeping Beauty is allowed to make bets every time she is woken up, I claim she should bet as though she believes the coin came up with probability ¹⁄₃ for Heads and ²⁄₃ for Tails (≈because over many iterations she’ll find herself betting in situations where Tails is true twice as often as in situations where Heads is true).
  
  I don’t understand what your solution means for Sleeping Beauty. The best operationalization I can think of for “Ω={Heads&Monday, Tails&Monday&Tuesday}” is something like:

  “Thanks for participating in the experiment Ms. Beauty, and just FYI it’s Tuesday”

  “𝕬𝖓𝖉 𝕸𝖔𝖓𝖉𝖆𝖞”

  “No, it’s just Tuesday”

  “𝕬𝖓𝖉 𝕸𝖔𝖓𝖉𝖆𝖞 𝖆𝖘 𝖜𝖊𝖑𝖑. 𝕿𝖍𝖊𝖗𝖊 𝖈𝖆𝖓 𝖇𝖊 𝖓𝖔 𝕿𝖚𝖊𝖘𝖉𝖆𝖞 𝖜𝖎𝖙𝖍𝖔𝖚𝖙 𝖆 𝕸𝖔𝖓𝖉𝖆𝖞”

  “Yep, it was Monday yesterday. Now it’s Tuesday”

  “𝕭𝖚𝖙 𝖎𝖙 𝖎𝖘 𝖆𝖑𝖘𝖔 𝕸𝖔𝖓𝖉𝖆𝖞, 𝖋𝖔𝖗 𝕴 𝖊𝖝𝖎𝖘𝖙 𝖆𝖘 𝖔𝖓𝖊 𝖎𝖓 𝖆𝖑𝖑 𝖕𝖔𝖎𝖓𝖙𝖘 𝖔𝖋 𝖙𝖎𝖒𝖊 𝖆𝖓𝖉 𝖘𝖕𝖆𝖈𝖊”

  “Wow! That explains the unearthly blue glow!”

  “𝕴𝖓𝖉𝖊𝖊𝖉”

• robo 25 Jan 2024 12:07 UTC

  2 points

  0

  on: Will quan­tum ran­dom­ness af­fect the 2028 elec­tion?

  Daniel Kahneman notes that if sperm are randomized, the chance of Hitler, Stalin, and Mao all being born boys is ¹⁄₈. Re-run the 20th century with any of them being female and you get vastly different results. That thought experiment makes me suspect our intuitions for the inevitability of history are faulty.