Lionel Levine

• Lionel LevineFeb 20, 2025, 3:03 PM

  2 points

  0

  on: A com­pu­ta­tional no-co­in­ci­dence principle

  Suppose I hand you a circuit C that I sampled uniformly at random from the set of all depth-n reversible circuits satisfying P. What is a reason to believe that there exists a short heuristic argument for the fact that this particular C satisfies P?

• Lionel LevineJan 21, 2025, 7:05 PM

  27 points

  8

  on: Some les­sons from the OpenAI-Fron­tierMath debacle

  mathematicians who worked on FrontierMath would possibly have not contributed to this if they knew about the funding and exclusive access.

  I’m a mathematician who contributed to FrontierMath.

  Speaking just for myself: From the beginning it was clear reading between the lines that the project had an industry sponsor, with OAI being an obvious guess. I judged the project as having a less favorable safety/​capabilities tradeoff than my other research, to the point where I drafted, but did not ultimately send, an email bowing out. In hindsight, I think staying in was the right call: It was a good learning experience for me to work on something directly impactful, even if the sign of the impact is ambiguous.

  My main updates are:

  • Benchmarks are super impactful! My main research is on dispositional AI safety, so now I’m thinking about dispositional benchmarks.

  • Proximity to power is confusing! It’s good to start wrestling with that confusion now, while the stakes are still low.

• Lionel LevineAug 4, 2022, 10:14 PM

  3 points

  0

  on: «Boundaries», Part 1: a key miss­ing con­cept from util­ity theory

  So, boundaries enable cooperation, by protecting BATNA.

  Would you say there is a boundary between cell and mitochondria?

  In the limit of perfect cooperation, the BATNA becomes minus infinity and the boundary dissolves.

• Lionel LevineDec 7, 2021, 2:02 PM

  1 point

  on: Cor­nell Meetup

  Thanks all for responding! The meetup will be this Thursday, any other Ithaca locals DM for details!

[Question] Cor­nell Meetup

• Lionel LevineJul 16, 2021, 4:57 PM

  1 point

  on: Danger­ous op­ti­mi­sa­tion in­cludes var­i­ance minimisation

  This is an obvious point, but: Any goal is likely to include some variance minimization as a subgoal, if only because of the possibility that another entity (rival AI, nation state, company) with different goals could take over the world. If an AI has the means to take over the world, then it probably takes seriously the scenario that a rival takes over the world. Could it prevent that scenario without taking over itself?

• Lionel LevineJul 16, 2021, 4:35 PM

  1 point

  in reply to: Daniel Kokotajlo’s comment on: We need a stan­dard set of com­mu­nity ad­vice for how to fi­nan­cially pre­pare for AGI

  money is probably much less valuable after AGI than before, indeed practically worthless.

  I think this overstates the case against money. Humans will always value services provided by other humans, and these will still be scarce after AGI. Services provided by humans will grow in value (as measured by utility to humans) if AGI makes everything else cheap. It seems plausible that money (in some form) will still be the human-to-human medium of exchange, so it will still have value after AGI.

• Lionel LevineMay 26, 2021, 6:32 PM

  3 points

  on: De­cou­pling de­liber­a­tion from competition

  If Alice and Bob are talking to each other as they deliberate

  I think this is a typo, it should say “compete” instead of “deliberate”.

  I worry about persuasion becoming so powerful that it blocks deliberation: How can Alice know whether Bob (or his delegated AI) is deliberating in good faith or trying to manipulate her?

  In this scenario, small high-trust communities can still deliberate, but mutual mistrust prevents them from communicating their insights to the rest of the world.

• Lionel LevineMay 16, 2021, 7:51 PM

  2 points

  on: Agency in Con­way’s Game of Life

  I think this is possible and it doesn’t require AI. It only requires a certain kind of “infectious Turing machine” described below.

  Following Gwern’s comment, let’s consider first the easier problem of writing a program on a small portion of a Turing machine’s tape, which draws a large smiley face on the rest of the tape. This is easy even with the *worst case* initialization of the rest of the tape. Whereas our problem is not solvable in worst case, as pointed out by Richard_Kennaway.

  What makes our problem harder is errors caused by the random environment. We could model these errors by spontaneous random changes in the letters written on the Turing machine’s tape. But it’s easy to make a computation robust to such errors: A naive way to do it is to repeat each square 100 times and assign a repair bot to repeatedly scan the 100 squares, compute majority, and convert the minority squares back to the majority letter. This is not so different from how your laptop repeatedly scans its memory to prevent errors.

  Now we get to what I think is the hard part of the problem: How to organize a random environment into squares of Turing machine tape? I don’t know how to do it in Conway’s Life, but I would guess it’s possible. An interesting question is whether *any* cellular automaton that supports a Turing machine can also support an infectious Turing machine. Again I would guess yes.

  Does an infectious Turning machine require AI? There is certainly something lifelike about a device that gradually converts its environment into squares of its own tape. The tape squares reproduce, but they hardly need to be intelligent.

  But okay, we’re setting the bar low by competing against a *random* environment. As Oscar_Cunningham points out, a random environment is unlikely to produce anything smart. If we set the bar higher by asking the infection to compete against an environment containing life or intelligence, then it might need to be significantly smarter in order to take over the rest.