Jordan Taylor

• Jordan TaylorOct 5, 2023, 12:15 PM

  3 points

  0

  in reply to: Alex K. Chen (parrot)’s comment on: Graph­i­cal ten­sor no­ta­tion for interpretability

  This is an interesting and useful overview, though it’s important not to confuse their notation with the Penrose graphical notation I use in this post, since lines in their notation seem to represent the message-passing contributions to a vector, rather than the indices of a tensor.

  That said, there are connections between tensor network contractions and message passing algorithms like Belief Propagation, which I haven’t taken the time to really understand. Some references are:

  Duality of graphical models and tensor networks—Elina Robeva and Anna Seigal
  Tensor network contraction and the belief propagation algorithm—R. Alkabetz and I. Arad
  Tensor Network Message Passing—Yijia Wang, Yuwen Ebony Zhang, Feng Pan, Pan Zhang
  Gauging tensor networks with belief propagation—Joseph Tindall, Matt Fishman

Graph­i­cal ten­sor no­ta­tion for interpretability

• Jordan TaylorAug 26, 2023, 11:37 PM

  1 point

  0

  in reply to: Tom DAVID’s comment on: A list of core AI safety prob­lems and how I hope to solve them

  I guess the shutdown timer would be most important in the training stage, so that it (hopefully) learns only to care about the short term.

• Jordan TaylorMay 31, 2023, 1:59 PM

  3 points

  0

  on: An­nounc­ing Apollo Research

  Seems great! I’m excited about potential interpretability methods for detecting deception.
  
  I think you’re right about the current trade-offs on the gain of function stuff, but it’s good to think ahead and have precommitments for the conditions under which your strategies there should change.
  
  It may be hard to find evals for deception which are sufficiently convincing when they trigger, yet still give us enough time to react afterwards. A few more similar points here: https://​​www.lesswrong.com/​​posts/​​pckLdSgYWJ38NBFf8/​​?commentId=8qSAaFJXcmNhtC8am
  
  Building good tools for detecting deceptive alignment seems robustly good though, even after you reach a point where you have to drop the gain of function stuff.

• Jordan TaylorMar 15, 2023, 2:25 AM

  13 points

  7

  in reply to: paulfchristiano’s comment on: GPT-4

  Potential dangers of future evaluations /​ gain-of-function research, which I’m sure you and Beth are already extremely well aware of:

  1. Falsely evaluating a model as safe (obviously)

  2. Choosing evaluation metrics which don’t give us enough time to react (After evaluation metrics switch would from “safe” to “not safe”, we should like to have enough time to recognize this and do something about it before we’re all dead)

  3. Crying wolf too many times, making it more likely that no one will believe you when a danger threshold has really been crossed

  4. Letting your methods for making future AIs scarier be too strong given the probability they will be leaked or otherwise made widely accessible. (If the methods /​ tools are difficult to replicate without resources)

  5. Letting your methods for making AIs scarier be too weak, lest it’s too easy for some bad actors to go much further than you did

  6. Failing to have a precommitment to stop this research when models are getting scary enough that it’s on balance best to stop making them scarier, even if no-one else believes you yet
     

  What links here?

  • Jordan Taylor's comment on An­nounc­ing Apollo Research by Marius Hobbhahn (May 31, 2023, 1:59 PM; 3 points)

• Jordan TaylorMar 7, 2023, 6:26 AM

  15 points

  1

  in reply to: Vladimir_Nesov’s comment on: Are we too con­fi­dent about un­al­igned AGI kil­ling off hu­man­ity?

  unless that’s an objective

  I think this is too all-or-nothing about the objectives of the AI system. Following ideas like shard theory, objectives are likely to come in degrees, be numerous and contextually activated, having been messily created by gradient descent.
  
  Because “humans” are probably everywhere in its training data, and because of naiive safety efforts like RLHF, I expect AGI to have a lot of complicated pseudo-objectives /​ shards relating to humans. These objectives may not be good—and if they are they probably won’t constitute alignment, but I wouldn’t be surprised if it were enough to make it do something more complicated than simply eliminating us for instrumental reasons.
  
  Of course the AI might undergo a reflection process leading to a coherent utility function when it self-improves, but I expect it to be a fairly complicated one, assigning some sort of valence to humans. We might also have some time before it does that, or be able to guide this values-handshake between shards collaboratively.

• Jordan TaylorDec 20, 2022, 3:15 AM

  8 points

  1

  on: In­ner and outer al­ign­ment de­com­pose one hard prob­lem into two ex­tremely hard problems

  I just wanted to say thanks for writing this. It is important, interesting, and helping to shape and clarify my views.

  I would love to hear a training story where a good outcome for humanity is plausibly achieved using these ideas. I guess it’d rely heavily on interpretability to verify what shards /​ values are being formed early in training, and regular changes to the training scenario and reward function to change them before the agent is capable enough to subvert attempts to be changed.
  
  Edit: I forgot you also wrote A shot at the diamond-alignment problem, which is basically this. Though it only assumes simple training techniques (no advanced interpretability) to solve a simpler problem.

• Jordan TaylorOct 18, 2022, 2:45 AM

  1 point

  0

  on: A re­view of the Bio-An­chors report

  One small thing: When you first use the word “power”, I thought you were talking about energy use rather than computational power. Although you clarify in “A closer look at the NN anchor”, I would get the wrong impression if I just read the hypotheses:

  … TAI will run on an amount of power comparable to the human brain …

  … neural network which would use that much power …

  Maybe change “power” to “computational power” there? I expect biological systems to be much more strongly selected to minimize energy use than TAI systems would be, but the same is not true for computational power.