Esben Kran

• Esben KranJan 21, 2023, 2:08 PM

  3 points

  0

  in reply to: the gears to ascension’s comment on: Gen­er­al­iz­abil­ity & Hope for AI [MLAISU W03]

  Thank you for pointing this out! It seems I wasn’t informed enough about the context. I’ve dug a bit deeper and will update the text to:

  • Another piece reveals that OpenAI contracted Sama to use Kenyan workers with less than $2 /​ hour wage ($0.5 /​ hour average in Nairobi) for toxicity annotation for ChatGPT and undisclosed graphical models, with reports of employee trauma from the explicit and graphical annotation work, union breaking, and false hiring promises. A serious issue.

  For some more context, here is the Facebook whistleblower case (and ongoing court proceedings in Kenya with Facebook and Sama) and an earlier MIT Sloan report that doesn’t find super strong positive effects (but is written as such, interestingly enough). We’re talking pay gaps from relocation bonuses, forced night shifts, false hiring promises, supposedly human trafficking as well? Beyond textual annotation, they also seemed to work on graphical annotation.

Gen­er­al­iz­abil­ity & Hope for AI [MLAISU W03]

Ro­bust­ness & Evolu­tion [MLAISU W02]

AI im­prov­ing AI [MLAISU W01!]

• Esben KranJan 5, 2023, 2:14 PM

  2 points

  0

  on: The case against AI alignment

  I recommend reading Blueprint: The Evolutionary Origins of a Good Society about the science behind the 8 base human social drives where 7 are positive and the 8th is the outgroup hatred that you mention as fundamental. I have not read much up on the research on outgroup exclusion but I talked to an evolutionary cognitive psychologist who mentioned that this is receiving a lot of scientific scrutiny as a “basic drive” from evolution’s side.

  Axelrod’s The Evolution of Cooperation also finds that collaborative strategies work well in evolutionary prisoner’s dilemma game-theoretic simulations, though hard and immediate reciprocity for defection is also needed, which might lead to the outgroup hatred you mention.

• Esben KranJan 5, 2023, 2:00 PM

  −1 points

  0

  in reply to: andrew sauer’s comment on: The case against AI alignment

  An interesting solution here is radical voluntarism where an AI philosopher king runs the immersive reality where all humans are in and you can only be causally influenced upon if you want to. This means that you don’t need to do value alignment, just very precise goal alignment. I was originally introduced to this idea Carado.

Re­sults from the AI test­ing hackathon

• Esben KranDec 21, 2022, 4:01 AM

  1 point

  0

  in reply to: Magdalena Wache’s comment on: Will Machines Ever Rule the World? MLAISU W50

  The summary has been updated to yours for both the public newsletter and this LW linkpost. And yes, they seem exciting. Connecting FFS to interpretability was a way to contextualize it in this case, until you would provide more thoughts on the use case (given your last paragraph in the post). Thank you for writing, always appreciate the feedback!

• Esben KranDec 21, 2022, 3:54 AM

  1 point

  0

  in reply to: Noosphere89’s comment on: Will Machines Ever Rule the World? MLAISU W50

  I think we agree here. Those both seem like updates against scaling is all you need, i.e. (in this case) “data for DL in ANNs on GPUs is all you need”.

• Esben KranDec 18, 2022, 10:03 AM

  1 point

  0

  in reply to: Noosphere89’s comment on: Will Machines Ever Rule the World? MLAISU W50

  What might not be obvious from the post is that I definitely disagree with the “AGI near-impossible” as well, for the same reasons. These are the thoughts of GPU R&D engineers I talked with. However, the GPU performance increase limitation is a significant update on the ladder of assumptions towards “scaling is all you need” leading to AGI.

Will Machines Ever Rule the World? MLAISU W50

Join the AI Test­ing Hackathon this Friday

ML Safety at NeurIPS & Paradig­matic AI Safety? MLAISU W49

NeurIPS Safety & ChatGPT. MLAISU W48

• Esben KranNov 26, 2022, 6:31 PM

  5 points

  1

  on: The limited up­side of interpretability

  Thank you for this critique! They are always helpful to hone in on the truth.

  So as far as I understand your text, you argue that fine-grained interpretability loses out against “empiricism” (running the model) because of computational intractability.

  I generally disagree with this. beren points out many of the same critiques of this piece as I would come forth with. Additionally, the arguments seem too undefined, like there is not in-depth argumentation enough to support the points you make. Strong upvote for writing them out, though!

  You emphasize the Human Brain Project (HBP) quite a lot, even in the comments, as an example of a failed large-scale attempt to model a complex system. I think this characterization is correct but it does not seem to generalize beyond the project itself. It seems just as much like a project management and strategy problem as so much else. Benes’ comment is great for more reasoning into this and why ANNs seem significantly more tractable to study than the brain.

  Additionally, you argue that interpretability and ELK won’t succeed simply because of the intractability of fine-grained interpretability. I have two points against this view:

  1. Mechanistic interpretability have clearly already garnered quite a lot of interesting and novel insights into neural networks and causal understanding since the field’s inception 7 years ago.

  It seems premature to disregard the plausibility of the agenda itself, just as it is premature to disregard the project of neuroscience based on HBP. Now, arguing that it’s a matter of speed seems completely fine but this is another argument and isn’t emphasized in the text.

  2. Mechanistic interpretability does not seem to be working on fine-grained interpretability (?).

  Maybe it’s just my misunderstanding of what you mean by fine-grained interpretability, but we don’t need to figure out what neurons do, we literally design them. So the inspections happen at feature level, which is much more high-level than investigating individual neurons (sometimes these features seem represented in singular neurons of course). The circuits paradigm also generally looks at neural networks like systems neuroscience does, interpreting causal pathways in the models (of course with radical methodological differences because of the architectures and computation medium). The mechanistic interpretability project does not seem misguided by an idealization of neuron-level analysis and will probably adopt any new strategy that seems promising.

  For example work in this paradigm that seems promising, see interpretability in the wild, ROME, the superpositions exposition, the mathematical understanding of transformers and the results from the interpretability hackathon.

  For an introduction to features as the basic building blocks as compared to neurons, see Olah et al.’s work (2020).

  When it comes to your characterization of the “empirical” method, this seems fine but doesn’t conflict with interpretability. It seems you wish to make game theory-like understanding of the models or have them play in settings to investigate their faults? Do you want to do model distillation using circuits analyses or do you want AI to play within larger environments?

  I falter to understand the specific agenda from this that isn’t done by a lot of other projects already, e.g. AI psychology and building test environments for AI. I do see potential in expanding the work here but I see that for interpretability as well.

  Again, thank you for the post and I always like when people cite McElreath, though I don’t see his arguments apply as well to interpretability since we don’t model neural networks with linear regression at all. Not even scaling laws use such simplistic modeling, e.g. see Ethan’s work.

• Esben KranNov 24, 2022, 3:39 PM

  24 points

  12

  in reply to: Ben Pace’s comment on: Con­jec­ture: a ret­ro­spec­tive af­ter 8 months of work

  Agreed, the releases I’ve seen from Conjecture has made me incredibly hopeful of what they can achieve and interested in their approach to the problem more generally.

  When it comes to the coordination efforts, I’m generally of the opinion that we can speak the truth while being inviting and ease people into the arguments. From my chats with Conjecture, it seems like their method is not “come in, the water is fine” but “come in, here are the reasons why the water is boiling and might turn into lava”.

  If we start by speaking the truth “however costly it may be” and this leads to them being turned off by alignment, we have not actually introduced them to truth but have achieved the opposite. I’m left with a sense that their coordination efforts are quite promising and follow this line of thinking, though this might be a knowledge gap from my side (I know of 3-5 of their coordination efforts). I’m looking forward to seeing more posts about this work, though.

  As per the product side, the goal is not to be profitable fast, it is to be attractive to non-alignment investors (e.g. AirBnB is not profitable). I agree with the risks, of course. I can see something like Loom branching out (haha) as a valuable writing tool with medium risk. Conjecture’s corporate structure and core team alignment seems to be quite protective of a product branch being positive, though money always have unintended consequences.

  I have been a big fan of the “new agendas” agenda and I look forward to read about their unified research agenda! The candidness of their contribution to the alignment problem has also updated me positively on Conjecture and the organization-building startup costs seems invaluable and inevitable. Godspeed.

Re­sults from the in­ter­pretabil­ity hackathon

• Esben KranNov 10, 2022, 2:50 PM

  9 points

  1

  in reply to: dkirmani’s comment on: I Con­verted Book I of The Se­quences Into A Zoomer-Read­able Format

  Thank you for making this, I think it’s really great!

  The idea of the attention-grabbing video footage is that you’re not just competing between items on the screen, you’re also competing with the videos that come before and after your video. Therefore, yours has to be visually engaging just for that zoomer (et al.) dopamine rush.

  Subway Surfers is inherently pretty active and as you mention, the audio will help you here, though you might be able to find a better voice synthesizer that can be a bit more engaging (not sure if TikTok supplies this). So my counterpoint to Trevor1′s is that we probably want to keep something like Subway Surfers in the background but that can of course be many things such as animated AI generated images or NASA videos of the space race. Who really knows—experimentation is king.

• Esben KranOct 31, 2022, 3:47 PM

  1 point

  on: List of good AI safety pro­ject ideas?

  We have developed AI Safety Ideas which is a collaborative AI safety research platform with a lot of research project ideas.

[Book] In­ter­pretable Ma­chine Learn­ing: A Guide for Mak­ing Black Box Models Explainable