Sebastian Schmidt

• Sebastian Schmidt 6 Sep 2024 15:43 UTC

  1 point

  0

  in reply to: KristianRonn’s comment on: The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

  Interesting. I wasn’t aware of the specific equations, but at a first glance, that does seem like a reasonable argument—thanks!

  Also, to follow up on my second comment (sorry, the formatting was a bit confusing but I just edited it): I think it would’ve been valuable for the blog post to end with a more concrete “bridge” to the solutions as the current version is fairly generic.

• Sebastian Schmidt 6 Sep 2024 8:49 UTC

  2 points

  1

  on: The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

  As mentioned in a previous comment, it seems that many of the concepts and claims are fairly well-established in these fora already (though I appreciated the concrete examples of the Darwinian demons and Darwinians angels throughout history and some of the exhibits provided in favor of the Fragility of Life Hypotheses).

  What do you think are the best arguments against the Fragility of Life Hypotheses?

  we have the chance to shape our future with rationality, empathy, and foresight—a potential savior not just for ourselves, but for all life.

  This struck me as a bit vague, but I’m guessing that you’ll be much more specific about potential solutions in your upcoming blog post.

• Sebastian Schmidt 31 Aug 2024 10:39 UTC

  2 points

  0

  in reply to: Akash’s comment on: Would catch­ing your AIs try­ing to es­cape con­vince AI de­vel­op­ers to slow down or un­de­ploy?

  Could you share more about this:

  “my own experiences suggest that many policymakers appear to be very interested in better understanding loss of control scenarios.”

  For example, what proportion of people you’ve interacted with gave you this impression and the context in which it happened (I understand that you’d have to keep details sparse for obvious reasons).

• Sebastian Schmidt 27 Aug 2024 7:53 UTC

  5 points

  3

  on: Dar­wi­nian Traps and Ex­is­ten­tial Risks

  Congrats on publishing the book. It seems that many of the concepts you discuss seem well-known (especially within EA and LW). However, am I right to say that the novelty/​contribution of the work lies in the following (caveat—obviously, this is a shallow summary):
  1. Making multipolar traps, stemming from the evolutionary optimization process, the central concept (instead of doing the most good or becoming less wrong).
  2. Outlining how it’s the root cause of key priorities such as existential risks (and perhaps things such as the Fermi paradox).
  3. Describing how multipolar traps can be avoided via global coordination in the form of value chains and related mechanisms. If so, this might be the most novel part of the book.
  
  

• Sebastian Schmidt 21 Jul 2024 10:04 UTC

  4 points

  3

  on: Towards more co­op­er­a­tive AI safety strategies

  I think I disagree with some of the claims in this post and I’m mostly sympathetic with the points Akash raised in his comments. Relatedly, I’d like to see a more rigorous comparison between the AI safety community (especially EA/​Rationality parts) and relevant reference class movements such as the climate change community.
  
  That said, I think it’s reasonable to have a high prior on people ending up aiming for inappropriate levels of power-seeking when taking ambitious actions in the world so it’s important to keep these things in mind.
  
  In addition to your two “recommendations” of focusing on legitimacy and competence, I’d add two additional candidates:
  1. Being careful about what de facto role models or spokespeople the AI safety community “selects”. It seems crucial to avoid another SBF.
  2. Enabling currently underrepresented perspectives to contribute in well-informed, competent ways.

• Sebastian Schmidt 10 Jun 2024 8:45 UTC

  3 points

  0

  in reply to: Akash’s comment on: Re­sponse to Aschen­bren­ner’s “Si­tu­a­tional Aware­ness”

  I agree that international coordination seems very important.
  I’m currently unsure about how to best think about this. One way is to focus on bilateral coordination between the US and China, as it seems that they’re the only actors who can realistically build AGI in the coming decade(s).
  
  Another way, is to attempt to do something more inclusive by also focusing on actors such as UK, EU, India, etc.
  
  An ambitious proposal is the Multinational AGI Consortium (MAGIC). It clearly misses many important components and considerations, but I appreciate the intention and underlying ambition.

• Sebastian Schmidt 28 May 2024 8:42 UTC

  3 points

  2

  on: AI com­pa­nies aren’t re­ally us­ing ex­ter­nal evaluators

  Thanks for this—I wasn’t aware. This also makes me more disappointed with the voluntary commitments at the recent AI Safety Summit. As far as I can tell (based on your blog post), the language used around external evaluations was also quite soft:
  ”They should also consider results from internal and external evaluations as appropriate, such as by independent third-party evaluators, their home governments^[3], and other bodies their governments deem appropriate.”
  
  I wonder if it’d be possible to have much stronger language around this at next year’s Summit.
  
  The EU AI Act, is the only regulatory rules that isn’t voluntary (i.e., companies have to follow them) and even that doesn’t require external evaluations (they use the wording Conformity Assessment) for all high-risk systems.
  
  Any takes on what our best bet is for making high-quality external evals mandatory for frontier models at this stage?

• Sebastian Schmidt 25 May 2024 6:40 UTC

  1 point

  0

  in reply to: habryka’s comment on: Miti­gat­ing ex­treme AI risks amid rapid progress [Linkpost]

  This is a valid concern, but I’m fairly certain Science (the journal—not the field) handled this well. Largely because they’re somewhat incentivized to do so (otherwise it could have very bad optics for them) and must have experienced this several times before. I also happen to know one of the senior authors who is significantly above average in conscientiousness.

• Sebastian Schmidt 23 May 2024 6:30 UTC

  4 points

  0

  in reply to: sanxiyn’s comment on: New vol­un­tary com­mit­ments (AI Seoul Sum­mit)

  Good point. Just checked—here’s the ranking of Chinese LLMs among the top 20:
  7. Yi-Large (01 AI)
  12. Qwen-Max-0428 (Alibaba)
  15. GLM-4-0116 (Zhipu AI)
  16. Qwen1.5-110B-Chat (Alibaba)
  19. Qwen1.5-72B-chat (Alibaba)
  
  Of the three, only Zhipu signed the agreement.

• Sebastian Schmidt 22 May 2024 6:52 UTC

  2 points

  0

  on: New vol­un­tary com­mit­ments (AI Seoul Sum­mit)

  Thanks for this analysis—I found it quite helpful and mostly agree with your conclusions.
  Looking at the list of companies who signed the commitments, I wonder why Baidu isn’t on there. Seems very important to have China be part of the discussion, which, AFAIK, mainly means Baidu.

• Sebastian Schmidt 2 May 2024 12:00 UTC

  3 points

  0

  on: Launch­ing ap­pli­ca­tions for AI Safety Ca­reers Course In­dia 2024

  I am honored to be part of enabling more people from around the world to contribute to the safe and responsible development of AI.

Re­lease of UN’s draft re­lated to the gov­er­nance of AI (a sum­mary of the Si­mon In­sti­tute’s re­sponse)

• Sebastian Schmidt 1 Jan 2024 18:02 UTC

  2 points

  0

  on: In­tro­duc­ing The ‘From Big Ideas To Real-World Re­sults’: A Series for Effec­tive Per­sonal Development

  Thanks for posting.
  
  The hindrances you mention seem relevant but also highly related. In particular, the
  ”Poorly-tested implementation: A disconnection between engaging educational content and its practical application, underscoring the importance of field-tested guidance that works in varied real-life situations.”
  
  I’m excited to see advice that has been tested via working closely with individuals. I’m reminded of the Scott Miller (who played a major role within evidence-based psychotherapy) claim on a podcast (paraphrasing): “We have an implementation problem”.

• Sebastian Schmidt 13 Nov 2023 18:00 UTC

  1 point

  0

  in reply to: DivineMango’s comment on: Men­tal Health and the Align­ment Prob­lem: A Com­pila­tion of Re­sources (up­dated April 2023)

  Great! I’d expect most people on there are. I know for sure that Paul Rohde and James Norris (the founder) are aware. My rates depends on the people I work with but $200-$300 is the standard rate.

• Sebastian Schmidt 8 May 2023 15:35 UTC

  2 points

  0

  on: Men­tal Health and the Align­ment Prob­lem: A Com­pila­tion of Re­sources (up­dated April 2023)

  Thank you. This is a really excellent post. I’d like to add a few resources and providers:
  1. EA mental health navigator: https://​​www.mentalhealthnavigator.co.uk/​​.
  2. Overview of providers on EA mental health navigator (not everyone familiar with alignment in significant ways). https://​​www.mentalhealthnavigator.co.uk/​​providers
  3. Upgradable has some providers that are quite informed around alignment. https://​​www.upgradable.org/​​
  4. If permissible, I’d like to add myself as a provider (coach) though I don’t take on any coachees at present.

• Sebastian Schmidt 7 Mar 2023 16:51 UTC

  12 points

  1

  on: The Sim­ple Truth

  So I need different names for the thingies that determine my predictions and the thingy that determines my experimental results. I call the former thingies ‘belief,’ and the latter thingy ’reality.

  Re-reading this after four years. It is still so elegantly beautiful. I don’t understand why this wasn’t part of sequences highlights. I would have put under “thinking better on purpose” or “the laws governing beliefs”.

• Sebastian Schmidt 7 Mar 2023 16:47 UTC

  1 point

  on: The Im­por­tance of Say­ing “Oops”

  Interesting that you seem to see rationality (as opposed to traditional rationality) as a more effective and efficient version of seeking the truth (~epistemic rationality). In that sense, it does seem somewhat similar to doing what EA is trying to do for altruism.