davekasten

• davekastenOct 7, 2024, 2:20 PM

  3 points

  0

  in reply to: Nathan Helm-Burger’s comment on: A Nar­row Path: a plan to deal with AI ex­tinc­tion risk

  Looking forward to it! (Should rules permit, we’re also happy to discuss privately at an earlier date)

A Nar­row Path: a plan to deal with AI ex­tinc­tion risk

• davekastenOct 7, 2024, 3:46 AM

  9 points

  8

  on: dav­ekas­ten’s Shortform

  Has anyone thought about the things that governments are uniquely good at when it comes to evaluating models?
  
  Here are at least 3 things I think they have as benefits:
  1. Just an independent 3rd-party perspective generally

  2. The ability to draw insights across multiple labs’ efforts, and identify patterns that others might not be able to

  3. The ability to draw on classified threat intelligence to inform its research (e.g., Country X is using model Y for bad behavior Z) and to test the model for classified capabilities (bright line example: “can you design an accurate classified nuclear explosive lensing arrangement”).

  Are there others that come to mind?

• davekastenSep 29, 2024, 4:01 PM

  5 points

  4

  in reply to: Ruby’s comment on: Ruby’s Short­form Feed

  I think this can be true, but I don’t think it needs to be true:

  “I expect that a lot of regulation about what you can and can’t do stops being enforceable once the development is happening in the context of the government performing it.”

  I suspect that if the government is running the at-all-costs-top-national-priority Project, you will see some regulations stop being enforceable. However, we also live in a world where you can easily find many instances of government officials complaining in their memoirs that laws and regulations prevented them from being able to go as fast or as freely as they’d want on top-priority national security issues. (For example, DoD officials even after 9-11 famously complained that “the lawyers” restricted them too much on top-priority counterterrorism stuff.)
  

• davekastenSep 26, 2024, 6:32 PM

  4 points

  0

  on: [Com­pleted] The 2024 Petrov Day Scenario

  Gentlemen, it’s been a pleasure playing with you tonight

• davekastenSep 23, 2024, 7:14 PM

  1 point

  0

  in reply to: davekasten’s comment on: dav­ekas­ten’s Shortform

  I suspect this won’t get published until November at the earliest, but I am already delightfully pleased with this bit:

  
  Canada geese fly overhead, honking. Your inner northeast Ohioan notices that you are confused; it’s the wrong season for them to migrate this far south, and they’re flying westwards, anyways.

  A quick Google discovers that some Canada geese have now established themselves non-migratorily in the Bay Area:

  “The Migratory Bird Treaty Act of 1918 banned hunting or the taking of eggs without a permit. These protections, combined with an increase in desirable real estate—parks, golf course and the like—spurred a dramatic turnaround for the species. Canada geese began breeding in the Bay Area—the southern end of their range – in the late 1950s.”

  You nod, approvingly; this clearly is another part of the East Bay’s well-known, long-term philanthropic commitment to mitigating Acher-Risks.

• davekastenSep 23, 2024, 5:00 AM

  2 points

  0

  on: dav­ekas­ten’s Shortform

  Preregistering intent to write “Every Bay Area Walled Compound” (hat tip: Emma Liddell)
  
  Unrelatedly, I am in Berkeley through Wednesday afternoon, let me know if you’re around and would like to chat

• davekastenSep 17, 2024, 11:18 PM

  12 points

  5

  in reply to: Ben Pace’s comment on: Akash’s Shortform

  Generally, it is difficult to understate how completely the PRC is seen as a bad-faith actor in DC these days. Many folks saw them engage in mass economic espionage for a decade while repeatedly promising to stop; those folks are now more senior in their careers than those formative moments. Then COVID happened, and while not everyone believes in the lab leak hypothesis, basically everyone believes that the PRC sure as heck reflexively covered up whether or not they were actually culpable.
  
  (Edit: to be clear, reporting, not endorsing, these claims)

• davekastenSep 13, 2024, 2:57 AM

  7 points

  6

  in reply to: evhub’s comment on: TurnTrout’s short­form feed

  Basic question because I haven’t thought about this deeply: in national security stuff, we often intentionally elide the difference between capabilities and intentions. The logic is: you can’t assume a capability won’t be used, so you should plan as-if it is intended to be used.
  
  Should we adopt such a rule for AGI with regards to policy decision-making? (My guess is...probably not for threat assessment but probably yes for contingency planning?)

• davekastenSep 9, 2024, 1:02 AM

  2 points

  2

  in reply to: Ben Pace’s comment on: Zach Stein-Per­l­man’s Shortform

  I think, having been raised in a series of very debate- and seminar-centric discussion cultures, that a quick-hit question like that is indeed contributing something of substance. I think it’s fair that folks disagree, and I think it’s also fair that people signal (e.g., with karma) that they think “hey man, let’s go a little less Socratic in our inquiry mode here.”
  
  But, put in more rationalist-centric terms, sometimes the most useful Bayesian update you can offer someone else is, “I do not think everyone is having the same reaction to your argument that you expected.” (Also true for others doing that to me!)
  
  (Edit to add two words to avoid ambiguity in meaning of my last sentence)

• davekastenSep 8, 2024, 11:11 PM

  −2 points

  1

  in reply to: Ben Pace’s comment on: Zach Stein-Per­l­man’s Shortform

  Yes, I would agree that if I expected a short take to have this degree of attention, I would probably have written a longer comment.
  
  Well, no, I take that back. I probably wouldn’t have written anything at all. To some, that might be a feature; to me, that’s a bug.
  

• davekastenSep 8, 2024, 10:04 PM

  3 points

  0

  in reply to: Ben Pace’s comment on: Zach Stein-Per­l­man’s Shortform

  It is genuinely a sign that we are all very bad at predicting others’ minds that it didn’t occur to me that if I said effectively “OP asked for ‘takes’, here’s a take on why I think this is pragmatically a bad idea” would also mean that I was saying “and therefore there is no other good question here”. That’s, as the meme goes, a whole different sentence.

• davekastenSep 8, 2024, 5:06 PM

  2 points

  2

  in reply to: Orpheus16’s comment on: Zach Stein-Per­l­man’s Shortform

  I think it’s bad for discourse for us to pretend that discourse doesn’t have impacts on others in a democratic society. And I think the meta-censoring of discourse by claiming that certain questions might have implicit censorship impacts is one of the most anti-rationality trends in the rationalist sphere.
  
  I recognize most users of this platform will likely disagree, and predict negative agreement-karma on this post.

• davekastenSep 8, 2024, 5:01 PM

  −5 points

  0

  in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Per­l­man’s Shortform

  Ok, then to ask it again in your preferred question format: is this where we think our getting-potential-employees-of-Anthropic-to-consider-the-value-of-working-on-safety-at-Anthropic points are best spent?

• davekastenSep 6, 2024, 10:18 PM

  0 points

  −10

  in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Per­l­man’s Shortform

  Is this where we think our pressuring-Anthropic points are best spent ?

• davekastenSep 4, 2024, 2:52 AM

  3 points

  2

  in reply to: habryka’s comment on: Habryka’s Short­form Feed

  I personally endorse this as an example of us being a community that Has The Will To Try To Build Nice Things.

• davekastenSep 4, 2024, 2:52 AM

  1 point

  −8

  in reply to: Zach Stein-Perlman’s comment on: The Check­list: What Suc­ceed­ing at AI Safety Will In­volve

  To say the obvious thing: I think if Anthropic isn’t able to make at least somewhat-roughly-meaningful predictions about AI welfare, then their core current public research agendas have failed?

• davekastenSep 2, 2024, 6:01 PM

  1 point

  2

  in reply to: Buck’s comment on: Buck’s Shortform

  Fair enough!

• davekastenSep 2, 2024, 3:23 PM

  1 point

  0

  AF

  in reply to: Buck’s comment on: Buck’s Shortform

  Possibly misguided question given the context—I see you incorporating imperfect information in “the attack fails silently”, why not also a distinction between “the attack succeeds noisily, the AI wins and we know it won” and “the attack succeeds silently, the AI wins and we don’t know it won” ?

• davekastenAug 31, 2024, 6:43 PM

  3 points

  0

  on: Ver­ifi­ca­tion meth­ods for in­ter­na­tional AI agreements

  I would suggest that the set of means available to nation-states to unilaterally surveil another nation state is far more expansive than the list you have. For example, the good-old-fashioned “Paying two hundred and eighty-two thousand dollars in a Grand Cayman banking account to a Chinese bureaucrat”* appears nowhere in your list.
  
  
  *If you get that this is a reference to the movie Spy Game, you are cool. If you don’t, go watch Spy Game. It has a worldview on power that is extremely relevant to rationalists.