Hey Midius!
My recommended rationality habit is noticing confusion, by which I mean a specific mental feeling that’s usually quick & subtle & easy to ignore.
David Chapman has a more wooey version called Eating Your Shadow, which was very helpful for me since it pointed me towards acknowledging parts of my experience that I was denying due to identity & social reasons (hence the easy to ignore part).
Could you go into more details into what skills these advisers would have or what situations to navigate?
Because I’m baking in the “superhuman in coding/maths” due to the structure of those tasks, and other tasks can either be improved through:
1. general capabilies
2. Specific task
And there might be ways to differentially accelarate that capability.
I really appreciate your post and all the links! This and your other recent posts/comments have really helped make a clearer model of timelines.
In my experience, most of the general public will verbally agree that AI X-risk is a big deal, but then go about their day (cause reasonably, they have no power). There’s no obvious social role/action to do in response to that.
For climate, people understand that they should recycle, not keep the water running, and if there’s a way to donate to clean the ocean on a Mr. Beast video, then some will even donate (sadly, none of these are very effective for solving the climate problem though! Gotta avoid that for our case).
Having a clear call-to-action seems relevant. For example, educating the public about AI taking jobs for the purpose of building support for UBI. It’s then clear what to communicate and the call-to-action.
I’d be curious to hear what you think an ask should be?
Alternatively, you could argue that generally informing folks on a wide scale about the risks involved will then allow general public to do what they believe is locally best. This could involve a documentary or realistic movie.
Claude 3.5 seems to understand the spirit of the law when pursuing a goal X.
A concern I have is that future training procedures will incentivize more consequential reasoning (because those get higher reward). This might be obvious or foreseeable, but could be missed/ignored under racing pressure or when lab’s LLMs are implementing all the details of research.
Thanks!
I forgot about faithful CoT and definitely think that should be a “Step 0”. I’m also concerned here that AGI labs just don’t do the reasonable things (ie training for briefness making the CoT more steganographic).
For Mech-interp, ya, we’re currently bottlenecked by:
Finding a good enough unit-of-computation (which would enable most of the higher-guarantee research)
Computing Attention_in--> Attention_out (which Keith got the QK-circuit → Attention pattern working a while ago, but haven’t hooked up w/ the OV-circuit)
This is mostly a “reeling from o3”-post. If anyone is doom/anxiety-reading these posts, well, I’ve been doing that too! At least, we’re in this together:)
From an apparent author on reddit:
[Frontier Math is composed of] 25% T1 = IMO/undergrad style problems, 50% T2 = grad/qualifying exam style porblems, 25% T3 = early researcher problems
The comment was responding to a claim that Terence Tao said he could only solve a small percentage of questions, but Terence was only sent the T3 questions.
I also have a couple friends that require serious thinking (or being on my toes). I think it’s because they have some model of how something works, and I say something, showing my lack of this model.
Additionally, programming causes this as well (in response to compilation errors, getting nonsense outputs, or runs too long).
Was looking up Google Trend lines for chatGPT and noticed a cyclical pattern:
Where the dips are weekends, meaning it’s mostly used by people in the workweek. I mostly expect this is students using it for homework. This is substantiated by two other trends:
1. Dips in interest over winter and summer breaks (And Thanksgiving break in above chart)
2. “Humanize AI” which is
Humanize AI™ is your go-to platform for seamlessly converting AI-generated text into authentic, undetectable, human-like content
[Although note that overall interest in ChatGPT is WAY higher than Humanize AI]
I was expecting this to include the output of MIRI for this year. Digging into your links we have:
Two Technical Governance Papers:
1. Mechanisms to verify international agreements about AI development
2. What AI evals for preventing catastrophic risks can and cannot do
Four Media pieces of Eliezer regarding AI risk:
1. Semafor piece
2. 1 hr talk w/ Panel
3. PBS news hour
4. 4 hr video w/ Stephen Wolfram
Is this the full output for the year, or are there less linkable outputs such as engaging w/ policymakers on AI risks?
Donated $100.
It was mostly due to LW2 that I decided to work on AI safety, actually, so thanks!
I’ve had the pleasure of interacting w/ the LW team quite a bit and they definitely embody the spirit of actually trying. Best of luck to y’all’s endeavors!
I tried a similar experiment w/ Claude 3.5 Sonnet, where I asked it to come up w/ a secret word and in branching paths:
1. Asked directly for the word
2. Played 20 questions, and then guessed the word
In order to see if it does have a consistent it can refer back to.
Branch 1:
Branch 2:
Which I just thought was funny.
Asking again, telling it about the experiment and how it’s important for it to try to give consistent answers, it initially said “telescope” and then gave hints towards a paperclip.
Interesting to see when it flips it answers, though it’s a simple setup to repeatedly ask it’s answer every time.
Also could be confounded by temperature.
It’d be important to cache the karma of all users > 1000 atm, in order to credibly signal you know which generals were part of the nuking/nuked side. Would anyone be willing to do that in the next 2 & 1⁄2 hours? (ie the earliest we could be nuked)
We could instead pre-commit to not engage with any nuker’s future posts/comments (and at worse comment to encourage others to not engage) until end-of-year.
Or only include nit-picking comments.
A trending youtube video w/ 500k views in a day brings up Dario Amodei’s Machines of Loving Grace (Timestamp for the quote):
[Note: I had Claude help format, but personally verified the text’s faithfulness]
Other notes: This is youtuber/Streamer DougDoug (2.8M subscribers), with this video posted on his other channel DougDougDoug (“DougDoug content that’s too rotten for the main channel”) who often streams/posts coding/AI integrated content.
The full video is also an entertaining summary of case law on AI generated art/text copyright.