Software engineer transitioned into AI safety, teaching and strategy. Particularly interested in psychology, game theory, system design, economics.
Jonathan Claybrough
Karma: 528
(aside : I generally like your posts’ scope and clarity, mind saying how long it takes you to write something of this length?)
Self modeling is a really important skill, and you can measure how good you are at it by writing predictions about yourself. (Modelling A notably important one for people who have difficulty with motivation is predicting your own motivation—will you be motivated to do X in situation Y?
If you can answer that one generally, you can plan to actually anything you could theoretically do, using the following algorithm : from current situation A, to achieve wanted outcome Z, find a predecessor situation Y from which you’ll be motivated to get to Z (eg. have written 3 paragraphs of 4 of an essay), and a predecessor situational X from which you’ll get to Y, iterate til you get to A (or forward chain, from A to Z). Check that indeed you’ll be motivated each step of the way.
How can the above plan fail? Either you were mistaken about yourself, or about the world. Figure out which and iterate.
Appreciate the highlight of identity as this import/crucial self fulfilling prophecy, I use that frame a lot.
What does the title mean? Since they all disagree I don’t see one as being more of a minority than the other.
Nice talk!
When you talk about the most important interventions for the three scenarios, I wanna highlight that in the case of nationalization, you can also, if you’re a citizen of one of these countries nationalizing AI, work for the government and be on those teams working and advocating for safe AI.
In my case I should have measurable results like higher salary, higher life satisfaction, more activity, more productivity as measured by myself and friends/flatmates. I was very low so it’ll be easy to see progress. The difficulty was finding something that’d work, but it won’t be measuring if it does.
Some people have short ai timelines based inner models that don’t communicate well. They might say “I think if company X trains according to new technique Y it should scale well and lead to AGI, and I expect them to use technique Y in the next few years”, and the reasons for why they think technique Y should work are some kind of deep understanding built from years of reading ml papers, that’s not particularly easy to transmit or debate.
In those cases, I want to avoid going into details and arguing directly, but would suggest that they use their deep knowledge of ML to predict existing recent results before looking at them. This would be easy to cheat, so I mostly suggest this for people to check themselves, or check people you trust to be honorable. Concretely, it’d be nice if when some new ml paper with a new technique comes out, someone compilés a list of questions answered by that paper (eg is technique A better than technique B for a particular result) and posts it to LW so people can track how well they understand ML, and thus (to some extent) short timelines.
For example a recent paper examinés how data affects performance on a bunch of benchmarks, and notably tested training either on an duplicated dataset (a bunch of common crawls), or deduplixated (the same except remove same documents that were shared between crawls). Do you expect deduplication in this case raises or lowers performance on benchmarks? If we could have similar questions when new results come out it’s be nice.
Thank you for sharing, it really helps to pile on these stories (and nice to have some trust they’re real, more difficult to get from reddit—on which note are there non doxing receipts you can show for this story being true? I have no reason to doubt you in particular but I guess it’s good hygiene when on the internet to ask for evidence)
It also makes me wanna share a bit of my story. I read The Mind Illuminated, I did only small amounts of meditation, yet the framing the book offers has been changing my thinking and motivational systems. There aren’t many things I’d call info hazards, but in my experience even just reading the book seems to be enough to contribute to profound changes, that would not be obviously be considered positive by the previous me. (They’re not obviously negative either, I happen to be hopeful, but I’m waiting on results another year later to say)
Might be good to have a dialogue format with other people who agree/disagree to flesh out scenarios and countermeasures
Hi, I’m currently evaluating the cost effectiveness of various projects and would be interested in knowing, if you’re willing to disclose, approximately how much this program costs MATS in total? By this I mean the summer cohort, includings ops before and after necessary for it to happen, but not counting the extension.
“It’s true that we don’t want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says ‘Don’t ask a woman out the first time you meet her’, then we’ll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway.”
This seems a weak response to me, at best only defensible considering yourself to be on the margin and without thought for longterm growth and your ability to clarify intentions (you have more than 3 words when interacting with people irl).
To be clear explicitly writing “don’t ask women out the first time you meet her” would be terrible writing and if that’s the best writing members of that group can do for guidelines, then maybe nothing is better than that. Still, it reeks of “we’ve tried for 30 seconds and are all out of ideas” energy.
A guidelines document can give high level guidance to the vibe you want (eg. truth seeking, not too much aggressiveness, giving space when people feel uncomfortable, communicating around norms explicitly), all in the positive way (eg. you say what you want, not what you don’t want), and can refer to sub-documents to give examples and be quite concrete if you have socially impaired people around that need to learn this explicitly.
The case for training frontier AIs on Sumerian-only corpus
Note Existential is a term of art different from Extinction.
The Precipice cites Bostrome and defines it such:
”An existential catastrophe is the destruction of humanity’s longterm potential.
An existential risk is a risk that threatens the destruction of humanity’s longterm potential.”Disempowerment is generally considered an existential risk in the literature.
I participated in the previous edition of AISC and found it very valuable to my involvement in AI Safety. I acquired knowledge (on standards and the standards process), got experience, contacts. I appreciate how much coordination AISC enables, with groups forming, which enable many to have their first hands on experience and step up their involvement.
Thanks, and thank you for this post in the first place!
Would be up for this project. As is, I downvoted Trevor’s post for how rambly and repetitive it is. There’s a nugget of idea, that AI can be used for psychological/information warfare that I was interested in learning about, but the post doesn’t seem to have much substantive argument to it, so I’d be interested in someone both doing an incredibly shorter version which argued for its case with some sources.
It’s a nice pic and moment, I very much like this comic and the original scene. It might be exaggerating a trait (here by having the girl be particularly young) for comedic effect but the Hogfather seems right.
I think I was around 9 when I got my first sword, around 10 for a sharp knife. I have a scar in my left palm from stabbing myself with that sharp knife as a child while whittling wood for a bow. It hurt for a bit, and I learned to whittle away from me or do so more carefully. I’m pretty sure my life is better for it and (from having this nice story attached to it) I like the scar.
This story still presents the endless conundrum between avoiding hurt and letting people learn and gain skills.
Assuming the world was mostly the same as nowadays, by the time your children are parenting, would they have the skills to notice sharp corners if they never experienced them ?I think my intuitive approach here would be to put some not too soft padding (which is effectively close to what you did, it’s still an unpleasant experience hitting against that even with the cloth).
What’s missing is how to teach against existential risks. There’s an extent to which actually bleeding profusely from a sharp corner can help learn walking carefully, anticipating dangers, and that these skills do generalize to many situations and allows one to live a long fruitful life. (This last sentence does not pertain to the actual age of your children and doesn’t address ideal ages at which you can actually learn the correct and generalizable thing). If you have control on the future, remove all the sharp edges forever.
If you don’t, you remove the hard edges when they’re young, instore them again when they can/should learn to recognize what typically are hard edges and must be accounted for.
Are people losing ability to use and communicate in previous ontologies after getting Insight from meditation ? (Or maybe they never had the understanding I’m expecting of them ?) Should I be worried myself, in my practice of meditation ?
Today I reread Kensho by @Valentine, which presents Looking, and the ensuing conversation in the comments between @Said Achmiz and @dsatan, where Said asks for concrete benefits we can observe and mostly fails to get them. I also noticed interesting comments by @Ruby who in contrast was still be able to communicate in the more typical LW ontology, but hadn’t meditated to the point of Enlightenment. Is Enlightenment bad? Different ?
My impression is that people don’t become drastically better (at epistemology, rationality, social interaction, actually achieving your goals and values robustly) very fast through meditating or getting Enlightened, though they may acquire useful skills that could help to get better. If that’s the case, it’s safe for me to continue practicing meditation, getting into Jhanas, Insight etc (I’m following The Mind Illuminated), as the failings of Valentine/dsatan to communicate their points could just be attributed to them not being able to before either.
But I remain wary that people spend so much time engaging and believing in the models and practices taught in meditation material that they actually change their minds for the worse in certain respects. It looks like meditation ontologies/practices are Out to Get You and I don’t want to get Got.
Oh wow, makes sense. It felt weird that you’d spend so much time on posts, yet if you didn’t spend much time it would mean you write at least as fast as Scott Alexander. Well, thanks for putting in the work. I probably don’t publish much because I want it to not be much work to do good posts but you’re reassuring it’s normal it does.