I think that pleading total agnosticism towards the simulators’ goals is not enough. I write “one common interest of all possible simulators is for us to cede power to an AI whose job is to figure out the distribution of values of possible simulators as best as it can, then serve those values.” So I think you need a better reason to guard against being influenced then “I can’t know what they want, everything and its opposite is equally likely”, because the action proposed above is pretty clearly more favored by the simulators than not doing it.

Btw, I don’t actually want to fully “guard against being influenced by the simulators”, I would in fact like to make deals with them, but reasonable deals where we get our fair share of value, instead of being stupidly tricked like the Oracle and ceding all our value for one observable turning out positively. I might later write a post about what kind of deals I would actually support.

The simulators can just use a random number generator to generate the events you use in your decision-making. They lose no information by this, your decision based on leaves falling on your face would be uncorrelated anyway with all other decisions anyway from their perspective, so they might as well replace it with a random number generator. (In reality, there might be some hidden correlation between the leaf falling on your left face, and another leaf falling on someone else’s face, as both events are causally downstream of the weather, but given that the process is chaotic, the simulators would have no way to determine this correlation, so they might as well replace it with randomness, the simulation doesn’t become any less informative.)

Separately, I don’t object to being sometimes forked and used in solipsist branches, I usually enjoy myself, so I’m fine with the simulators creating more moments of me, so I have no motive to try schemes that make it harder to make solipsist simulations of me.

I’m generally pretty good. I’m way better at communicating when I am having a problem. Plus, with my meds I made it through the end of a ten-year relationship without falling into a deep depression. I haven’t had a weird melt down in some time.

As you say, I can predict certain things, and prepare. Routines help me not forget essential things. For instance, I don’t brush my teeth until after I have taken my pills so I can immediately tell if I already have.

I use way-stations for cleaning up, like by the door to my room, I have a spot on a table for things that’s I want to take out of the room the next time I go, that way I don’t have to interrupt whatever I am doing at the time. I have stations like this in most rooms of my house.

I also compensate by motivating myself to take care of things immediately. “Later is a lie,” has become sort of mantra for me. When I hear myself saying our thinking “later,” I try to remind myself of that mantra. It works for my son too.

All of my bills are on autopay. I put appointments in my calendar so I don’t have to remember them, and set timers all the time for things. It’s generally better than the reminder apps. My phone is basically the third lobe of my brain. I’ve written routines for when to set my alarm, so I don’t forget to so.

I also work within my limits. For instance, if a project will take more than a weekend, I know I won’t be same to sustain interest in it. I can make exceptions—to varying levels of success—but they have to be things that are really important. I won’t get a pet, and I have just one plant that’s right by my sink.

I don’t generally have problems with willpower; I have problems with the word and concept.

If you think you might be in a solipsist simulation, you might try to add some chaotic randomness to your decisions. For example, go outside under some trees and wait till any kind of tree leaf or seed or anything hits your left half of the face, choose one course of action. If it hits the other half of your face, choose another course of action. If you do this multiple times in your life, each of your decisions will depend on the state of the whole earth and on all your previous decisions, since weather is chaotic. And thus the simulators will be unable to get good predictions about you using a solipsist simulation. A potential counterargument is that they analyze your thinking and hardcode this binary random choice, i.e. hardcode the memory of the seed hitting your left side. But then there would need to be an intelligent process analyzing your thinking to try and isolate the randomness. But then you could make the dependence of your strategy on randomness even more complicated.

It seems that 76.6% originally came from the GPT-4o announcement blog post. I’m not sure why it dropped to 60.3% by the time of o1′s blog post.

Years ago when I was hanging out with day traders there was a heuristic they all seemed to hold. If their trading model was producing winning trades two out of three times they thought the model was good and could be used. No one ever suggested why that particular rate was the shared meme/​norm—why not 4 out of 5 or 3 out of 5. I wonder if empirically (or just intuitively over time) they simply approximated the results in this post.

Or maybe just a coincidence, but generally when money is at stake I think the common practices will tend to reflect some fundamental fact of the environment.

I think it is an obvious yes. I tend to think of intelligence as being efficient with energy, fast, accounting for any possibly useful stimuli or information and creativity. Now creativity, relies very much so on memory. One of the functions of the nervous system is to navigate and keep safe of the body from dying basically, keeping the system on. If that has to be done, then it is useful to have nervous system that properly, or barely or enough properly maps the environment, collects information and keeps a record of it too so when that encounter happens again or you want to relate that info to something else, you don’t have to spend too much time and energy thinking what to do about it, you can be fast in processing if the recurring stimuli lits related substrates of brain. I would say this is partly creativity, connections amongst neurons. That is how one can utilise memory, otherwise it is not much use, unless one wants to woo their parents and teachers. And besides, author stated in article that genes for intelligence will be chosen based on iq or equivalent, eg SAT. Of course it would serve people better if there were distinction among these cortical traits by measuring them individually, instead of relying solely on iq. If it will be based of iq only, then it will make the masses slightly homogeneous.

Oh to be clear I don’t think it was bad for you to post this as-is. Just that I’d like to see more followup

You’re right. I just like the phrase “postmodern warfare” because I think it’s funny.

The content/​minute rate is too low, it follows 1960s film standards where audiences weren’t interested in science fiction films unless concepts were introduced to them very very slowly (at the time they were quite satisfied by this due to lower standards, similar to Shakespeare).

As a result it is not enjoyable (people will be on their phones) unless you spend much of the film either thinking or talking with friends about how it might have affected the course of science fiction as a foundational work in the genre (almost every sci-fi fan and writer at the time watched it).

I haven’t decided yet whether to write up a proper “Why Not Just...” for the post’s proposal, but here’s an overcompressed summary. (Note that I’m intentionally playing devil’s advocate here, not giving an all-things-considered reflectively-endorsed take, but the object-level part of my reflectively-endorsed take would be pretty close to this.)

Charlie’s concern isn’t the only thing it doesn’t handle. The only thing this proposal does handle is an AI extremely similar to today’s, thinking very explicitly about intentional deception, and even then the proposal only detects it (as opposed to e.g. providing a way to solve the problem, or even a way to safely iterate without selecting against detectability). And that’s an extremely narrow chunk of the X-risk probability mass—any significant variation in the AI breaks it, any significant variation in the threat model breaks it. The proposal does not generalize to anything.

Charlie’s concern is just one specific example of a way in which the proposal does not generalize. A proper “Why Not Just...” post would list a bunch more such examples.

And as with Charlie’s concern, the meta-level problem is that the proposal also probably wouldn’t get us any closer to handling those more-general situations. Sure, we could make some very toy setups (like the chess thing), and see what the shoggoth+face AI does on those very toy setups, but we get very few bits, and the connection is very tenuous to both other threat models and AIs with any significant differences from the shoggoth+face. Accounting for the inevitable failure to measure what we think we’re measuring (with probability close to 1), such experiments would not actually get us any closer to solving any of the problems which constitute the bulk of the X-risk probability mass. It’s not “a start”, because “a start” would imply that the experiment gets us closer, i.e. that the problem gets easier after doing the experiment. If you try to think about the You Are Not Measuring What You Think You Are Measuring problem as “well, we got at least some tiny epsilon of evidence, right?”, then you will shoot yourself in the foot; such reasoning is technically correct, but the correct value of epsilon is small enough that the correct update from it is not distinguishable from zero in practice.

Thanks for clarifying. By “policy” and “standards” and “compelled speech” I thought you meant something more than community norms and customs. This is traditionally an important distinction to libertarians and free speech advocates. I think the distinction carves reality at the joints, and I hope you agree. I agree that community norms and customs can be unwelcoming.

Yeah, IMO we should just add a bunch of functionality for integrating alignment forum stuff more with academic things. It’s been on my to do list for a long time.

As mentioned in another reply, I’m planning to do a lot more research and interviews on this topic, especially with people who are more hawkish on China. I also think it’s important that unsupported claims with large stakes get timely pushback, which is in tension with the type of information gathering you’re recommending (which is also really important, TBC!).

Claiming that China as a country is racing toward AGI != Chinese AI companies aren’t fast following US AI companies, which are explicitly trying to build AGI. This is a big distinction!

Tenet (2020) by George Nolan revolves around recursive thinking and responding to unreasonably difficult problems. Nolan introduces the time-reversed material as the core dynamic, then iteratively increases the complexity from there, in ways specifically designed to ensure that as much of the audience as possible picks up as much recursive thinking as possible.

This chart describes the movement of all key characters plot elements through the film; it is actually very easy to follow for most people. But you can also print out a bunch of copies and hand them out before the film (it isn’t a spoiler so long as you don’t look closely at the key).

Most of the value comes from Eat the Instructions-style mentality, as both the characters and the viewer pick up on unconventional methods to exploit the time reversing technology, only to be shown even more sophisticated strategies and are walked through how they work and their full implications.

It also ties into scope sensitivity, but it focuses deeply on the angles of interfacing with other agents and their knowledge, and responding dynamically to mistakes and failures (though not anticipating them), rather than simply orienting yourself to mandatory number crunching.

The film touches on cooperation and cooperation failures under anomalous circumstances, particularly the circumstances introduced by the time reversing technology.

The most interesting of these was also the easiest to miss:

Hey Seth, appreciate the detailed engagement. I don’t think the 2017 report is the best way to understand what China’s intentions are WRT to AI, but there was nothing in the report to support Helberg’s claim to Reuters. I also cite multiple other sources discussing more recent developments (with the caveat in the piece that they should be taken with a grain of salt). I think the fact that this commission was not able to find evidence for the “China is racing to AGI” claim is actually pretty convincing evidence in itself. I’m very interested in better understanding China’s intentions here and plan to deep dive into it over the next few months, but I didn’t want to wait until I could exhaustively search for the evidence that the report should have offered while an extremely dangerous and unsupported narrative takes off.

I also really don’t get the error pushback. These really were less technical errors than basic factual errors and incoherent statements. They speak to a sloppiness that should affect how seriously the report should be taken. I’m not one to gatekeep ai expertise, but idt it’s too much to expect a congressional commission with a top recommendation to commence in a militaristic AI arms race to have SOMEONE read a draft who knows that chatgpt-3 isn’t a thing.

So one of the themes of sequences is that deliberate self-deception or thought censorship—deciding to prevent yourself from “knowing” or learning things you would otherwise learn—is almost always irrational. Reality is what it is, regardless of your state of mind, and at the end of the day whatever action you’re deciding to take—for example, not talking about dragons—you could also be doing if you knew the truth. So when you say:

But if I decided to look into it I might instead find myself convinced that dragons do exist. In addition to this being bad news about the world, I would be in an awkward position personally. If I wrote up what I found I would be in some highly unsavory company. Instead of being known as someone who writes about a range of things of varying levels of seriousness and applicability, I would quickly become primarily known as one of those dragon advocates. Given the taboos around dragon-belief, I could face strong professional and social consequences.

It’s not a reason not to investigate. You could continue to avoid these consequences you speak of by not writing about Dragons regardless of the results of your investigation. My guess is that what you’re also avoiding, is guilt that might come from knowing the truth and remaining silent. But through your decision not to investigate, the world is going to carry the burden of that silence either way.

Another theme of the sequences is that self-deception, deliberate agnosticism, and motivated reasoning are a source of surprising amounts of human suffering. Richard explains one way it goes horribly wrong here. Whatever subject you’re talking about, I’m sure there a lot of other people in your position who have chosen not to look into it for the same reasons. But if all of those people had looked into it, and faced whatever conclusion that resulted squarely, you yourself might not be in the position of having to face a harmful taboo in the first place. So the form of information hiding you endorse in the post is self-perpetuating, and is part of what helps keep the taboo strong in the first place.

“A space Odyssey” is not watchable without discussing historical context

… why? I’ve watched this movie, and I… don’t think I’m aware of any special “historical context” that was relevant to it. (Or, at any rate, I don’t know what you mean by this.) It seemed to work out fine…

Say more?