Director at AI Impacts.
Richard Korzekwa
Karma: 1,442
My weak downvotes are +1 and my strong downvotes are −9. Upvotes are all positive.
I agree that in the context of an explicit “how soon” question, the colloquial use of fast/slow often means sooner/later. In contexts where you care about actual speed, like you’re trying to get an ice cream cake to a party and you don’t want it to melt, it’s totally reasonable to say “well, the train is faster than driving, but driving would get me there at 2pm and the train wouldn’t get me there until 5pm”. I think takeoff speed is more like the ice cream cake thing than the flight to NY thing.
That said, I think you’re right that if there’s a discussion about timelines in a “how soon” context, then someone starts talking about fast vs slow takeoff, I can totally see how someone would get confused when “fast” doesn’t mean “soon”. So I think you’ve updated me toward the terminology being bad.
I agree. I look at the red/blue/purple curves and I think “obviously the red curve is slower than the blue curve”, because it is not as steep and neither is its derivative. The purple curve is later than the red curve, but it is not slower. If we were talking about driving from LA to NY starting on Monday vs flying there on Friday, I think it would be weird to say that flying is slower because you get there later. I guess maybe it’s more like when people say “the pizza will get here faster if we order it now”? So “get here faster” means “get here sooner”?
Of course, if people are routinely confused by fast/slow, I am on board with using different terminology, but I’m a little worried that there’s an underlying problem where people are confused about the referents, and using different words won’t help much.
Yeah! I made some lamps using sheet aluminum. I used hot glue to attach magnets, which hold it onto the hardware hanging from the ceiling in my office. You can use dimmers to control the brightness of each color temperature strip separately, but I don’t have that set up right now.
why do you think s-curves happen at all? My understanding is that it’s because there’s some hard problem that takes multiple steps to solve, and when the last step falls (or a solution is in sight), it’s finally worthwhile to toss increasing amounts of investment to actually realize and implement the solution.
I think S-curves are not, in general, caused by increases in investment. They’re mainly the result of how the performance of a technology changes in response to changes in the design/methods/principles behind it. For example, with particle accelerators, switching from Van der Graaff generators to cyclotrons might give you a few orders of magnitude once the new method is mature. But it takes several iterations to actually squeeze out all the benefits of the improved approach, and the first few and last few iterations give less of an improvement than the ones in the middle.
This isn’t to say that the marginal return on investment doesn’t factor in. Once you’ve worked out some of the kinks with the first couple cyclotrons, it makes more sense to invest in a larger one. This probably makes S-curves more S-like (or more step like). But I think you’ll get them even with steadily increasing investment that’s independent of the marginal return.
Neurons’ dynamics looks very different from the dynamics of bits.
Maybe these differences are important for some of the things brains can do.
This seems very reasonable to me, but I think it’s easy to get the impression from your writing that you think it’s very likely that:
The differences in dynamics between neurons and bits are important for the things brains do
The relevant differences will cause anything that does what brains do to be subject to the chaos-related difficulties of simulating a brain at a very low level.
I think Steven has done a good job of trying to identify a bit more specifically what it might look like for these differences in dynamics to matter. I think your case might be stronger if you had a bit more of an object level description of what, specifically, is going on in brains that’s relevant to doing things like “learning rocket engineering”, that’s also hard to replicate in a digital computer.
(To be clear, I think this is difficult and I don’t have much of an object level take on any of this, but I think I can empathize with Steven’s position here)
The Trinity test was preceded by a full test with the Pu replaced by some other material. The inert test was designed to test whether they were getting the needed compression. (My impression is this was not publicly known until relatively recently)
Regardless, most definitions [of compute overhang] are not very analytically useful or decision-relevant. As of April 2023, the cost of compute for an LLM’s final training run is around $40M. This is tiny relative to the value of big technology companies, around $1T. I expect compute for training models to increase dramatically in the next few years; this would cause how much more compute labs could use if they chose to to decrease.
I think this is just another way of saying there is a very large compute overhang now and it is likely to get at least somewhat smaller over the next few years.
Keep in mind that “hardware overhang” first came about when we had no idea if we would figure out how to make AGI before or after we had the compute to implement it.
Drug development is notably different because, like AI, it’s a case where the thing we want to regulate is an R&D process, not just the eventual product
I agree, and I think I used “development” and “deployment” in this sort of vague way that didn’t highlight this very well.
But even if we did have a good way of measuring those capabilities during training, would we want them written into regulation? Or should we have simpler and broader restrictions on what counts as good AI development practices?
I think one strength of some IRB-ish models of regulation is that you don’t rely so heavily on a careful specification of the thing that’s not allowed, because instead of meshing directly with all the other bureaucratic gears, it has a layer of human judgment in between. Of course, this does pass the problem to “can you have regulatory boards that know what to look for?”, which has its own problems.
I put a lid on the pot because it saves energy/cooks faster. Or maybe it doesn’t, I don’t know, I never checked.
I checked and it does work.
Seems like the answer with pinball is to avoid the unstable processes, not control them.
Regarding the rent for sex thing: The statistics I’ve been able to find are all over the place, but it looks like men are much more likely to not have a proper place to sleep than women. My impression is this is caused by lots of things (I think there are more ways for a woman to be eligible for government/non-profit assistance, for example), but it does seems like evidence that women are exchanging sex for shelter anyway (either directly/explicitly or less directly, like staying in a relationship where the main thing she gets is shelter and the main thing the other person gets is sex).
Wow, thanks for doing this!
I’m very curious to know how this is received by the general public, AI researchers, people making decisions, etc. Does anyone know how to figure that out?
With the caveats that this is just my very subjective experience, I’m not sure what you mean by “moderately active” or “an athlete”, and I’m probably taking your 80⁄20 more literally than you intended:
I agree there’s a lot of improvement from that first 20% of effort (or change in habits or time or whatever), but I think it’s much less than than 80% of the value. Like, say 0% effort is the 1-2 hours/week of walking I need do to get to work and buy groceries and stuff, 20% is 2-3 hours of walking + 1-2 hours at the gym or riding a bike, and 100% is 12 hours/week of structured training on a bicycle. I think 20% gets me maybe 40-50% of the benefit for doing stuff that requires thinking clearly that 100% gets me. Where the diminishing returns really kick in is around 6-8 hours/week of structured training (so 60%?), which seems to get me about 80-90% of the benefit.
That said: Anecdotally, I seem to need more intense exercise than a lot of people. Low-to-moderate intensity exercise, even in significant quantity, has a weirdly small effect on my mood and my (subjectively judged by me) cognitive ability.
Right, but being more popular than the insanely popular thing would be pretty notable (I suppose this is the intuition behind the “most important chart of the last 100 years” post), and that’s not what happened.
The easiest way to see what 6500K-ish sunlight looks like without the Rayleigh scattering is to look at the light from a cloudy sky. Droplets in clouds scatter without the strong wavelength dependence that air molecules do, so it’s closer to the unmodified solar spectrum (though there is still atmospheric absorption).
If you’re interested in (somewhat rudimentary) color measurements of some natural and artificial light sources, you can see them here.
It’s maybe fun to debate about whether they had mens rea, and the courts might care about the mens rea after it all blows up, but from our perspective, the main question is what behaviors they’re likely to engage in, and there turn out to be many really bad behaviors that don’t require malice at all.
I agree this is the main question, but I think it’s bad to dismiss the relevance of mens rea entirely. Knowing what’s going on with someone when they cause harm is important for knowing how best to respond, both for the specific case at hand and the strategy for preventing more harm from other people going forward.
I used to race bicycles with a guy who did some extremely unsportsmanlike things, of the sort that gave him an advantage relative to others. After a particularly bad incident (he accepted a drink of water from a rider on another team, then threw the bottle, along with half the water, into a ditch), he was severely penalized and nearly kicked off the team, but the guy whose job was to make that decision was so utterly flabbergasted by his behavior that he decided to talk to him first. As far as I can tell, he was very confused about the norms and didn’t realize how badly he’d been violating them. He was definitely an asshole, and he was following clear incentives, but it seems his confusion was a load-bearing part of his behavior because he appeared to be genuinely sorry and started acting much more reasonably after.
Separate from the outcome for this guy in particular, I think it was pretty valuable to know that people were making it through most of a season of collegiate cycling without fully understanding the norms. Like, he knew he was being an asshole, but he didn’t really get how bad it was, and looking back I think many of us had taken the friendly, cooperative culture for granted and hadn’t put enough effort into acculturating new people.
Again, I agree that the first priority is to stop people from causing harm, but I think that reducing long-term harm is aided by understanding what’s going on in people’s heads when they’re doing bad stuff.
Note that research that has high capabilities externalities is explicitly out of scope:
”Proposals that increase safety primarily as a downstream effect of improving standard system performance metrics unrelated to safety (e.g., accuracy on standard tasks) are not in scope.”I think the language here is importantly different from placing capabilities externalities as out of scope. It seems to me that it only excludes work that creates safety merely by removing incompetence as measured by standard metrics. For example, it’s not clear to me that this excludes work that improves a model’s situational awareness or that creates tools or insights into how a model works with more application to capabilities than to safety.
I agree that college is an unusually valuable time for meeting people, so it’s good to make the most of it. I also agree that one way an event can go badly is if people show up wanting to get to know each other, but they do not get that opportunity, and it sounds like it was a mistake for the organizers of this event not to be more accommodating of smaller, more organic conversations. And I think that advice on how to encourage smaller discussions is valuable.
But I think it’s important to keep in mind that not everyone wants the same things, not everyone relates to each other or their interests in the same way, and small, intimate conversations are not the be-all-end-all of social interaction or friend-making.
Rather, I believe human connection is more important. I could have learned about Emerson much more efficiently by reading the Stanford Encyclopedia of Philosophy, or been entertained more efficiently by taking a trip to an amusement park, for two hours.
For some people, learning together is a way of connecting, even in a group discussion where they don’t get to say much. And, for some people, mutual entertainment is one of their ways of connecting. Another social benefit of a group discussion or presentation is that you have some specific shared context to go off of—everyone was there for the same discussion, which can provide an anchor of common experience. For some people this is really important.
Also, there are social dynamics in larger groups that cannot be replicated in smaller groups. For example:
Suppose a group of 15 people is talking about Emerson. Alice is trying to get an idea across, but everyone seems to be misunderstanding. Bob chimes in and asks just the right questions to show that he understands and wants Alice’s idea to get through. Alice smiles at Bob and thanks him. Alice and Bob feel connection.
Another example:
In the same discussion, Carol is high status and wrote her PhD dissertation on Emerson. Debbie wants to ask her a question, but is intimidated by the thought of having a one-on-one conversation with her. Fortunately, the large group discussion environment gives Debbie the opportunity to ask a question without the pressure of a having a full conversation. Carol reacts warmly and answers Debbie’s question in a thoughtful way. This gives Debbie the confidence to approach Carol during the social mingling part of the discussion later on.
Most people prefer talking to listening.
But not everyone! Some people really like listening, watching, and thinking. And, among people who do prefer talking, many don’t care that much if they sometimes don’t get to talk that much, especially if there are other benefits.
(Also, I’m a little suspicious when someone argues for event formats that are supposed to make it easier to “get to know people”, and one of the main features is that they get to spend more time talking and less time listening)
Different hosts have different goals for their events, and that’s fine. I just value human connection a lot.
I think it’s important not to look at an event that fails to create social connection for you and assume that it does not create connection for others. This is both because not everyone connects the same way and because it’s hard to look at how an event and say whether it resulted in personal connection (it would not have been hard for an observer to miss the Alice-Bob connection, for example). That said, I do think it gets easier to tell as a group has been hosting events with the same people for longer. If people are consistently treating each other as strangers or acquaintances from week to week, this is a bad sign.
Another victory for trend extrapolation!