I also write at https://splittinginfinity.substack.com/
harsimony
Karma: 498
I feel like people are dismissing this study out of hand without updating appropriately. If there’s at least a chance that this result replicates, that should shift our opinions somewhat.
First, a few reasons why the common counterarguments aren’t strong enough to dismiss the study:
I’ve been seeing arguments against this result based on vibes or claims that the next generation of LLM’s will overturn this result. But that is directly contradicted by the results of this study, people’s feelings are poor indicators of actual productivity.
On Cursor experience, I think Joel Becker had a reasonable response here. Essentially, many of the coders had tried cursor, had some experience with it, and had a lot of experience using LLM’s for programming. Is the learning curve really so steep that we shouldn’t see them improve over the many tasks? See image below. Perhaps the fact that these programmers don’t use it and saw little improvement is a sign that Cursor isn’t very helpful.
While this is a challenging environment for LLM coding tools, this is the sort of environment I want to see improvement in for AI to have a transformative impact on coding. Accelerating experienced devs is where a lot of the value of automating coding will come from.
That aside, how should we change our opinions with regard to the study?
Getting AI to be useful in a particular domain is tricky, you have to actually run tests and establish good practices.
Anecdotes about needing discipline to stay on task with coding tools and the cursor learning curve suggest that AI adoption has frictions and requires tacit knowledge to use.
Coding is one of the cleanest, most data-rich, most LLM-developer-supported domains. As of yet, AI automation is not a slam dunk, even here. Every other domain will require its own iteration, testing, and practice to see a benefit.
If this holds, the points above slow AI diffusion, particularly when used as a tool for humans. Modelling the impact of current and near-future AI’s should take this into account.
Yeah I think sleep probably serves other roles, I just don’t see why those roles require 7 hours of sleep rather than say 5 hours.
I do agree that basic research is what will actually get sleep need reduction therapies to work at scale. I’m hoping that citizen science and discussion of the topic will encourage more work on this.
Oh I see. So the hypothesis is “In a healthy animal, stress is a highly-informative signal that inhibits risk-taking. Sleep ensures the stress system continues to inhibit risk taking appropriately.”
Makes sense. It’s consistent with sleep deprivation raising the level of cortisol and the brain developing a tolerance to high levels of certain hormones.
Oh I think sleep probably plays other roles today! But I don’t think those roles require exactly 7 hours of sleep.
And agreed, need to look at long term effects of sleep need reduction too. My vision is more that people have 3-4 nights of 1-2 hours less sleep and then take a break for 3 nights rather than taking a drug to stop sleeping entirely.
We are fundraising for a self-experiment soon!
I think there’s a substantial chance that orexin agonists are “just stimulants” and you can’t reduce sleep need much with them. But short sleepers prove it’s biologically possible and I want to encourage people to start working on this.
The point about children is a good one, I have to think on it more. But it seems consistent with children needing more calories to grow (and they are too young to gather their own calories), so they rest more.
It might be something more complex like: (1) Animals that aren’t careful tend to take a lot of risks that result in them dying. (2) There’s a process that builds up stress that’s about taking less risks. (3) Sleep exists to process that build-up stress and resolve it.
By this do you mean: “when stress builds up, animals take more risks. Risk-taking animals die. Sleep relieves stress, thus enabling less risky behavior?”
It’s a reasonable hypothesis, and I’m open to it happening to some degree. But I think all the arguments against “neurons need rest” apply here just as well. Aren’t there brain regions more or less exposed to stress? Don’t some neurons experience constant (metabolic) stress simply by being active all the time? Are animals with more stress (prey) sleeping more? Do more stressed people adapt by sleeping more?
Why? Orexin-A is perfectly capable of crossing the blood-brain barrier. If you create a gene therapy to produce more of it, it doesn’t need to be produced in the brain.
I should have provided more context, the proposals of Minjune Song and Issak Freeman center on short sleep mutations found in the literature. These are typically mutations of receptors found on neurons in the brain.
Overproducing orexin-A is fine, but a gene therapy for a hormone that can cross the BBB is overkill when you can supply it exogenously right?
By the way, Orexin-A supplementation is the subject of the proposal I mentioned at the end. Should come out today.
Why do you believe that? For Orexin-A? You can buy it these days from a neurotropic store if you want as a nasal spray.
I think you’re mostly asking about production costs right? Yes, you can buy orexin peptides and custom RNA, but the cost-per-unit-effect is orders of magnitude higher than small molecules. It’s hard to beat the ~$100/kg that you can get with generic drug manufacturing.
Good to have a number for this. Though I think a better counterfactual is between sleeping and actively foraging. Foraging + thermoregulation costs even more calories.
But let’s say for the sake of argument that being awake + foraging takes 20% more calories compared to sleeping. Would sleeping actually get selected for? I think so. Evolution can make pretty fine distinctions given enough generations.
For example, cavefish (who live in an environment without light) quickly evolve less pigmentation and underdeveloped eyes to save energy. This is a convergent trait, it’s been observed in several different species. Though I’m not sure what kind of energy penalty eyes and pigment have.
I’m not as familiar with insomnia treatments, but orexin antagonists seem to be an improvement over existing meds. Probably the biggest improvement is the lower risk of abuse and tolerance compared to other medications. Belsomra has been around for over 10 years and seems to be well tolerated and effective. Though it doesn’t work for everyone.
The argument that orexin antagonists could help people sleep more without making you sleepy during the day makes sense to me, with one caveat. If the half-life is long enough, the antagonist could block the orexin signal your body normally produces in the morning, making you feel sleepier than you otherwise would have. But perhaps this would be outweighed by the alertness one gets from a good nights sleep.
As for side effects (beyond daytime sleepiness and things we already know to watch for) I guess I would look for changes in motivation. Orexin has a (tenuous) link to reward seeking, addiction, and motivation. If you feel less energy and motivation during the day that might be something to look into. Indeed the point of a good nights sleep is to feel good and energetic the following day, so it’s good to check if an insomnia treatment actually delivers on that.
You may also want to consider adding psychological approaches like CBT-i or paradoxical intention. It seems like some insomnia features a self-reinforcing loop of bad sleep leading to frustration which begets more bad sleep.
(ofc, none of this is medical advice)
Wonderful to get more numbers on this!
These examples seem to contradict note 2 where D/N falls for larger C. Now I’m not sure what the trend should be.
It feels like you could derive a rule of thumb based on the loss and the entropy of the dataset e.g. “If my model starts at a loss of 4 bits/token and the asymptote is 2 bits/token, I need X tokens of data to fully specify a model with Y bits stored in the parameters.”
Oh that makes sense!
If the predictors can influence the world in addition to making a prediction, they would also have an incentive to change the world in ways that make their predictions more accurate than their opponents right? For example, if everyone else thinks Bob is going to win the presidency, one of the predictors can bribe Bob to drop out and then bet on Alice winning the presidency.
Is there work on this? To be fair, it seems like every AI safety proposal has to deal with something like this.
This is super cool stuff, thank you for posting!
I may have missed this, but do these scoring rules prevent agents from trying to make the environment more un-predictable? In other words, if you’re competing against other predictors, it may make sense to influence the world to be more random and harder to understand.
I think this prediction market type issue has been discussed elsewhere but I can’t find a name for it.
Thanks for this! I misinterpreted Lucius as saying “use the single highest and single lowest eigenvalues to estimate the rank of a matrix” which I didn’t think was possible.
Counting the number of non-zero eigenvalues makes a lot more sense!
You can absolutely harvest potential energy from the solar system to spin up tethers. ToughSF has some good posts on this:
https://toughsf.blogspot.com/2018/06/inter-orbital-kinetic-energy-exchanges.html https://toughsf.blogspot.com/2020/07/tethers-all-way.html
Ideally your tether is going to constantly adjust its orbit so it says far away from the atmosphere, but for fun I did a calculation of what would happen if a 10K tonne tether (suitable for boosting 100 tonne payloads) fell to the Earth. Apparently it just breaks up in the atmosphere and produces very little damage. More discussion here:
The launch cadence is an interesting topic that I haven’t had a chance to tackle. The rotational frequency limits how often you can boost stuff.
Since time is money you would want a shorter and faster tether, but a shorter time of rotation means that your time window to dock with the tether is smaller, so there’s an optimization problem there as well.
It’s a little easier when you’ve got catapults on the moon’s surface. You can have two running side by side and transfer energy between them electrically. So load up catapult #1, spin it up, launch the payload, and then transfer the remaining energy to catapult #2. You can get much higher launch cadence that way.
Oops yes, that should read “Getting oxygen from the moon to LEO requires less delta V than going from the Earth to LEO!”. I edited the original comment.
Lunar tethers actually look like they will be feasible sooner than Earth tethers! The lack of atmosphere, micrometeorites, and lower gravity (g) makes them scale better.
In fact, you can even put a small tether system on the lunar surface to catapult payloads to orbit: https://splittinginfinity.substack.com/p/should-we-get-material-from-the-moon
Whether tethers are useful on the moon depends on the mission you want to do. Like you point out, low delta-V missions probably don’t need a tether when rockets work just fine. But if you want to take lunar material to low earth orbit or send it to Mars, a lunar tether is a great option.
The near-term application I’m most excited about is liquid oxygen. Getting oxygen from the moon to LEO requires less delta V than going from the Earth to LEO! Regolith is ~45% oxygen by mass and a fully-fueled Starship is 80% LOX by mass. So refueling ships in LEO with lunar O2 could be viable.
Even better, the falling lunar oxygen can spin up a tether in LEO which can use that momentum to boost a Starship to other parts of the solar system.
Thanks for the comments! Going point-by-point:
-
I think both fiberglass and carbon fiber use organic epoxy that’s prone to UV (and atomic oxygen) degradation? One solution is to avoid epoxy entirely using parallel strands or something like a Hoytether. The other option is to remove old epoxy and reapply over time, if its economical vs just letting the tether degrade.
-
I worry that low-thrust options like ion engines and sails could be too expensive vs catching falling mass, but I could be convinced either way!
-
Yeah, some form of vibration damping will be important, I glossed over this. Bending modes are particularly a problem for glass. Though I would guess that vibrations wouldn’t make the force along the tether any higher?
-
Catching the projectile is a key engineering challenge here! One that I probably can’t solve from my armchair. As for missing the catch, I guess I don’t see this as a huge issue? If the rocket can re-land, missing the catch means that the only loss is fuel. Though colliding with the tether would be a big problem.
-
Yeah I think low orbits are too challenging for tethers, so they’re definitely going to be at risk of micrometeorite impacts. I see this as a key role of the “safety factor”. Tether should be robust to ~10-50% of fibers being damaged, and there should be a way to replace/repair them as well.
-
Right, though tethers can’t help satellites get to LEO, they can help them get to higher orbits which seems useful. But the real value-add comes when you want to get to the Moon and beyond.
-
Good to know! I would love to see more experiments on glass fibers pulled in space, small-scale catches, and data on what kinds of defects form on these materials in orbit.
-
Yeah, my overall sense is that using falling mass to spin the tether back up is the most practical. But solar sails and ion drives might contribute too, these are just much slower which hurts launch cadence and costs.
The fact that you need a regular supply of falling mass from e.g. the moon is yet another reason why tethers need a mature space industry to become viable!
That makes sense, I guess it just comes down to an empirical question of which is easier.
Question about what you said earlier: How can you use the top/bottom eigenvalues to estimate the rank of the Hessian? I’m not as familiar with this so any pointers would be appreciated!
Nicely written proposal, thank you.
In truth, I’m quite concerned with such a proposal being implemented. Part of this is, as you mentioned, the risk of lock in. In particular, a global entity with a monopoly on violence is alarmingly close to global authoritarianism. This is an existential risk all on its own.
Such an entity would have to:
Limit access to space (to avoid self-sustaining expansion faster than everyone else).
Limit/monitor the uses of AI and AI training.
Possess a credible monopoly on violence.
Surveil enough to prevent surprise attacks and overthrow attempts.
Considering the risk that such a system is corrupted or misaligned or permanent, I feel better about a future that emphasized freedom and acceleration of defensive technologies.
(I could be convinced that the “multiple night watchmen overseeing each other” is viable. Rather than oversee each other, it might be better to give them completely separate jurisdictions. Federalism and free movement allows people to choose night watchmen that suit their needs. Risk of war between jurisdictions is low since they both have watchmen. Some watchmen may allow AI’s to develop and entities to leak into space, but this laxity a feature to avoid global totalitarianism.)