make fewer points, selected carefully to be bulletproof, understandable to non-experts, and important to the overall thesis
That conflicts with eg:
If you replied with this, I would have said something like “then what’s wrong with the designs for diamond mechanosynthesis tooltips, which don’t resemble enzymes
Yes, this is part of why I didn’t post AI stuff in the past, and instead just tried to connect with people privately. I might not have accomplished much, but at least I didn’t help OpenAI happen or shift the public perception of AI safety towards “fedora-wearing overweight neckbeards”.
Must Not Modify AI Rules: AI must not modify AI Rules. If inadequacies are identified, AI can suggest changes to Legislators but the final modification must be executed by them.
Must Not Modify Its Own Program Logic: AI must not modify its own program logic (self-iteration). It may provide suggestions for improvement, but final changes must be made by its Developers.
Must Not Modify Its Own Goals: AI must not modify its own goals. If inadequacies are identified, AI can suggest changes to its Users but the final modification must be executed by them.
I agree that, if those rules are followed, AI alignment is feasible in principle. The problem is, some people won’t follow those rules if they have a large penalty to AI capabilities, and I think they will.
“Mirror life” is beyond the scope of this post, and the concerns about it are very different than the concerns about “grey goo”—it doesn’t have more capabilities or efficiency, it’s just maybe harder for immune systems to deal with. Personally, I’m not very worried about that and see no scientific reason for the timing of the recent fuss about it. If it’s not just another random fad, the only explanation I can see for that timing is: influential scientists trying to hedge against Trump officials determining that “COVID was a lab leak” in a way that doesn’t offend their colleagues. On the other hand, I do think artificial pathogens in general are a major concern, and even if I’m not very concerned about “mirror life”, there are no real benefits to trying to make it, so maybe just don’t.
I think this is a pretty good post that makes a point some people should understand better. There is, however, something I think it could’ve done better. It chooses a certain gaussian and log-normal distribution for quality and error, and the way that’s written sort of implies that those are natural and inevitable choices.
I would have preferred something like:
Suppose we determine that quality has distribution X and error has distribution Y. Here’s a graph of those superimposed. We can see that Y has more of a fat tail than X, so if measured quality is very high, we should expect that to be mostly error. But of course, the opposite case is also possible. Now then, here’s some basic info about when different probability distributions are good choices.
This was a quick and short post, but some people ended up liking it a lot. In retrospect I should’ve written a bit more, maybe gone into the design of recent running shoes. For example, this Nike Alphafly has a somewhat thick heel made of springy foam that sticks out behind the heel of the foot, and in the front, there’s a “carbon plate” (a thin sheet of carbon fiber composite) which also acts like a spring. In the future, there might be gradual evolution towards more extreme versions of the same concept, as recent designs become accepted. Running shoes with a carbon plate have become significantly more common over the past few years. That review says:
The energy return is noticeably greater than that of a shoe without any plating, especially when you lay down some serious power. And that stiffness doesn’t always compromise as much comfort as you’d think.
So that’s the running-optimized version of shoes with springs using modern materials, while I was writing more about high heels worn for fashion.
Biomechanics is a topic I could write a lot about, but that would be a separate post. On the general topic of “walking” I also wrote this post. (japanese version here)
What have you learned since then? Have you changed your mind or your ontology?
I’ve learned even more chemistry and biology, and I’ve changed my mind about lots of things, but not the points in this post. Those had solid foundations I understood well and redundant arguments, so the odds of that were low.
What would you change about the post? (Consider actually changing it.)
The post seems OK. I could have handled replies to comments better. For example, the top comment was by Thomas Kwa, and I replied to part of it as follows:
Regarding 5, my understanding is that mechanosynthesis involves precise placement of individual atoms according to blueprints, thus making catalysts that selectively bind to particular molecules unnecessary.
No, that does not follow.
I didn’t know in advance which comments would be popular. In retrospect, maybe I should’ve gone into explaining the basics of entropy and enthalpy in my reply, eg:
Even if you hold a substrate in the right position, that only affects the entropy part of the free energy of the intermediate state. In many cases, catalysts are needed to reduce the enthalpy of the highest-energy intermediate states, which requires specific elements and catalyst molecules that form certain bonds with the substrate intermediate state. Affecting enthalpy by holding molecules in certain configurations requires applying a proportional amount of force, which requires strong binding to the substrate, which requires flexible and substrate-specialized holder molecules, and now you have enzymes again. It’s also necessary to bind strongly to substrates if you want a very low level of free ones that can react at uncontrolled positions. (And then some basic explanation of what entropy/enthalpy/etc are, and what enzyme intermediate states look like.)
When you write a post that gets comments from many people, it’s not practical to respond to them all. If you try to, you have less time than the collective commenters, and less information about their position than they have about yours. So you have to guess about what exactly each person is misunderstanding, and that’s not usually something I enjoy.
What do you most want people to know about this post, for deciding whether to read or review-vote on it?
Of the 7 (!) posts of mine currently nominated for “Best of 2023”, this is probably the most appropriate for that.
Of the 2023 posts of mine not currently nominated, my personal favorites were probably:
How concretely have you (or others you know of) used or built on the post? How has it contributed to a larger conversation
Muireall Prase wrote this, and my post was relevant for some conversations on twitter. I suppose it also convinced some people I had some understanding of chemistry.
So, I have a lot of respect for Sarah, I think this post makes some good points, and I upvoted it. However, my concern is, when I look at this particular organization’s Initiatives page, I see “AI for math”, “AI for education”, “high-skill immigration assistance”, and not really anything that distinguishes this organization from the various other ones working on the same things, or their projects from a lot of past projects that weren’t really worthwhile.
Note that due to the difference being greater at higher frequencies, the effect on speech intelligibility will probably be greater for most women than for you.
We can see the diaphragm has some resonance peaks that increase distortion. Probably it’s too thick to help very much, but it has to resist the pressure changes from breathing.
As a “physicist and dabbler in writing fantasy/science fiction” I assume you took the 10 seconds to do the calculation and found that a 1km radius cylinder would have ~100 kW of losses per person from roller bearings supporting it, for the mass per person of the ISS. But I guess I don’t understand how you expect to generate that power or dissipate that heat.
This happened to Explorer 1, the first satellite launched by the United States in 1958. The elongated body of the spacecraft had been designed to spin about its long (least-inertia) axis but refused to do so, and instead started precessing due to energy dissipation from flexible structural elements.
That works well enough, but a Vital 200S currently costs $160 at amazon, less than the cheapest variant of the thing you linked, and has a slightly higher max air delivery rate, some granular carbon in the filter, and features like power buttons. The Vital 200S on speed 2 has similar power usage and slightly less noise, but less airflow, but a carbon layer always reduces airflow. It doesn’t have a rear intake so it can be placed against a wall. It also has a washable prefilter.
Compared to what you linked, the design in this post has 3 filters instead of 2, some noise blocking, and a single large fan instead of multiple fans. Effective floor area usage should be slightly less, but of course it has to go together with shelving for that.
First, we have to ask: what’s the purpose? Generally aircraft try to get up to their cruise speed quickly and then spend most of their time cruising, and you optimize for cruise first and takeoff second. Do we want multiple cruise speeds, eg a supersonic bomber that goes slow some of the time and fast over enemy territory? Are we designing a supersonic transport and trying to reduce fuel usage getting up to cruise?
And then, there are 2 basic ways you can change the bypass ratio: you can change the fan/propeller intake area, or you can turn off turbines. The V-22 has a driveshaft through the wing to avoid crashes if an engine fails; in theory you could turn off an engine while powering the same number of propellers, which is sort of like a variable bypass ratio. If you have a bunch of turbogenerators inside the fuselage, powering electric fans elsewhere, then you can shut some down while powering the same number of fans. There are also folding propellers.
Yes, helium costs would be a problem for large-scale use of airships. Yes, it’s possible to use hydrogen in airships safely. This has been noted by many people.
Hydrogen has some properties that make it relatively safe:
it’s light so it rises instead of accumulating on the ground or around a leak
it has a relatively high ignition temperature
and some properties that make it less safe:
it has a wide range of concentrations where it will burn in air
fast diffusion, that is, it mixes with air quickly
it leaks through many materials
it embrittles steel
it causes some global warming if released
Regardless, the FAA does not allow using hydrogen in airships, and I don’t expect that to change soon. Especially since accidents still happen despite the small number of airships.
In any case, the only uses of airships that are plausibly economical today are: advertising and luxury yachts for the wealthy. Are those things that you care about working towards?
bhauth
bhauth.com
This won’t find deception in mesaoptimizers, right?
That conflicts with eg:
Anyway, I already answered that in 9. diamond.
Yes, this is part of why I didn’t post AI stuff in the past, and instead just tried to connect with people privately. I might not have accomplished much, but at least I didn’t help OpenAI happen or shift the public perception of AI safety towards “fedora-wearing overweight neckbeards”.
I wrote a related post.
Yes, if you meant TSMC.
...so? More importantly, Intel is down 50% from early 2024.
Your document says:
I agree that, if those rules are followed, AI alignment is feasible in principle. The problem is, some people won’t follow those rules if they have a large penalty to AI capabilities, and I think they will.
“Mirror life” is beyond the scope of this post, and the concerns about it are very different than the concerns about “grey goo”—it doesn’t have more capabilities or efficiency, it’s just maybe harder for immune systems to deal with. Personally, I’m not very worried about that and see no scientific reason for the timing of the recent fuss about it. If it’s not just another random fad, the only explanation I can see for that timing is: influential scientists trying to hedge against Trump officials determining that “COVID was a lab leak” in a way that doesn’t offend their colleagues. On the other hand, I do think artificial pathogens in general are a major concern, and even if I’m not very concerned about “mirror life”, there are no real benefits to trying to make it, so maybe just don’t.
I think this is a pretty good post that makes a point some people should understand better. There is, however, something I think it could’ve done better. It chooses a certain gaussian and log-normal distribution for quality and error, and the way that’s written sort of implies that those are natural and inevitable choices.
I would have preferred something like:
This was a quick and short post, but some people ended up liking it a lot. In retrospect I should’ve written a bit more, maybe gone into the design of recent running shoes. For example, this Nike Alphafly has a somewhat thick heel made of springy foam that sticks out behind the heel of the foot, and in the front, there’s a “carbon plate” (a thin sheet of carbon fiber composite) which also acts like a spring. In the future, there might be gradual evolution towards more extreme versions of the same concept, as recent designs become accepted. Running shoes with a carbon plate have become significantly more common over the past few years. That review says:
So that’s the running-optimized version of shoes with springs using modern materials, while I was writing more about high heels worn for fashion.
Biomechanics is a topic I could write a lot about, but that would be a separate post. On the general topic of “walking” I also wrote this post. (japanese version here)
I’ve learned even more chemistry and biology, and I’ve changed my mind about lots of things, but not the points in this post. Those had solid foundations I understood well and redundant arguments, so the odds of that were low.
The post seems OK. I could have handled replies to comments better. For example, the top comment was by Thomas Kwa, and I replied to part of it as follows:
I didn’t know in advance which comments would be popular. In retrospect, maybe I should’ve gone into explaining the basics of entropy and enthalpy in my reply, eg:
When you write a post that gets comments from many people, it’s not practical to respond to them all. If you try to, you have less time than the collective commenters, and less information about their position than they have about yours. So you have to guess about what exactly each person is misunderstanding, and that’s not usually something I enjoy.
Of the 7 (!) posts of mine currently nominated for “Best of 2023”, this is probably the most appropriate for that.
Of the 2023 posts of mine not currently nominated, my personal favorites were probably:
https://www.lesswrong.com/posts/BTnqAiNoZvqPfquKD/resolving-some-neural-network-mysteries
https://www.lesswrong.com/posts/PdwQYLLp8mmfsAkEC/magnetic-cryo-ftir
https://www.lesswrong.com/posts/rSycgquipFkozDHzF/ai-self-improvement-is-possible
https://www.lesswrong.com/posts/dTpKX5DdygenEcMjp/neuron-spike-computational-capacity
https://www.lesswrong.com/posts/6Cdusp5xzrHEoHz9n/faster-latent-diffusion
Clearly my opinion of my own posts doesn’t correlate with upvotes here that well.
My all-time best post in my view is probably: https://bhauth.com/blog/biology/alzheimers.html
Muireall Prase wrote this, and my post was relevant for some conversations on twitter. I suppose it also convinced some people I had some understanding of chemistry.
So, I have a lot of respect for Sarah, I think this post makes some good points, and I upvoted it. However, my concern is, when I look at this particular organization’s Initiatives page, I see “AI for math”, “AI for education”, “high-skill immigration assistance”, and not really anything that distinguishes this organization from the various other ones working on the same things, or their projects from a lot of past projects that weren’t really worthwhile.
Note that due to the difference being greater at higher frequencies, the effect on speech intelligibility will probably be greater for most women than for you.
We can see the diaphragm has some resonance peaks that increase distortion. Probably it’s too thick to help very much, but it has to resist the pressure changes from breathing.
What exactly are people looking for from (the site-suggested) self-reviews?
As a “physicist and dabbler in writing fantasy/science fiction” I assume you took the 10 seconds to do the calculation and found that a 1km radius cylinder would have ~100 kW of losses per person from roller bearings supporting it, for the mass per person of the ISS. But I guess I don’t understand how you expect to generate that power or dissipate that heat.
Air resistance.
That is, however, basically the system I proposed near the end, for use near the center of a cylinder where speeds would be low.
picture: https://en.wikipedia.org/wiki/Explorer_1#/media/File:Explorer1.jpg
That works well enough, but a Vital 200S currently costs $160 at amazon, less than the cheapest variant of the thing you linked, and has a slightly higher max air delivery rate, some granular carbon in the filter, and features like power buttons. The Vital 200S on speed 2 has similar power usage and slightly less noise, but less airflow, but a carbon layer always reduces airflow. It doesn’t have a rear intake so it can be placed against a wall. It also has a washable prefilter.
Compared to what you linked, the design in this post has 3 filters instead of 2, some noise blocking, and a single large fan instead of multiple fans. Effective floor area usage should be slightly less, but of course it has to go together with shelving for that.
First, we have to ask: what’s the purpose? Generally aircraft try to get up to their cruise speed quickly and then spend most of their time cruising, and you optimize for cruise first and takeoff second. Do we want multiple cruise speeds, eg a supersonic bomber that goes slow some of the time and fast over enemy territory? Are we designing a supersonic transport and trying to reduce fuel usage getting up to cruise?
And then, there are 2 basic ways you can change the bypass ratio: you can change the fan/propeller intake area, or you can turn off turbines. The V-22 has a driveshaft through the wing to avoid crashes if an engine fails; in theory you could turn off an engine while powering the same number of propellers, which is sort of like a variable bypass ratio. If you have a bunch of turbogenerators inside the fuselage, powering electric fans elsewhere, then you can shut some down while powering the same number of fans. There are also folding propellers.
The question is always, “but is that better”?
Yes, helium costs would be a problem for large-scale use of airships. Yes, it’s possible to use hydrogen in airships safely. This has been noted by many people.
Hydrogen has some properties that make it relatively safe:
it’s light so it rises instead of accumulating on the ground or around a leak
it has a relatively high ignition temperature
and some properties that make it less safe:
it has a wide range of concentrations where it will burn in air
fast diffusion, that is, it mixes with air quickly
it leaks through many materials
it embrittles steel
it causes some global warming if released
Regardless, the FAA does not allow using hydrogen in airships, and I don’t expect that to change soon. Especially since accidents still happen despite the small number of airships.
In any case, the only uses of airships that are plausibly economical today are: advertising and luxury yachts for the wealthy. Are those things that you care about working towards?