Running Lightcone Infrastructure, which runs LessWrong and Lighthaven.space. You can reach me at habryka@lesswrong.com.
(I have signed no contracts or agreements whose existence I cannot mention, which I am mentioning here as a canary)
habryka
Karma: 43,730
Yeah, that’s the case I find most compelling. I think the key thing that makes me not sold on this being a defeater, even if swarm tactics dominate, is just the ability for the U.S. to direct it’s extremely strong and powerful open market to this problem. My guess is if the U.S. was buying military drones from private U.S. companies en-masse, we would see enormous scale-up, and my guess is more responsively than the Chinese economy would, since the market is healthier. I am not sure of this, but this is how it’s gone in many other domains.
I didn’t listen closely to the lyrics two weeks ago, but I’m finding time now. “Friendly Fire” is particularly affecting. I don’t see a co-author listed, no obvious hits on the language. Was that you? How are you doing?
All the new songs are written by me with original lyrics! (with the exception of Dance of the Doomsday Clock which was written and created by Ben)
I am doing well! Though the feelings expressed in Friendly Fire sure are a lot of what I am thinking about these days.
Note: “You Have Not Been A Good User” seems to be missing, with 7 seconds of silence as a placeholder.
Yep, that one isn’t on Suno. You can go to the relevant section of this video to listen to it:
This proves too much. If you consistently require there be no “serious personal and professional consequences” before you trust a source, you’d have to dismiss almost all of them.
I think the heuristic of “do not trust a source to accurately report X if it faces serious personal and professional consequences for many plausible beliefs about X” is not a particularly weird heuristic? That seems extremely normal to me, and I am confused what’s going on here. Most people, especially in the US do not face serious personal and professional consequences for most beliefs they express, and when they do, you should absolutely dismiss them as a source.
Yes, I would predict that. My understanding for high-end military drones, which to be clear cost $100k+ each, the US is undisputedly the world leader. You linked to a random subreddit for consumer drones, which of course have almost nothing to do with the specific point of the U.S. being ahead on the cutting edge frontier.
My understanding is that American military technology is extremely expensive, and also at the frontier miles ahead of the competition. The thing you linked at are not at all in a comparable market (and again, yes, if mass production might turn out to be a bottleneck things are different, but I am disputing the cutting-edge point, not the mass production point).
I don’t understand? Russia does not have stable rule of law. If you praise the US military as a Russian military official, you would almost certainly face serious personal and professional consequences, this just seems really obvious from how Russia operates.
I agree with many of these things, but of course, bad commenters have driven away many more commenters and authors than banning has driven good commenters away. I don’t think the current ban-system is generally causing serious, effective critics to be banned, and on-net is increasing the number of serious and effective critics.
If we had a better karma system, I think there are some tools that I might want to make available to people that are better than banning. Maybe things like karma thresholds, or some other way of making it so a user needs to be in particular good standing to leave a comment. But unfortunately, our current karma system is not robust enough for that, and indeed, leaving many bad comments, is still unfortunately a way to get lots of karma.
The feature was not designed for this purpose, it was mostly designed so that people who are interested in LW can generally see what kind of moderation actions are happening. I agree that if banning was more frequent I would add a more specific list (which is what I said, and you seem to have just ignored).
I don’t think a full list makes sense, just because of clutter, but a number seems pretty reasonable (and ideally some way of highlighting the UI element if indeed there is some kind of relevant thing happening).
Yeah, I agree this would be bad if it happened. I don’t currently think it’s happening, but see my response to sunwillrise on what I would do if it turned out to be an issue.
I also am really not interested in this discussion with you in-particular. You alone are like 30% of the reason why a ban system on this site is necessary. I think this site might have literally died if we had you and not a ban system, so it appears to me that you in-particular seem to particularly fail to understand the purpose of a ban system. I could not in good faith encourage someone to post on LW if they did not have the option of banning you and an extremely small number of other users from their post.
Makes sense! I hadn’t read that subthread, so was additionally confused.
But they are not acting badly enough that I confidently assume bad faith
I like a lot of your comment, but this feels like a total non-sequitur. Did anyone involved in this conversation say that Anthropic was acting under false pretenses? I don’t think anyone brought up concerns that rest on assumptions of bad faith (though to be clear, Anthropic employees have mostly told me I should assume something like bad faith from Anthropic as an institution, and that people should try to hold it accountable the same way any other AI lab, and to not straightforwardly trust statements Anthropic makes without associated commitments, so I do think I would assume bad faith, but it mostly just feels besides the point in this discussion).
The thing that it changes is the degree to which the author’s popularity or quality is being used to give a platform to other people, which I think makes a lot of sense to give the author some substantial control over. If you have a critique that people care about, you can make a top-level post, and if it’s good it can stand on its own and get its own karma.
If it’s really important for a critique to end up directly associated with a post, you can just ask someone who isn’t banned to post a link to it under that post. If you can’t find anyone who thinks it’s worth posting a link to it who isn’t yourself, then I think it’s not that sad for your critique to not get seen.
Yes, this puts up a few trivial inconveniences, but the alternative of having people provide a platform to anyone without any choice in the matter, whose visibility gets multiplied proportional to their own reach and quality, sucks IMO a lot more.
My engagement with LW is kind of weird and causes me to not write as many top-level posts as I like, and the ones I do write are site announcements, but if I was trying to be a more standard author on LW, I wouldn’t use it without the ability to ban (and for example, would almost certainly ban Said, who has been given multiple warnings by the moderation team, is by far the most heavily complained user on the site, and I would actively encourage many authors to ban if they don’t want to have a kind of bad time, but I think is providing enough value to the site in other contexts that I don’t think a site-wide ban would make sense).
Otherwise, I’d suspect a large part of the readers of the post will not even know there is such a response disagreeing with the original (because they just stumbled upon the post, or they clicked on a link to it from elsewhere on LW, or were just browsing some tag or the “recent activity” tab, etc).
I think some UI thing in the space wouldn’t be crazy. If banning was going to be used more frequently, it’s something I would consider building (I would put it at the bottom of the comment section, but that’s still a reasonable place to find it).
(Not to mention that posts don’t even have a sticker at the bottom saying “the author has banned the following users from commenting on their posts: [...]”, which should absolutely appear if the point of allowing authors to ban commenters was actually to improve the record and discourse.
We have something kind of close. At the bottom of every comment section you see a link to the moderation log, which allows you to see who is banned from whose posts. If banning was a thing that happened reasonably frequently, changing it to say “Moderation Log (3 users banned from this post, 2 comments deleted)” or something like that, would be reasonable to me. But it happens so rarely that I currently don’t think it’s worth the development effort (but if people really care, I would accept a PR that adds something like that).
That video seems like a kind of terrible source to me. A russian drone developer isn’t going to be neutral on the state of U.S. drone manufacturing, he might literally face persecution if he praises the U.S. military.
(The technological areas where the US does seem ahead (i.e., say, quiet nuclear submarine technologies) are areas where the US has been manufacturing actively for 80 years, and where we don’t have a history of manufacturing in China; but even this isn’t a guarantee, as a handful of smaller, cheaper unmanned subs sidestep this advantage, in the same way they can sidestep other things.)
I thought the U.S. was also ahead in fighter jet manufacturing, missile manufacturing, aircraft manufacturing, aircraft carrier design and capacity, and many other things that seem like they would more directly translate into drone manufacturing. In as much as I am wrong about that, that would be a substantial update, but my sense is despite its pathologies, the U.S. is really where a huge fraction of cutting edge military technology gets developed and built, in basically every domain.
In general this story of “most new cutting-edge stuff comes from refinement and so if you make a lot you will also make the best” really doesn’t seem true to me. The U.S. produces the best software for approximately every single domain, even if the industry in which that software is used is much smaller in the U.S. than anywhere else. A far better predictor of whether you will produce the cutting edge stuff is whether you have an industry specialized in producing the cutting edge stuff. China and India have been copying American innovation for decades in dozens of industries, from software, to medical, to manufacturing, to construction, and they have not generally been the drivers of innovation in those domains, despite their markets for those things being much larger than the U.S.
I think whether you have a healthy industry that incentivizes innovation and can build new things will be much more indicative of who will be at the frontier here (as it’s been in basically every other industry). The strongest argument against this mattering in the drone case is that volume of production is more important, but my guess is the U.S. is in a better position to incentivize large volume production of drones than China, because the U.S. has a functioning market economy where the U.S. can incentivize things by paying for them, in a way that China cannot reliably.
I don’t think “number of drones produced” is a good proxy for “aggregate quality and usefulness of drones produced if a country decided it’s important”.
I thought the U.S. had by far the world’s most advanced military manufacturing industry, with approximately all cutting edge military technologies (including most drone designs) being developed here. Seems like this would apply straightforwardly to drones. There is possibly an unspoken argument here that drones do not require much technological innovation to make good, or less technological sophistication, as it’s more important to just mass produce them, but I don’t currently buy it. In as much as drones will be a really crucial military technology, I expect you will get substantial returns to quality, and the U.S. won’t be bottlenecked on literal volume of production.
Banning someone does not generally silence their harshest critics. It just asks those critics to make a top-level post, which generally will actually have any shot at improving the record and discourse in reasonable ways compared to nested comment replies.
The thing that banning does is make it so the author doesn’t look like he is ignoring critics (which he hasn’t by the time he has consciously decided to ban a critic).
My guess would be the US is in a much better position to build large numbers of cutting-edge drones? Am I missing something?
Just as a specific prediction, does this mean you expect we will very substantially improve the cheating/lying behavior of current RL models? It’s plausible to me, though I haven’t seen any approach that seems that promising to me (and it’s not that cruxy for my overall takeoff beliefs). Right now, I would describe the frontier thinking models as cheating/lying on almost every response (I think approximately every time I use o3 it completely makes up some citation or completely makes up some kind of quote).
I think there are few people who have beliefs as considered on this topic as I do! And maybe no one who has as much evidence as I do (which doesn’t mean I am right, people come to dumb beliefs while being exposed to lots of evidence all the time).
I’ve conducted informal surveys, have done hundreds of user interviews, have had conversations about their LessWrong posting experienes with almost every core site contributor over the years, have had conversations with hundreds of people who decided not to post on LW but instead post somewhere else, conversations with people who moved from other platforms to LW, conversations with people who moved from LW to other platforms, and many more.
I have interviewed people in charge of handling these tradeoffs at many of the other big content platforms out there, as well as dozens of people who run smaller forums and online communities. I have poured over analytics and stats and graphs trying to understand what causes people to write here instead of other places, and what causes them to grow as both a commenter and writer.
All of these form a model of how things work that suggests to me that yes, it is true that bad commenters drive away many more good critics than our current threat of banning does.
I have been working on understanding the dynamics here now for almost a full decade, with really a lot of my time. I absolutely could know that, in the same way we know many many things that we cannot directly observe.
I didn’t put an explicit probability on this, and the exact probability would differ based on the exact operationalization, but IDK, it’s my current belief with like 85% probability.
Again, I don’t really want to continue this conversation with you, so please choose somewhere else to make this kind of comment.