I’m an independent researcher currently working on a sequence of posts about consciousness. You can send me anonymous feedback here: https://www.admonymous.co/rafaelharth. If it’s about a post, you can add [q] or [nq] at the end if you want me to quote or not quote it in the comment section.
Rafael Harth
Karma: 4,929
I probably shouldn’t have clicked on this but I did. FYI it slightly raised my opinion of Kelsey because I was expecting some level of incriminating evidence against her, and this is well below that level. If the worst thing about someone is that they did not write about a thing, that’s a pretty good sign.
Another quick way to get at my skepticism for LLM capability: the ability do differentiate good from bad ideas. ImE the only way LLMs can do this (or at least seem to do this) is the prevailing views are different depending on context—like if there’s an answer popular among lay people and a better answer popular among academics, and your tone makes it clear that you belong in the second category, then it will give you the second one, and that can make it seem like it can tell good and bad takes apart. But this clearly doesn’t count. If the most popular view is the same across contexts it’ll echo that view no matter how dumb it is. I’d guess most people here have rolled their eyes before at GPT regurgitating bad popular takes.
If there’s an update to LLMs where you can ask it a question and it’ll just outright tell you that the prevailing view on this is dumb (and it is in fact dumb) then I shall be properly terrified. Of course this is another non-objective benchmark, but as I said in my post, I think being not-objective may just be a property that all the actually important milestones have.
No I definitely think thought assessment has more to it than just attention. In fact I think you could argue that LLMs’ attention equivalent is already more powerful/accurate than human attention.
The only evidence I can provide at this point is the similarity of LLMs to humans who don’t pay attention (as first observed in Sarah’s post that I linked in the text). If you want to reject the post based on the lack of evidence for this claim, I think that’s fair.
I think we have specialized architectures for consciously assessing thoughts, whereas LLMs do the equivalent of rattling off the first thing that comes to mind, and reasoning models do the equivalent of repeatedly feeding back what comes to mind into the input (and rattling off the first thing that comes to mind for that input).
Trump says a lot of stuff that he doesn’t do, the set of specific things that presidents don’t do is larger than the set of things they do, and tariffs didn’t even seem like they’d be super popular with his base if in fact they were implemented. So “~nothing is gonna happen wrt tariffs” seemed like the default outcome with not enough evidence to assume otherwise.
I was also not paying a lot of attention to what he was saying. After the election ended, I made a conscious decision to tune out of politics to protect my mental health. So it was a low information take—but I don’t know if paying more attention would have changed my prediction. I still don’t think I actually know why Trump is doing the tariffs, especially to such an extreme extent..
I’ve also noticed this assumption. I myself don’t have it, at all. My first thought has always been something like “If we actually get AGI then preventing terrible outcomes will probably require drastic actions and if anything I have less faith in the US government to take those”. Which is a pretty different approach from just assuming that AGI being developed by government will automatically lead to a world with values of government . But this a very uncertain take and it wouldn’t surprise me if someone smart could change my mind pretty quickly.
Yeah.
These are very poor odds, to the point that they seem to indicate a bullish rather than a bearish position on AI.
If you think the odds of something are , but lots of other people think they are with , then the rational action is not to offer bets at a point close to ; it’s to find the closest number to possible. Why would you bet at 1:5 odds if you have reason to believe that some people would be happy to bet at 1:7 odds?
You could make an argument that this type of thinking is too mercenary/materialistic or whatever, but then critique should be about that. In any case the inference that offering a bet close to indicates beliefs close to is just not accurate.
I’m glad METR did this work, and I think their approach is sane and we should keep adding data points to this plot.
It sounds like you also think the current points on the plot are accurate? I would strongly dispute this, for all the reasons discussed here and here. I think you can find sets of tasks where the points fit on an exponential curve, but I don’t think AI can do 1 hour worth of thinking on all, or even most, practically relevant questions.
In the last few months, GPT models have undergone a clear shift toward more casual language. They now often close a post by asking a question. I strongly dislike this from both a ‘what will this do to the public’s perception of LLMs’ and ‘how is my personal experience as a customer’ perspective. Maybe this is the reason to finally take Gemini seriously.
I unfortunately don’t think this proves anything relevant. The example just shows that there was one question where the market was very uncertain. This neither tells us how certain the market is in general (that depends on its confidence on other policy questions), nor how good this particular estimate was (that, I would argue, depends on how far along the information chart it was, which is not measurable—but even putting my pet framework aside, it seems intuitively clear “it was 56% and then it happened” doesn’t tell you how much information the market utilized).
The point is that even if voters did everything right and checked prediction markets as part of their decision making algorithm, it wouldn’t help.
This depends on the first point, which again requires looking at a range of policy markets, not just one. And actually, I personally didn’t expect Trump to do any tarrifs at all (was 100% wrong there), so for me, the market would have updated me significantly into the right direction.
Thanks. I’ve submitted my own post on the ‘change our mind form’, though I’m not expecting a bounty. I’d instead be interested in making a much bigger bet (bigger than Cole’s 100 USD), gonna think about what resolution criterion is best.
I might be misunderstanding how this works, but I don’t think I’m gonna win the virtue of The Void anytime soon. Or at all.
Yeah, valid correction.
If people downvoted because they thought the argument wasn’t useful, fine—but then why did no one say that? Why not critique the focus or offer a counter? What actually happened was silence, followed by downvotes. That’s not rational filtering. That’s emotional rejection.
Yeah, I do not endorse the reaction. The situation pattern-matches to other cases where someone new writes things that are so confusing and all over the place that making them ditch the community (which is often the result of excessive downvoting) is arguably a good thing. But I don’t think this was the case here. Your essays look to me to be coherent (and also probably correct). I hadn’t seen any of them before this post but I wouldn’t have downvoted. My model is that most people are not super strategic about this kind of thing and just go “talking politics → bad” without really thinking through whether demotivating the author is good in this case.
So if I understand you correctly: you didn’t read the essay, and you’re explaining that other people who also didn’t read the essay dismissed it as “political” because they didn’t read it.
Yes—from looking at it, it seems like it’s something I agree with (or if not, disagree for reasons that I’m almost certain won’t be addressed in the text), so I didn’t see a reason to read. I mean reading is a time investment, you have to give me a reason to invest that time, that’s how it works. But I thought the (lack of) reaction was unjustified, so I wanted to give you a better model of what happened, which also doesn’t take too much time.
Most people say capitalism makes alignment harder. I’m saying it makes alignment structurally impossible.
The point isn’t to attack capitalism. It’s to explain how a system optimised for competition inevitably builds the thing that kills us.
I mean that’s all fine, but those are nuances which only become relevant after people read, so it doesn’t really change the dynamic I’ve outlined. You have to give people a reason to read first, and then put more nuances into the text. Idk if this helps but I’ve learned this lesson the hard way by spending a ridiculous amount of time on a huge post that was almost entirely ignored (this was several years ago).
(It seems like you got some reactions now fwiw, hope this may make you reconsider leaving.)
I think you probably don’t have the right model of what motivated the reception. “AGI will lead to human extinction and will be built because of capitalism” seems to me like a pretty mainstream position on LessWrong. In fact I strongly suspect this is exactly what Eliezer Yudkowsky believes. The extinction part has been well-articulated, and the capitalism part is what I would have assumed is the unspoken background assumption. Like, yeah, if we didn’t have a capitalist system, then the entire point about profit motives, pride, and race dynamics wouldn’t apply. So… yeah, I don’t think this idea is very controversial on LW (reddit is a different story).
I think the reason that your posts got rejected is that the focus doesn’t seem useful. Getting rid of capitalism isn’t tractable, so what is gained by focusing on this part of the causal chain? I think that’s the part your missing. And because this site is very anti-[political content], you need a very good reason to focus on politics. So I’d guess that what happened is that people saw the argument, thought it was political and not-useful, and consequently downvoted.
Sorry, but isn’t this written by an LLM? Especially since milan’s other comments ([1], [2], [3]) are clearly in a different style, the emotional component goes from 9⁄10 to 0⁄10 with no middle ground.
I find this extremely offensive (and I’m kinda hard to offend I think), especially since I’ve ‘cooperated’ with milan’s wish to point to specific sections in the other comment. LLMs in posts is one thing, but in comments, yuck. It’s like, you’re not worthy of me even taking the time to respond to you.
The guidelines don’t differentiate between posts and comments but this violates them regardless (and actually the post does as well) since it very much does have the stereotypical writing style of an AI assistant, and the comment also seems copy-pasted without a human element at all.
A rough guideline is that if you are using AI for writing assistance, you should spend a minimum of 1 minute per 50 words (enough to read the content several times and perform significant edits), you should not include any information that you can’t verify, haven’t verified, or don’t understand, and you should not use the stereotypical writing style of an AI assistant.
The sentence you quoted is a typo, it’s is meant to say that formal languages are extremely impractical.
Not the OP, but here is an attempt.
If people with realist intuitions argue against direct realism, they usually start by taking the existence of mental objects (such as images) at face value. Their argument usually goes something like “well clearly those mental objects can’t be the external objects because xyz”, where xyz can be about the brain’s processing pipeline or about visual illusions. This xyz stuff is in fact pretty hard to argue with, which makes the entire thing look a little silly/trivial.
However the actual controversial step here is completely unacknowledged, which is to assume that mental objects exist at all. A Dennettian-aligned skeptic can simply argue that our talk of seeing visual images is just a clunky/confused way to describe the limited access we have to our visual processing pipeline—which is in fact something like a highly sparse representation, or maybe even a list of features—and nothing like a 2d pixel image (or even vector image) is ever actually created.
Under this view, insofar as “what we see” refers to any object at all, well it kind of does refer to the external physical object. Not in the sense that we have some magical ability to read off real properties of the physical object, but just in the sense that there is nothing else causally upstream of our reports about seeing visual images.
So what about visual illusions? Well, visual illusions are cases where our reports aren’t accurate. But so what? This just means we’re making a mistake in describing the external thing, not that we’re describing anything else. I don’t think that the frequency of illusions or the magnitude of the mistake is actually all that relevant.
My hot take is that realists generally don’t understand this and that’s why virtually every post arguing against direct realism fails to address the actual crux.