Yeah it’s pretty bad. There are some professors who are better than just reading the textbook, but unfortunately they’re the exceptions. My undergrad experience got a lot more productive once I started picking my courses based on the *teacher* more than on the *subject*.
David Hornbein
Karma: 1,292
Maybe Shear was lying. Maybe the board lied to Shear, and he truthfully reported what they told him. Maybe “The board did *not* remove Sam over any specific disagreement on safety” but did remove him over a *general* disagreement which, in Sutskever’s view, affects safety. Maybe Sutskever wanted to remove Altman for a completely different reason which also can’t be achieved after a mass exodus. Maybe different board members had different motivations for removing Altman.
My guess: Sutskever was surprised by the threatened mass exodus. Whatever he originally planned to achieve, he no longer thinks he can succeed. He now thinks that falling on his sword will salvage more of what he cares about than letting the exodus happen.
I note that the comments here include a lot of debate on the implications of this post’s thesis and on policy recommendations and on social explanations for why the thesis is true. No commenter has yet disagreed with the actual thesis itself, which is that this paper is a representative example of a field that is “more advocacy than science”, in which a large network of Open Philanthropy Project-funded advocates cite each other in a groundless web of footnotes which “vastly misrepresents the state of the evidence” in service of the party line.
I can hail a robotaxi. So can anyone living in San Francisco, Phoenix, Beijing, Shanghai, Guangzhou, Shenzhen, Chongqing or Wuhan. The barriers to wider rollout are political and regulatory, not technological.
Waymo cars, and I believe Apollo and Cruise as well, are “level 4” autonomous vehicles, i.e. there is no human involved in driving them whatsoever. There is a “human available to provide assistance” in roughly the same sense that a member of the American Automobile Association has an offsite human available to provide assistance in case of crashes, flat tires, etc.
I don’t see any reason to think AGI is imminent but this particular argument against it doesn’t go through. Robotaxi tech is very good and improving swiftly.
Seems also worth mentioning that Gretta Duleba and Eliezer Yudkowsky are in a romantic relationship, according to Yudkowsky’s Facebook profile.
At the risk of butting in, I also didn’t participate because none of the options reflected my own views on the important virtue of Petrov Day, which is more like “Do not start a fucking nuclear war”. You can try to distill that down to nicely categorized principles and virtues, and some of those might well be good things on their own, but at this level of abstraction it doesn’t capture what’s special about Petrov Day to me.
Trying to file down the Petrov Day narrative into supporting some other hobbyhorse, even if it’s a wonderful hobbyhorse which I otherwise support like resisting social pressure, is a disservice to what Stanislav Petrov actually did. The world is richer and more complex than that.
I personally preferred the past Petrov Day events, with the button games and standoffs between different groups and all that. They didn’t perfectly reflect the exact dilemma Petrov faced, sure, but what could. They were messy and idiosyncratic and turned on weird specific details. That feels like a much closer reflection of what makes Petrov’s story compelling, to me. Maybe the later stages of this year’s event would’ve felt more like that, if I’d seen them at the time, but reading the description I suspect probably not.
I like that you guys are trying a bunch of different stuff and it’s fine if this one thing didn’t land for me.
One very common pattern is, most people oppose a technology when it’s new and unfamiliar, then later once it’s been established for a little while and doesn’t seem so strange most people think it’s great.
Offering a specific amount of pay, in cash and in kind, and then not doing the accounting to determine whether or not that amount was actually paid out. If I’m charitable to the point of gullibility, then this is unethical and culpable negligence. Probably it’s just fraud. (Assuming this allegation is true, of course, and
AFAIK it is not yet disputed.)Screenshots of threats to retaliate for speaking up.
EDIT: Nonlinear has now replied and disputed many of the allegations. I am persuaded that the allegation of fraud/negligence around payment is simply false. As for the screenshots of threats to retaliate, my opinion is that retaliation or threats to retaliate are perfectly justified in the face of the behavior which Nonlinear alleges. Nonlinear also provides longer chatlogs around one of the screenshotted texts which they argue recontextualizes it.
think about how bad you expect the information would be if I selected for the worst, credible info I could share
Alright. Knowing nothing about Nonlinear or about Ben, but based on the rationalist milieu, then for an org that’s weird but basically fine I’d expect to see stuff like ex-employees alleging a nebulously “abusive” environment based on their own legitimately bad experiences and painting a gestalt picture that suggests unpleasant practices but without any smoking-gun allegations of really egregious concrete behavior (as distinct from very bad effects on the accusers); allegations of nepotism based on social connections between the org’s leadership and their funders or staff; accusations of shoddy or motivated research which require hours to evaluate; sources staying anonymous for fear of “retaliation” but without being able to point to any legible instances of retaliation or concrete threats to justify this; and/or thirdhand reports of lying or misdirection around complicated social situations.
[reads post]
This sure has a lot more allegations of very specific and egregious behavior than that, yeah.
EDIT: Based on Nonlinear’s reply and the thorough records they provide, it seems that the smoking-gun allegations of really egregious concrete behavior are probably just false. This leaves room for unresolvable disagreement on the more nebulous accusations, but as I said initially, that’s the pattern I’d expect to see if Nonlinear were weird but basically fine.
I have seen many people try to become more ambitious. I have also seen many people in fact become more ambitious. But I don’t know that I’ve ever seen anyone try to become more ambitious and succeed, leaving aside the sort of thing you call “hoops” and “fake ambition”.
There’s this thing, this spark, you could call it “independent ambition” or being “agenty” or “strategic”, and other subcultures have other words. But whatever you call it, you can’t get it by aiming at it directly. When I’ve seen someone get it, she gets it by aiming at something else, and if there happens to be a path to her deeper goal just by jumping through the right hoops—well, that’s much better, actually, it’s more reliable and usually saves her a tremendous amount of trouble. But if the only way to get her deeper goal is to break away from the usual paths and go do some bizarre novel thing, then sometimes she’ll do that instead. But of course there can be no hoop for breaking free of hoops, there must be some impetus from outside the system of hoops.
Back in the ancient days we called all this stuff “media”.
I don’t understand the logic of this. Does seem like game-theoretically the net-payout is really what matters. What would be the argument for something else mattering?
BEORNWULF: A messenger from the besiegers!
WIGMUND: Send him away. We have nothing to discuss with the norsemen while we are at war.
AELFRED: We might as well hear them out. This siege is deadly dull. Norseman, deliver your message, and then leave so that we may discuss our reply.
MESSENGER: Sigurd bids me say that if you give us two thirds of the gold in your treasury, our army will depart. He reminds you that if this siege goes on, you will lose the harvest, and this will cost you more dearly than the gold he demands.
The messenger exits.
AELFRED: Ah. Well, I can’t blame him for trying. But no, certainly not.
BEORNWULF: Hold on, I know what you’re thinking, but this actually makes sense. When Sigurd’s army first showed up, I was the first to argue against paying him off. After all, if we’d paid right at the start, then he would’ve made a profit on the attack, and it would only encourage more. But the siege has been long and hard for us both. If we accept this deal *now*, he’ll take a net loss. We’ve spent most of the treasury resisting the siege—
WIGMUND: As we should! Millions for defense, but not one cent for tribute!
BEORNWULF: Certainly. But the gold we have left won’t even cover what they’ve already spent on their attack. Their net payout will still be negative, so game-theoretically, it doesn’t make sense to think of it as “tribute”. As long as we’re extremely sure they’re in the red, we should minimize our own costs, and missing the harvest would be a *huge* cost. People will starve. The deal is a good one.
WIGMUND: Never! if once you have paid him the danegeld, you never get rid of the Dane!
BEORNWULF: Not quite. The mechanism matters. The Dane has an incentive to return *only if the danegeld exceeds his costs*.
WIGMUND: Look, you can mess with the categories however you like, and find some clever math that justifies doing whatever you’ve already decided you want to do. None of that constrains your behavior and so none of that matters. What matters is, take away all the fancy definitions and you’re still just paying danegeld.
BEORNWULF: How can I put this in language you’ll understand—it doesn’t matter whether the definitions support what *I* want to do, it matters whether the definitions reflect the *norsemen’s* decision algorithm. *They* care about the net payout, not the gross payout.
AELFRED: Hold on. Are you modeling the norsemen as profit-maximizers?
BEORNWULF: More or less? I mean, no one is perfectly rational, but yeah, everyone *approximates* a rational profit-maximizer.
WIGMUND: They are savage, irrational heathens! They never even study game theory!
BEORNWULF: Come on. I’ll grant that they don’t use the same jargon we do, but they attack because they expect to make a profit off it. If they don’t expect to profit, they’ll stop. Surely they do *that* much even without explicit game theoretic proofs.
AELFRED: That affects their decision, yes, but it’s far from the whole story. The norsemen care about more than just gold and monetary profit. They care about pride. Dominance. Social rank and standing. Their average warrior is a young man in his teens or early twenties. When he decides whether to join the chief’s attack, he’s not sitting down with spreadsheets and a green visor to compute the expected value, he’s remembering that time cousin Guthrum showed off the silver chalice he looted from Lindisfarne. Remember, Sigurd brought the army here in the first place to avenge his brother’s death—
BEORNWULF: That’s a transparent pretext! He can’t possibly blame us for that, we killed Agnarr in self-defense during the raid on the abbey.
WIGMUND: You can tell that to Sigurd. If it had been my brother, I’d avenge him too.
AELFRED: Among their people, when a man is murdered, it’s not a *tragedy* to his family, it’s an *insult*. It can only be wiped away with either a weregeld payment from the murderer or a blood feud. Yes, Sigurd cares about gold, but he also cares tremendously about *personally knowing he defeated us*, in order to remove the shame we dealt him by killing Agnarr. Modeling his decisions as profit-maximizing will miss a bunch of his actual decision criteria and constraints, and therefore fail to predict the norsemen’s future actions.
WIGMUND: You’re overcomplicating this. If we pay, the norsemen will learn that we pay, and more will come. If we do not pay, they will learn that we do not pay, and fewer will come.
BEORNWULF: They don’t care if we *pay*, they care if it’s *profitable*. This is basic accounting.
AELFRED: They *do* care if we pay. Most of them won’t know or care what the net-payout is. If we pay tribute, this will raise Sigurd’s prestige in their eyes no matter how much he spent on the expedition, and he needs his warriors’ support more than he needs our gold. Taking a net loss won’t change his view on whether he’s avenged the insult to his family, and we do *not* want the Norsemen to think they can get away with coming here to avenge “insults” like killing their raiders in self-defense. On the other hand, if Sigurd goes home doubly shamed by failing to make us submit, they’ll think twice about trying that next time.
BEORNWULF: I don’t care about insults. I don’t care what Sigurd’s warriors think of him. I don’t care who can spin a story of glorious victory or who ends up feeling like they took a shameful defeat. I care about how many of our people will die on norse spears, and how many of our people will die of famine if we don’t get the harvest in. All that other stuff is trivial bullshit in comparison.
AELFRED: That all makes sense. You still ought to track those things instrumentally. The norsemen care about all that, and it affects their behavior. If you want a model of how to deter them, you have to model the trivial bullshit that they care about. If you abstract away what they *actually do* care about with a model of what you think they *ought* to care about, then your model *won’t work*, and you might find yourself surprised when they attack again because they correctly predict that you’ll cave on “trivial bullshit”. Henry IV could swallow his pride and say “Paris is well worth a mass”, but that was because he was *correctly modeling* the Parisians’ pride.
WIGMUND: Wait. That is *wildly* anachronistic. Henry converted to Catholicism in 1593. This dialogue is taking place in, what, probably the 9th century?
AELFRED: Hey, I didn’t make a fuss when you quoted Kipling.
“he didn’t end up with more money than he started with after the whole miricult thing” is such a weirdly specific way to phrase things.
My speculation from this is that MIRI paid Helm or his lawyers some money, but less money than Helm had spent on the harassment campaign, and among people who know the facts there is a semantic disagreement about whether this constitutes a “payout”. Some people say something like “it’s a financial loss for Helm, so game-theoretically it doesn’t provide an incentive to blackmail, therefore it’s fine” and others say something like “if you pay out money in response to blackmail, that’s a blackmail payout, you don’t get to move the bar like that”.
I would appreciate it if someone who knows what happened can confirm or deny this.
(AFAICT the only other possibility is that somewhere along the line, at least one of the various sources of contradictory-sounding rumors was just lying-or-so-careless-as-to-be-effectively-lying. Which is very possible, of course, that happens with rumors a lot.)
Yudkowsky says it’s now “short-term publishable, fundable, ‘relatable’ topics affiliated with academic-left handwringing”
I assume this means, like, Timnit Gebru and friends.
If you want people to stop calling doomers “doomers”, you need to provide a specific alternative. Gesturing vaguely at the idea of alternatives isn’t enough. “Thou shalt not strike terms from others’ expressive vocabulary without suitable replacement.”
Doomers used to call themselves the “AI safety community” or “AI alignment community”, but Yudkowsky recently led a campaign to strike those terms and replace them with “AI notkilleveryoneism”. Unfortunately the new term isn’t suitable and hasn’t been widely adopted (e.g. it’s not mentioned in the OP), which leaves the movement without a name its members endorse.
People are gonna use *some* name for it, though. A bunch of people are spending tens of millions of dollars per year advocating for a very significant political program! Of course people will talk about it! So unless and until doomers agree on a better name for themselves (which is properly the responsibility of the doomers, and not the responsibility of their critics) my choices are calling it “AI safety” and getting told that no, that’s inaccurate, “AI safety” now refers to a different group of people with a different political program, or else I can call it “doomers” and get told I’m being rude. I don’t want to be inaccurate or rude, but if you make me pick one of the two, then I’ll pick rude, so here we are.
If the doomers were to agree on a new name and adopt it among themselves, I would be happy to switch. (Your “AI pessimist” isn’t a terrible candidate, although if it caught on then it’d be subject to the same entryism which led Yudkowsky to abandon “AI safety”.) Until then, “doomer” remains the most descriptive word, in spite of all its problems.
That’s correct.
I guess if they found themselves in a similar situation then I’d *want* them to ask me for help. I have a pretty easy time saying no to people and wouldn’t feel bad about sympathetically rejecting a request if that felt like the right call, and maybe that’s not the norm, idk. But in any case, I offered one-off gifts, and it was always understood that way.
Once per. That’s not a policy, I just haven’t yet felt moved to do it twice, so I haven’t really thought about it.
I’ve actually done this and it worked incredibly well, so I’m not persuaded by your vague doubts that it’s possible. If you insist on using “opinion of established organizations” as your yard stick then I’ll add that a strong majority of the people I supported would later go on to get big grants and contracts from well-respected organizations, always after years of polishing and legibilizing the projects which I’d supported in their infancy.
Certainly it wouldn’t work for the median person on Earth. But “LW reader who’s friends with a bunch of self-starting autodidacts and has enough gumption to actually write a check” is not the median person on Earth, and people selected that hard will often know some pretty impressive people.
I really don’t think you can justify putting this much trust in the NYT’s narrative of events and motivations here. Like, yes, Toner did publish the paper, and probably Altman did send her an email about it. Then the NYT article tacitly implies but *doesn’t explicitly say* this was the spark that set everything off, which is the sort of haha-it’s-not-technically-lying that I expect from the NYT. This post depends on that implication being true.