In some of his books on evolution, Dawkins also said very similar things when commenting on Darwin vs Wallace, basically saying that there’s no comparison, Darwin had a better grasp of things, justified it better and more extensively, didn’t have muddled thinking about mechanisms, etc.
Lukas_Gloor
Karma: 3,179
We might be missing some key feature of AI takeoff; it’ll probably seem like “we could’ve seen this coming”
Very cool! I used to think Hume was the most ahead of his time, but this seems like the same feat if not better.
Yeah, you need an enormous bankroll to play $10,000 tournaments. What a lot of pros do is sell action. Let’s say you’re highly skilled and have a, say, 125% expected return on investment. If you find someone with a big bankroll and they’re convinced of your skills, you can you sell them your action at a markup somewhere between 1 and 1.2 to incentivize them to make a profit. I’d say something like 1.1 markup is fairest, so you’re paying them a good prize to weather the variance for you. At 1.1 markup, they pay 1.1x whatever it costs you to buy into the tournament. You can sell a large part of your action but not quite all of it to keep an incentive to play well (if you sold everything at $11,000, you could, if you were shady, just pocket the extra $1,000, go out early on purpose, and register the next tournament where you sold action for another round of instant profit).
So, let’s say they paid you $8,800 to get 80% of your winnings, so they make an expected profit of ($8,000 * 1.25) - $8,000, which is $1,200. And then you yourself still have 20% of your own action, for which you only paid $1,200 (since you got $800 from the 1.1 markup and you invest that into your tournament). Now, you’re only in for $1,200 of your own money, but you have 20% of the tournament, so you’d already be highly profitable if you were just breaking even. In addition, as we stipulated, you have an edge on the field expecting 125% ROI, so in expectation, that $1,200 is worth $2,000*1.25, which is $2,500. This still comes with a lot of variance, but your ROI is now so high that Kelly allows you to play a big tournament in this way even if your net worth is <$100k.
(This analysis simplified things assuming there’s no casino fee. In reality, if a tournament is advertized as a $10k tournament, the buy in tends to be more like $10,500, and $500 is just the casino fee that doesn’t go into the prize pool. This makes edges considerably smaller.)
Regarding busting a tournament with a risky bluff: In the comment above I was assuming we’re playing cash game where chips are equivalent to real dollars and you can leave the table at any point. In tournaments, at least if they are not “winner takes it all” format (which they almost never are), there’s additional expected value in playing a little more conservative than the strategy “maximizing expected value in chips.” Namely, you have to figure out how “having lots of chips” translates into “probabilities of making various pay jumps.” If you’re close to the money, or close to a big pay jump when you’re already in the money (and at a big final table, every pay jump tends to be huge!), you actually make money by folding, since every time you fold, there’s a chance that some other player will go out (either behind you at your table, or at some other table in a tournament where there are still many tables playing). If someone else goes out and you make the pay jump, you get more money without having to risk your stack. So, in tournaments, you gotta be more selective with the big bluffs for multiples of what is already in the pot, especially if you think you have an edge on the field and if the pay jumps are close.
You also quote this part of the article:
Theo Boer, a healthcare ethics professor at Protestant Theological University in Groningen, served for a decade on a euthanasia review board in the Netherlands. “I entered the review committee in 2005, and I was there until 2014,” Boer told me. “In those years, I saw the Dutch euthanasia practice evolve from death being a last resort to death being a default option.” He ultimately resigned.
I found a submission by this Theo Boer for the UK parliament, where he explains his reasons for now opposing euthanasia in more detail.
He writes:
It is well known that British advocates of assisted dying argue for a more restricted law than is found in the low countries. Here is my prediction: any law that allows assisted dying will come to be experienced as an injustice and will be challenged in the courts. Why only euthanasia for terminally ill patients, who have access to an ever widening array of palliative care and whose suffering will be relatively short, whereas chronic patients may suffer more intensely and much longer? Why exclude psychiatric patients, many of whom are suffering most heartbreakingly of all? Why only an assisted death for people suffering from a disease, and not for those suffering from irremediable meaninglessness, alienation, loneliness, from life itself? We are presently seeing how in the years 2016-2023 Canada’s Medical Assistance in Dying (MAiD), from being euthanasia for terminal patients only, has evolved into an assisted death for patients whose chronic disease has become unbearable due to shortage of healthcare(Douthat 2022).
This is a “slope” of sorts, but I think it’s not a bad one. The arguments for extending the practice all seem reasonable. What matters is, “are people suffering?” and, “are they right that there’s not enough hope for them to justify continued suffering?”
Regarding pressure/being pressured, I thought this part was interesting:
This brings me to the second question: how to protect vulnerable citizens? Different from what is presently going on in Canada, I do not yet see a specific risk for citizens who by many are considered vulnerable – homeless, underinsured, people on welfare, people with disabilities. Although these groups are present in those who get euthanasia in the Netherlands, it is not my impression that they are overrepresented. If any group is well represented in the euthanasia numbers, it is the better-off, the healthy-aging population, the higher educated. In our research on practice variation, we found that in regions where the average experienced health is higher, the euthanasia numbers are also higher. In places where people on average are better off, obviously serious threats to their wellbeing tend to be more often a reason for a euthanasia request than in places where people are more used to dealing with life’s different hardships. This leads me to adopt a different definition of vulnerability, a vulnerability that may be found in all social and economic groups, from top to bottom: one of despair, meaninglessness, social isolation, feeling redundant. It may apply to wealthy citizens in a villa with woodblock floors and a grand piano, whose children have their businesses elsewhere and whose friends are either dead or institutionalized, just as much as to a single disabled woman on welfare. Anyone under this shadow of despair may make a euthanasia request, and there is no way a government can prevent this kind of vulnerability to motivate a euthanasia request, since the autonomous citizens are not under any other pressure than their own, that is, their own incapacity to face life’s harder episodes. “Life has always been a feast for me,” an elderly man whose euthanasia I assessed, “and that’s how it should end for me.”
I’d be curious to figure out why it is exactly that requests for euthanasia are higher in demographs where people tend to be better off/suffering less.
That said, I’m not at all convinced that this would prove that there’s something horribly wrong going on with these developments after legalization of assisted suicide. (Still, I’d be curious to investigate this further.)
Reading this account, it feels to me like Theo Boer has a problem with death intrinsically, as opposed to only having a problem with death when a person has for-themselves good/strong reasons to continue to want to live. That’s not an outlook I agree with.
“Their own incapacity to face life’s harder episodes” is a question-begging phrasing. For all we know, many people who choose assisted suicide would voluntarily chose to continue with their suffering if there was more at stake that they cared about! For instance, if they learned that by continuing to suffer, they’d solve world poverty, they might continue to suffer. It seems wrong, then, to say they’re “incapable,” when the real reason is more about how they don’t want it enough. It’s their life, so their decision.
“Since the autonomous citizens are not under any other pressure than their own” – this is also an interesting perspective. He seems to be conceding that no matter how much society and relatives try to reassure chronically ill or disabled elderly people that they’re still valued and cared about (something we absolutely must emphasize or work towards if it isn’t everywhere the case!), those people will struggle with worries of being a burden. That’s unfortunate, but also very natural. It’s how I would feel too. But people who feel that way don’t necessarily jump right towards considering assisted suicide! Consider two different cases:
You still enjoy life/are happy.
You have been feeling suicidal for many years, and there’s no realistic hope for things to get better.
In which of these cases is “worries about being a burden” (even if you know on some level that these worries probably don’t accurately reflect the reality of the views of your caretakers or loved ones) a bigger reason to sway your decision towards wanting euthanasia? Obviously, it is in the second case, where you lack positive reasons to stay alive, so negative reasons weigh more comparatively, even if they’re quite weak in absolute terms. In fact, what if “what will other people think” had been the primary motivation that kept you wanting to live for a long time, as long as you could provide more value for your relatives or loved ones? Is there not also something disconcerting about that? (To underscore this point, many people who are depressed write that the main reason they don’t consider suicide more seriously is because of what it would do to their relatives and loved ones. If you want to see for yourself, you can read reddit threads on this for examples of people’s suicidal ideation.)
“Life has always been a feast for me,” an elderly man whose euthanasia I assessed, “and that’s how it should end for me.”
This quote comes at the end of a passage that was all about “pressure,” but it has nothing to do with pressure anymore, nor does it have to do with being “incapable of facing life’s hardship.” Instead, it just sounds like this person disagrees about the view that “facing life’s hardship” for no upside is something that’s a virtue or otherwise important/right to do. This is more like an expression of a philosophy that life can be completed (see also the ending of the series ‘The Good Place,’) or, somewhat differently, that there’s no need to prolong it after the best (and still-good) times are now over. If that’s someone’s attitude, let them have it.
Overall, I respect Theo Boer for both the work he’s done for terminally ill patients in the early stages of the assisted suicide program in the Netherlands and for speaking out against the assisted suicide practice after it went in a direction that he no longer could support. At the same time, I think he has an attitude towards the topic that I don’t agree with. In my view, he doesn’t seem to take seriously how bad it is to suffer, and especially, how bad and pointlesss it is to suffer for no good reason.
It might well be true that doctors and mental health professionals are now okay with assisted suicide as a solution too quickly (without trying other avenues first), but I’m not sure I’d trust Theo Boer’s judgment on this, given the significant differences in our points of view. In any case, I acknowledge this is a risk and that we should take steps to make sure this doesn’t occur or doesn’t become too strong of an issue (and people who decide to go through with it should be well-informed about other options and encouraged to try these other options in case they haven’t already been doing this to no success for many years).
Assisted Suicide Watch
A psychiatrist overstepping their qualifications by saying “It’s never gonna get any better” ((particularly when the source of the suffering is at least partly BPD, for which it’s commonly known that symptoms can get better in someone’s 40s)) clearly should never happen.
However, I’d imagine that most mental health professionals would be extremely careful when making statements about whether there’s hope for things to get better. In fact, there are probably guidelines around that.
Maybe it didn’t happen this way at all: I notice I’m confused.
This could just be careless reporting by the newspaper.
The article says:
She recalled her psychiatrist telling her that they had tried everything, that “there’s nothing more we can do for you. It’s never gonna get any better.”
Was it really the psychiatrist who added “It’s never gonna get any better,” or was it just that the psychiatrist said “There’s nothing more we can do for you,” and then Zoraya herself (the person seeking assisted suicide) told the reporters her conclusion “It’s never going to get any better,” and the reporters wrote it as though she ascribed those words to the psychiatrist?
In any case, this isn’t a proper “watch” (“assisted suicide watch”) if you only report when you find articles that make the whole thing seem slippery-slopy. (And there’s also a question of “how much is it actually like that?” vs “How much is it in the reporting” – maybe the reporter had their own biases in writing it like that. For all we know, this person, Zoraya, has had this plan for ever since she was a teenager, and gave herself 25 years to stop feeling suicidal, and now it’s been enough. And the reporter just chose to highlight a few things that sound dramatic, like the bit about not wanting to inconvenience the boyfriend with having to keep the grave tidy.)
I feel like the response here should be: Think hard about what sorts of guidelines we can create for doctors or mental health professionals to protect against risks of sliding down a slippery slope. It’s worth taking some risks because it seems really bad as well to err in the other direction (as many countries and cultures still do). Besides, it’s not straightforwardly evidence of a slippery slope simply because the numbers went up or seem “startling,” as the article claims. These developments can just as plausibly be viewed as evidence for, “startlingly many people suffer unnecessarily and unacceptably without these laws.” You have to look into the details to figure out which one it is, and it’s gonna be partly a values question rather than something we can settle empirically.
There are other written-about cases like Lauren Hoeve quite recently, also from Netherlands, who’d suffered from debilitating severe myalgic encephalomyelitis (ME) for five years and began her assisted suicide application in 2022. Anyone interested in this topic should probably go through more of these accounts and read sources directly from the people themselves (like blogposts explaining their decision) rather than just media reporting about it.
If you know you have a winning hand, you do not want your opponent to fold, you want them to match your bet. So you kinda have to balance optimizing for the maximum pool at showdown with limiting the information you are leaking so there is a showdown. Or at least it would seem like that to me, I barely know the rules.
This is pretty accurate.
For simplicity, let’s assume you have a hand that has a very high likelihood of winning at showdown on pretty much any runout. E.g., you have KK on a flop that is AK4, and your opponent didn’t raise you before the flop, so you can mostly rule out AA. (Sure, if an A comes then A6 now beats you, or maybe they’ll have 53s for a straight draw to A2345 with a 2 coming, or maybe they somehow backdoor into a different straight or flush depending on the runout and their specific hand – but those outcomes where you end up losing are unlikely enough to not make a significant difference to the math and strategy.)The part about information leakage is indeed important, but rather than adjusting your bet sizing to prevent information leakage (i.e., “make the bet sizing smaller so it’s less obvious that I’ve got a monster”), you simply add the right number of bluffs to your big-bet line to make your opponent’s bluff-catching hands exactly indifferent. So, instead of betting small with KK to “keep them in” or “disguise the strength of your hand,” you still bomb it, but you’d play the same way with a hand like J5ss (can pick up a 1-to-a-straight draw on all of the following turns: 2,3, T, Q; and can pick up a flush draw on any turn with a spade if there was one spade already on the flop).
To optimize for the maximum pot at showdown and maximum likelihood of getting called for all the chips, you want to bet the same proportion of the pot on each street (flop, turn, and river) to get all-in with the last bet. (This is forcing your opponent to defend the most; if you make just one huge bet of all-in right away, your opponent mathematically has to call you with fewer hands to prevent you from automatically profiting with every hand as a bluff.)
So, if the pot starts out at 6 big blinds (you raise 2.75x, get called by the big blind, and there’s a 0.5 small blind in there as well). Your stack was100 big blinds to start. If you were to bet 100% of the pot on each street, this would be 6+18+54, which is 78, so slightly too small (you want it to sum to 100 or [100 minus the preflop raise of 2.75 big blinds or whatever]). So, with your very best hands, it’s slightly better here to bet a little larger on each street, like 110% pot. (So you get something like 7 into 6 on the flop, 22 into 20 on the turn, and 70ish into 64 on the river for the rest of your stack.)
Let’s say the board runs out very dry AK469, no flushes possible. You have KK, your opponent has A2 as a bluffcatcher (or AT, doesn’t make a difference here because you wouldn’t want to take this line with anything worse for value than AQ probably). If you get the ratio of bluffs-to-value exactly right, then your opponent is now faced with a choice: Either forfeit what’s in the pot right now (zero further EV for them), or look you up with a bluffcatcher (also zero EV – they win when you’re bluffing, but they lose when you’ve got it). If they overfold, you always win what’s already in the pot and your bluffs are printing money. If they overcall, your bluffs become losing plays (and if you knew the opponent was overcalling, you’d stop bluffing!), but your value hands get paid off more often than the should in game theory, so the overall outcome is the same.Of course, what is a proper “bluff-catching hand” depends on the board and previous action. If someone thinks any pair is a good bluffcatcher, but some of your bluffs include a pair that beats their pair, then they’re committing a huge blunder if they call. (E.g., they might consider bluffcatching with the pocket pair 88 on this AK469 runout, but you might be bluffing J9s on the river, which is actually reasonable since you get remaining Kx to fold and some Ax to fold and your 9 is making it less likely that they backed into an easy-to-call two pair with K9 or A9. So, if you bluff 9x on the river, they’re now giving you a present by ever calling with 88 [even if you have some other bluffs that they beat, their call is still losing lots of money in expectation because the bluff-catching ratio is now messed up.]) Or, conversely, if they fold a hand that they think is a bluffcatcher, but it actually beats some of your thinnest value hands (or at least ties with them), they’d again be blundering by folding. So, for instance, if someone folds AQ here, it could be that you went for a thin value bet with AQ yourself and they now folded something that not only beats all your bluffs, but also ties with some value (again changing the ratio favorably to make the call fairly highly +EV). (Note that a call can be +EV even if you’re losing a little more than 50% of the time, because there’s already a fair bit of money in the pot before the last bet).
The above bet-sizing example had some properties that made the analysis easy:
Your KK didn’t block the hands you most want to get value from (Ax hands).
[KK is actually a less clear example for betting big than 44, because at least on flop and turn, you should get value by lots of Kx hands. So 44 prefers betting big even more than KK because it gets paid more.]The board AK4 was pretty static – hands that are good on the flop tend to be still good by the river. (Compare this to the “wet” board 8s75s, the “ss” signifying that there’s a flush draw. Say you have 88 on this board for three of a kind 8s. It’s unlikely your opponent has exactly 96 or 64 for a straight, so you’re likely ahead at the moment. But will you still be ahead if a 6, a T, or a 4 comes, or if the flush comes in? Who knows, so things can change radically with even just one more card!).
If we assume that you raised before the flop and got merely called by the big blind (everyone else folded), you have the advantage of top hands on this board because the big blind is really supposed to always raise AK, KK, and AA. “Trapping” them for surprise value would simply not be worth it: those hands win a lot more by making the pot bigger before the flop. Having the advantage in top hands allows you to bet really big.
These three bullet points all highlight separate reasons why a hand that is great on the flop shouldn’t always go for the sizing to get all-in by the river with three proportional big bets (“geometric sizing”). To express them in simple heuristics:
Bet smaller (or throw in a check) when you block the main hands you want to get called by.
Bet smaller (or throw in a check) when a lot can still change later in the hand because of how “wet” the board is.
Bet smaller (or throw in a check) if your opponent has the advantage of top hands – this means they should in theory do a lot of the betting for you and bet big and include bluffs. (Of course, if your opponent is generally too passive, it’s better for you to do the betting yourself even if it’s not following optimal theory.)
There’s also a concept where you want to make sure you still have some good hands in nodes of the game where you only make a small bet or even check, so your opponent cannot always pounce on these signs of weakness. However, it’s often sufficient to allocate your second tier hands there, you don’t need to “protect your checking range” with something as strong as KK. This would be like allocating Achilles to defending the boats at the beach when the rest of your army storms against the walls of Troy. You want someone back there with the boats, but it doesn’t have to be your top fighter.
Lastly, one other interesting situation where you want to check strong hands is if you think that the best way to get the money in is “check-in-order-to-check-raise” rather than “bet big outright.” That happens when your “out of position” (your opponent will still have the option to bet after you check) and your opponent’s range is capped to hands that are worth one big bet, but are almost never worth enough to raise against your big bet. In that situation, you get extra money from all their bluffs if you check, and the hands that would’ve called your big bet had you bet, those hands will bet for you anyway, and then have to make a though decision against your check-raise. So, let’s say you’re the big blind now on AK4 and you hold 99. This time, your opponent checks back the flop, indicating that they’re unlikely to have anything very strong (except maybe AA that can’t easily get paid). After he checks back, the turn is a 6. You both check once again. At this point, you’re pretty sure that the best hands in your opponent’s range are one pair at best, because it just wouldn’t make sense for them to keep the pot small and give you a free river csard with a hand that can get paid off. The river is a 9 (bingo for you!). In theory, your opponent could have 99 themselves or back into two pair with K9, 96s, or A9 that weirdly enough didn’t bet neither the turn nor the flop. However, since you have two 9s, you’re blocking those strong hands your opponent could have pretty hard. As a result, if you were to bet big here with 99, your most likely outcome is getting called at best. That’s a disaster and you want to get more value. So, you check. Your opponent will now bet KQ for value (since most of your Ax would’ve bet the river) and will bet all their weak Ax for value that they got there in the check-flop, check-turn line. Against this bet, your 99 now (and also A9 to some degree, but having the A isn’t ideal because you want to get called by an A) can jam all-in for 10 times the size of the pot to make all your opponent’s hands except rivered A9 indifferent. Any smaller bet sizing wouldn’t make a lot of sense because you’re never getting raised, so why not go for maximum value (or maximum pressure, in case you’re bluffing). Your bluffs to balance this play should all contain a 9 themselves, so you again make it less likely that your opponent holds 99 or A9/K9. (Note that 9x can also function as a breakeven call against a river bet after you check, catching your opponent’s river bluffs. However, since 9x never beats value, so you’re not really wasting your handstrength by turning it into a bluff some of the time.) If you have the right ratio of bluffs-to-value for your 10x-the-pot overbet, your opponent is supposed to fold all Kx and most Ax hands, but some Ax has to defend because it blocks the A9 that you also go for value with. So, the opponent has to occasionally call this crazy big raise with just an A. If they never do this, and you know they never do it, you can check-jam every single 9x on the river and it makes for a +EV bluffs. (You probably cannot just check-jam every single hand because if you don’t have a 9, the times your opponent has A9 or 99 or K9 are now drastically increased, and you’ll always get called by those.)
Of course, if your opponent correctly guesses that you’re jamming all your 9x there, they realize that you have a lot more 9x-that-is-just-one-pair than 99/A9/K9, so they’ll now exploitatively call all their Ax. Since you now bet 10x the pot as a bluff with too many bluffs in expectation, you’re losing way more than you were standing to gain, so you have to be especially careful with exploits here when betting many times the pot on the river.)
I really liked this post! I will probably link to it in the future.
Edit: Just came to my mind that these are things I tend to think of under the heading “considerateness” rather than kindness, but it’s something I really appreciate in people either way (and the concepts are definitely linked).
FWIW, one thing I really didn’t like about how he came across in the interview is that he seemed to be engaged in framing the narrative one-sidedly in an underhanded way, sneakily rather than out in the open. (Everyone tries to frame the narrative in some way, but it becomes problematic when people don’t point out the places where their interpretation differs from others, because then listeners won’t easily realize that there are claims that they still need to evaluate and think about rather than just take for granted and something that everyone else already agrees about.)
He was not highlighting the possibility that the other side’s perspective still has validity; instead, he was shrugging that possibility under the carpet. He talked as though (implicitly, not explicitly) it’s now officially established or obviously true that the board acted badly (Lex contributed to this by asking easy questions and not pushing back on anything too much). He focused a lot on the support he got during this hard time and people saying good things about him (eulogy while still alive comparison, highlighting that he thinks there’s no doubt about his character) and said somewhat condescending things about the former board (about how he thinks they had good intentions, said in that slow voice and thoughtful tone, almost like they had committed a crime) and then emphasized their lack of experience.
For contrast, here are things he could have said that would have made it easier for listeners to come to the right conclusions (I think anyone who is morally scrupulous about whether they’re in the right in situations when many others speak up against them would have highlighted these points a lot more, so the absence of these bits in Altman’s interview is telling us something.)
Instead of just saying that he believes the former board members came from a place of good intentions, also say if/whether he believes that some of the things they were concerned about weren’t totally unreasonable from their perspective. E.g., acknowledge things he did wrong or things that, while not wrong, understandably would lead to misunderstandings.
Acknowledge that just because a decision had been made by the review committee, the matter of his character and suitability for OpenAI’s charter is not now settled (esp. given that the review maybe had a somewhat limited scope?). He could point out that it’s probably rational (or, if he thinks this is not necesarily mandated, at least flag that he’d understand if some people now feel that way) for listeners of the youtube interview to keep an eye on him, while explaining how he intends to prove that the review committee came to the right decision.
He said the board was inexperienced, but he’d say that in any case, whether or not they were onto something. Why is he talking about their lack of experience so much rather than zooming in on their ability to assess someone’s character? It could totally be true that the former board was both inexperienced and right about Altman’s unsuitability. Pointing out this possibility himself would be a clarifying contribution, but instead, he chose to distract from that entire theme and muddle the waters by making it seem like all that happened was that the board did something stupid out of inexperience, and that’s all there was.
Acknowledge that it wasn’t just an outpouring of support for him; there were also some people who used to occasion to voice critical takes about him (and the Y Combinator thing came to light).
(Caveat that I didn’t actually listen to the full interview and therefore may have missed it if he did more signposting and perspective taking and “acknowledging that for-him-inconvenient hypotheses are now out there and important if true and hard to dismiss entirely for at the very least the people without private info” than I would’ve thought from skipping through segments of the interview and Zvi’s summary.)
In reaction to what I wrote here, maybe it’s a defensible stance to go like, “ah, but that’s just Altman being good at PR; it’s just bad PR for him to give any air of legitimacy to the former board’s concerns.”
I concede that, in some cases when someone accuses you of something, they’re just playing dirty and your best way to make sure it doesn’t stick is by not engaging with low-quality criticism. However, there are also situations where concerns have enough legitimacy that shrugging them under the carpet doesn’t help you seem trustworthy. In those cases, I find it extra suspicious when someone shrugs the concerns under the carpet and thereby misses the opportunity to add clarity to the discussion, make themselves more trustworthy, and help people form better views on what’s the case.
Maybe that’s a high standard, but I’d feel more reassured if the frontier of AI research was steered by someone who could talk about difficult topics and uncertainty around their suitability in a more transparent and illuminating way.
There are realistic beliefs Altman could have about what’s good or bad for AI safety that would not allow Zvi to draw that conclusion. For instance:
Maybe Altman thinks it’s really bad for companies’ momentum to go through CEO transitions (and we know that he believes OpenAI having a lot of momentum is good for safety, since he sees them as both adequately concerned about safety and more concerned about it than competitors).
Maybe Altman thinks OpenAI would be unlikely to find another CEO who understands the research landscape well enough while also being good at managing, who is at least as concerned about safety as Altman is.
Maybe Altman was sort of willing to “put that into play,” in a way, but his motivation to do so wasn’t a desire for power, nor a calculated strategic ploy, but more the understandable human tendency to hold a grudge (esp. in the short term) against the people who just rejected and humiliated him, so he understandably didn’t feel a lot of motivational pull to want help them look better about the coup they had just attempted for what seemed to him as unfair/bad reasons. (This still makes Altman look suboptimal, but it’s a lot different from “Altman prefers power so much that he’d calculatedly put the world at risk for his short-term enjoyment of power.”)
Maybe the moments where Altman thought things would go sideways were only very brief, and for the most part, when he was taking actions towards further escalation, he was already very confident that he’d win.
Overall, the point is that it seems maybe a bit reckless/uncharitable to make strong inferences about someone’s rankings of priorities just based on one remark they made being in tension with them pushing in one direction rather than the other in a complicated political struggle.
Small edges are why there’s so much money gambled in poker.
It’s hard to reach a skill level where you make money 50% of the night, but it’s not that hard to reach a point where you’re “only” losing 60% of the time. (That’s still significantly worse than playing roulette, but compared to chess competitions where hobbyists never win any sort of prize, you’ve at least got chances.)
You criticize Altman for pushing ahead with dangerous AI tech, but then most of what you’d spend the money on is pushing ahead with tech that isn’t directly dangerous. Sure, that’s better. But it doesn’t solve the issue that we’re headed into an out-of-control future. Where’s the part where we use money to improve the degree to which thoughtful high-integrity people (or prosocial AI successor agents with those traits) are able to steer where this is all going?
(Not saying there are easy answers.)
I mean, personality disorders are all about problems in close interpersonal relationships (or lack of interest in such relationships, in schizoid personality disorder), and trust is always really relevant in such relationships, so I think this could be a helpful lens of looking at things. At the same time, I’d be very surprised if you could derive new helpful treatment approaches from this sort of armchair reasoning (even just at the level of hypothesis generation to be subjected to further testing).
Also, some of these seem a bit strained:Narcissistic personality disorder seems to be more about superiority and entitlement than expecting others to be trusting. And narcissism is correlated with Machiavellianism, where a feature of that is having a cynical worldview (i.e., thinking people in general aren’t trustworthy). If I had to frame narcissism in trust terms, I’d maybe say it’s an inability to value or appreciate trust?
Histrionic personality disorder has a symptom criterion of “considers relationships to be more intimate than they actually are.” I guess maybe you could say “since (by your hypothesis) they expect people to not care, once someone cares, a person with histrionic personality disorder is so surprised that they infer that the relationship must be deeper than it is.” A bit strained, but maybe can be made to fit.
Borderline: I think there’s more of pattern to splitting than randomness (e.g., you rarely have splitting in the early honeymoon stage of a relationship), so maybe something like “fluctuating” would fit better. But also, I’m not sure what fluctuates is always about trust. Sure, sometimes splitting manifests in accusing the partner of cheating out of nowhere, but in other cases, the person may feel really annoyed at the partner in a way that isn’t related to trust. (Or it could be related to trust, but going in a different direction: they may resent the partner for trusting them because they have such a low view of themselves that anyone who trusts them must be unworthy.)
Dependent: To me the two things you write under it seem to be in tension with each other.
Edit:
Because it takes eight problems currently considered tied up with personal identy and essentially unsolvable [...]
I think treatment success probabilities differ between personality disorders. For some, calling them “currently considered essentially unsolvable” seems wrong.
And not sure how much of OCPD is explained by calling it a persistent form of OCD – they seem very different. You’d expect “persistent” to make something worse, but OCPD tends to be less of an issue for the person who has it (but can be difficult for others around them). Also, some symptoms seem to be non-overlapping, like with OCPD I don’t think intrusive thoughts play a role (I might be wrong?), whereas intrusive thoughts are a distinct and telling feature of some presentations of OCD.
Dilemma:
If the Thought Assessors converge to 100% accuracy in predicting the reward that will result from a plan, then a plan to wirehead (hack into the Steering Subsystem and set reward to infinity) would seem very appealing, and the agent would do it.
If the Thought Assessors don’t converge to 100% accuracy in predicting the reward that will result from a plan, then that’s the very definition of inner misalignment!
[...]
The thought “I will secretly hack into my own Steering Subsystem” is almost certainly not aligned with the designer’s intention. So a credit-assignment update that assigns more positive valence to “I will secretly hack into my own Steering Subsystem” is a bad update. We don’t want it. Does it increase “inner alignment”? I think we have to say “yes it does”, because it leads to better reward predictions! But I don’t care. I still don’t want it. It’s bad bad bad. We need to figure out how to prevent that particular credit-assignment Thought Assessor update from happening.
[...]
I think there’s a broader lesson here. I think “outer alignment versus inner alignment” is an excellent starting point for thinking about the alignment problem. But that doesn’t mean we should expect one solution to outer alignment, and a different unrelated solution to inner alignment. Some things—particularly interpretability—cut through both outer and inner layers, creating a direct bridge from the designer’s intentions to the AGI’s goals. We should be eagerly searching for things like that.
Yeah, there definitely seems to be something off about that categorization. I’ve thought a bit about how this stuff works in humans, particularly in this post of my moral anti-realism sequence. To give some quotes from that:
One of many takeaways I got from reading Kaj Sotala’s multi-agent models of mind sequence (as well as comments by him) is that we can model people as pursuers of deep-seated needs. In particular, we have subsystems (or “subagents”) in our minds devoted to various needs-meeting strategies. The subsystems contribute behavioral strategies and responses to help maneuver us toward states where our brain predicts our needs will be satisfied. We can view many of our beliefs, emotional reactions, and even our self-concept/identity as part of this set of strategies. Like life plans, life goals are “merely” components of people’s needs-meeting machinery.[8]
Still, as far as components of needs-meeting machinery go, life goals are pretty unusual. Having life goals means to care about an objective enough to (do one’s best to) disentangle success on it from the reasons we adopted said objective in the first place. The objective takes on a life of its own, and the two aims (meeting one’s needs vs. progressing toward the objective) come apart. Having a life goal means having a particular kind of mental organization so that “we” – particularly the rational, planning parts of our brain – come to identify with the goal more so than with our human needs.[9]
[...]
There’s a normative component to something as mundane as choosing leisure activities. [E.g., going skiing in the cold, or spending the weekend cozily at home.] In the weekend example, I’m not just trying to assess the answer to empirical questions like “Which activity would contain fewer seconds of suffering/happiness” or “Which activity would provide me with lasting happy memories.” I probably already know the answer to those questions. What’s difficult about deciding is that some of my internal motivations conflict. For example, is it more important to be comfortable, or do I want to lead an active life? When I make up my mind in these dilemma situations, I tend to reframe my options until the decision seems straightforward. I know I’ve found the right decision when there’s no lingering fear that the currently-favored option wouldn’t be mine, no fear that I’m caving to social pressures or acting (too much) out of akrasia, impulsivity or some other perceived weakness of character.[21]We tend to have a lot of freedom in how we frame our decision options. We use this freedom, this reframing capacity, to become comfortable with the choices we are about to make. In case skiing wins out, then “warm and cozy” becomes “lazy and boring,” and “cold and tired” becomes “an opportunity to train resilience / apply Stoicism.” This reframing ability is a double-edged sword: it enables rationalizing, but it also allows us to stick to our beliefs and values when we’re facing temptations and other difficulties.
[...]
Visualizing the future with one life goal vs. anotherWhether a given motivational pull – such as the need for adventure, or (e.g.,) the desire to have children – is a bias or a fundamental value is not set in stone; it depends on our other motivational pulls and the overarching self-concept we’ve formed.
Lastly, we also use “planning mode” to choose between life goals. A life goal is a part of our identity – just like one’s career or lifestyle (but it’s even more serious).We can frame choosing between life goals as choosing between “My future with life goal A” and “My future with life goal B” (or “My future without a life goal”). (Note how this is relevantly similar to “My future on career path A” and “My future on career path B.”)
[...]
It’s important to note that choosing a life goal doesn’t necessarily mean that we predict ourselves to have the highest life satisfaction (let alone the most increased moment-to-moment well-being) with that life goal in the future. Instead, it means that we feel the most satisfied about the particular decision (to adopt the life goal) in the present, when we commit to the given plan, thinking about our future. Life goals inspired by moral considerations (e.g., altruism inspired by Peter Singer’s drowning child argument) are appealing despite their demandingness – they can provide a sense of purpose and responsibility.So, it seems like we don’t want “perfect inner alignment,” at least not if inner alignment is about accurately predicting reward and then forming the plan of doing what gives you most reward. Also, there’s a concept of “lock in” or “identifying more with the long-term planning part of your brain than with the underlying needs-meeting machinery.” Lock in can be dangerous (if you lock in something that isn’t automatically corrigible), but it might also be dangerous not to lock in anything (because this means you don’t know what other goals form later on).
Idk, the whole thing seems to me like brewing a potion in Harry Potter, except that you don’t have a recipe book and there’s luck involved, too. “Outer alignment,” a minimally sufficient degree thereof (as in: the agent tends to gets rewards when it takes actions towards the intended goal), increases the likelihood that you get broadly pointed you in the right direction, so the intended goal maybe gets considered among things the internal planner considers reinforcing itself around / orienting itself towards. But then, whether the intended gets picked over other alternatives (instrumental requirements for general intelligence, or alien motivations the AI might initially have), who knows. Like with raising a child, sometimes they turn out the way the parents intend, sometimes not at all. There’s probably a science to finding out how outcomes become more likely, but even if we could do that with human children developing into adults with fixed identities, there’s then still the question of how to find analogous patterns in (brain-like) AI. Tough job.
Conditioned Taste Aversion (CTA) is a phenomenon where, if I get nauseous right now, it causes an aversion to whatever tastes I was exposed to a few hours earlier—not a few seconds earlier, not a few days earlier, just a few hours earlier. (I alluded to CTA above, but not its timing aspect.) The evolutionary reason for this is straightforward: a few hours is presumably how long it typically takes for a toxic food to induce nausea.
That explains why my brother no longer likes mushrooms. When we were little, he liked them and we ate mushrooms at a restaurant, then were driven through curvy mountain roads later that day with the family. He got car sick and vomited, and afterwards he had an intense hatred for mushrooms.
Is that sort of configuration even biologically possible (or realistic)? I have no deep immunology understanding, but I think bad reactions to vaccines have little to nothing to do with whether you’re up-to-date on previous vaccines. So far, I’m not sure we’re good at predicting who reacts with more severe side effects than average (and if we did, it’s not like it’s easy to tweak the vaccine, except for tradeoff-y things like lowering the vaccination dose).
My point is that I have no evidence that he ended up reading most of the relevant posts in their entirety. I don’t think people who read all the posts in their entirety should just go ahead and unilaterally dox discussion participants, but I feel like people who have only read parts of it (or only secondhand sources) should do it even less.
Also, at the time, I interpreted Roko’s “request for a summary” more as a way for him to sneer at people. His “summary” had a lot of loaded terms and subjective judgments in it. Maybe this is a style thing, but I find that people should only (at most) write summaries like that if they’re already well-informed. (E.g., Zvi’s writing style can be like that, and I find it fine because he’s usually really well-informed. But if I see him make a half-assed take on something he doesn’t seem to be well-informed on, I’d downvote.)
See my comment here.
Kat and Emerson were well-known in the community and they were accused of something that would cause future harm to EA community members as well. By contrast, Chloe isn’t particularly likely to make future false allegations even based on Nonlinear’s portrayal (I would say). It’s different for Alice, since Nonlinear claim she has a pattern. (But with Alice, we’d at least want someone to talk to Nonlinear in private and verify how reliable they seem about negative info they have about Alice, before simply taking their word for it based on an ominous list of redacted names and redacted specifics of accusations.)Theoretically Ben could have titled his post, “Sharing Information About [Pseudonymous EA Organization]”, and requested the mods enforce anonymity of both parties, right?
That would miss the point, rendering the post almost useless. The whole point is to prevent future harm.
but not for Roko to unilaterally reveal the names of Alice and Chloe?
Alice and Chloe had Ben, who is a trusted community member, look into their claims. I’d say Ben is at least somewhat “on the hook” for the reliability of the anonymous claims.
By contrast, Roko posted a 100 word summary of the Nonlinear incident that got some large number of net downvotes, so he seems to be particularly poorly informed about what even happened.
Some conditions for when I think it’s appropriate for an anonymous source to make a critical post about a named someone on the forum:
Is the accused a public person or do they run an organization in the EA or rationality ecosystem?
Or: Is the type of harm the person is accused of something that the community benefits from knowing?
Did someone who is non-anonymous and trusted in the community talk to the anonymous accuser and verify claims and (to some degree*) stake their reputation for them?
*I think there should be a role of “investigative reporter:” someone verifies that the anonymous person is not obviously unreliable. I don’t think the investigative reporter is 100% on the hook for anything that will turn out to be false or misleading, but they are on the hook for things like doing a poor job at verifying claims or making sure there aren’t any red flags about a person.
(It’s possible for anonymous voices to make claims without the help of an “investigative reporter;” however, in that case, I think the appropriate community reaction should be to give little-to-no credence to such accusations. After all, they could be made by someone who already has had their reputation justifiably tarnished.)
On de-anonymizing someone (and preventing an unfair first-mover advantage):In situations where the accused parties are famous and have lots of influence, we can view anonymity protection as evening the playing field rather than conferring an unfair advantage. (After all, famous and influential people already have a lot of advantages on their side – think of Sam Altman in the conflict with the OpenAI board.)
If some whistleblower displays a pattern/history of making false accusations, that implies potential for future harm, so it seems potentially appropriate to warn others about them (but you’d still want to be cautious, take your time to evaluate evidence carefully, and not fall prey to a smear campaign by the accused parties – see DARVO).
If there’s no pattern/history of false accusations, but the claims by a whistleblower turn out to be misleading in more ways than one would normally expect in the heat of things (but not egregiously so), then the situation is going to be unsatisfying, but personally I’d err on the side of protecting anonymity. (I think this case is strongest the more the accused parties are more powerful/influential than the accusers.) I’d definitely protect anonymity if the accusations continue to seem plausible but are impossible to prove/there remains lots of uncertainty.
I think de-anonymization, if it makes sense under some circumstances, should only be done after careful investigation, and never “in the heat of the movement.” In conflicts that are fought publicly, it’s very common for different sides to gain momentum temporarily but then lose it again, depending on who had the last word.
I lean towards agreeing with the takeaway; I made a similar argument here and would still bet on the slope being very steep inside the human intelligence level.