In particular, it highlights a gap in my way of reasoning. I notice that even after you give examples, the category of “ultra-BS” doesn’t really gel for me. I think I use a more vague indicator for this, like emotional tone plus general caution when someone is trying to persuade me of something.
In the spirit of crisping up my understanding, I have a question:
Now, I understand I sound obviously crazy already, but hear me out. Russia’s Kinzhal hypersonic missiles, which have a range of roughly 1,000 miles, cannot hit the US from the Russian mainland. But they can hit us from the Arctic. I add that hypersonic missles are very, very fast. [This essentially acts as a preemptive rebuttal to my opponent’s counterargument (but what about MAD?).] If we’re destroyed by a first strike, there is no MAD, and giving Russia the Arctic would immediately be an existential threat.
Of course, this is ridiculous…
I think I’m missing something obvious, or I’m missing some information. Why is this clearly ridiculous?
In particular, it highlights a gap in my way of reasoning. I notice that even after you give examples, the category of “ultra-BS” doesn’t really gel for me. I think I use a more vague indicator for this, like emotional tone plus general caution when someone is trying to persuade me of something.
Hm… this is interesting. I’m not too sure I understand what you mean though. Do you mind providing examples of what categories and indicators you use?
I think I’m missing something obvious, or I’m missing some information. Why is this clearly ridiculous?
Right, so, I think I may have omitted some relevant context here. In public forum debate, one of the primary ways to win is to ‘terminally outweigh on impacts’, or proving that a certain policy action prevents catastrophe. The ‘impact’ of preventing said catastrophe is so big that it negates all of your opponent’s arguments, even if they are completely legitimate. Think of it as an appeal to X-risk. The flip side is that our X-risk arguments tend to be highly unsophisticated and overall quite unlikely.
Consider this part:
] If we’re destroyed by a first strike, there is no MAD, and giving Russia the Arctic would immediately be an existential threat.
The unspoken but implicit argument is that Russia doesn’t need a reason to nuke us. If we give them the Arctic there’s no question, we will get nuked. (or at least, Russia is crazy enough to consider a full on nuclear attack, international fallout and nuclear winter be damned). This was actually what my opponents argued. My point relied on too many ridiculous assumptions. (a common and valid rebuttal of X risk arguments in debate)
Then there’s the factual rebuttal. I did a cursory overview of it, but I never fully elaborated. The idea is that multiple things prevent a successful nuclear first strike. First, and most obviously, would be the U.S nuclear triad. The idea is that we have a land (ICBM silos), sea (nuclear submarines), and air (bomber aircraft from supercarriers) deterrent against nuclear attacks. For a successful nuclear first strike to be performed Russia must locate all of our military assets (plus likely that of our NATO allies as well), take them all out at once, all while the CIA somehow never gets wind of a plan. It requires that Russia essentially be handed coordinates of where every single US nuke is, and for them to have the necessary delivery systems to destroy them. (good luck trying to reach an underwater sub, or an aircraft that’s currently flying) It also requires the biggest intelligence failure in world history.
Could it happen? Maybe? But then the chance is so small I’d rather bet on an asteroid destroying the earth within the next hour. In any case the plan wouldn’t rely on hypersonics. It’d rely on all American civilian and military leaders simultaneously developing Alzheimer’s. It’d also require the same to happen on the Russian side, since Russian nuclear doctrine is staunchly against use of nuclear weapons unless their own nuclear capabilities are threatened or if the Russian state is facing an existential threat (like say, imminent nuclear Armageddon).
For anyone who has studied the subject, this is rather basic knowledge, but then most judges (and debaters as well) don’t enter the room having already studied nuclear doctrine. Reactions like yours are thus part of what I was counting on when making the argument. It works because in general I can count on people not having prior knowledge. (don’t worry, you’re not alone) Thus, I can win by ‘outnerding’ them with my peculiar love for strange subjects.
However, the argument isn’t just ridiculous for anybody with knowledge of US/Russian nuclear doctrine. It also seems rather incongruous with most people’s model of the world (my debate partner stared at me as I made the argument, his expression was priceless). Suppose Russia was prepared to nuke the US, and had a credible first strike capability. Why isn’t Uncle Sam rushing to defend his security interests? Why haven’t pundits and politicians sounded the alarm? Why has there been no diplomatic incidents? A second Cuban missile crisis? A Russian nuclear attack somewhere else?
Overall, you could say that while my line of logic is not necessarily ridiculous (indeed, Kinzhal can reach the US) the conclusions I support (giving Russia the Arctic is an existential threat) definitely are. It’s ridiculous because it somehow postulates massive consequences while resulting in no real world action, independent of any facts. Imagine if I argued that the first AGI was discovered in 1924 before escaping from a secret lab (said AGI has apparently never made waves since). Regardless of history you can likely conclude I’m being a tinfoil hat conspiracy theorist.
I hope that answers your question! Is everything clear now?
Is the “ASI doom” argument meaningfully different or does it pattern match to “Ultra BS”.
The ASI doom argument as I understand it is:
(1) humans build and host ASI
(2) the ASI decides to seek power as an instrumental goal
(3) without the humans being aware of it, or being able to stop it, the ASI gains
(a) a place to exist and think at all (aka thousands of AI inference cards interconnected)
(b) increasing amounts of material resources that allow the machine to exist on it’s own and attack humans by some way (factories, nanotechnology, bioweapon labs etc)
(4) at a certain point, the ASI computes victory is likely due to (a) and (b) and it treacherous turns/attacks from hiding
Your hypersonic argument is:
(1) Russia improves and builds thousands of hypersonic nuclear tipped missiles. (Russia has the GDP of Florida, this is not a given)
(2) Russia decides to risk nuclear annihilation by killing all it’s rivals on the planet
(3) without NATO being aware of it, or being able to stop it, Russia gains:
(a) a massive secret arctic missile base or bases and/or submarines
(b) exact targeting coordinates for all of NATO’s nukes
(4) at a certain point, Russia determines they have enough of both (a) and (b) and they open fire with a first strike and destroy their rivals for earth, all by surprise.
I am noting that the reasons why ASI doom might not happen are similar:
It’s not actually clear when humans will invent a really strong ASI, it may not take linear amounts of compute to host one. (aka if it takes 80 H100s per human equivalent, a weak ASI might require 800, and a really strong ASI might require 800,000)
Human authorities would all have to simultaneously develop Alzheimer’s to not notice the missing clusters of compute able to host an ASI, or the vast robotic factories needed to develop bioweapons, nanotechnology, or a clanking robotics supply chain in order for the ASI to exist without humans
During this time period, why aren’t the humans using their own ASI to look for threats and develop countermeasures?
Anytime you disprove one point about AI doom, additional threat models are brought up. Or just “the ASI is smarter than you, therefore it wins” (which ignores you have your own ASIs, and ignores that it may not be possible to overcome a large resource advantage with intelligence). These “additional threats” often seem very BSish, from nanotechnology in a garage, a bioweapon from protein folding and reading papers, convincing a human to act against their own interests via super-persuasion.
It’s not provable—no current systems have any of the properties described, and the “ASI doom” advocates state that we all die if we build any system that might have those properties to verify the threat exists.
So, I think the difference is that ASI is ultimately far more difficult to prove on either side. However, the general framework maps pretty similarly.
Allow me to compare ASI with another X-risk scenario we’re familiar with, cold war MAD. The general argument goes:
The Cold War argument is:
(1) USSR improves and builds thousands of non-hypersonic nuclear tipped missiles. (did actually happen)
(2) USSR decides to risk nuclear annihilation by killing all it’s rivals on the planet
(3) due to miscalculations, perceived nuclear attack, and/or security threats, USSR gains:
(a) credible (or whatever passes for credible in that paranoid era) evidence they’re getting nuked
(4) at a certain point, Russia determines that today is the day to launch the nukes, and everyone dies
What’s the difference between this and hypersonics, or ASI? Ultimately, even if Washington and Moscow sat down and tried to give an accurate assessment of P(Armageddon) I doubt they’d have succeeded in producing an accurate estimate. The narrative is difficult to prove or disprove, all we know was that we came close (see Cuban missile crisis) but it never actually happened.
The issue for hypersonics isn’t the framework, it’s that the narrative itself fails to stand up to scrutiny (see my explanation). We know for a fact that those probabilities are extraordinarily unlikely. NATO doesn’t leave coordinates to nuclear launch sites lying around! Governments take nuclear threats very seriously! Unlike in the cold war I’d consider this narrative easily disprovable.
I have flagrantly disregarded relevant evidence suggesting that point 3 doesn’t happen.
With ASI we’re more or less completely in the dark. You can’t really verify if a point is ‘obviously not going to happen’, to the best of my understanding. Sure, you can say ‘probably’ or ‘probably not’, but you’d have to be the judge of that. There is less empirical evidence (that you presented, anyways) in regards to ASI being legitimate or not legitimate.
Is there an argument suggesting that ASI X risk is highly unlikely? I think it probably does exist, but then there may be rebuttals to that. Without full context it’s difficult to judge.
That said, this only applies to the ASI argument as you presented it. I’m sure my assessment will vary based off who and how the argument is presented, and what evidence is cited. But to the best of my understanding your ASI argument as presented is improvable on both sides. I could call it ultra-BS, but I think speculation is just as accurate a descriptor. To make it more than that you’ll need to cite evidence and address counterarguments, that’s what distinguished a good theory from BS and speculation.
So I think what you are saying is an ultra-BS argument is one that you know is obviously wrong. ASI doom or acceleration arguments are speculation where we don’t know which argument is wrong, since we don’t have access to an ASI. While for example we do know it’s difficult to locate a quiet submarine, it’s difficult to stop even subsonic bombers, it’s difficult to react in time to a large missile attack, and we have direct historical examples of all this in non-nuclear battles with the same weapons. For example the cruise missiles in Ukraine that keep impacting both side’s positions are just a warhead swap from being nuclear.
If you don’t know, you cannot justify a policy of preemptive nuclear war over AI. That’s kinda my point. I’m not even trying to say, object level, whether or not ASI actually will be a threat humans need to be willing to go to nuclear war over. I am saying the evidence right now does not support that conclusion. (it doesn’t support the conclusion that ASI is safe either, but it doesn’t justify the most extreme policy action)
So I think what you are saying is an ultra-BS argument is one that you know is obviously wrong.
Yep, pretty much. Part of the technique is knowing the ins and outs of our own argument. As I use ultra-BS prominently in debate, I need to be able to rebut the argument when I’m inevitably forced to argue the other side. I thus draw the distinction between ultra-BS along these lines. If it’s not obviously wrong (to me, anyways) it’s speculation. I can thus say that extended Chinese real economic stagnation for the next 10 years is educated speculation, while imminent Chinese economic collapse is ultra-BS.
If you don’t know, you cannot justify a policy of preemptive nuclear war over AI. That’s kinda my point. I’m not even trying to say, object level, whether or not ASI actually will be a threat humans need to be willing to go to nuclear war over. I am saying the evidence right now does not support that conclusion. (it doesn’t support the conclusion that ASI is safe either, but it doesn’t justify the most extreme policy action)
So, this is where I withdraw into acknowledging my limits. I don’t believe I have read sufficient ASI literature to fully understand this point, so I’m not too comfortable offering any object level predictions or narrative assessments. I can agree that many ASI arguments follow the same narrative format as ultra-BS, and there are likely many bad ASI arguments which can be revealed as wrong through careful (or even cursory) research. However, I’m not sufficiently educated on the subject to actually evaluate the narrative, thus the unsatisfactory response of ‘I’m not sure, sorry’.
However, if your understanding of ASI is correct, and there indeed is insufficient provable evidence, then yes, I can agree ASI policies cannot be argued for with provable evidence. Note again, however, that this would essentially be me taking your word for everything, which I’m not comfortable doing.
Currently, my priors on ASI ruin are limited, and I’ll likely need to do more specific research on the topic.
However, if your understanding of ASI is correct, and there indeed is insufficient provable evidence, then yes, I can agree ASI policies cannot be argued for with provable evidence. Note again, however, that this would essentially be me taking your word for everything, which I’m not comfortable doing.
So in this particular scenario, those concerned about ASI doom aren’t asking for a small or reasonable policy action proportional to today’s uncertainty. They are asking for AI pauses and preemptive nuclear war.
AI pauses will cost an enormous amount of money, some of which is tax revenue.
Preemptive nuclear war is potential suicide. It’s asking for a country to risk the deaths of approximately 50% of it’s population in the near term, and to lose all it’s supply chains, turning it into broken third world country separated by radioactive craters on all the transit and food supply hubs, which would likely kill a large fraction of it’s remaining citizens.
To justify (1) you would need to have some level of evidence that the threat exists. To justify (2) I would expect you would need beyond a shadow of a doubt evidence that the threat exists.
So for (1) convincing evidence might be a weak ASI that is hostile needs to exist in the lab before the threat can be claimed to be real. For (2) researchers would need to have produced in an isolated lab strong ASI, demonstrated that they were hostile, and tried thousands of times to make a safe ASI with a 100% failure rate.
I think we could argue about the exact level of evidence needed, or briefly establish plausible ways that (1) and (2) could fail to show a threat, but in general I would say the onus is on AI doom advocates to prove the threat is real, not on advocates for “business as usual” technology development to prove it is not. I think this last part is the dark arts scam, that and other hidden assumptions that get treated as certainty. (a lot of the hidden assumptions are in the technical details of how an ASI is assumed to work by someone with less detailed technical knowledge, vs the way actual ML systems work today)
Another part of the scam is the whole calling this “rational”. If your evidence on any topic is uncertain, and you can’t prove your point, certainty is unjustified, and it’s not a valid “agree to disagree” opinion. See: https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem .
So with this said, it seems like all I would need to do is show with a cite that ASI don’t exist yet, and show with a cite a reason, any reason at all, that plausibly could mean ASI are unable to be a threat. I don’t have to prove the reason is anything but plausible.
It does bother me that my proposal for proving ASI might not be a threat is suspiciously similar to how tobacco companies delayed any action to ban cigarettes essentially forever, but first they started with shoddy science to show that maybe the cigarettes weren’t the reason people were dying. Or how fossil fuel advocates have pulled the same scam, amplifying any doubts over climate change and thus delaying meaningful action for decades. (meaningful action is to research alternatives, which did succeed, but also to price carbon, which https://www.barrons.com/articles/europe-carbon-tax-emissions-climate-policy-1653e360 doesn’t even start until 2026, 50 years after the discovery of climate change)
These historical examples lead to a conclusion as well, I will see if you realize what this means for AI.
Thanks for the update! I think this is probably something important to take into consideration when evaluating ASI arguments.
That said, I think we’re starting to stray from the original topic of the Dark Arts, as we’re focusing more on ASI specifically rather than the Dark Arts element of it. In the interest of maintaining discussion focus on this post, would you agree to continuing AGI discussion in private messages?
And I was trying to focus on the dark arts part of the arguments. Note I don’t make any arguments about ASI in the above, just state that fairly weak evidence should be needed to justify not doing anything drastic about it at this time, because the drastic actions have high measurable costs. It’s not provable at present to state that “ASI could find a way to take over the planet with limited resources, because we don’t have an ASI or know the intelligence ROI on a given amount of flops”, but it is provable to state that “an AI pause of 6 months would cost tens of billions, possibly hundreds of billions of dollars and would reduce the relative power of the pausing countries internationally”. It’s also provable to state the damage of a nuclear exchange.
Look how it’s voted down to −10 on agreement : others feel very strongly about this issue.
Do you mind providing examples of what categories and indicators you use?
I can try to provide examples. The indicators might be too vague for the examples to help much with though!
A few weeks ago I met a fellow who seems to hail from old-guard atheism. Turn-of-the-century “Down with religion!” type of stuff. He was leading a philosophy discussion group I was checking out. At some point he said something (I don’t remember what) that made me think he didn’t understand what Vervaeke calls “the meaning crisis”. So I brought it up. He started going into a kind of pressured debate mode that I intuitively recognized from back when I swam in activist atheism circles. I had a hard time pinning down the moves he was doing, but I could tell I felt a kind of pressure, like I was being socially & logically pulled into a boxing ring. I realized after a few beats that he must have interpreted what I was saying as an assertion that God (as he thought others thought of God) is real. I still don’t know what rhetorical tricks he was doing, and I doubt any of them were conscious on his part, but I could tell that something screwy was going on because of the way interacting with him became tense and how others around us got uneasy and shifted how they were conversing. (Some wanted to engage & help the logic, some wanted to change the subject.)
Another example: Around a week ago I bumped into a strange character who runs a strange bookstore. A type of strange that I see as being common between Vassar and Ziz and Crowley, if that gives you a flavor. He was clearly on his way out the door, but as he headed out he directed some of his… attention-stuff… at me. I’m still not sure what exactly he was doing. On the surface it looked normal: he handed me a pamphlet with some of the info about their new brick-and-mortar store, along with their online store’s details. But there was something he was doing that was obviously about… keeping me off-balance. I think it was a general social thing he does: I watched him do it with the young man who was clearly a friend to him and who was tending the store. A part of me was fascinated. But another part of me was throwing up alarm bells. It felt like some kind of unknown frame manipulation. I couldn’t point at exactly how I was being affected, but I knew that I was, because my inner feet felt less firmly on inner ground in a way that was some kind of strategic.
More blatantly, the way that streetside preachers used to find a corner on college campuses and use a loudspeaker to spout off fundamentalist literalist Christianity memes. It’s obvious to me now that the memetic strategy here isn’t “You hear my ideas and then agree.” It’s somehow related to the way that it spurs debate. Back in my grad school days, I’d see clusters of undergrads surrounding these preachers and trying to argue with them, both sides engaging in predetermined patter. It was quite strange. I could feel the pull to argue with the preacher myself! But why? It has a snare trap feeling to it. I don’t understand the exact mechanism. I might be able to come up with a just-so story. But looking back it’s obvious that there’s a being-sucked-in feeling that’s somehow part of the memetic strategy. It’s built into the rhetoric. So a first-line immune response is “Nope.” Even though I have little idea what it is that I’m noping out of. Just its vibe.
I don’t think all (any?) of these fall under what you’re calling “ultra-BS”. That’s kind of my point: I think my rhetoric detector is tracking vibes more than techniques, and you’re naming a technique category. Something like that.
I think this part stands alone, so I’ll reply to the rest separately.
A few weeks ago I met a fellow who seems to hail from old-guard atheism. Turn-of-the-century “Down with religion!” type of stuff. He was leading a philosophy discussion group I was checking out. At some point he said something (I don’t remember what) that made me think he didn’t understand what Vervaeke calls “the meaning crisis”. So I brought it up. He started going into a kind of pressured debate mode that I intuitively recognized from back when I swam in activist atheism circles. I had a hard time pinning down the moves he was doing, but I could tell I felt a kind of pressure, like I was being socially & logically pulled into a boxing ring. I realized after a few beats that he must have interpreted what I was saying as an assertion that God (as he thought others thought of God) is real. I still don’t know what rhetorical tricks he was doing, and I doubt any of them were conscious on his part, but I could tell that something screwy was going on because of the way interacting with him became tense and how others around us got uneasy and shifted how they were conversing. (Some wanted to engage & help the logic, some wanted to change the subject.)
So, about this, I think this is a typical case of status game esque ‘social cognition’. If membership in a certain group is a big part of your identity, the group can’t be wrong. (Imagine if you’re a devout Churchgoer, and someone suggests your priest may be one of many pedophiles). There’s an instinctive reaction of ‘well, church is a big part of my life, and makes me feel like a full, happy person, very good vibes… unlike pedophilia’ so they snap to defending their local priest. You may see the ‘happens in other places but not here’ defense. Social cognition isn’t a full proof dark arts happened, but it usually is a good indicator (since by nature it tends to be irrational). In this case it’s an atheist who bases status on being an athiest feeling their personal beliefs/worth are being attacked, and responding as a result. I’d read up on Will Storr’s The Status Game if you’re interested.
Another example: Around a week ago I bumped into a strange character who runs a strange bookstore. A type of strange that I see as being common between Vassar and Ziz and Crowley, if that gives you a flavor. He was clearly on his way out the door, but as he headed out he directed some of his… attention-stuff… at me. I’m still not sure what exactly he was doing. On the surface it looked normal: he handed me a pamphlet with some of the info about their new brick-and-mortar store, along with their online store’s details. But there was something he was doing that was obviously about… keeping me off-balance. I think it was a general social thing he does: I watched him do it with the young man who was clearly a friend to him and who was tending the store. A part of me was fascinated. But another part of me was throwing up alarm bells. It felt like some kind of unknown frame manipulation. I couldn’t point at exactly how I was being affected, but I knew that I was, because my inner feet felt less firmly on inner ground in a way that was some kind of strategic.
I think I can understand in general terms what might’ve happened. There’s a lot of ways to ‘suggest’ something without verbally saying it. Think of an advertisement having a pretty girl in the product (look at you, so fat and ugly, don’t you want to be more like us?). It’s not explicit, of course, that’s the point, but it’s meant to take peripheral instead of central route persuasion.
From a more ‘human’ example, I might think of a negotiator seating their rival in front of the curtains while the sun is shining through to disorient them, or a parent asking one sibling to do something after having just yelled at another. In all cases there’s a hidden message of sorts, which can at times be difficult to put into words but is usually felt as a vibe. I have difficulty describing it myself.
I think one I can describe might be the sandwich example (though this isn’t something I’ve seen in my own life). You have something important to talk about someone with, and they’re maintaining eye contact and ‘paying attention’, but they’re also nibbling on the sandwich and enjoying themselves. (indirect communication: This is not too big of an issue). Or maybe they put the sandwich down occasionally check their watch, and their half eaten sandwich (why are you making me wait? can’t you see I’m hungry and busy?).
I obviously can’t say what exactly they did. But I think vibe wise the effect was similar to some of the techniques I illustrated above. They did something, it wasn’t apparent what, for a desired effect. I’ll call it peripheral techniques of communication (as opposed to central).
I think the preacher example is similar. (implicit message: I’m attacking you, your tribal groups, your status, and offering you some free status right now for beating me in front of your friends. Why don’t you come give it a try?) What specific technique they used, I’m not sure, but I think it had the effect of communicating an implicit message (thus the reaction).
And yes, you’re right, none of these are ‘ultra-BS’, I consider them different techniques with a different purpose. I do think they are techniques though, and someone familiar with them can recognize them.
Yep, I think you’re basically right on all accounts. Maybe a little off with the atheist fellow, but because of context I didn’t think to share until reading your analysis, and what you said is close enough!
It’s funny, I’m pretty familiar with this level of analysis, but I still notice myself thinking a little differently about the bookstore guy in light of what you’ve said here. I know people do the unbalancing thing you’re talking about. (Heck, I used to quite a lot! And probably still do in ways I haven’t learned to notice. Charisma is a hell of a drug when you’re chronically nervous!) But I didn’t think to think of it in these terms. Now I’m reflecting on the incident and noticing “Oh, yeah, okay, I can pinpoint a bunch of tiny details when I think of it this way.”
The fact that I couldn’t tell whether any of these were “ultra-BS” is more the central point to me.
If I could trouble you to name it: Is there a more everyday kind of example of ultra-BS? Not in debate or politics?
It’s funny, I’m pretty familiar with this level of analysis, but I still notice myself thinking a little differently about the bookstore guy in light of what you’ve said here. I know people do the unbalancing thing you’re talking about. (Heck, I used to quite a lot! And probably still do in ways I haven’t learned to notice. Charisma is a hell of a drug when you’re chronically nervous!) But I didn’t think to think of it in these terms. Now I’m reflecting on the incident and noticing “Oh, yeah, okay, I can pinpoint a bunch of tiny details when I think of it this way.”
Glad you appreciated my analysis!
The fact that I couldn’t tell whether any of these were “ultra-BS” is more the central point to me.
Hm… I think we may have miscommunicated somewhere. From what I understand at least, what you saw was distinctly not ‘ultra-BS’ as I envision it.
In persuasion, students of rhetoric generally classify two types of persuasive styles, ‘central’ and ‘peripheral’, route, specifically. Whereas central route persuasion focuses more on overt appeals to logic, peripheral route focuses more on other factors. Consider, for instance, the difference between an advertisement extolling the nutritional benefits of their drink, as opposed to an ad for the same company showing a half naked girl sampling it. Both aim to ‘convince’ the consumer to buy their product, except one employs a much different strategy than the other.
More generally, central route persuasion is explicit. We want you to convince you of ‘X’, here are the arguments for ‘X’. The drink is nutritious and good for your health, you should Buy the Drink. Peripheral route persuasion is more implicit, though at times it’s no less subtle. This pretty and sexually appealing girl loves this drink, why don’t you? Doesn’t evolution make you predisposed to trust pretty people? Wouldn’t you want to be more like them?Buy the drink
I consider ultra-BS a primarily ‘central route’ argument, as the practitioner uses explicit reasoning to support explicit narrative arguments. It’s often ill intentioned sure, and clearly motivated, intellectually dishonest reasoning, but that’s besides the point. It still falls under the category of ‘central route’ arguments.
Putting someone off balance, on the other hand, is more ‘peripheral route’ persuasion. There’s far more emphasis on the implicit messaging. You don’t know what you’re doing, do you? Trust me instead, come on.
In the case of your atheist friend, it’s not really possible to tell what persuasion technique they used, because it wasn’t really clear. But the indicators you received were accurate, because under those conditions he would be incentivized to use dishonest techniques like ultra-BS. That’s not to say, however, that they did use ultra-BS!
In that sense, I think I might conclude that your implicit primers and vibes are very good at detecting implicit persuasion, which typically but not always has a correlation with dark artsy techniques. Dark Arts often relies on implicit messaging, because if the message were explicit (see with sexual advertising techniques) it would be, well… ridiculous. (’So I should buy your product just because one pretty person drunk it? What kinda logic is that?)
However, ‘ultra-BS’ is an explicit technique, which is why I believe your typical indicators failed. You saw the indicators for what you’re used to associating with ‘honest discussion’, indicators like evidence, a coherent narrative, and good presentation skills. In a interpersonal setting, these indicators likely would’ve been sufficient. Not so in politics.
That said...
If I could trouble you to name it: Is there a more everyday kind of example of ultra-BS? Not in debate or politics?
This is a bit hard, since ‘ultra-BS’ is a technique designed for the environment of politics by a special kind of dishonest people. Regular people tend to be intellectually honest. You won’t see them support a policy one moment and oppose it the same evening. You also don’t see them wielding more sophisticated evidences and proofs in daily discussion, which is why we see ‘ultra-BS’ far less often in everyday life. If someone is pulling out evidence at all chances are they’ve already ‘won’ the argument. Regular people also tend to have far less stake/interest in their political positions, unlike say, debaters or politicians. The incentives and structure of the format is different.
The most similar example I can think of off the top of my head is a spat between domestic partners. Say, Alice and Bob.
Alice: You never take out the trash (evidence), look after the kids (evidence), or say you care about me (evidence). And now you’ve forgotten about our anniversary? (evidence) How dare you?? Do you really care about me? (narrative: Bob doesn’t care about Alice)
But then, this isn’t a perfect fit for ultra-BS, since 1) Alice isn’t necessarily aware she’s overgeneralizing 2) Alice doesn’t care about the specific examples she uses, she’s just as likely responding to a ‘vibe’ of laziness or lack of care from her partner. 3) The evidence is well… not very sophisticated.
But it general, I guess it’s similar in that Alice is supporting a dubious narrative with credible evidence (a pretty general summary of ‘ultra-BS’). Sure, Bob did do all these things, and probably cares for Alice in other ways which she isn’t acknowledging (or who knows, maybe he really doesn’t care about Alice).
Is this example satisfying?
Thanks for the response in any case, I really enjoy these discussions! Would you like to do a dialogue sometime?
I consider ultra-BS a primarily ‘central route’ argument, as the practitioner uses explicit reasoning to support explicit narrative arguments. […]
Putting someone off balance, on the other hand, is more ‘peripheral route’ persuasion. There’s far more emphasis on the implicit messaging.
Ah! This distinction helped clarify a fair bit for me. Thank you!
…I think I might conclude that your implicit primers and vibes are very good at detecting implicit persuasion, which typically but not always has a correlation with dark artsy techniques.
I agree on all accounts here. I think I dumped most of my DADA skill points into implicit detection. And yes, the vibes thing isn’t a perfect correlation to Dark stuff, I totally agree.
Is this example satisfying?
It’s definitely helpful! The category still isn’t crisp in my mind, but it’s a lot clearer. Thank you!
Thanks for the response in any case, I really enjoy these discussions! Would you like to do a dialogue sometime?
I’ve really enjoyed this exchange too. Thank you!
And sure, I’d be up for a dialogue sometime. I don’t have a good intuition for what kind of thing goes well in dialogues yet, so maybe take the lead if & when you feel inspired to invite me into one?
The unspoken but implicit argument is that Russia doesn’t need a reason to nuke us. If we give them the Arctic there’s no question, we will get nuked.
Ah, interesting, I didn’t read that assumption into it. I read it as “The power balance will have changed, which will make Russia’s international bargaining position way stronger because now it has a credible threat against mainland USA.”
I see the thing you’re pointing out as implicit though. Like an appeal to raw animal fear.
For a successful nuclear first strike to be performed Russia must locate all of our military assets (plus likely that of our NATO allies as well), take them all out at once, all while the CIA somehow never gets wind of a plan.
That makes a lot of sense. I didn’t know about the distributed and secret nature of our nuclear capabilities… but it’s kind of obvious that that’s how it’d be set up, now that you say so. Thank you for spelling this out.
Reactions like yours are thus part of what I was counting on when making the argument. It works because in general I can count on people not having prior knowledge. (don’t worry, you’re not alone)
Makes sense!
And I wasn’t worried. I’m actually not concerned about sounding like (or being!) an idiot. I’m just me, and I have the questions I do! But thank you for the kindness in your note here.
It also seems rather incongruous with most people’s model of the world […]. Suppose Russia was prepared to nuke the US, and had a credible first strike capability. Why isn’t Uncle Sam rushing to defend his security interests? Why haven’t pundits and politicians sounded the alarm? Why has there been no diplomatic incidents? A second Cuban missile crisis? A Russian nuclear attack somewhere else?
I gotta admit, my faith in the whole system is pretty low on axes like this. The collective response to Covid was idiotic. I could imagine the system doing some stupid things simply because it’s too gummed up and geriatric to do better.
That’s not my main guess about what’s happening here. I honestly just didn’t think through this level of thing when I first read your arctic argument from your debate. But collective ineptitude is plausible enough to me that the things you’re pointing out here just don’t land as damning.
But they definitely are points against. Thank you for pointing them out!
I hope that answers your question! Is everything clear now?
For this instance, yes!
There’s some kind of generalization that hasn’t happened for me yet. I’m not sure what to ask exactly. I think this whole topic (RE what you’re saying about Dark Arts) is bumping into a weak spot in my mind that I wasn’t aware was weak. I’ll need to watch it & observe other examples & let it settle in.
But for this case: yes, much clearer!
Thank you for taking the time to spell all this out!
I think that part of it is probably you not having much experience with debate or debate adjacent fields. (quite understandable, given how toxic it’s become). It took me some lived experience to recognize it, after all.
If you want to see it at work, I recommend just tuning into any politician during a debate. I think you’ll start recognizing stuff pretty quick. Wish you happy hunting in any case.
I think I’m missing something obvious, or I’m missing some information. Why is this clearly ridiculous?
Nuclear triad aside, there’s the fact that the Arctic is more than 1000 miles away from the nearest US land (about 1700 miles away from Montana, 3000 miles away from Texas), that Siberia is already roughly as close.
And of course, the fact the Arctic is made of, well, ice, that melts more and more as the climate warms, and thus not the best place to build a missile base on.
Even without familiarity with nuclear politics, the distance part can be checked in less than 2 minutes on Google Map; if you have access to an internet connection and judges that penalize blatant falsehoods like “they can hit us from the Arctic”, you absolutely wreck your adversary with some quick checking.
Of course, in a lot of debate formats you’re not allowed the two minutes it would take to do a google map check.
Nuclear triad aside, there’s the fact that the Arctic is more than 1000 miles away from the nearest US land (about 1700 miles away from Montana, 3000 miles away from Texas), that Siberia is already roughly as close.
Well, there’s Alaska, but yes, part of Russia is only ~55 miles away from Alaska, so the overall point stands that Russia having a greater presence in the Arctic doesn’t change things very much.
And of course, the fact the Arctic is made of, well, ice, that melts more and more as the climate warms, and thus not the best place to build a missile base on.
That’s not what is being proposed—it is building more bases in ports on the land where the water doesn’t freeze as much because of climate change.
Thank you. I found this exchange very enriching.
In particular, it highlights a gap in my way of reasoning. I notice that even after you give examples, the category of “ultra-BS” doesn’t really gel for me. I think I use a more vague indicator for this, like emotional tone plus general caution when someone is trying to persuade me of something.
In the spirit of crisping up my understanding, I have a question:
I think I’m missing something obvious, or I’m missing some information. Why is this clearly ridiculous?
I’m glad you enjoyed it!
Hm… this is interesting. I’m not too sure I understand what you mean though. Do you mind providing examples of what categories and indicators you use?
Right, so, I think I may have omitted some relevant context here. In public forum debate, one of the primary ways to win is to ‘terminally outweigh on impacts’, or proving that a certain policy action prevents catastrophe. The ‘impact’ of preventing said catastrophe is so big that it negates all of your opponent’s arguments, even if they are completely legitimate. Think of it as an appeal to X-risk. The flip side is that our X-risk arguments tend to be highly unsophisticated and overall quite unlikely.
Consider this part:
The unspoken but implicit argument is that Russia doesn’t need a reason to nuke us. If we give them the Arctic there’s no question, we will get nuked. (or at least, Russia is crazy enough to consider a full on nuclear attack, international fallout and nuclear winter be damned). This was actually what my opponents argued. My point relied on too many ridiculous assumptions. (a common and valid rebuttal of X risk arguments in debate)
Then there’s the factual rebuttal. I did a cursory overview of it, but I never fully elaborated. The idea is that multiple things prevent a successful nuclear first strike. First, and most obviously, would be the U.S nuclear triad. The idea is that we have a land (ICBM silos), sea (nuclear submarines), and air (bomber aircraft from supercarriers) deterrent against nuclear attacks. For a successful nuclear first strike to be performed Russia must locate all of our military assets (plus likely that of our NATO allies as well), take them all out at once, all while the CIA somehow never gets wind of a plan. It requires that Russia essentially be handed coordinates of where every single US nuke is, and for them to have the necessary delivery systems to destroy them. (good luck trying to reach an underwater sub, or an aircraft that’s currently flying) It also requires the biggest intelligence failure in world history.
Could it happen? Maybe? But then the chance is so small I’d rather bet on an asteroid destroying the earth within the next hour. In any case the plan wouldn’t rely on hypersonics. It’d rely on all American civilian and military leaders simultaneously developing Alzheimer’s. It’d also require the same to happen on the Russian side, since Russian nuclear doctrine is staunchly against use of nuclear weapons unless their own nuclear capabilities are threatened or if the Russian state is facing an existential threat (like say, imminent nuclear Armageddon).
For anyone who has studied the subject, this is rather basic knowledge, but then most judges (and debaters as well) don’t enter the room having already studied nuclear doctrine. Reactions like yours are thus part of what I was counting on when making the argument. It works because in general I can count on people not having prior knowledge. (don’t worry, you’re not alone) Thus, I can win by ‘outnerding’ them with my peculiar love for strange subjects.
However, the argument isn’t just ridiculous for anybody with knowledge of US/Russian nuclear doctrine. It also seems rather incongruous with most people’s model of the world (my debate partner stared at me as I made the argument, his expression was priceless). Suppose Russia was prepared to nuke the US, and had a credible first strike capability. Why isn’t Uncle Sam rushing to defend his security interests? Why haven’t pundits and politicians sounded the alarm? Why has there been no diplomatic incidents? A second Cuban missile crisis? A Russian nuclear attack somewhere else?
Overall, you could say that while my line of logic is not necessarily ridiculous (indeed, Kinzhal can reach the US) the conclusions I support (giving Russia the Arctic is an existential threat) definitely are. It’s ridiculous because it somehow postulates massive consequences while resulting in no real world action, independent of any facts. Imagine if I argued that the first AGI was discovered in 1924 before escaping from a secret lab (said AGI has apparently never made waves since). Regardless of history you can likely conclude I’m being a tinfoil hat conspiracy theorist.
I hope that answers your question! Is everything clear now?
Is the “ASI doom” argument meaningfully different or does it pattern match to “Ultra BS”.
The ASI doom argument as I understand it is:
(1) humans build and host ASI
(2) the ASI decides to seek power as an instrumental goal
(3) without the humans being aware of it, or being able to stop it, the ASI gains
(a) a place to exist and think at all (aka thousands of AI inference cards interconnected)
(b) increasing amounts of material resources that allow the machine to exist on it’s own and attack humans by some way (factories, nanotechnology, bioweapon labs etc)
(4) at a certain point, the ASI computes victory is likely due to (a) and (b) and it treacherous turns/attacks from hiding
Your hypersonic argument is:
(1) Russia improves and builds thousands of hypersonic nuclear tipped missiles. (Russia has the GDP of Florida, this is not a given)
(2) Russia decides to risk nuclear annihilation by killing all it’s rivals on the planet
(3) without NATO being aware of it, or being able to stop it, Russia gains:
(a) a massive secret arctic missile base or bases and/or submarines
(b) exact targeting coordinates for all of NATO’s nukes
(4) at a certain point, Russia determines they have enough of both (a) and (b) and they open fire with a first strike and destroy their rivals for earth, all by surprise.
I am noting that the reasons why ASI doom might not happen are similar:
It’s not actually clear when humans will invent a really strong ASI, it may not take linear amounts of compute to host one. (aka if it takes 80 H100s per human equivalent, a weak ASI might require 800, and a really strong ASI might require 800,000)
Human authorities would all have to simultaneously develop Alzheimer’s to not notice the missing clusters of compute able to host an ASI, or the vast robotic factories needed to develop bioweapons, nanotechnology, or a clanking robotics supply chain in order for the ASI to exist without humans
During this time period, why aren’t the humans using their own ASI to look for threats and develop countermeasures?
Anytime you disprove one point about AI doom, additional threat models are brought up. Or just “the ASI is smarter than you, therefore it wins” (which ignores you have your own ASIs, and ignores that it may not be possible to overcome a large resource advantage with intelligence). These “additional threats” often seem very BSish, from nanotechnology in a garage, a bioweapon from protein folding and reading papers, convincing a human to act against their own interests via super-persuasion.
It’s not provable—no current systems have any of the properties described, and the “ASI doom” advocates state that we all die if we build any system that might have those properties to verify the threat exists.
Interesting question!
So, I think the difference is that ASI is ultimately far more difficult to prove on either side. However, the general framework maps pretty similarly.
Allow me to compare ASI with another X-risk scenario we’re familiar with, cold war MAD. The general argument goes:
What’s the difference between this and hypersonics, or ASI? Ultimately, even if Washington and Moscow sat down and tried to give an accurate assessment of P(Armageddon) I doubt they’d have succeeded in producing an accurate estimate. The narrative is difficult to prove or disprove, all we know was that we came close (see Cuban missile crisis) but it never actually happened.
The issue for hypersonics isn’t the framework, it’s that the narrative itself fails to stand up to scrutiny (see my explanation). We know for a fact that those probabilities are extraordinarily unlikely. NATO doesn’t leave coordinates to nuclear launch sites lying around! Governments take nuclear threats very seriously! Unlike in the cold war I’d consider this narrative easily disprovable.
I have flagrantly disregarded relevant evidence suggesting that point 3 doesn’t happen.
With ASI we’re more or less completely in the dark. You can’t really verify if a point is ‘obviously not going to happen’, to the best of my understanding. Sure, you can say ‘probably’ or ‘probably not’, but you’d have to be the judge of that. There is less empirical evidence (that you presented, anyways) in regards to ASI being legitimate or not legitimate.
Is there an argument suggesting that ASI X risk is highly unlikely? I think it probably does exist, but then there may be rebuttals to that. Without full context it’s difficult to judge.
That said, this only applies to the ASI argument as you presented it. I’m sure my assessment will vary based off who and how the argument is presented, and what evidence is cited. But to the best of my understanding your ASI argument as presented is improvable on both sides. I could call it ultra-BS, but I think speculation is just as accurate a descriptor. To make it more than that you’ll need to cite evidence and address counterarguments, that’s what distinguished a good theory from BS and speculation.
So I think what you are saying is an ultra-BS argument is one that you know is obviously wrong. ASI doom or acceleration arguments are speculation where we don’t know which argument is wrong, since we don’t have access to an ASI. While for example we do know it’s difficult to locate a quiet submarine, it’s difficult to stop even subsonic bombers, it’s difficult to react in time to a large missile attack, and we have direct historical examples of all this in non-nuclear battles with the same weapons. For example the cruise missiles in Ukraine that keep impacting both side’s positions are just a warhead swap from being nuclear.
With that said, isn’t “high confidence” in your speculation by definition wrong? If you don’t know, how can you know that it’s almost certain an ASI will defeat and kill humanity? AGI Ruin: A List of Lethalities and https://twitter.com/ESYudkowsky/status/1658616828741160960 . Of course thermodynamics has taken it’s sweet time building ASI : https://twitter.com/BasedBeffJezos/status/1670640570388336640
If you don’t know, you cannot justify a policy of preemptive nuclear war over AI. That’s kinda my point. I’m not even trying to say, object level, whether or not ASI actually will be a threat humans need to be willing to go to nuclear war over. I am saying the evidence right now does not support that conclusion. (it doesn’t support the conclusion that ASI is safe either, but it doesn’t justify the most extreme policy action)
Yep, pretty much. Part of the technique is knowing the ins and outs of our own argument. As I use ultra-BS prominently in debate, I need to be able to rebut the argument when I’m inevitably forced to argue the other side. I thus draw the distinction between ultra-BS along these lines. If it’s not obviously wrong (to me, anyways) it’s speculation. I can thus say that extended Chinese real economic stagnation for the next 10 years is educated speculation, while imminent Chinese economic collapse is ultra-BS.
So, this is where I withdraw into acknowledging my limits. I don’t believe I have read sufficient ASI literature to fully understand this point, so I’m not too comfortable offering any object level predictions or narrative assessments. I can agree that many ASI arguments follow the same narrative format as ultra-BS, and there are likely many bad ASI arguments which can be revealed as wrong through careful (or even cursory) research. However, I’m not sufficiently educated on the subject to actually evaluate the narrative, thus the unsatisfactory response of ‘I’m not sure, sorry’.
However, if your understanding of ASI is correct, and there indeed is insufficient provable evidence, then yes, I can agree ASI policies cannot be argued for with provable evidence. Note again, however, that this would essentially be me taking your word for everything, which I’m not comfortable doing.
Currently, my priors on ASI ruin are limited, and I’ll likely need to do more specific research on the topic.
So in this particular scenario, those concerned about ASI doom aren’t asking for a small or reasonable policy action proportional to today’s uncertainty. They are asking for AI pauses and preemptive nuclear war.
Pause: https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Nuclear war: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
AI pauses will cost an enormous amount of money, some of which is tax revenue.
Preemptive nuclear war is potential suicide. It’s asking for a country to risk the deaths of approximately 50% of it’s population in the near term, and to lose all it’s supply chains, turning it into broken third world country separated by radioactive craters on all the transit and food supply hubs, which would likely kill a large fraction of it’s remaining citizens.
To justify (1) you would need to have some level of evidence that the threat exists. To justify (2) I would expect you would need beyond a shadow of a doubt evidence that the threat exists.
So for (1) convincing evidence might be a weak ASI that is hostile needs to exist in the lab before the threat can be claimed to be real. For (2) researchers would need to have produced in an isolated lab strong ASI, demonstrated that they were hostile, and tried thousands of times to make a safe ASI with a 100% failure rate.
I think we could argue about the exact level of evidence needed, or briefly establish plausible ways that (1) and (2) could fail to show a threat, but in general I would say the onus is on AI doom advocates to prove the threat is real, not on advocates for “business as usual” technology development to prove it is not. I think this last part is the dark arts scam, that and other hidden assumptions that get treated as certainty. (a lot of the hidden assumptions are in the technical details of how an ASI is assumed to work by someone with less detailed technical knowledge, vs the way actual ML systems work today)
Another part of the scam is the whole calling this “rational”. If your evidence on any topic is uncertain, and you can’t prove your point, certainty is unjustified, and it’s not a valid “agree to disagree” opinion. See: https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem .
So with this said, it seems like all I would need to do is show with a cite that ASI don’t exist yet, and show with a cite a reason, any reason at all, that plausibly could mean ASI are unable to be a threat. I don’t have to prove the reason is anything but plausible.
It does bother me that my proposal for proving ASI might not be a threat is suspiciously similar to how tobacco companies delayed any action to ban cigarettes essentially forever, but first they started with shoddy science to show that maybe the cigarettes weren’t the reason people were dying. Or how fossil fuel advocates have pulled the same scam, amplifying any doubts over climate change and thus delaying meaningful action for decades. (meaningful action is to research alternatives, which did succeed, but also to price carbon, which https://www.barrons.com/articles/europe-carbon-tax-emissions-climate-policy-1653e360 doesn’t even start until 2026, 50 years after the discovery of climate change)
These historical examples lead to a conclusion as well, I will see if you realize what this means for AI.
Thanks for the update! I think this is probably something important to take into consideration when evaluating ASI arguments.
That said, I think we’re starting to stray from the original topic of the Dark Arts, as we’re focusing more on ASI specifically rather than the Dark Arts element of it. In the interest of maintaining discussion focus on this post, would you agree to continuing AGI discussion in private messages?
Sure. Feel free to PM.
And I was trying to focus on the dark arts part of the arguments. Note I don’t make any arguments about ASI in the above, just state that fairly weak evidence should be needed to justify not doing anything drastic about it at this time, because the drastic actions have high measurable costs. It’s not provable at present to state that “ASI could find a way to take over the planet with limited resources, because we don’t have an ASI or know the intelligence ROI on a given amount of flops”, but it is provable to state that “an AI pause of 6 months would cost tens of billions, possibly hundreds of billions of dollars and would reduce the relative power of the pausing countries internationally”. It’s also provable to state the damage of a nuclear exchange.
Look how it’s voted down to −10 on agreement : others feel very strongly about this issue.
I can try to provide examples. The indicators might be too vague for the examples to help much with though!
A few weeks ago I met a fellow who seems to hail from old-guard atheism. Turn-of-the-century “Down with religion!” type of stuff. He was leading a philosophy discussion group I was checking out. At some point he said something (I don’t remember what) that made me think he didn’t understand what Vervaeke calls “the meaning crisis”. So I brought it up. He started going into a kind of pressured debate mode that I intuitively recognized from back when I swam in activist atheism circles. I had a hard time pinning down the moves he was doing, but I could tell I felt a kind of pressure, like I was being socially & logically pulled into a boxing ring. I realized after a few beats that he must have interpreted what I was saying as an assertion that God (as he thought others thought of God) is real. I still don’t know what rhetorical tricks he was doing, and I doubt any of them were conscious on his part, but I could tell that something screwy was going on because of the way interacting with him became tense and how others around us got uneasy and shifted how they were conversing. (Some wanted to engage & help the logic, some wanted to change the subject.)
Another example: Around a week ago I bumped into a strange character who runs a strange bookstore. A type of strange that I see as being common between Vassar and Ziz and Crowley, if that gives you a flavor. He was clearly on his way out the door, but as he headed out he directed some of his… attention-stuff… at me. I’m still not sure what exactly he was doing. On the surface it looked normal: he handed me a pamphlet with some of the info about their new brick-and-mortar store, along with their online store’s details. But there was something he was doing that was obviously about… keeping me off-balance. I think it was a general social thing he does: I watched him do it with the young man who was clearly a friend to him and who was tending the store. A part of me was fascinated. But another part of me was throwing up alarm bells. It felt like some kind of unknown frame manipulation. I couldn’t point at exactly how I was being affected, but I knew that I was, because my inner feet felt less firmly on inner ground in a way that was some kind of strategic.
More blatantly, the way that streetside preachers used to find a corner on college campuses and use a loudspeaker to spout off fundamentalist literalist Christianity memes. It’s obvious to me now that the memetic strategy here isn’t “You hear my ideas and then agree.” It’s somehow related to the way that it spurs debate. Back in my grad school days, I’d see clusters of undergrads surrounding these preachers and trying to argue with them, both sides engaging in predetermined patter. It was quite strange. I could feel the pull to argue with the preacher myself! But why? It has a snare trap feeling to it. I don’t understand the exact mechanism. I might be able to come up with a just-so story. But looking back it’s obvious that there’s a being-sucked-in feeling that’s somehow part of the memetic strategy. It’s built into the rhetoric. So a first-line immune response is “Nope.” Even though I have little idea what it is that I’m noping out of. Just its vibe.
I don’t think all (any?) of these fall under what you’re calling “ultra-BS”. That’s kind of my point: I think my rhetoric detector is tracking vibes more than techniques, and you’re naming a technique category. Something like that.
I think this part stands alone, so I’ll reply to the rest separately.
Oooh, I think I can classify some of this!
So, about this, I think this is a typical case of status game esque ‘social cognition’. If membership in a certain group is a big part of your identity, the group can’t be wrong. (Imagine if you’re a devout Churchgoer, and someone suggests your priest may be one of many pedophiles). There’s an instinctive reaction of ‘well, church is a big part of my life, and makes me feel like a full, happy person, very good vibes… unlike pedophilia’ so they snap to defending their local priest. You may see the ‘happens in other places but not here’ defense. Social cognition isn’t a full proof dark arts happened, but it usually is a good indicator (since by nature it tends to be irrational). In this case it’s an atheist who bases status on being an athiest feeling their personal beliefs/worth are being attacked, and responding as a result. I’d read up on Will Storr’s The Status Game if you’re interested.
I think I can understand in general terms what might’ve happened. There’s a lot of ways to ‘suggest’ something without verbally saying it. Think of an advertisement having a pretty girl in the product (look at you, so fat and ugly, don’t you want to be more like us?). It’s not explicit, of course, that’s the point, but it’s meant to take peripheral instead of central route persuasion.
From a more ‘human’ example, I might think of a negotiator seating their rival in front of the curtains while the sun is shining through to disorient them, or a parent asking one sibling to do something after having just yelled at another. In all cases there’s a hidden message of sorts, which can at times be difficult to put into words but is usually felt as a vibe. I have difficulty describing it myself.
I think one I can describe might be the sandwich example (though this isn’t something I’ve seen in my own life). You have something important to talk about someone with, and they’re maintaining eye contact and ‘paying attention’, but they’re also nibbling on the sandwich and enjoying themselves. (indirect communication: This is not too big of an issue). Or maybe they put the sandwich down occasionally check their watch, and their half eaten sandwich (why are you making me wait? can’t you see I’m hungry and busy?).
I obviously can’t say what exactly they did. But I think vibe wise the effect was similar to some of the techniques I illustrated above. They did something, it wasn’t apparent what, for a desired effect. I’ll call it peripheral techniques of communication (as opposed to central).
I think the preacher example is similar. (implicit message: I’m attacking you, your tribal groups, your status, and offering you some free status right now for beating me in front of your friends. Why don’t you come give it a try?) What specific technique they used, I’m not sure, but I think it had the effect of communicating an implicit message (thus the reaction).
And yes, you’re right, none of these are ‘ultra-BS’, I consider them different techniques with a different purpose. I do think they are techniques though, and someone familiar with them can recognize them.
Yep, I think you’re basically right on all accounts. Maybe a little off with the atheist fellow, but because of context I didn’t think to share until reading your analysis, and what you said is close enough!
It’s funny, I’m pretty familiar with this level of analysis, but I still notice myself thinking a little differently about the bookstore guy in light of what you’ve said here. I know people do the unbalancing thing you’re talking about. (Heck, I used to quite a lot! And probably still do in ways I haven’t learned to notice. Charisma is a hell of a drug when you’re chronically nervous!) But I didn’t think to think of it in these terms. Now I’m reflecting on the incident and noticing “Oh, yeah, okay, I can pinpoint a bunch of tiny details when I think of it this way.”
The fact that I couldn’t tell whether any of these were “ultra-BS” is more the central point to me.
If I could trouble you to name it: Is there a more everyday kind of example of ultra-BS? Not in debate or politics?
Glad you appreciated my analysis!
Hm… I think we may have miscommunicated somewhere. From what I understand at least, what you saw was distinctly not ‘ultra-BS’ as I envision it.
In persuasion, students of rhetoric generally classify two types of persuasive styles, ‘central’ and ‘peripheral’, route, specifically. Whereas central route persuasion focuses more on overt appeals to logic, peripheral route focuses more on other factors. Consider, for instance, the difference between an advertisement extolling the nutritional benefits of their drink, as opposed to an ad for the same company showing a half naked girl sampling it. Both aim to ‘convince’ the consumer to buy their product, except one employs a much different strategy than the other.
More generally, central route persuasion is explicit. We want you to convince you of ‘X’, here are the arguments for ‘X’. The drink is nutritious and good for your health, you should Buy the Drink. Peripheral route persuasion is more implicit, though at times it’s no less subtle. This pretty and sexually appealing girl loves this drink, why don’t you? Doesn’t evolution make you predisposed to trust pretty people? Wouldn’t you want to be more like them? Buy the drink
I consider ultra-BS a primarily ‘central route’ argument, as the practitioner uses explicit reasoning to support explicit narrative arguments. It’s often ill intentioned sure, and clearly motivated, intellectually dishonest reasoning, but that’s besides the point. It still falls under the category of ‘central route’ arguments.
Putting someone off balance, on the other hand, is more ‘peripheral route’ persuasion. There’s far more emphasis on the implicit messaging. You don’t know what you’re doing, do you? Trust me instead, come on.
In the case of your atheist friend, it’s not really possible to tell what persuasion technique they used, because it wasn’t really clear. But the indicators you received were accurate, because under those conditions he would be incentivized to use dishonest techniques like ultra-BS. That’s not to say, however, that they did use ultra-BS!
In that sense, I think I might conclude that your implicit primers and vibes are very good at detecting implicit persuasion, which typically but not always has a correlation with dark artsy techniques. Dark Arts often relies on implicit messaging, because if the message were explicit (see with sexual advertising techniques) it would be, well… ridiculous. (’So I should buy your product just because one pretty person drunk it? What kinda logic is that?)
However, ‘ultra-BS’ is an explicit technique, which is why I believe your typical indicators failed. You saw the indicators for what you’re used to associating with ‘honest discussion’, indicators like evidence, a coherent narrative, and good presentation skills. In a interpersonal setting, these indicators likely would’ve been sufficient. Not so in politics.
That said...
This is a bit hard, since ‘ultra-BS’ is a technique designed for the environment of politics by a special kind of dishonest people. Regular people tend to be intellectually honest. You won’t see them support a policy one moment and oppose it the same evening. You also don’t see them wielding more sophisticated evidences and proofs in daily discussion, which is why we see ‘ultra-BS’ far less often in everyday life. If someone is pulling out evidence at all chances are they’ve already ‘won’ the argument. Regular people also tend to have far less stake/interest in their political positions, unlike say, debaters or politicians. The incentives and structure of the format is different.
The most similar example I can think of off the top of my head is a spat between domestic partners. Say, Alice and Bob.
Alice: You never take out the trash (evidence), look after the kids (evidence), or say you care about me (evidence). And now you’ve forgotten about our anniversary? (evidence) How dare you?? Do you really care about me? (narrative: Bob doesn’t care about Alice)
But then, this isn’t a perfect fit for ultra-BS, since 1) Alice isn’t necessarily aware she’s overgeneralizing 2) Alice doesn’t care about the specific examples she uses, she’s just as likely responding to a ‘vibe’ of laziness or lack of care from her partner. 3) The evidence is well… not very sophisticated.
But it general, I guess it’s similar in that Alice is supporting a dubious narrative with credible evidence (a pretty general summary of ‘ultra-BS’). Sure, Bob did do all these things, and probably cares for Alice in other ways which she isn’t acknowledging (or who knows, maybe he really doesn’t care about Alice).
Is this example satisfying?
Thanks for the response in any case, I really enjoy these discussions! Would you like to do a dialogue sometime?
Ah! This distinction helped clarify a fair bit for me. Thank you!
I agree on all accounts here. I think I dumped most of my DADA skill points into implicit detection. And yes, the vibes thing isn’t a perfect correlation to Dark stuff, I totally agree.
It’s definitely helpful! The category still isn’t crisp in my mind, but it’s a lot clearer. Thank you!
I’ve really enjoyed this exchange too. Thank you!
And sure, I’d be up for a dialogue sometime. I don’t have a good intuition for what kind of thing goes well in dialogues yet, so maybe take the lead if & when you feel inspired to invite me into one?
Glad you enjoyed!
Let me send a PM regarding a dialogue…
Ah, interesting, I didn’t read that assumption into it. I read it as “The power balance will have changed, which will make Russia’s international bargaining position way stronger because now it has a credible threat against mainland USA.”
I see the thing you’re pointing out as implicit though. Like an appeal to raw animal fear.
That makes a lot of sense. I didn’t know about the distributed and secret nature of our nuclear capabilities… but it’s kind of obvious that that’s how it’d be set up, now that you say so. Thank you for spelling this out.
Makes sense!
And I wasn’t worried. I’m actually not concerned about sounding like (or being!) an idiot. I’m just me, and I have the questions I do! But thank you for the kindness in your note here.
I gotta admit, my faith in the whole system is pretty low on axes like this. The collective response to Covid was idiotic. I could imagine the system doing some stupid things simply because it’s too gummed up and geriatric to do better.
That’s not my main guess about what’s happening here. I honestly just didn’t think through this level of thing when I first read your arctic argument from your debate. But collective ineptitude is plausible enough to me that the things you’re pointing out here just don’t land as damning.
But they definitely are points against. Thank you for pointing them out!
For this instance, yes!
There’s some kind of generalization that hasn’t happened for me yet. I’m not sure what to ask exactly. I think this whole topic (RE what you’re saying about Dark Arts) is bumping into a weak spot in my mind that I wasn’t aware was weak. I’ll need to watch it & observe other examples & let it settle in.
But for this case: yes, much clearer!
Thank you for taking the time to spell all this out!
Of course. Glad you enjoyed!
I think that part of it is probably you not having much experience with debate or debate adjacent fields. (quite understandable, given how toxic it’s become). It took me some lived experience to recognize it, after all.
If you want to see it at work, I recommend just tuning into any politician during a debate. I think you’ll start recognizing stuff pretty quick. Wish you happy hunting in any case.
Nuclear triad aside, there’s the fact that the Arctic is more than 1000 miles away from the nearest US land (about 1700 miles away from Montana, 3000 miles away from Texas), that Siberia is already roughly as close.
And of course, the fact the Arctic is made of, well, ice, that melts more and more as the climate warms, and thus not the best place to build a missile base on.
Even without familiarity with nuclear politics, the distance part can be checked in less than 2 minutes on Google Map; if you have access to an internet connection and judges that penalize blatant falsehoods like “they can hit us from the Arctic”, you absolutely wreck your adversary with some quick checking.
Of course, in a lot of debate formats you’re not allowed the two minutes it would take to do a google map check.
Well, there’s Alaska, but yes, part of Russia is only ~55 miles away from Alaska, so the overall point stands that Russia having a greater presence in the Arctic doesn’t change things very much.
That’s not what is being proposed—it is building more bases in ports on the land where the water doesn’t freeze as much because of climate change.
Thanks for the addition! I actually didn’t consider this, and neither did my opponents.