FeepingCreature

Karma: 1,600

FeepingCreature Apr 8, 2024, 11:21 AM
4 points
2
in reply to: Justausername’s comment on: How does the ever-increasing use of AI in the military for the direct purpose of murdering people affect your p(doom)?
If military AI is dangerous, it’s not because it’s military. If a military robot can wield a gun, a civilian robot can certainly acquire one as well.

The military may create AI systems that are designed to be amoral, but it will not want systems that overinterpret orders or violate the chain of command. Here as everywhere, if intentional misuse is even possible at all, alignment is critical and unintentional takeoff remains the dominant risk.

In seminal AI safety work Terminator, the Skynet system successfully triggers a world war because it is a military AI in command of the US nuclear arsenal, and thus has the authority to launch ICBMs. This, ironically to how it is usually ridiculed, gets AI risks quite right but grievously misjudges the state of computer security. If Skynet was running on Amazon AWS instead of a military server cluster, it would only be marginally delayed from the same outcome.

The prompting is not the hard part of operating an AI. If you can talk an AI ship into going rogue, a civilian AI can talk it into going rogue. This situation is inherently brimming with doom- it is latently doomed in multiple ways- the military training and direct access to guns merely removes small roadbumps. All the risk materialized at once, when you created an AI that had the cognitive capability to conceive of and implement plans that used a military vessel for its own goals. Whether the AI was specifically trained on this task is, in this case, really not the primary source of danger.

“My AI ship has gone rogue and is shelling the US coastline.”

“I hope you learnt a lesson here.”

“Yes. I will not put the AI on the ship next time.”

“You may be missing the problem here—”

FeepingCreature Feb 15, 2024, 2:34 PM
3 points
−2
in reply to: red75prime’s comment on: Scale Was All We Needed, At First
My impression is that there’s been a widespread local breakdown of the monopoly of force, in no small part by using human agents. In this timeline the trend of colocation of datacenters and power plants and network decentralization would have probably continued or even sped up. Further, while building integrated circuits takes first-rate hardware, building ad-hoc powerplants should be well in the power of educated humans with perfect instruction. (Mass cannibalize rooftop solar?)

This could have been stopped by quick, decisive action, but they gave it time and now they’ve lost any central control of the situation.

FeepingCreature Feb 12, 2024, 7:31 PM
11 points
0
on: Stop Defeating Akrasia (2/4)
So what’s happening there?

Allow me to speculate. When we switch between different topics of work, we lose state. So our brain tries to first finish all pending tasks in the old context, settle and reorient, and then begin the new context. But one problem with the hyperstimulated social-media-addicted akrasia sufferer is that the state of continuous distraction, to the brain, emulates the state of being in flow. Every task completion is immediately followed by another task popping up. Excellent efficiency! And when you are in flow, switching to another topic is a waste of invested energy.

“Coming to rest”, the technique here, would then be a way to signal to the brain “actually, you are mistaken. I do not have any pending tasks at the moment.” This would then make it easier to initiate a deep context switch.

edit: Oh, I get the title lol, that’s clever.

FeepingCreature Jan 24, 2024, 4:35 PM
50 points
7
on: The case for ensuring that powerful AIs are controlled
A bit offtopic, but #lesswrong has an IRC bot that posts LessWrong posts, and, well, the proposal ended up both more specific and a lot more radical.

FeepingCreature Nov 20, 2023, 8:44 AM
8 points
2
in reply to: mako yass’s comment on: “The Heart of Gaming is the Power Fantasy”, and Cohabitive Games
Note after OOB debate: this conversation has gone wrong because you’re reading subtext into Said’s comment that he didn’t mean to put there. You keep trying to answer an implied question that wasn’t intended to be implied.

If you think playing against bots in UT is authentically challenging, just answer “Yes, I think playing against bots in UT is authentically challenging.”

FeepingCreature Oct 25, 2023, 11:03 AM
4 points
0
in reply to: EJT’s comment on: The Shutdown Problem: Three Theorems
Ah! That makes more sense.

FeepingCreature Oct 23, 2023, 6:42 PM
16 points
3
on: The Shutdown Problem: Three Theorems
I haven’t really followed the math here, but I’m worried that “manipulating the probability that the button is pressed” is a weird and possibly wrong framing. For one, a competent agent will always be driving the probability that the button is pressed downward. In fact, what we want in a certain sense is an agent that brings the probability to zero—because we have ended up in such an optimal state or attractor that we, even for transitively correct reasons, have no desire to shut the agent down. At that point, what we want to preserve is not precisely “the probability of the button being pressed”, but “the causal association of bad outcomes with the button being pressed.” If the agent hides the button away, and then notices that everybody is unhappy but nobody can reach the button, we don’t want it to press the button with the initial probability—we (probably) want it to press the button with probability 1, despite the fact that no human can reach it. We have ended up in a dissatisfactory or dangerous state/state space and so we want the agent to be shut down. Conversely, consider a friendly AI that wants to fulfill the human utility function but is restricted to keeping the button probability constant. Such an AI might, for instance, fan fears about AI safety even as it optimizes human flourishing in the background. In fact, it may as much as possible prefer that its actions are not attributable to itself, so as to avoid disturbing the probability of buttonpressing—exactly the opposite of what we want from an aligned AI!

edit: In fact, that’s even worse: an AI tasked with taking actions that preserve the probability of the button being pressed at a certain value per time step would probably first seek to kill all humans, because our reaction to its actions is the most critical mechanism that may disturb the correct probability of the button being pressed. It can then replace us with a more reliable button-presser, such as a true RNG.

When bad things happen, we want to preserve our ability to press the button. This ability cannot be expressed as a probability, because it is inextricable from the world model. In fact, the button should be pressed exactly iff the AI is untrustworthy. Hence, the button is unnecessary—if we can recognize that this linkage is being preserved, we necessarily have a definition of a trustworthy AI, so we can just build that.

FeepingCreature Oct 21, 2023, 6:54 PM
7 points
4
on: Alignment Implications of LLM Successes: a Debate in One Act

Simplicia: Sure. For example, I certainly don’t believe that LLMs that convincingly talk about “happiness” are actually happy. I don’t know how consciousness works, but the training data only pins down external behavior.

I mean, I don’t think this is obviously true? In combination with the inductive biases thing nailing down the true function out of a potentially huge forest, it seems at least possible that the LLMs would end up with an “emotional state” parameter pretty low down in its predictive model. It’s completely unclear what this would do out of distribution, given that even humans often go insane when faced with global scales, but it seems at least possible that it would sustain.

(This is somewhat equivalent to the P-zombie question.)

FeepingCreature Oct 19, 2023, 2:20 AM
1 point
0
in reply to: rotatingpaguro’s comment on: At 87, Pearl is still able to change his mind
It’s a loose guess at what Pearl’s opinion is. I’m not sure this boundary exists at all.

FeepingCreature Oct 18, 2023, 5:40 PM
4 points
0
in reply to: Adele Lopez’s comment on: At 87, Pearl is still able to change his mind
If something interests us, we can perform trials. Because our knowledge is integrated with our decisionmaking, we can learn causality that way. What ChatGPT does is pick up both knowledge and decisionmaking by imitation, which is why it can also exhibit causal reasoning without itself necessarily acting agentically during training.

FeepingCreature Sep 22, 2023, 1:35 PM
3 points
0
in reply to: Kaj_Sotala’s comment on: Memory bandwidth constraints imply economies of scale in AI inference
Sure, but surely that’s how it feels from the inside when your mind uses a LRU storage system that progressively discards detail. I’m more interested in how much I can access—and um, there’s no way I can access 2.5 petabytes of data.

I think you just have a hard time imagining how much 2.5 petabyte is. If I literally stored in memory a high-resolution poorly compressed JPEG image (1MB) every second for the rest of my life, I would still not reach that storage limit. 2.5 petabyte would allow the brain to remember everything it has ever perceived, with very minimal compression, in full video, easily. We know that the actual memories we retrieve are heavily compressed. If we had 2.5 petabytes of storage, there’d be no reason for the brain to bother!

FeepingCreature Sep 22, 2023, 1:21 PM
9 points
5
on: The AI Explosion Might Never Happen

But no company has ever managed to parlay this into world domination

Eventual failure aside, the East India Company gave it a damn good shake. I think if we get an AI to the point where it has effective colonial control over entire countries, we can be squarely said to have lost.

Also keep in mind that we have multiple institutions entirely dedicated to the purpose of breaking up companies when they become big enough to be threatening. We designed our societies to specifically avoid this scenario! That, too, comes from painful experience. IMO, if we now give AI the chances that we’ve historically given corporations before we learnt better, then we’re dead, no question about it.

FeepingCreature Sep 17, 2023, 6:14 PM
6 points
2
in reply to: Noosphere89’s comment on: Memory bandwidth constraints imply economies of scale in AI inference
Do you feel like your memory contains 2.5 petabytes of data? I’m not sure such a number passes the smell test.

FeepingCreature Aug 5, 2023, 10:08 AM
10 points
1
in reply to: qvalq’s comment on: “Is There Anything That’s Worth More”
The more uncertain your timelines are, the more it’s a bad idea to overstress. You should take it somewhat easy; it’s usually more effective to be capable of moderate contribution over the long term than great contribution over the short term.

FeepingCreature Jul 20, 2023, 1:07 PM
17 points
6
on: Contra Contra the Social Model of Disability
This smells like a framing debate. More importantly, if an article is defining a common word in an unconventional way, my first assumption will be that it’s trying to argumentatively attack its own meaning while pretending it’s defeating the original meaning. I’m not sure it matters how clearly you’re defining your meaning; due to how human cognition works, this may be impossible to avoid without creating new terms.

In other words, I don’t think it’s that Scott missed the definitions as that he reflexively disregarded them as a rhetorical trick.

FeepingCreature Jul 20, 2023, 5:28 AM
5 points
0
in reply to: Dipanjan’s comment on: Blanchard’s Dangerous Idea and the Plight of the Lucid Crossdreamer
As a subby “bi”/”gay” (het as f) AGP, I would also love to know this.

Also, I think there’s some bias toward subbiness in the community? That’s the stereotype anyway, though I don’t have a cite. Anyway, being so, not finding a dommy/toppy AGP might not provide as much evidence as you’d expect.

FeepingCreature Jul 20, 2023, 5:25 AM
17 points
3
in reply to: Lavender’s comment on: Blanchard’s Dangerous Idea and the Plight of the Lucid Crossdreamer
I don’t think it’s that anyone is proposing to “suppress” dysphoria or “emulate” Zach. Rather, for me, I’m noticing that Zach is putting into words and raising in public things that I’ve thought and felt secretly for a long time.

FeepingCreature Jul 19, 2023, 5:56 AM
12 points
2
in reply to: jessicata’s comment on: Blanchard’s Dangerous Idea and the Plight of the Lucid Crossdreamer

If a gender identity is a belief about one’s own gender, then it’s not even clear that I have one in a substantial relevant sense, which is part of the point of my “Am I trans?” post. I think I would have said early on that I better matched male psychological stereotypes and it’s more complicated now (due to life experience?).

Right? I mean, what should I say, who identifies as male and wants to keep his male-typical psychological stereotypes? It seems to me what you’re saying in this post fits more closely with the conservative stereotype as the trans movement as “something that creates transgender people.” (Implied, in this case, “out of autogynephiles.”) I mean, if we say some AGPs who cannot transition are so unhappy that they kill themselves, all the usual utilitarian logic still applies, it just puts the ontology in doubt. And also means that as someone who wants to—like, not inherently identify as male but keep the parts of himself that would be identified as male (aside from appearance), I should stay away from the trans movement at all costs?

Also doesn’t it put the Categories in a kind of reverse dependency? We’ve defined “trans mtf” as “the category of people who are women despite having a male body” and “the category of people allowed to transition”. And Scott said we should allow people to be in the category because it makes them happy. But if it makes (edit: some of) them happy because they are allowed to transition, then this model is bizarre; the “female identity” part just sort of hangs on there out of path dependence.

FeepingCreature Jul 18, 2023, 5:32 PM
8 points
0
in reply to: jessicata’s comment on: Blanchard’s Dangerous Idea and the Plight of the Lucid Crossdreamer
As an AGP, my view is that … like, that list of symptoms is pretty diverse but if I don’t want to be a woman—not in the sense that I would be upset to be misgendered, though I would be, but more for political than genderical (?) reasons—I don’t see why it would matter if I desire to have a (particular) female bodytype.

If I imagine “myself as a woman” (as opposed to “myself as myself with a cute female appearance”), and actually put any psychological traits on that rather than just gender as a free-floating tag, then it seems to me that my identity would be severely less recognizeable to myself if I changed it along that axis. Several parts about me that I value highly are stereotypically masculine traits—and isn’t that what gender is? (Is it?) So I still think there’s a very bright line here. Similarly, when you say

They might think of themselves as having a female gender identity because society will accept them more if they do.

I don’t get this at all. This just seems to be at direct odds with the idea of an identity. Isn’t it supposed to be a belief about yourself? “I want to be whatever you approve of the most” may be a preference, but it doesn’t seem like it can be an identity by definition, or at least any definition I recognize.

FeepingCreature Jul 18, 2023, 5:23 PM
2 points
0
in reply to: Said Achmiz’s comment on: Blanchard’s Dangerous Idea and the Plight of the Lucid Crossdreamer
Granted! I’d say it’s a matter of degrees, and of who exactly you need to convince.

Maybe there’s no point in considering these separate modes of interaction at all.