Noosphere89

Karma: 2,342

Noosphere89 22 Nov 2024 17:46 UTC
3 points
1
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
I don’t buy that o1 has actually given people expert-level bioweapons, so my actions here are more so about preparing for future AI that is very competent at bioweapon building.

Also, even with the current level of jailbreak resistance/adversarial example resistance, assuming no open-weights/open sourcing of AI is achieved, we can still make AIs that are practically hard to misuse by the general public.

See here for more:

https://www.lesswrong.com/posts/KENtuXySHJgxsH2Qk/managing-catastrophic-misuse-without-robust-ais

Noosphere89 22 Nov 2024 17:40 UTC
3 points
1
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
The answers to this question is actually 2 things:
1. This is why I expect we will eventually have to fight to ban open-source, and we will have to get the political will to ban both open-source and open-weights AI.
2. This is where the unlearning field comes in. If we could make the AI unlearn knowledge, an example being nuclear weapons, we could possibly distribute AI safely without causing novices to create dangerous stuff.
More here:

https://www.lesswrong.com/posts/mFAvspg4sXkrfZ7FA/deep-forgetting-and-unlearning-for-safely-scoped-llms

https://www.lesswrong.com/posts/9AbYkAy8s9LvB7dT5/the-case-for-unlearning-that-removes-information-from-llm

But the solutions are intentionally going to make AI safe without relying on alignment.

Noosphere89 22 Nov 2024 17:39 UTC
2 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
The threat model is plausible enough that some political actions should be done, like banning open-source/open-weight models, and putting in basic Know Your Customer checks.

Noosphere89 22 Nov 2024 16:56 UTC
2 points
3
in reply to: sunwillrise’s comment on: Bogdan Ionut Cirstea’s Shortform
I’d say the big factor that makes AI controllable right now is that the compute necessary to build AI that can do very good AI research to automate R&D and then the economy is locked behind TSMC/NVidia and ASML, and their processes are both nearly irreplaceable and very expensive to make, so it’s way easier to intervene on the checkpoints requiring AI development than gain-of function research.

Noosphere89 22 Nov 2024 15:20 UTC
3 points
0
in reply to: sunwillrise’s comment on: LLM chatbots have ~half of the kinds of “consciousness” that humans believe in. Humans should avoid going crazy about that.
Yeah, this theory definitely needs far better methodologies for testing this theory, and while I wouldn’t be surprised by at least part of the answer/solution to the Hard Problem or problems of Consciousness being that we have unnecessarily conflated various properties that occur in various humans in the word consciousness because of political/moral reasons, whereas AIs don’t automatically have all the properties of humans here, so we should create new concepts for AIs, it’s still methodologically bad.

But yes, this post at the very least relies on a theory that hasn’t been tested, and while I suspect it’s at least partially correct, the evidence in the conflationary alliances post is basically 0 evidence for the proposition.

Noosphere89 22 Nov 2024 0:11 UTC
2 points
0
in reply to: dr_s’s comment on: What are the good rationality films?
Nor do we have the ability to bend probabilities arbitrarily for arbitrary statements, which was a core power in Gurren Lagann movies, if I recall correctly.

Noosphere89 21 Nov 2024 23:39 UTC
2 points
0
in reply to: David Matolcsi’s comment on: “The Solomonoff Prior is Malign” is a special case of a simpler argument
This part IMO is a crux, in that I don’t truly believe an objective measure/magical reality fluid can exist in the multiverse, if we allow the concept to be sufficiently general, ruining both probability and expected value/utility theory in the process.

Heck, in the most general cases, I don’t believe any coherent measure exists at all, which basically ruins probability and expected utility theory at the same time.

Maybe we have some deeper disagreement here. It feels plausible to me that there is a measure of “realness” in the Multiverse that is an objective fact about the world, and we might be able to figure it out.

Noosphere89 21 Nov 2024 20:32 UTC
3 points
0
in reply to: Hastings’s comment on: Hastings’s Shortform
The most important thing to realize about AI safety is that basically all versions of practically safe AI must make certain assumptions that no one does a specific action (mostly related to misuse reasons, but for some specific plans, can also be related to misalignment reasons).

Another way to say it is that I believe that in practice, these two categories are the same category, such that basically all work that’s useful in the field will require someone not to do something, so the costs of sharing are practically 0, and the expected value of sharing insights is likely very large.

Specifically, I’m asserting that these 2 categories are actually one category for most purposes:

Actually make AI safe and another, sadder but easier field of “Make AI safe as long as no one does the thing.”

Noosphere89 21 Nov 2024 19:35 UTC
4 points
0
in reply to: abramdemski’s comment on: Why Don’t We Just… Shoggoth+Face+Paraphraser?
My most plausible explanation of your position is that you think this is aligned, IE, you think providing plausible URLs is the best it can do when it doesn’t know. I disagree: it can report that it doesn’t know, rather than hallucinate answers. Actively encouraging itself to give plausible-but-inaccurate links is misalignment. It’s not just missing a capability; it’s also actively self-prompting to deceive the user in a case where it seems capable of knowing better (if it were deliberating with the user’s best interest in mind.
I agree that this is actually a small sign of misalignment, and o1 should probably fail more visibly by admitting it’s ignorance rather than making stuff up, so at this point, I’ve come to agree that o1′s training probably induced at least a small amount of misalignment, which is bad news.
Also, thanks for passing my ITT here.
Capabilities issues and alignment issues are not mutually exclusive. It appears that capabilities issues (not being able to remember all valid URLs) is causing an alignment issue in this case (actively planning to give deceptive URLs rather than admit its ignorance). This causes me to predict that even if this particular capability issue is resolved, the training will still have this overall tendency to turn remaining capability issues into alignment issues. How are you seeing this differently?
This is a somewhat plausible problem, and I suspect the general class of solutions to these problems will probably require something along the lines of making the AI fail visibly rather than invisibly.
The basic reason I wanted to distinguish them is because the implications of a capabilities issue mean that the AI labs are incentivized to solve it greatly, and while alignment incentives are real and do exist, they can be less powerful than the general public wants.

Noosphere89 20 Nov 2024 19:43 UTC
4 points
0
in reply to: abramdemski’s comment on: Why Don’t We Just… Shoggoth+Face+Paraphraser?
What do you mean? I don’t get what you are saying is convincing.
I’m specifically referring to this answer, combined with a comment that convinced me that the o1 deception so far is plausibly just a capabilities issue:
https://www.lesswrong.com/posts/3Auq76LFtBA4Jp5M8/why-is-o1-so-deceptive#L5WsfcTa59FHje5hu
https://www.lesswrong.com/posts/3Auq76LFtBA4Jp5M8/why-is-o1-so-deceptive#xzcKArvsCxfJY2Fyi
Is this damning in the sense of providing significant evidence that the technology behind o1 is dangerous? That is: does it provide reason to condemn scaling up the methodology behind o1? Does it give us significant reason to think that scaled-up o1 would create significant danger to public safety? This is trickier, but I say yes. The deceptive scheming could become much more capable as this technique is scaled up. I don’t think we have a strong enough understanding of why it was deceptive in the cases observed to rule out the development of more dangerous kinds of deception for similar reasons.
I think this is the crux.
To be clear, I am not saying that o1 rules out the ability of more capable models to deceive naturally, but I think 1 thing blunts the blow a lot here:
1. As I said above, the more likely explanation is that there’s an asymmetry in capabilities that’s causing the results, where just knowing what specific URL the customer wants doesn’t equal the model having the capability to retrieve a working URL, so this is probably at the heart of this behavior.
So for now, what I suspect is that o1′s safety when scaled up mostly remains unknown and untested (but this is still a bit of bad news).
Is this damning in the sense that it shows OpenAI is dismissive of evidence of deception?
- However, I don’t buy the distinction they draw in the o1 report about not finding instances of “purposefully trying to deceive the user for reasons other than satisfying the user request”. Providing fake URLs does not serve the purpose of satisfying the user request. We could argue all day about what it was “trying” to do, and whether it “understands” that fake URLs don’t satisfy the user. However, I maintain that it seems at least very plausible that o1 intelligently pursues a goal other than satisfying the user request; plausibly, “provide an answer that shallowly appears to satisfy the user request, even if you know the answer to be wrong” (probably due to the RL incentive).
I think the distinction is made to avoid confusing capability and alignment failures here.
I agree that it doesn’t satisfy the user’s request.
- More importantly, OpenAI’s overall behavior does not show concern about this deceptive behavior. It seems like they are judging deception case-by-case, rather than treating it as something to steer hard against in aggregate. This seems bad.
Yeah, this is the biggest issue of OpenAI for me, in that they aren’t trying to steer too hard against deception.

Noosphere89 20 Nov 2024 17:27 UTC
3 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
The problem with that plan is that there are too many valid moral realities, so which one you do get is once again a consequence of alignment efforts.

To be clear, I’m not stating that it’s hard to get the AI to value what we value, but it’s not so brain-dead easy that we can make the AI find moral reality and then all will be well.

Noosphere89 20 Nov 2024 16:31 UTC
2 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
Not always, but I’d say often.
I’d also say that at least some of the justification for changing values in philosophers/humans is because they believe the new values are closer to the moral reality/truth, which is an instrumental incentive.
To be clear, I’m not going to state confidently that this will happen (maybe something like instruction following ala @Seth Herd is used instead, such that the pointer is to the human giving the instructions, rather than having values instead), but this is at least reasonably plausible IMO.

Noosphere89 20 Nov 2024 16:05 UTC
3 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
Yes, I admittedly want to point to something along the lines of preserving your current values being a plausibly major drive of AIs.

Noosphere89 20 Nov 2024 15:46 UTC
2 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
In this case, it would mean the convergence to preserve your current values.

Noosphere89 20 Nov 2024 15:01 UTC
3 points
0
in reply to: Dakara’s comment on: If we solve alignment, do we die anyway?
The answer to this is that we’d rely on instrumental convergence to help us out, combined with adding more data/creating error-correcting mechanisms to prevent value drift from being a problem.

Noosphere89 20 Nov 2024 2:24 UTC
4 points
2
in reply to: Steven Byrnes’s comment on: Evolution’s selection target depends on your weighting
Oh, I was responding to something different, my apologies.

Noosphere89 20 Nov 2024 1:25 UTC
11 points
0
in reply to: Steven Byrnes’s comment on: Evolution’s selection target depends on your weighting

Neither of those claims has anything to do with humans being the “winners” of evolution. I don’t think there’s any real alignment-related claim that does. Although, people say all kinds of things, I suppose. So anyway, if there’s really something substantive that this post is responding to, I suggest you try to dig it out.

The evolution analogy of how evolution failed to birth intelligent minds that valued what evolution valued is an intuition pump that does get used in explaining outer/inner alignment failures, and is part of why in some corners there’s a general backdrop of outer/ inner alignment being so hard.

It’s also used in the sharp left turn, where the capabilities of an optimization process like humans outstripped their alignment to evolutionary objectives, and the worry is that an AI could do the same to us, and evolutionary analogies do get used here.

Both Eliezer Yudkowsky and Nate Soares use arguments that rely on evolution failing to get a selection target inside us, thus misaligning us to evolution:

https://www.lesswrong.com/posts/wAczufCpMdaamF9fy/my-objections-to-we-re-all-gonna-die-with-eliezer-yudkowsky#Yudkowsky_argues_against_AIs_being_steerable_by_gradient_descent_

https://www.lesswrong.com/posts/GNhMPAWcfBCASy8e6/a-central-ai-alignment-problem-capabilities-generalization

Noosphere89 20 Nov 2024 1:13 UTC
4 points
1
in reply to: Nathan Helm-Burger’s comment on: Why Don’t We Just… Shoggoth+Face+Paraphraser?
This might actually be a useful idea, thanks for your idea.

Noosphere89 20 Nov 2024 0:49 UTC
4 points
1
in reply to: Marius Hobbhahn’s comment on: Training AI agents to solve hard problems could lead to Scheming
I definitely agree that conditioning on AI catastrophe, I think the 4 step chaotic catastrophe is the most likely way an AI catastrophe leads to us being extinct or at least in a very bad position.

I admit the big difference is that I do think that 2 is probably incorrect, as we have some useful knowledge of how models form goals, and I expect this to continue.

Noosphere89 20 Nov 2024 0:08 UTC
6 points
3
in reply to: Daniel Kokotajlo’s comment on: Why Don’t We Just… Shoggoth+Face+Paraphraser?
That’s actually a pretty good argument, and I actually basically agree that hiding CoT from the users is a bad choice from an alignment perspective now.