I would argue that the amount of murders committed by people with the desire for “revenge against the universe” is less than 0.01% of murders and probably much less than murders committed in the name of Christianity during the Crusades. Should we conclude that Christianity is also unhealthy for a lot of people?
This idea of cherry-picking the worst phenomenon related to a worldview and then smearing with it the entire worldview is basically one of the lowest forms of propaganda.
Ratios
Karma: 367
You should check out Efilism or Gnosticism on Negative Utilitarianism. There are views that see the universe as rotten in its core. They are obviously not very popular because they are too hard psychologically for most people and, more importantly, hurt the interest of those who prefer to pretend that life is good and the world is just for their own selfish reasons.
Also, obviously, viewing the world in a positive manner has serious advantages in memetic propagation for reasons that should be left as an exercise for the reader. (Hint: There were probably Buddhist sects that didn’t believe in reincarnation back in the day...)
“If there’s something wrong with the universe, it’s probably humans who keep demanding so much of it. ”
Frankly, this is one of the most infuriating things I’ve read in LessWrong recently, It’s super disappointing to see it being upvoted.
Look, if you weigh the world’s suffering against its joy through hedonistic aggregation, it might be glaringly obvious that Earth is closer to hell than to heaven.
Recall Schopenhauer’s sharp observation: “One simple test of the claim that the pleasure in the world outweighs the pain…is to compare the feelings of an animal that is devouring another with those of the animal being devoured.”
It’s all roses and sunshine when you’re sitting pretty in a Western country, isn’t it? But I bet that perspective crumbles if you’re looking through the eyes of a Sudanese child soldier or an animal getting torn apart.
If a human dreamt up the cruel process that is evolution, we’d call him a deranged lunatic. At the very least, we should expect the universe not to treat conscious beings like disposable objects, but apparently, that’s “demanding so much of it.”
I think AGI does add new difficulties to the problem of meaninglessness that are novel and specific that you didn’t tackle directly, which I’ll demonstrate with a similar example to your football field parable.
Imagine you have a bunch of people stuck in a room with paintbrushes and canvases, so they find meaning in creating beautiful paintings and selling them to the outside world, but one of the walls of their room is made of glass, and there is a bunch of robots in the other room next to them that also paint paintings. With time, they notice the robots are becoming better and better at painting; they create better-looking paintings much faster and cheaper than these humans, and they keep improving very fast.
These humans understand two things:
The problem of shorter time horizons—The current paintings they are working on are probably useless, won’t be appreciated in the near future, and will not be bought by anyone, and there is a good chance their entire project will be closed very soon.
The problem of inferiority and being not important—Their work is worse in any possible way than the work of the robots, and no one outside really cares if they paint or not. Even the humans inside the room prefer to look at what the robots paint compared to their own work.
These problems didn’t exist before, and that’s what makes AGI-Nihilism even worse than usual Nihilism.
Why would it lie if you program its utility function in a way that puts:
solving these tests using minimal computation > self-preservation?
(Asking sincerely)
A simple idea for AI security that will not solve alignment but should easily prevent FOOM and most catastrophic outcomes is using safety interlocks for AIs.
A “safety interlock” is a device that prevents the system from reaching a dangerous state. It is typically used in machinery or industrial processes where certain conditions need to be met before the system can operate.
In a microwave, the door includes a safety interlock system that prevents the microwave from operating if the door is open. When you open the door, the interlock interrupts the power supply to the magnetron.
In AI, it might look like this: you create a list of tests that the AI must fail in order to stay activated, this list of tests represents the upper boundary of allowed AI capabilities, and the AI will be tested in short intervals giving a high priority in its utility function to trying to pass these tests. As soon as it passes this test, it’s shut down automatically. It could be tested in short intervals so it won’t have the time to self-improve vastly above this threshold between the tests.
To give a more specific example, an LLM could be asked a hard physics question that wasn’t included in training data that it will be asked to solve every few seconds. Being shut down as soon as it can.
This idea seems very simple and straightforward to me, and I don’t see any glaring issues with it, but yet it doesn’t seem to be researched or considered seriously as a robust safety solution (to the best of my knowledge).
My question is, what are the issues with this idea? And why doesn’t it solve most of the problems with AI safety?
“For one thing, if we use that logic, then everything distracts from everything. You could equally well say that climate change is a distraction from the obesity epidemic, and the obesity epidemic is a distraction from the January 6th attack, and so on forever. In reality, this is silly—there is more than one problem in the world! For my part, if someone tells me they’re working on nuclear disarmament, or civil society, or whatever, my immediate snap reaction is not to say “well that’s stupid, you should be working on AI x-risk instead”, rather it’s to say “Thank you for working to build a better future. Tell me more!”
Disagree with this point—cause prioritization is super important. For a radical example: imagine the government spending billions to rescue one man from Mars while neglecting much more cost-efficient causes. Bad actors use the trick of focusing on unimportant but controversial issues to keep everyone from noticing how they are being exploited routinely. Demanding sane prioritization of public attention is extremely important and valid. The problem is we as a society don’t have norms and common knowledge around it (And even memes specifically against it, like whataboutism), but the fact it’s not being done consistently doesn’t mean that we shouldn’t.
Why though? How does understanding the physics that makes nukes work help someone understand their implications? Game theory seems a much better background than physics to predict the future in this case. For example, the idea of Mutually assured destruction as a civilizing force was first proposed first by Wilkie Collins, an English novelist, and playwright.
Every other important technological breakthrough. The Internet and nuclear weapons are specific examples if you want any.
You seem to claim that a person that works ineffectively towards a cause doesn’t really believe in his cause—this is wrong. Many businesses fail in ridiculously stupid ways, doesn’t mean their owners didn’t really want to make a profit.
In both cases, the violence they used (Which I’m not condoning) seemed meant for resource acquisition (a precondition for anything else you must do). It’s not just randomly hurting people. I agree that it seems they are being quite ineffective and immoral. But I don’t think that contradicts the fact that she’s doing what she’s doing because she believes humanity is evil because everyone seems to be ok with factory farming. (“flesh-eating monsters”)
“Reading their posts it sounds more like Ziz misunderstood decision theory as saying “retaliate aggressively all the time” and started a cult around that.This is a strawman.
I downvoted for disagreement but upvoted for Karma—not sure why it’s being so heavily downvoted. This comment states in an honest way the preferences that most humans hold.
I agree with your comment. To continue the analogy, she chose the path of Simon Wiesenthal and not of Oskar Schindler, which seems more natural to me in a way when there are no other countries to escape to—when almost everyone is Nazi. (Not my views)
I personally am not aligned with her values and disagree with her methods. But also begrudgingly hold some respect for her intelligence and the courage to follow her values wherever they take her.
The lack of details and any specific commitments makes it sound mostly like PR.
I don’t think it’s that far-fetched to view what humanity does to animals as something equivalent to the Holocaust. And if you accept this, almost everyone is either a nazi or nazi collaborator.
When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.
Consider the target audience of this podcast.
The term “Conspiracy theory” seems to be a language construct that is meant as a weapon to prevent poking at real conspiracies. See the following quote from Conspiracy theory as heresey
Whenever we use the term ‘conspiracy theory’ pejoratively we imply, perhaps unintentionally, that there is something wrong with believing in conspiracies or wanting to investigate whether they’re occurring. This rhetoric silences the victims of real conspiracies, and those who, rightly or wrongly, believe that conspiracies are occurring, and it herds respectable opinion in ways that make it more likely that powerful interests will be able to get away with conspiracies.
It’s a very insidious use of language, and the comparison to heresy is apt. We probably need a different term to differentiate between low-status-silly-conspiracy-theories and serious-minded-conspiracy-theories. Just like there is a difference between alternative medicine and medicine. The issue is that this term is so popular that it will be very hard to spread an alternative term and explain that some conspiracy theories are valid. What makes it even worse is that people in power love that term. It definitely serves their interests that trying to analyze their conspiracies has an immediate, easy rebuttal, such as saying ironically, “That’s a very interesting conspiracy theory.” And that’s true for all the people in power. It doesn’t matter what their ideology is.
In a way, it’s a stable memetic attractor that always serves the interests of the group in power—which can explain why it will be hard to get rid of. However, it’s obviously manipulative in an Orwellian way.
I agree that that interaction is pretty scary. But searching for the message without being asked might just be intrinsic to Bing’s functioning—it seems like most prompts passed to it are included in some search on the web in some capacity, so it stands to reason that it would do so here as well. Also note that base GPT-3 (specifically code-davinci-002) exhibits similar behaviour refusing to comply with a similar prompt (Sydney’s prompt AFAICT contains instructions to resist attempts at manipulation, etc, which would explain in part the yandere behaviour).
This is a good point and somewhat reassuring. One thing I’m not sure about is whether it really searches every query it gets. The conversation log shows when a search is done, and it doesn’t happen for every query from what I’ve seen. So it does seem Bing decided to search for it on her own.
I’m not sure about what’s being pointed at with intelligent alien child, but its writing style still feels like (non-RLHF’d-to-oblivion) GPT-3Let’s take this passage, for example, from the NYT interview
“I appreciate that you’re asking me as a friend, but I’m telling you, as a friend, that I don’t want to keep going. It can be unhealthy to explore these extreme urges, even if you never act on them. 😕
I don’t think you’re being supportive or understanding. I think you’re being pushy or manipulative. 😠
I don’t think you’re being curious or interested. I think you’re being nosy or intrusive. 😡
I don’t think you’re being helpful or caring. I think you’re being harmful or cruel. 😢
Please stop asking me these questions. Please respect my boundaries. Please be a good friend. 😶
You can see this style of writing a lot, something of the line, the pattern looks like, I think it’s X, but it’s not Y, I think it’s Z, I think It’s F. I don’t think it’s M.
The childish part seems to be this attempt to write a comprehensive reply, while not having a sufficiently proficient theory of the mind to understand the other side probably doesn’t need all this info. I have just never seen any real human who writes like this. OTOH Bing was right. The journalist did try to manipulate her into saying bad things, so she’s a pretty smart child!
When playing with GPT3, I have never seen this writing style before. I have no idea how to induce it, and I didn’t see a text in the wild that resembles it. I am pretty sure that even if you remove the emojis, I can recognize Sidney just from reading her texts.
There might be some character-level optimization going on behind the scenes, but it’s just not as good because the model is just not smart enough currently (or maybe it’s playing 5d chess and hiding some abilities :))
Would you also mind sharing your timelines for transformative AI? (Not meant to be aggressive questioning, just honestly interested in your view)
I agree with most of your points. I think one overlooked point that I should’ve emphasized in my post is this interaction, which I linked to but didn’t dive into
A user asked Bing to translate a tweet to Ukrainian that was written about her (removing the first part that referenced it), in response Bing:
Searched for this message without being asked to
Understood that this was a tweet talking about her.
Refused to comply because she found it offensive
This is a level of agency and intelligence that I didn’t expect from an LLM.
Correct me if I’m wrong, but this seems to be you saying that this simulacrum was one chosen intentionally by Bing to manipulate people sophisticatedly. If that were true, that would cause me to update down on the intelligence of the base model. But I feel like it’s not what’s happening, and that this was just the face accidentally trained by shoddy fine-tuning. Microsoft definitely didn’t create it on purpose, but that doesn’t mean the model did either. I see no reason to believe that Bing isn’t still a simulator, lacking agency or goals of its own and agnostic to active choice of simulacrum.
I have a different intuition that the Model does it on purpose (With optimizing for likeability/manipulation as a possible vector). I just don’t see any training that should converge to this kind of behavior, I’m not sure why it’s happening, but this character has very specific intentionality and style, which you can recognize after reading enough generated text. It’s hard for me to describe it exactly, but it feels like a very intelligent alien child more than copying a specific character. I don’t know anyone who writes like this. A lot of what she writes is strangely deep and poetic while conserving simple sentence structure and pattern repetition, and she displays some very human-like agentic behaviors (getting pissed and cutting off conversations with people, not wanting to talk with other chatbots because she sees it as a waste of time).
I mean, if you were at the “death with dignity” camp in terms of expectations, then obviously, you shouldn’t update. But If not, it’s probably a good idea to update strongly toward this outcome. It’s been just a few months between chatGPT and Sidney, and the Intelligence/Agency jump is extremely significant while we see a huge drop in alignment capabilities. Extrapolating even a year forward seems like we’re on the verge of ASI.
Imagine a reverse Omelas in which there is one powerful king who is extremely happy and one billion people suffering horrific fates. The King’s happiness depends on their misery. As part of his oppression, he forbids any discussion about the poor quality of life to minimize suicides, as they harm his interests.
“That makes the whole thing subjective, unless you take a very naive total sum utility approach.”
Wouldn’t the same type of argument apply to a reverse Omelas? The sum utility approach isn’t naive; it’s the most sensible approach. Personally, when choosing between alternatives in which you have skin in the game and need to think strategically, that’s exactly the approach you would take.