quanticle

Karma: 2,744

quanticle Apr 8, 2023, 3:40 AM
3 points
1
on: ChatGTP “Writing ” News Stories for The Guardian?
It’s not that odd. Ars Technica has a good article on why generative AIs have such a strong tendency to confabulate. The short answer is that, given a prompt (consisting of tokens, which are similar to, but not quite the same as words), GPT will come up with new tokens that are more or less likely to come after the given tokens in the prompt. This is subject to a temperature parameter, which dictates how “creative” GPT is allowed to be (i.e. allowing GPT to pick less probable next-tokens with some probability). The output token is added to the prompt, and the whole thing is then fed back into GPT in order to generate another new token.

In other words, GPT is incapable of “going backwards”, as a human might and editing its previous output in to correct inaccuracies or inconsistencies. Instead, what it has to do is take the previous output as a given, and try to come up with new tokens that are likely to be generated given the already generated incorrect tokens. This is how GPT ends up with confabulated citations. Given the prompt, GPT generates some tokens, representing an author, for example. It then tries to generate the most likely words associated with that author, and the rest of the prompt, which is presumably asking for citations. As it generates a title, it chooses a word that doesn’t exist in any existing article titles written by that author. But it doesn’t “know” that, and it has no way of going back and editing prior output in order to correct itself. Instead GPT presses on, generating more tokens that are deemed to be likely given the mixture of correct and incorrect tokens that it has generated.

Scott Alexander has a great post, about human psychology, which touches on a similar theme, called The Apologist and the Revolutionary. Using the terms of that post, a GPT is 100% apologist, 0% revolutionary. No matter how nonsensical its previous output, GPT, by its very design, must take that previous output as axiomatic, and generate new output based upon that. That is what leads to uncanny results when GPT is asked for specific facts.

quanticle Apr 8, 2023, 3:17 AM
1 point
1
on: Is there a fundamental distinction between simulating a mind and simulating *being* a mind? Is this a useful and important distinction?

This creates a recursive loop such that each of them experiences what it is like to experience being them experiencing what it is like to be the other, on and on to whatever degree is desired by either of them.

Why should this be the case? When I encounter a potentially hostile piece of programming, I don’t run it on my main computer. I run it in a carefully isolated sandbox until I’ve extracted whatever data or value I need from that program. Then I shut down the sandbox. If the AI is superintelligent enough to scan human minds as its taking humans apart (and why should it do that?), what prevents it from creating a similar isolated environment to keep any errant human consciousnesses away from its vital paper-clip optimizing computational resources?

quanticle Apr 8, 2023, 3:05 AM
30 points
21
on: Consider The Hand Axe

I’m the idiot holding the hand axe. I’m the imbecile mangling my shins with rock debris. Why bother?

Because no matter how obsolete hand axes or hand-forged iron spoons, or hand-built CPUs are, it’s still cool to make one yourself.

quanticle Apr 6, 2023, 12:13 AM
20 points
21
on: The Orthogonality Thesis is Not Obviously True

If realism is false, nothing matters, so it’s not bad that everyone dies

That’s a misunderstanding of moral realism. Moral realism is a philosophical argument that states that moral arguments state true facts about the world. In other words, when I say that “Murder is bad,” that is a fact about the world, as true as $2 + 2 = 4$ or the Pythagorean theorem.

It’s entirely possible for me to think that moral realism is false (i.e. morality is a condition of human minds) while also holding, as a member of humanity, a view that the mass extinction of all humanity is an undesirable state. Denying moral realism isn’t the same as saying, “Nothing matters.” It’s closer to claiming, “Rocks don’t have morality.” And an AI, insofar as it is a fancy thinking rock, won’t have morality by default either. We could, of course, give it morality, by ensuring that it is aligned to human values. But that would be the result of humans taking positive steps to impart their moral reasoning onto an otherwise amoral reality.

quanticle Apr 5, 2023, 10:12 PM
2 points
0
in reply to: YafahEdelman’s comment on: Excessive AI growth-rate yields little socio-economic benefit.

This would have a significant impact on the everyday lives of people within a month.

It would have a drastic impact on your life in a month. However, you are a member of a tiny fraction of humanity, sufficiently interested and knowledgeable about AI to browse and post on a forum that’s devoted to a particularly arcane branch of AI research (i.e. AI safety). You are in no way representative. Neither am I. Nor is, to a first approximation, anyone who posts here.

The average American (who, in turn, isn’t exactly representative of the world) has only a vague idea of what ChatGPT is. They’ve seen some hype videos on TV or on YouTube. Maybe they’ve seen one of their more technically sophisticated friends or relatives use it. But they don’t really know what it’s good for, they don’t know how it would integrate into their lives, and they’re put off by the numerous flaws they’ve heard about with regards to generative models. If OpenAI came out with GPT-5 tomorrow, and it fixed all, or almost all, of the flaws in GPT-4, it would still take years, at least, and possibly decades before it was integrated into the economy in a way that the average American would be able to perceive.

This has nothing to do with the merits of the technology. It has to do with the psychology of people. People’s appetite for novelty varies, like many other psychological traits, along a spectrum. On one hand, you have people like Balaji Srinivasan, who excitedly talk about every new technology, whether it be software, financial, or AI. At the other end, you have people like George R. R. Martin, who’re still using software written in the 1980s for their work, just because it’s what they’re familiar and competent with. Most people, I’d venture to guess, are somewhere in the middle. LessWrong is far towards the novelty-seeking end of the spectrum, at, or possibly farther ahead than Balaji Srinivasan. New advancements in AI affect us because we are open to being affected by them, in a way that most people are not.

quanticle Apr 5, 2023, 6:45 PM
8 points
5
in reply to: Ppxl’s comment on: Excessive AI growth-rate yields little socio-economic benefit.
That very same argument could have been (and was!) made for steam engines, electric motors, and (non-AI) computers. In 2023, we can look back and say, “Of course, as workers were displaced out of agriculture and handicrafts, they went into manufacturing and services.” But it wasn’t apparent in 1823 that such a transition would occur. Indeed, in 1848, one of the motivations for Marx to write The Communist Manifesto was his fervent belief that industrial manufacturing and services would not be able to absorb all the surplus workers, leading to significant alienation and, eventually, revolution.

quanticle Apr 5, 2023, 5:58 PM
2 points
0
on: On The Impossibility of AI Alignment

The best metaphor I have here is that in some sense, weak superintelligences already exist. Corporations and governments are generally smarter than individual humans.

Corporations and governments have vastly more resources than any single human, but it’s not at all clear to me that they’re smarter. On the contrary, we have numerous examples of corporations and governments acting in ways that are less intelligent than any of their individual component humans, because they have to deal with things like principal-agent problems and inefficient communications, all of which serve to limit the ability of the broader whole to act in a “coherent” fashion. “Coherent”, here, is defined as having a common goal, and coordinating among themselves efficiently to reach that goal. Corporations and governments give up this coherence in order to have more resources, under the reasoning that the loss of coherence in action can be more than compensated for by brute force. Often this is true, but sometimes (e.g. when a startup is able to outmaneuver a lumbering conglomerate) it’s not.

A superintelligent AI would be akin to a corporation that could act with the coherence and unity of purpose of a single human. We don’t have any experience with such a thing.

quanticle Apr 1, 2023, 6:38 PM
13 points
8
on: Exposure to Lizardman is Lethal
I’m surprised that no one has mentioned “tenure”, because this is exactly the problem that academic tenure was designed to solve. The point of having professors be relatively unfireable, after they’ve demonstrated a basic minimum standard of academic output, was precisely to allow them to explore controversial opinions without having that investigation be immediately shut down because it was offensive to broader society.

It seems like you’re advocating for more tenure, or at least tenure-like insulation across all of society.

quanticle Mar 31, 2023, 6:28 PM
2 points
0
in reply to: Lucas Pfeifer’s comment on: Limit intelligent weapons
To repeat what I said above: even a total launch of all the nuclear weapons in the world will not be sufficient to ensure human extinction. However, AI driven social, economic, and environmental changes could ensure just that.

If an AI got hold of a few nuclear weapons and launched them, that would, in fact, probably be counterproductive from the AI’s perspective, because in the face of such a clear warning sign, humanity would probably unite and shut down AI research and unplug its GPU clusters.

quanticle Mar 29, 2023, 11:44 PM
15 points
5
on: Run Posts By Orgs
I disagree with this post. At the very least, I feel like there should be some kind of caveat or limit regarding the size of the organization or distance that one has from the organization. For example, if I’m writing a post or comment about some poor experience I had with Amazon, do I have a moral obligation to run that post by Amazon’s PR beforehand? No. Amazon is a huge company, and I’m not really connected to them in any way, so I do not and should not feel any obligation towards them prior to sharing my experiences with their products or services.

quanticle Mar 28, 2023, 9:17 PM
11 points
0
on: Chatbot convinces Belgian to commit suicide
An archived, un-paywalled version of the article is available here.

quanticle Mar 28, 2023, 9:10 PM
2 points
0
in reply to: Lucas Pfeifer’s comment on: Limit intelligent weapons

My point is this: we should focus first on limiting the most potent vectors of attack: those which involve conventional ‘weapons’.

That’s exactly where I disagree. Conventional weapons aren’t all that potent compared to social, economic, or environmental changes.

quanticle Mar 28, 2023, 12:03 AM
2 points
0
on: Bermudez Interface Problem

Does comparing neurons firing with beliefs/desires involve a type distinction that renders belief/desires irreducible to patterns of neuron activity?

I don’t think it does, but I do think that the difference in scale between a neuron firing and an entire belief forming makes the reduction very difficult, and possibly pointless. It’s a bit like reducing the spray of water out of a garden hose to the movement of individual water molecules. It’s very difficult, given that each water molecule’s motion contributes only an infinitesimal amount to the movement of the water as whole. Furthermore, even if you could map particular interactions between water molecules to specific motions of water droplets or the water stream as a whole, would you learn anything new thereby? Would it help solve any of the problems you’re interested in? A lot of the time, it’s better to work at higher levels of abstraction.

quanticle Mar 27, 2023, 10:33 PM
2 points
0
in reply to: Lucas Pfeifer’s comment on: Limit intelligent weapons

Let’s define a weapon as any tool which could be used to mount an attack.

Why? That broadens the definition of “weapon” to mean literally any tool, technology, or tactic by which one person or organization can gain an advantage over another. It’s far broader than and connotationally very different from the implied definition of “weapon” given by “building intelligent machines that are designed to kill people” and the examples of “suicide drones”, “assassin drones” and “robot dogs with mounted guns”.

Redefining “weapon” in this way turns your argument into a motte-and-bailey, where you’re redefining a word that connotes direct physical harm (e.g. robots armed with guns, bombs, knives, etc) to mean any machine that can, on its own, gain some kind of resource advantage over humans. Most people would not, for example, consider a superior stock-trading algorithm to be a “weapon”, but your (re)definition, it would be.

quanticle Mar 26, 2023, 8:15 PM
2 points
0
in reply to: Lucas Pfeifer’s comment on: Limit intelligent weapons

However, weapons provide the most dangerous vector of attack for a rogue, confused, or otherwise misanthropic AI.

I’m not sure why you think that. Human weapons, as horrific as they are, can only cause localized tragedies. Even if we gave the AI access to all of our nuclear weapons, and it fired them all, humanity would not be wiped out. Millions (possibly billions) would perish. Civilization would likely collapse or be set back by centuries. But human extinction? No. We’re tougher than that.

But an AI that competes with humanity, in the same way that Homo sapiens competed with Homo neanderthalis? That could wipe out humanity. We wipe out other species all the time, and only in a small minority of cases is it because we’ve turned our weapons on them and hunted them into extinction. It’s far more common for species to go extinct because humanity needed the habitat and other natural resources that that species needed to survive, and outcompeted that species for access to those resources.

quanticle Mar 25, 2023, 3:46 PM
13 points
11
in reply to: Lucas Pfeifer’s comment on: Limit intelligent weapons

Human mercenaries causing a societal collapse? That would mean a large number of individuals who are willing to take orders from a machine to actively harm their communities. Very unlikely.

I’m wondering how you can hold that position given all the recent social disorder we’ve seen all over the world where social media driven outrage cycles have been a significant accelerating factor. People are absolutely willing to “take orders from a machine” (i.e. participate in collective action based on memes from social media) in order to “harm their communities” (i.e. cause violence and property destruction).

quanticle Mar 23, 2023, 10:12 PM
3 points
1
on: Limit intelligent weapons
What is an “intelligent” machine? What is a machine that is “designed” to kill people? Why should a machine with limited intelligence that is “designed” to kill, such as an AIM-9 be more of a threat than a machine with vast intelligence that is designed to accomplish a seemingly innocuous goal, that has the destruction of humanity as an unintended side-effect.
Currently, leading militaries around the world are developing and using:
- Drone swarms
- Suicide drones
- Assassin drones
- Intelligent AI pilots for fighter jets
- Targeting based on facial recognition
- Robot dogs with mounted guns
None of these things scare me as much as GPT-4. Militaries are overwhelmingly staid and conservative institutions. They are the ones that are most likely to require extensive safeguards and humans-in-the-loop. What does scare me is the notion of a private entity developing a superintelligence, or an uncontrolled iterative process that will lead to a superintelligence and letting it loose accidentally.

quanticle Mar 21, 2023, 11:38 PM
3 points
1
in reply to: MiguelDev’s comment on: Why Carl Jung is not popular in AI Alignment Research?
I fail to see how Jungian analysis can actually debug LLMs better than the approach that Robert_AIZI used in their analysis of the “SolidGoldMagikarp” glitch token.

quanticle Mar 20, 2023, 2:12 AM
4 points
2
in reply to: MiguelDev’s comment on: Why Carl Jung is not popular in AI Alignment Research?
It is a belief that doesn’t pay rent. Let’s assume that there is such a thing as a collective unconscious, which is the source of archetypes. What additional predictions does this enable? Why should I add the notion of a collective unconscious to my existing psychological theory? Why shouldn’t I trim away this epicycle with Occam’s Razor?

quanticle Mar 18, 2023, 8:32 PM
5 points
0
in reply to: MiguelDev’s comment on: Why Carl Jung is not popular in AI Alignment Research?
The idea that individuals are driven by subconscious or unconscious instincts is a well established fact of psychology. The idea of a collective unconscious, in the way the Jung described it, is the unfalsifiable woo.