I am using Trump merely as an illustrative example of techniques.
My more immediate concern is actually the ability of China to shape US opinion through TikTok.
RationalDino
Karma: 39
The simple reason to use Kelly is this.
With 100% odds, any other strategy will lose to Kelly in the long run.
This can be shown by applying the strong law of large numbers to the random walk that is the log of your net worth.
Now what about a finite game? It takes surprisingly few rounds before Kelly, with median performance, pulls ahead of alternate strategies. It takes rather more rounds before, say, you have a 90% chance of beating another strategy. So in the short to medium run, Kelly offers the top of a plateau for median returns. You can deviate fairly far from it and still do well on average.
So should you still bet Kelly? Well, if you bet less than Kelly, you’ll experience lower average returns and lower variance. If you bet more than Kelly, you’ll experience lower average returns and higher variance. Variance in the real world tends to translate into, “I don’t have enough left over for expenses and I’m broke.” Reducing variance is generally good. That’s why people buy insurance. It is a losing money bet that reduces variance. (And in a complex portfolio, can increase expected returns!) So it makes sense to bet something less than Kelly in practice.
There is a second reason to bet less than Kelly in practice. When we’re betting, we estimate the odds. We’re betting against someone else who is also estimating the odds. The average of many people betting is usually more accurate than individual bettors. We believe that we’re well-informed and have a better estimate than others. But we’re still likely biased towards overconfidence in our chances. That means that betting Kelly based on what we think the odds are means we’re likely betting too much.
Ideally you would have enough betting history tracked to draw a regression line to figure out the true odds based on the combination of what you think, and the market things. But most of us don’t have enough carefully tracked history to accurately make such judgments.
Social media has proven more than capable of creating effective social contexts for persuading people.
LLMs are perfectly capable of operating in these social contexts. Particularly if they have (as in the case of TikTok and China) the support of the owner of the site.
Do you have specific cause to believe that LLMs will fail to persuade in these social contexts?
You make a good point that Cruz et al may have different beliefs than they portray publicly. But if so, then Cruz must have had a good acting coach in late 2018.
About 70 million followers, you’re right to call me out for overstating it. But according to polling, that’s how many people believe that the 2020 election was stolen at the ballot box. So far he has lost dozens of election challenge cases, key members of his inner circle have admitted in court that there was no evidence, and he’s facing multiple sets of felony charges in multiple jurisdictions.
I think it is reasonable to call someone a devoted follower if they continue to accept his version in the face of such evidence.
On AI safety, we can mean two different things.
My concern is with the things that are likely to actually happen. Hence my focus on what is supported by tech companies, and what politicians are likely to listen to. That part I’m sure is mostly regulatory capture.
I did acknowledge, though not loudly enough, that there are people working in AI safety who truly believe in what they are doing. But to the extent that they don’t align with the vested interests, what they do will not matter. To the extent that they do align, their motivations don’t matter as much as the motivations of the vested interests. And in the meantime, I wish that they would investigate questions that I consider important.
For example, how easily can an LLM hypnotize people? Given the ability to put up images and play videos created by AI. Can it hypnotize people then? Can it implant posthypnotic suggestions? In other words, how easily can an existing social network, with existing technology, be used for mass hypnosis?
Update. I forgot to mention that Andrew Ng’s accomplishments in AI are quite impressive. Cofounder of Google Brain, taught machine learning to Sam Altman, and so on. I might wind up disagreeing with some of his positions, but I’ll generally default to trusting his thinking over mine on anything related to machine learning.
If you’re willing to pay, you can read a stronger version of his thoughts in Google Brain founder says big tech is lying about AI extinction danger.
With all due respect, I see no evidence that elites are harder to fool now than they were in the past. For concrete examples, look at the ones who flipped to Trump over several years. The Corruption of Lindsey Graham gives an especially clear portrayal about how one elite went from condemning Trump to becoming a die-hard supporter.
I dislike a lot about Mr. Graham. But there is no question that he was smart and well aware of how authoritarians gain power. He saw the risk posed by Trump very clearly. However he knew himself to be smart, and thought he could ride the tiger. Instead, his mind got eaten.
Moving on, I believe that you are underestimating the mass psychology stuff. Remember, I’m suggesting it as a floor to what could already be done. New capabilities and discoveries allow us to do more. But what should already be possible is scary enough.
However that is a big topic. I went into it in AI as Super-Demagogue which you will hopefully find interesting.
AI as Super-Demagogue
I disagree that people who do ML daily would be in a good position to judge the risks here. The key issue is not the capabilities of AI, but rather the level of vulnerability of the brain. Since they don’t study that, they can’t judge it.
It is like how scientists proved to be terrible at unmasking charlatans like Uri Geller. Nature doesn’t actively try to fool us, charlatans do. The people with actual relevant expertise were people who studied how people can be fooled. Which meant magicians like James Randi. Similarly, to judge this risk, I think you should look at how dictators, cult leaders, and MLM companies operate.
A century ago Benito Mussolini figured out how to use mass media to control the minds of a mass audience. He used this to generate a mass following, and become dictator of Italy.. The same vulnerabilities exploited the same way have become a staple for demagogues and would-be dictators ever since. But human brains haven’t been updated. And so Donald Trump has managed to use the same basic rootkit to amass about 70 million devoted followers. As we near the end of 2023, he still has a chance of successfully overthrowing our democracy if he can avoid jail.
Your thinking about zero days is a demonstration of how thinking in terms of computers can mislead you. What matters for an attack is the availability of vulnerable potential victims. In computers there is a correlation between novelty and availability. Before anyone knows about a vulnerability, everyone is available for your attack. Then it is discovered, a patch is created, and availability goes down as people update. But humans don’t simply upgrade to brain 2.1.8 to fix the vulnerabilities found in brain 2.1.7. People can be brainwashed today by the same techniques that the CIA was studying when they funded the Reverend Sun Moon back in the 1960s.
You do make an excellent point about the difficulty of building something that can work at scale in the real world. Which is why I focused my scenario on techniques that have worked, repeatedly, at scale. We know that they can work, because they have worked. We see it in operation whenever we study the propaganda techniques used by dictators like Putin.
Given these examples, the question stops being an abstract, “Can AI find vulnerabilities by which we can be exploited?” It then switches to, “Is AI capable of executing effectrive variants on the strategies that dictators, cult leaders and MLM founders already have shown works at scale against human minds?”
I think that the answer is a pretty clear yes. Properly directed, ChatGPT should be more than capable of doing this. We then have the hallmark of a promising technology, we know that nothing fundamentally new is required. It is just a question of execution.
Sorry, but you’re overthinking what’s required. Simply being able to reliably use existing techniques is more than enough to hack the minds of large groups of people, no complex new research needed.
Here is a concrete example.
First, if you want someone’s attention, just make them feel listened to. ELIZA could already successfully do this in the 1970s, ChatGPT is better. The result is what therapists call transference, and causes the person to wish to please the AI.
Now the AI can use the same basic toolkit mastered by demagogues throughout history. Use simple and repetitive language to hit emotional buttons over and over again. Try to get followers to form a social group. Switch positions every so often. Those that pay insufficient attention will have the painful experience of being attacked by their friends, and it forces everyone to pay more attention.
All of this is known and effective. What AI brings is that it can use individualized techniques, at scale, to suck people into many target groups. And once they are in those groups, it can use the demagogue’s techniques to erase differences and get them aligned into ever bigger groups.
The result is that, as Sam Altman predicted, LLMs will prove superhumanly persuasive. They can beat the demagogues at their own game by seeding the mass persuasion techniques by individual attention at scale.
Do you think that this isn’t going to happen? Social media accidentally did a lot of this at scale. Now it is just a question of weaponizing something like TikTok.
You would be amazed at what lengths many go to never learn.
Ever heard the saying (variously attributed) that A level people want to be around other A level people while B level people want to be around C level people?
A lot of those B level people are ones who stop getting better because they believe themselves to already be good. And they would prefer to surround themselves with people who confirm that belief than risk challenging themselves.
Furthermore, it is easier to maintain illusions of superior competency when it isn’t competitive. It was a lot easier for me to hide from ways in which I was a bad husband than to hide from the fact that I was losing at chess. There isn’t really an objective measure of being a poor husband. And continuing doing what I already did was constant evidence to me that I was a good husband. So my illusions continued until some of the same problems showed up in my next relationship.
One example is the kind of person who began to learn something, worked at it, and became good at it compared to their friends. Without context for what “good” really means in the outside world, it is easy to believe that you are good.
In my blog I gave the example of myself as a teenager in chess. I could usually beat everyone in my school except my brother, so I felt like a good player.
But my competitive rating would have probably been about 1200-1400. I still remember my first encounter with a good chess player. A master was sitting in public, playing simultaneously against everyone who wanted to play him. I sat down, promptly lost, played again and lost again. He gave me some advice beginning with, “Weak players like you should focus on...”
I took offense, despite having just received evidence that he knew what he was talking about when it came to chess.
While I learned better, I’ve now been on the other side of this interaction in a number of areas. Including ping-pong and programming. Which suggests that my younger self was hardly unique in my overestimation of my abilities.
I agree that when you feel sure of your reasoning, you are generally more likely right than when you aren’t sure.
But when you cross into feeling certain, you should suspect cognitive bias. And when you encounter other people who are certain, you should question whether they might also have cognitive bias. Particularly when they are certain on topics that other smart and educated people disagree with them on.
This is not a 100% rule. But I’ve found it a useful guideline.
Doubt Certainty
Add me to those who have been through the physics demonstration. So I’ll give it odds of, let’s say, 99.9999%.
But I also don’t like how most physicists think about this. In The Feynman Lectures on Physics, Richard Feynman taught it as energy and mass are the same thing, and c^2 is simply the conversion factor. But most physicists distinguish between rest mass and relativistic mass. And so think in terms of converting between mass and energy. And not simply between different forms of energy, one of which is recognized to be mass.
But let’s take a hydrogen atom. A hydrogen atom is an electron and proton. But the mass of a hydrogen atom is less than the mass of an electron plus the mass of a proton. It is less by (to within measurement error) the mass of the energy required to split a hydrogen atom apart. I find this easier to think about within Feynman’s formulation than what most physicists do.
I believe that AI safety is a real issue. There are both near term and long term issues.
I believe that the version of AI safety that will get traction is regulatory capture.
I believe that the AI safety community is too focused on what fascinating technology can do, and not enough on the human part of the equation.
On Andrew Ng, his point is that he doesn’t see how exactly AI is realistically going to kill all of us. Without a concrete argument that is worth responding to, what can he really say? I disagree with him on this, I do think that there are realistic scenarios to worry about. But I do agree with him on what is happening politically with AI safety.