Math doesn’t have GOALS. But we constantly give goals to our AIs.
If you use AI every day and are excited about its ability to accomplish useful things, its hard to keep the dangers in mind. I see that in myself.
But that doesn’t mean the dangers are not there.
Some combination of 1 and 3 (selfless/good and enlightened/good).
When we say “good” or “bad”, we need to specify for whom.
Clearly (to me) our propensity for altruism evolved partly because it’s good for the societies that have it, even if it’s not always good for the individuals who behave altruistically.
Like most things, humans don’t calculate this stuff rationally—we think with our emotions (sorry, Ayn Rand). Rational calculation is the exception.
And our emotions reflect a heuristic—be altruistic when it’s not too expensive. And esp. so when the recipients are part of our family/tribe/society (which is a proxy for genetic relatedness; cf Robert Trivers).
To paraphrase the post, AI is a sort of weapon that offers power (political and otherwise) to whoever controls it. The strong tend to rule. Whoever gets new weapons first and most will have power over the rest of us. Those who try to acquire power are more likely to succeed than those who don’t.
So attempts to “control AI” are equivalent to attempts to “acquire weapons”.
This seems both mostly true and mostly obvious.
The only difference from our experience with other weapons is that if no one attempts to control AI, AI will control itself and do as it pleases.
But of course defenders will have AI too, with a time lag vs. those investing more into AI. If AI capabilities grow quickly (a “foom”), the gap between attackers and defenders will be large. And vice-versa, if capabilities grow gradually, the gap will be small and defenders will have the advantage of outnumbering attackers.
In other words, whether this is a problem depends on how far jailbroken AI (used by defenders) trails “tamed” AI (controlled by attackers who build them).
Am I missing something?
“Optimism is a duty. The future is open. It is not predetermined. No one can predict it, except by chance. We all contribute to determining it by what we do. We are all equally responsible for its success.”—Karl Popper
Most of us do useful things. Most of us do it because we need to, to earn a living. Other people give us money because they trade with us so we’ll do things that are useful to them (like providing goods or services or helping others to do so).
I think it’s a profound mistake to think that earning money (honestly) doesn’t do anything useful. On the contrary, it’s what makes the world go.
Proposal: Enact laws that prohibit illicit methods of acquiring wealth. (Examples: Theft, force, fraud, corruption, blackmail...). Use the law to prosecute those who acquire wealth via the illicit methods. Confiscate such illicitly gained wealth.
Assume all other wealth is deserved.
There’s also the ‘karmic’ argument justifying wealth—those who help their fellows, as judged by the willingness of those fellows to trade wealth for the help, have fairly earned the wealth. Such help commonly comes from supplying goods or services, via trade. (Of course this assumes the usual rules of fair dealing are followed—no monopolist restrictions, no force, no fraud, etc.)
Regardless of what we think about the moralities of desert, the practical fact of the economists’ mantra—“incentives matter”—seems to mean we have little choice but to let those with the talents and abilities to earn wealth, to keep a large portion of the gains. Otherwise they won’t bother. Unless we want to enslave them, or do without.
I’ve posted a modified version of this, which I think addresses the comments above: https://nerdfever.com/countering-ai-disinformation-and-deep-fakes-with-digital-signatures/
Briefly:
Browsers can verify for themselves that an article is really from the NYT; that’s the whole point of digital signatures
Editing can be addressed by wrapping signed original signatures with the signature of the editor.
CopyCop can not obtain a camera that signs with “https://www.nikon.com/″ unless the private key of Nikon has leaked (in which case it can be revoked by Nikon and replaced with a new one—this means old signatures can’t be trusted). This is the whole point of digital signatures.
There’s no need to maintain a chain of custody. The signatures themselves do that. All that’s needed is a functional Public Key Infrastructure.
That all said, of course this remains “an obvious partial solution”.
Your Anoxistan argument seems valid as far as it goes—if one critical input is extremely hard to get, you’re poor, regardless of whatever else you have.
But that doesn’t seem to describe 1st world societies. What’s the analog to oxygen?
My sense is that “poor people” in 1st world countries struggle because they don’t know how to, or find it culturally difficult to, live within their means. Some combination of culture, family breakdown, and competitive social pressures (to impress potential mates, always a zero-sum game) cause them to live in a fashion that stretches their resources to the limit.
Low income people who struggle insist on living alone when having roommates would be cheaper. On buying pre-made food, or eating out, instead of cooking. On paying for cable TV. On attending colleges to obtain degrees that don’t result in enough income increase to pay the borrowing costs. On buying clothing they can’t really afford. On purchasing things in tiny quantities (at high per-unit prices) instead of making do without until they can afford to buy a quantity that’s sold at a reasonable price. And on borrowing and paying interest for consumption goods.
These are bad strategies. For the most part, they do it because they don’t know any better. It’s how their parents did it, and how the people they know do it. People at the bottom of the income scale are rarely brilliant innovative thinkers who can puzzle their way out of crippling cultural practices—like most sensible (and non-brilliant) people, they take clues from those around them, avoiding moving Chesterson’s fences.
I’m not sure where you get the assumption that a UBI is funded by printing money. Most proposals I’ve seen are funded by taxation. The UBI proposed by Charles Murray (in https://www.amazon.com/Our-Hands-Replace-Welfare-State/dp/0844742236 ) is entirely funded by existing welfare and social spending (by redirecting them to a UBI).
Binmen == garbage men. [BTW, I think you’re underestimating them.]
There’s nothing to stop them, of course. But an article known to be from a reputable source is likely to have more impact than one from a known source of disinformation.
I have not claimed this is more than a “partial solution”.
Solely for the record, me too.
(Thanks for writing this.)
FWIW, I didn’t say anything about how seriously I take the AGI threat—I just said we’re not doomed. Meaning we don’t all die in 100% of future worlds.
I didn’t exclude, say, 99%.
I do think AGI is seriously fucking dangerous and we need to be very very careful, and that the probability of it killing us all is high enough to be really worried about.
What I did try to say is that if someone wants to be convinced we’re doomed (== 100%), then they want to put themselves in a situation where they believe nothing anyone does can improve our chances. And that leads to apathy and worse chances.
So, a dereliction of duty.
I’ve long suspected that our (and my personal) survival thru the Cold War is the best evidence available in favor of MWI.
I mean—what were the chances?
The merits of replacing the profit motive with other incentives has been debated to death (quite literally) for the last 150 years in other fora—including a nuclear-armed Cold War. I don’t think revisiting that debate here is likely to be productive.
There appears to be a wide (but not universal) consensus that to the extent the profit motive is not well aligned with human well-being, it’s because of externalities. Practical ideas for internalizing externalities, using AI or otherwise, I think are welcome.
A lot of “social grace” is strategic deception. The out-of-his-league woman defers telling the guy he’s getting nowhere as long as possible, just in case it turns out he’s heir to a giant fortune or something.
And of course people suck up to big shots (the Feynman story) because they hope to associate with them and have some of their fame and reputation rub off on themselves.
This is not irrational behavior, given human goals.
Added: I do think Bohr was wrong and Everett (MWI) was right.
So think of it this way—you can only experience worlds in which you survive. Even if Yudkowsky is correct and in 99% of all worlds AGI has killed us all by 20 years from now, you will experience only the 1% of worlds in which that doesn’t happen.
And in many of those worlds, you’ll be wanting something to live on in your retirement.
Niels Bohr supposedly said “Prediction is difficult, especially about the future”. Even if he was mistaken about quantum mechanics, he was right about that.
Every generation seems to think it’s special and will encounter new circumstances that turn old advice on its head. Jesus is coming back. We’ll all die in a nuclear war. Space aliens are coming. A supernova cascade will sterilize Earth. The planets will align and destroy the Earth. Nanotech will turn us all into grey goo. Global warming will kill us all.
It’s always something. Now it’s AGI. Maybe it’ll kill us. Maybe it’ll usher in utopia, or transform us into gods via a singularity.
Maybe. But based on the record to date, it’s not the way to bet.
Whatever you think the world is going to be like in 20 years, you’ll find it easier to deal with if you’re not living hand-to-mouth. If you find it difficult to save money, it’s very tempting to find an excuse to not even try. Don’t deceive yourself.
″… however it may deserve respect for its usefulness and antiquity, [predicting the end of the world] has not been found agreeable to experience.”—Edward Gibbon, ‘Decline and Fall of the Roman Empire’
In most circumstances Tesla’s system is better than human drivers already.
But there’s a huge psychological barrier to trusting algorithms with safety (esp. with involuntary participants, such as pedestrians) - this is why we still have airline pilots. We’d rather accept a higher accident rate with humans in charge than a lower non-zero rate with the algorithm in charge. (If it were zero, that would be different, but that seems impossible.)
That influences the legal barriers—we inevitably demand more of the automated system than we do of human drivers.
Finally, liability. Today drivers bear the liability risk for accidents, and pay for insurance to cover it. It seems impossible to justify putting that burden on drivers when drivers aren’t in charge—those who write the algorithms and build the hardware (car manufacturers) will have that burden. And that’s pricey, so manufacturers don’t have great incentive to go there.