Just this guy, you know?
Dagon
Karma: 12,795
Do you think that the world is getting worse each year?
Good clarification question! My answer probably isn’t satisfying, though. “It’s complicated” (meaning: multidimensional and not ordinally comparable).On a lot of metrics, it’s better by far, for most of the distribution. On harder-to-operationally-define dimensions (sense of hope and agency for the 25th through 75th percentile of culturally normal people), it’s quite a bit worse.
would consider the end of any story a loss.
Unfortunately, now you have to solve the fractal-story problem. Is the universe one story, or does each galaxy have it’s own? Each planet? Continent? Human? Subpersonal individual goals/plotlines? Each cell?
I feel like you’re talking in highly absolutist terms here.
You’re correct, and I apologize for that. There are plenty of potential good outcomes where individual autonomy reverses the trend of the last ~70 years. Or where the systemic takeover plateaus at the current level, and the main change is more wealth and options for individuals. Or where AI does in fact enable many/most individual humans to make meaningful decisions and contributions where they don’t today.
I mostly want to point out that many disempowerment/dystopia failure scenarios don’t require a step-change from AI, just an acceleration of current trends.
Presumably, if The Observer has a truly wide/long view, then destruction of the Solar System, or certainly loss of all CHON-based lifeforms on earth, wouldn’t be a problem—there have got to be many other macroscopic lifeforms out there, even if The Great Filter turns out to be “nothing survives the Information Age, so nobody ever detects another lifeform”.
Also, you’re describing an Actor, not just an Observer. If has the ability to intervene, even if it rarely chooses to do so, that’s it’s salient feature.
This seems like it would require either very dumb humans, or a straightforward alignment mistake risk failure, to mess up.
I think “very dumb humans” is what we have to work with. Remember, it only requires a small number of imperfectly aligned humans to ignore the warnings (or, indeed, to welcome the world the warnings describe).
a lot of people have strong low-level assumptions here that a world with lots of strong AIs must go haywire.
For myself, it seems clear that the world has ALREADY gone haywire. Individual humans have lost control of most of our lives—we interact with policies, faceless (or friendly but volition-free) workers following procedure, automated systems, etc. These systems are human-implemented, but in most cases too complex to be called human-controlled. Moloch won.
Big corporations are a form of inhuman intelligence, and their software and operations have eaten the world. AI pushes this well past a tipping point. It’s probably already irreversable without a major civilizational collapse, but it can still get … more so.in worlds where AI systems have strong epistemics without critical large gaps, and can generally be controlled / aligned, things will be fine.
I don’t have good working definitions of “controlled/aligned” that would make this true. I don’t see any large-scale institutions or groups large and sane enough to have a reasonable CEV, so I don’t know what an AI could align with or be controlled by.
In non-trivial settings, (some but not all) structural differences between programs lead to differences in input/output behaviour, even if there is a large domain for which they are behaviourally equivalent.
I think this is a crux (of why we’re talking past each other; I don’t actually know if we have a substantive disagreement). The post was about detecting “smaller than a lookup table would support” implementations, which implied that the input/output functionally-identical-as-tested were actually tested in the broadest possible domain. I fully agree that “tested” and “potential” input/output pairs are not the same sets, but I assert that, in a black-box situation, it CAN be tested in a very broad set of inputs, so the distinction usually won’t matter. That said, nobody has built a pure lookup table anywhere near as complete as it would take to matter (unless the universe or my experience is simulated that way, but I’ll never know).
My narrower but stronger point is that “lookup table vs algorithm” is almost never as important as “what specific algorithm” for any question we want to predict about the black box. Oh, and almost all real-world programs are a mix of algorithm and lookup.
might be true if you just care about input and output behaviour
Yes, that is the assumption for “some computable function” or “black box which takes in strings and spits out other strings.”
I’m not sure your example (of an AI with a much wider range of possible input/output pairs than the lookup table) fits this underlying distinction. If the input/output sets are truly identical (or even identical for all tests you can think of), then we’re back to the “why do we care” question.
i don’t exactly disagree with the methodology, but I don’t find the “why do we care” very compelling. For most practical purposes, “calculating a function” is only and exactly a very good compression algorithm for the lookup table.
Unless we care about side-effects like heat dissipation or imputed qualia, but those seem like you need to distinguish among different algorithms more than just “lookup table or no”.
(I’m using time-sensitive words, even though we are stepping out of the spacetime of our universe for parts of this discussion.)
Maybe use different words, so as not to imply that there is a temporal, causal, or spacial relation.
Many people realize that, conceptually “below” or “before” any “base universe,” there is
I don’t realize or accept that. Anything that would be in those categories are inaccessible to our universe, and not knowable or reachable from within. They are literally imaginary.
“all” humans?
The vast majority of actual humans are already dead. The overwhelming majority of currently-living humans should expect 95%+ chance they’ll die in under a century.
If immortality is solved, it will only apply to “that distorted thing those humans turn into”. Note that this is something the stereotypical Victorian would understand completely—there may be biological similarities with today’s humans, but they’re culturally a different species.
When humans fall well below marginal utility compared to AIs, will their priorities matter to a system that has made them essentially obsolete?
The point behind my question is “we don’t know. If we reason analogously to human institutions (which are made of humans, but not really made or controlled BY individual humans), we have examples in both directions. AIs have less biological drive to care about humans than humans do, but also have more training on human writings and thinking than any individual human does.
My suspicion is that it won’t take long (in historical time measure; perhaps only a few decades, but more likely centuries) for a fully-disempowered species to become mostly irrelevant. Humans will be pets, perhaps, or parasites (allowed to live because it’s easier than exterminating them). Of course, there are plenty of believable paths that are NOT “computational intelligence eclipses biology in all aspects”—it may hit a wall, it may never develop intent/desire, it may find a way to integrate with biologicals rather than remaining separate, etc. Oh, and it may be fragile enough that it dies out along with humans.
Do we have a good story about why this hasn’t already happened to humans? Systems don’t actually care about the individuals they comprise, and certainly don’t care about the individuals that are neither taxpayers, selectorate, contributors, or customers.
Why do modern economies support so many non-participants? Let alone the marginal and slightly sub-marginal workers, which don’t cost much and may have option value or be useful to keep money moving in some way, there are a lot who are clearly a drain on resources.
There are a lot of good reasons to believe that stated human preferences correspond to real human preferences.
Can you name a few? I know of one: I assume that there’s some similarity with me in because of similar organic structures doing the preferring. That IS a good reason, but it’s not universally compelling or unassailable.
Actually, can you define ‘real preferences’ in some way that could be falsifiable for humans and observable for AIs?“Surely the AIs can be trained to say “I want hugs” or “I don’t want hugs,” just as easily, no?”
Just as easily as humans, I’m sure.
“Safety bar” and “most safety possible” both assume that safety is measurable. Is it?
No real knowledge of the factors involved, other than that it’s been tried before, somewhat recently. I wouldn’t bet either side of those markets, which means they’re reasonable to me.
Note that “plainly illegal means” is poorly-defined in this context. Laws change and get interpreted differently in diferent contexts. Things I’d call shenanigans, the Supreme Court may not, especially if one or two more justices get replaced. There will almost certainly be icky behavior that probably would have been illegal in other decades. There will almost certainly NOT be any convictions of Trump (or the eventual winner).
Edit to add: This is not just Trump. He’s old and he knows it. It’s the change of societal expectations, and the political technology of control.
We haven’t figured it out for humans, and only VERY recently in history has the idea become common that people not kin to you deserve empathy and care. Even so, it’s based on vibes and consensus, not metrics or proof. I expect it’ll take less than a few decades to start recognizing some person-hood for some AIs.
It’ll be interesting to see if the reverse occurs: the AIs that end up making decisions about humans could have some amount of empathy for us, or they may just not care.
Core Framework:
Endurists: Death as the ultimate evil (life at all costs).
Serenists: Suffering as the ultimate evil (non-existence over agony).
I don’t think I fit into this framework, and I reject the “ultimate evil” concept. Death is currently and generally disprefered. Suffering sucks as well and I’d like to minimize it. The question of which is better/worse depends on the distribution of possible futures, and there are (probably) cases I’d choose death, and (probably) cases I’d choose suffering. Especially in situations where suffering has option value—I can choose death later, when I have more information about the distribution of possible futures, or more information about my own mind and it’s reaction to suffering.
I also acknowledge that it’s very hard to measure suffering (or joy) in a rigorous comparable way—this means I have very little standing to judge someone on their choice for themselves. And I tend to be a bit assymetric in my advice and preferences-for-others: I recommend endurism to people who aren’t fully certain over a fairly long period of time (say, 2 human lifetimes...), and I’m not sure I’d choose it for some levels or kinds of suffering.
I have strong doubts as to the effectiveness and secrecy of a single-headed conspiracy. My version of this is “Most popular, open, discussion or news sources are heavily biased and many participants (including reporters with a byline or chat hosts) have a POV or purpose different than the naive cooperative view.”
Whether it’s a bot, a sock-puppet (alt account), a thrall (idiot parroting some demagogue), or just a low-value uninformed “real” participant, 90% of everything is crap, and has been since before Ted Sturgeon identified the phenomenon in 1956.
I’ve given some thought to this over the last few decades, and have yet to find ANY satisfying measures, let alone a good set. I reject the trap of “if it’s not objective and quantitative, it’s not important”—that’s one of the underlying attitudes causing the decline.
I definitely acknowledge that my memory of the last quarter of the previous century is fuzzy and selective, and beyond that is secondhand and not-well-supported. But I also don’t deny my own experience that the (tiny subset of humanity) people I am aware of as individuals have gotten much less hopeful and agentic over time. This may well be for reasons of media attention, but that doesn’t make it not real.