These things are either unlikely to succeed or just not that important.
Including dath ilan is especially strange. It is a work of fiction, it is not really appealing, and it is not realistic (that is, not internally consistent) either.
Lalartu
Karma: 360
That story of Mongol conquests is simply not true. Horse archer steppe nomads existed for many centuries before Mongols, and often tried to conquer their neighbors, with mixed success. What happened in the 1200s is that Mongols had a few exceptionally good leaders. After they died, Mongols lost their advantage.
Calling states like Khwarazmian or Jin Empire “small duchies” is especially funny.
As I understand, topicstarters claim is that civilization is not a chaotic system, and any temporary disturbances don’t affect long-term trajectory. Weather is a chaotic system.
It is always possible to say in retrospect that whatever happened was inevitable. The problem is, a world where individual actions don’t matter that much should be a predictable world. And ours very much isn’t.
Defaults only matter due to reputation. But stopping a weird practice which no one else does doesn’t really damage reputation.
So what happens when in 50 years the government just stops paying, without passing the law? Buyers of these instruments don’t care about that law, so they will not object much, and there will be no reputation loss.
80% for AGI solving aging is very optimistic. Even just one single possibility, that people who decide what values should AGI have happen to be anti-immortalist is imo >20%.
560M is not from the paper, it is from the post. The paper has graph with births per year stabilizing at around 5M, which can correspond to different population sizes depending on child mortality, but all of them are unrealistically low.
I don’t see any reason how the population can stabilize at the arbitrary level of 560 million. Either it will start to increase again at some point (due to cultural shift, natural selection or some other reason), or it will decline until the collapse of the industrial civilization. But in that case, with no universal education anymore, the demographic transition will reverse, and the population starts growing again to limits imposed by pre-industrial agriculture (which is somewhere around 2 to 3 billion, not half a billion).
Or, suppose there is an intelligent alien civilization that has been around for much longer than humans. Would you expect that they have definitely industrialized in some form? Or would it depend on the particular geology of their planet?I think the answers are “no” and “yes”. No fossil fuels, no cheap energy, no industrial revolution ever. There might be a different way through hydroelectric power, but that requires cheap copper.
Didn’t we pretty much always know it was going to come from one or a few giant companies or research labs?
Not at all. In that ancient era of early 2000s, the common position was that insight was far more important than resources. So the usual scenario was some bright guy building AI in his basement.
There is a counterargument to claims 6 and 7: there is only one known warm-blooded species with longer lifespan than humans (the bowhead whale), and it is large and has slow metabolism for a mammal. In contrast, there are plenty of cold-blooded long-lived species. So, it is entirely plausible that humans live about as long as possible for given metabolic level.
On Earth, yes. In a world where animals have projectile weapons, no.
There is a threshold where intelligence becomes much more useful, and this threshold is an ability to make better weapons than other animals have. In a world where this is not possible at all (for example, animals have zerg-level natural weapons, and there is no metal to make guns), having human-level intelligence is just not worth downsides from a big brain.
Because ability to believe whatever most people around believe was net beneficial. Also, the situation when going to war means risking your life and getting nothing valuable in return is rather modern.
That Malthusianism is wrong (for predicting the future). Prior to the demographic transition, arguments in favor of this view could be basically summarized as “somehow it will be fine”.
RLHF is a trial-and-error approach. For superhuman AGI, that amounts to letting it kill everybody, and then telling that this is bad, don’t do it again.
The Milgram experiments demonstrated Didn’t it fall victim to the replication crisis? I have read somewhere that with different groups outcomes of those experiments are wildly different.
I think this scenario is not even remotely realistic. If things really go this way (which is far from granted), the government will massively expand police and other security services (and they will have money for that due to AI productivity gains). When a large percentage of population are cops, riots aren’t that big problem.
This is based on assumption that defense is much easier than offense. This is not true, in fact in WWI attacker’s and defender’s losses were usually close (for example, ~140k vs ~160k KIA at Verdun).