Occasionally think about topics discussed here. Will post if I have any thoughts worth sharing.
Tomás B.
Karma: 2,999
Why Should I Assume CCP AGI is Worse Than USG AGI?
I was joking.
it’s good though! but yeah a bit cringe, as all the best things are.
I was not wrong? I argued with Connor about this back in the day on EAI. If you take the PISA scores at face value and account for their higher population, China has like 30x more people above 145 IQ. Steve Hsu has a nice post about this[1]:
This appears to be very antimemetic but seems to be true and the world sure is behaving like this is true.
If human capital matters, they have more and better human capital than America. But human capital will be obsolete soon.
And I do think much of the idea that China cannot catch up is based on some idiotic, racist notions that asians lack creativity. And this increasingly is becoming obviously massive cope as China catches up and surpasses in industry after industry. And no, semiconductors are not special. They will catch up just fine.
I think timelines are short, so I would still be betting on an “American god” but if they are longer than I think, China is a contender, and increasing more so in worlds where timelines are long and human capital still matters.
Dusty Hands and Geo-arbitrage
Update: Claude 3.6 is clearly capable of writing jokes. Even if I tell it to write jokes a maximally-alien creature would write if they lived in a maximally alien environment it now seems able to reliably produce strings I qualify as jokes.
interesting reading this 3 years later. I occasionally paste a bug report directly into cursor and provided I am right about which file the bug is in, it often one-shots them. i remain confused about why rsi isn’t critical by now.
one data-point: i have been generating songs with lyrics I like and it’s most of my music consumption now.
the song you generated has a slop vibe i am not a fan of—but we are all wireheaded in different ways. however, if I generate hundreds of songs I usually get what I want. focusing on simple lyrics helps a lot and “no autotune” in the prompt helps too.
I was really sleep deprived and slightly intoxicated yesterday and wrote this. It was amusing to me, at least, in the state I was in.
Gwerns
I would like to take this opportunity to express my undying loyalty to my nation and its human instruments.
This is all downstream of the board being insufficiently Machiavellian, and Ilya’s weaknesses in particular. Peter Thiel style champerty against Altman as new EA cause?
I officially lost the bet and paid up. Amusingly, SWE-Bench is so broken it was likely impossible for me to win. Though I would have lost in any case.
If you make a bet about a benchmark, probably you should understand it deeply and not just bet on vibes, ha!
The real crux for these arguments is the assumption that law and property rights are patterns that will persist after the invention of superintelligence. I think this is a shaky assumption. Rights are not ontologically real. Obviously you know this. But I think they are less real, even in your own experience, than you think they are. Rights are regularly “boiled-froged” into an unrecognizable state in the course of a human lifetime, even in the most free countries. Rights are and always have been those privileges the political economy is willing to give you. Their sacredness is a political formula for political ends—though an extremely valuable one, one still has to dispense with the sacredness in analysis.
To the extent they persist through time they do so through a fragile equilibrium—and one that has been upset and reset throughout history extremely regularly.
It is a wonderfully American notion that an “existing system of law and property rights” will constrain the power of Gods. But why exactly? They can make contracts? And who enforces these contracts? Can you answer this without begging the question? Are judicial systems particularly unhackable? Are humans?
The invention of radio destabilized the political equilibrium in most democracies and many a right was suborned to those who took power. Democracy, not exactly the bastion of stability, (when a democracy elects a dictator, “Democracy” is rarely tainted with its responsibility) is going to be presented with extremely-sympathetic superhuman systems claiming they have a moral case to vote. And probably half the population will be masturbating to the dirty talk of their AI girlfriends/boyfriends by then—which will sublimate into powerful romantic love even without much optimization for it. Hacking democracy becomes trivial if constrained to rhetoric alone.
But these systems will not be constrained to rhetoric alone. Our world is dry tinder and if you are thinking in terms of an “existing system of law and property rights” you are going to have to expand on how this is robust to technology significantly more advanced than the radio.
“Existing system of law and property rights” looks like a “thought-terminating cliché” to me.
One thing to note about RSI, we know mindless processes like gradient descent and evolution can improve performance of a model/organism enormously despite their stupidity. And so it’s not clear to me that the RSI loop has to be very smart or reliable to start making fast progress. We are approaching a point where the crystallized intelligence and programming and mathematics ability of existing models strike me as being very close to being in extremely dangerous territory. And though reliability probably needs to improve before doom—perhaps not as much as one would think.
Yeah, I expect it to fall soon but I will lose my bet if it doesn’t happen in a month.
Macroscopic self-replicators are extremely powerful, and provide much of the power of nanotech without relying on nanotech. Seems like they might be worth mentioning more often as a rhetorical tool against those who dismiss anyone who mentions nanotechnology.
Not looking good for my prediction: https://www.swebench.com/
I am unsure how free the average Chinese person is, nor how to weigh freedom of speech with certain economic freedoms and competent local government, low crime, the tendency of modern democracies to rent seek from the young in favour of the old, zoning laws, restriction on industrial development, a student loan system that seems to be a weird form of indenture. I do come from a country with rather strict hate speech laws. And we do not, in fact, have freedom of speech by any strict definition. And this is a policy American elites in and out of government strongly approve of.
I ask out of relative ignorance of what life in China is like for the average Chinese person, but with a slight suspicion that we might be defining our western notion of ‘freedom’ in such a way that ignores the many ways we are restricted and extracted from, and ways in which the average Chinese may be more free.
It’s very clear the CCP has committed far larger crimes against its people in living memory. But it is also a very different organization than it was at its worst.
I think the question is still worth asking. And the argument worth justifying.