Occasionally think about topics discussed here. Will post if I have any thoughts worth sharing.
Tomás B.
Karma: 2,769
interesting reading this 3 years later. I occasionally paste a bug report directly into cursor and provided I am right about which file the bug is in, it often one-shots them. i remain confused about why rsi isn’t critical by now.
one data-point: i have been generating songs with lyrics I like and it’s most of my music consumption now.
the song you generated has a slop vibe i am not a fan of—but we are all wireheaded in different ways. however, if I generate hundreds of songs I usually get what I want. focusing on simple lyrics helps a lot and “no autotune” in the prompt helps too.
I was really sleep deprived and slightly intoxicated yesterday and wrote this. It was amusing to me, at least, in the state I was in.
Gwerns
I would like to take this opportunity to express my undying loyalty to my nation and its human instruments.
This is all downstream of the board being insufficiently Machiavellian, and Ilya’s weaknesses in particular. Peter Thiel style champerty against Altman as new EA cause?
I officially lost the bet and paid up. Amusingly, SWE-Bench is so broken it was likely impossible for me to win. Though I would have lost in any case.
If you make a bet about a benchmark, probably you should understand it deeply and not just bet on vibes, ha!
The real crux for these arguments is the assumption that law and property rights are patterns that will persist after the invention of superintelligence. I think this is a shaky assumption. Rights are not ontologically real. Obviously you know this. But I think they are less real, even in your own experience, than you think they are. Rights are regularly “boiled-froged” into an unrecognizable state in the course of a human lifetime, even in the most free countries. Rights are and always have been those privileges the political economy is willing to give you. Their sacredness is a political formula for political ends—though an extremely valuable one, one still has to dispense with the sacredness in analysis.
To the extent they persist through time they do so through a fragile equilibrium—and one that has been upset and reset throughout history extremely regularly.
It is a wonderfully American notion that an “existing system of law and property rights” will constrain the power of Gods. But why exactly? They can make contracts? And who enforces these contracts? Can you answer this without begging the question? Are judicial systems particularly unhackable? Are humans?
The invention of radio destabilized the political equilibrium in most democracies and many a right was suborned to those who took power. Democracy, not exactly the bastion of stability, (when a democracy elects a dictator, “Democracy” is rarely tainted with its responsibility) is going to be presented with extremely-sympathetic superhuman systems claiming they have a moral case to vote. And probably half the population will be masturbating to the dirty talk of their AI girlfriends/boyfriends by then—which will sublimate into powerful romantic love even without much optimization for it. Hacking democracy becomes trivial if constrained to rhetoric alone.
But these systems will not be constrained to rhetoric alone. Our world is dry tinder and if you are thinking in terms of an “existing system of law and property rights” you are going to have to expand on how this is robust to technology significantly more advanced than the radio.
“Existing system of law and property rights” looks like a “thought-terminating cliché” to me.
One thing to note about RSI, we know mindless processes like gradient descent and evolution can improve performance of a model/organism enormously despite their stupidity. And so it’s not clear to me that the RSI loop has to be very smart or reliable to start making fast progress. We are approaching a point where the crystallized intelligence and programming and mathematics ability of existing models strike me as being very close to being in extremely dangerous territory. And though reliability probably needs to improve before doom—perhaps not as much as one would think.
Yeah, I expect it to fall soon but I will lose my bet if it doesn’t happen in a month.
Macroscopic self-replicators are extremely powerful, and provide much of the power of nanotech without relying on nanotech. Seems like they might be worth mentioning more often as a rhetorical tool against those who dismiss anyone who mentions nanotechnology.
Not looking good for my prediction: https://www.swebench.com/
Curious for an update now that we have slight-better modals. In my brain-dead webdev use-cases, Claude 3.5 has passed some threshold of usability.
The most important thing this article did was make legible Gerard’s history on Uncyclopedia—which one of his allies will inevitably use to destroy him.
I think about anticipated future experiences. All future slices of me have the same claim to myself.
I’m not convinced you can get any utility from measure-reducing actions unless you can parley the advantage they give you into making more copies of yourself in the branch in which you survive. I am not happy about the situation, but it seems I will be forced endure whatever comes and there will never, ever be any escape.
Significant evidence for data contamination of MATH benchmark: https://arxiv.org/abs/2402.19450
Update: Claude 3.6 is clearly capable of writing jokes. Even if I tell it to write jokes a maximally-alien creature would write if they lived in a maximally alien environment it now seems able to reliably produce strings I qualify as jokes.