Maybe the test case is to delete one chromosome and insert another a chromosome in a fruit fly. Only 4 pairs of chromosomes, already used for genetic modifications with CRISPR
Goal = complete the insertion and still develop a normal fruit fly. I bet this is a fairly inexpensive experiment, within reach of many people on LessWrong
Chromosome selection seems like the most consequential idea here if it’s possible
Is it possible now, even in animals? Can you isolate chromosomes without damaging them and assemble them into a viable nucleus?
Edit: also—strong upvoted because I want to see more of this on LW. Not directly AI but massively affects the gameboard
My model of “steering” the military is a little different from that It’s over a thousand partially autonomous headquarters, which each have their own interests. The right hand usually doesn’t know what the left is doing
Of the thousand+ headquarters, there’s probably 10 that have the necessary legitimacy and can get the necessary resources. Winning over any one of the 10 is a sufficient condition to getting the results I described above
In other words, you don’t have to steer the whole ship. Just a small part of it. I bet that can be done in 6 months
I don’t agree, because a world of misaligned AI is known to be really bad. Whereas a world of AI successfully aligned by some opposing faction probably has a lot in common with your own values
Extreme case: ISIS successfully builds the first aligned AI and locks in its values. This is bad, but it’s way better than misaligned AI. ISIS want to turn the world into an idealized 7th Century Middle East, which is a pretty nice place compared to much of human history. There’s still a lot in common with your own values
I bet that’s true
But it doesn’t seem sufficient to settle the issue. A world where aligning/slowing AI is a major US priority, which China sometimes supports in exchange for policy concessions sounds like a massive improvement over today’s world
The theory of impact here is that there’s a lot of policy actions to slow down AI, but they’re bottlenecked on legitimacy. The US military could provide legitimacy
They might also help alignment, if the right person is in charge and has a lot of resources. But even if 100% their alignment research is noise that doesn’t advance the field, military involvement could be a huge net positive
So the real question is:
Is the theory of impact plausible
Are their big risks that mean this does more harm than good
Because maximizing the geometric rate of return, irrespective of the risk of ruin, doesn’t reflect most peoples’ true preferences
In the scenario above with the red and blue lines, the full Kelly has a 9.3% chance of losing at least half your money, but the .4 Kelly only has a 0.58% chance of getting an outcome at least that bad
I agree. I think this basically resolves the issue. Once you’ve added a bunch of caveats:
The bet is mind-bogglingly favorable. More like the million-to-one, and less like the 51% doubling
The bet reflects the preferences of most of the world. It’s not a unilateral action
You’re very confident that the results will actually happen (we have good reason to believe that the new Earths will definitely be created)
Then it’s actually fine to take the bet. At that point, our natural aversion is based on our inability to comprehend the vast scale of a million Earths. I still want to say no, but I’d probably be a yes at reflective equilibrium
Therefore… there’s not much of a dilemma anymore
It doesn’t matter who said an idea. I’d rather just consider each idea on its own merits
I don’t think that solves it. A bounded utility function would stop you from doing infinite doublings, but it still doesn’t prevent some finite number of doublings in the million-Earths case
That is, if the first round multiplies Earth a millionfold, then you just have to agree that a million Earths is at least twice as good as one Earth
There’s definitely bet patterns superior to risking everything—but only if you’re allowed to play a very large number of rounds
If the pharmacist is only willing to play 10 rounds, then there’s no way to beat betting everything every round
As the number of rounds you play approaches infinity (assuming you can bet fractions of a coin), your chance of saving everyone approaches 1. But this takes a huge number of rounds. Even at 10,000 rounds, the best possible strategy only gives each person around a 1 in 20 chance of survival
Updated! Excuse the delay
I buy that… so many of the folks funded by Emergent Ventures are EAs, so directly arguing against AI risk might alienate his audience
Still, this Straussian approach is a terrible way to have a productive argument
My mistake! Fixed
Many thanks for the update… and if it’s true that you could write the very best primer, that sounds like a high value activity
I don’t understand the astroid analogy though. Does this assume the impact is inevitable? If so, I agree with taking no action. But in any other case, doing everything you can to prevent it seems like the single most important way to spend your days
Many thanks! It looks like EA was the right angle… found some very active English-speaking EA groups right next to where I’ll be
I bet you’re right that a perceived lack of policy options is a key reason people don’t write about this to mainstream audiences
Still, I think policy options exist
The easiest one is adding right right types AI capabilities research to the US Munitions List, so they’re covered under ITAR laws. These are mind-bogglingly burdensome to comply with (so it’s effectively a tax on capabilities research). They also make it illegal to share certain parts of your research publicly
It’s not quite the secrecy regime that Eliezer is looking for, but it’s a big step in that direction
I think 2, 3, and 8 are true but pretty easy to overcome. Just get someone knowledgeable to help you
4 (low demand for these essays) seems like a calibration question. Most writers probably would lose their audience if they wrote about it as often as Holden. But more than zero is probably ok. Scott Alexander seems to be following that rule, when he said that we was summarizing the 2021 MIRI conversations at a steady drip so as not to alienate the part of his audience that doesn’t want to see that
I think 6 (look weird) used to be true, but it’s not any more. It’s hard to know for sure without talking to Kelsey Piper or Ezra Klein, but I suspect they didn’t lose any status for their Vox/NYT statements
I agree that it’s hard, but there are all sorts of possible moves (like LessWrong folks choosing to work at this future regulatory agency, or putting massive amounts of lobbying funds into making sure the rules are strict)
If the alternative (solving alignment) seems impossible given 30 years and massive amounts of money, then even a really hard policy seems easy by comparison
Eliezer gives alignment a 0% chance of succeeding. I think policy, if tried seriously, has >50%. So it’s a giant opportunity that’s gotten way too little attention
I’m optimistic about policy for big companies in particular. They have a lot to lose from breaking the law, they’re easy to inspect (because there’s so few), and there’s lots of precedent (ITAR already covers some software). Right now, serious AI capabilities research just isn’t profitable outside of the big tech companies
Voluntary compliance is also a very real thing. Lots of AI researchers are wealthy and high-status, and they’d have a lot to lose from breaking the law. At the very least, a law would stop them from publishing their research. A field like this also lends itself to undercover enforcement
I think an agreement with China is impossible now, because prominent folks don’t even believe the threat exists. Two factors could change the art of the possible. First, if there were a widely known argument about the dangers of AI, on which most public intellectual agreed. Second, since the US has a technological lead, it could actually be to their advantage.
That implies the ability to mix and match human chromosomes commercially is really far off
I agree that the issues of avoiding damage and having the correct epigenetics seem like huge open questions, and successfully switching a fruit fly chromosome isn’t sufficient to settle them
Would this sequence be sufficient?
1. Switch a chromosome in a fruit fly
Success = normal fruit fly development
2a. Switch a chromosome in a rat
Success = normal rat development
2b. (in parallel, doesn’t depend on 2a) Combine several chromosomes in a fruit fly to optimize aggressively for a particular trait
Success = fruit fly develops with a lot of the desired trait, but without serious negative consequences
3. Repeat 2b on a rat
4. Repeat 2a and 2b on a primate
Can you think of a faster way? It seems like a very long time to get something commercially viable