AI engineer
alyssavance
Karma: 3,510
Most highly educated people lean left, but there really are just very few Stalinists. I quoted a poll above showing that just 8% of Americans would support an AOC hard-left party, and actual Stalinists are a small fraction of that. There’s no developed country where tankies get a serious fraction of the vote. See Contrapoints for why communist revolutionaries are super unlikely to take power: https://youtu.be/t3Vah8sUFgI
There are many crazy professors of various stripes, but universities aren’t states. They can’t shoot you, can’t throw you in jail, can’t seize your house or business, and are ultimately dependent on government funding to even exist.
The current constitution isn’t that old (although 65 years is still longer than most democracies), but with brief interruptions, France has been a democracy for around 150 years, which is far longer than most countries can claim.
Thanks for the response! Here are some comments:
- India, Turkey, and Hungary are widely referred to as “hybrid regimes” (https://en.wikipedia.org/wiki/Hybrid_regime), in which opposition still exists and there are still elections, but the state interferes with elections so as to virtually guarantee victory. In Turkey’s case, there have been many elections, but Erdogan always wins through a combination of mass arrests, media censorship, and sending his most popular opponent to prison for “insulting public officials” (https://www.bbc.com/news/world-europe-63977555). In India’s case, Modi is no doubt very popular, but elections are likewise hardly fair when the main opponent is disqualified and sent to prison for “defamation” (insulting Modi). Rather than being voted out, hybrid regimes usually transition to full dictatorships, as has happened in (eg.) Russia, Belarus, Venezuela, Nicaragua, Iran, etc.
- Of course nothing is certain, but France’s president is very powerful, and this article discusses in detail how Le Pen could manipulate the system to get a legislative supermajority and virtually unlimited power if elected: https://foreignpolicy.com/2022/04/20/france-election-le-pen-macron-constitution-separtism-law-state-of-emergency-referendum/
- I made a map here of roughly how much electoral support the far right has in each country worldwide (https://twitter.com/alyssamvance/status/1656882958903418880). However, it is tricky to forecast based on this, because far-right parties can appear from nothing and gain a wide support base very quickly (as happened with eg. Chile’s Republican Party).
I’m surprised by how strong the disagreement is here. Even if what we most need right now is theoretical/pre-paradigmatic, that seems likely to change as AI develops and people reach consensus on more things; compare eg. the work done on optics pre-1800 to all the work done post-1800. Or the work done on computer science pre-1970 vs. post-1970. Curious if people who disagree could explain more—is the disagreement about what stage the field is in/what the field needs right now in 2022, or the more general claim that most future work will be empirical?
I think saying “we” here dramatically over-indexes on personal observation. I’d bet that most overweight Americans have not only eaten untasty food for an extended period (say, longer than a month); and those that have, found that it sucked and stopped doing it. Only eating untasty food really sucks! For comparison, everyone knows that smoking is awful for your health, it’s expensive, leaves bad odors, and so on. And I’d bet that most smokers would find “never smoke again” easier and more pleasant (in the long run) than “never eat tasty food again”. Yet, the vast majority of smokers continue smoking:
https://news.gallup.com/poll/156833/one-five-adults-smoke-tied-time-low.aspx
https://transformer-circuits.pub/ seems impressive to me!
There are now quite a lot of AI alignment research organizations, of widely varying quality. I’d name the two leading ones right now as Redwood and Anthropic, not MIRI (which is in something of a rut technically). Here’s a big review of the different orgs by Larks:
Great post. I’m reminded of instructions from the 1944 CIA (OSS) sabotage manual:
”When possible, refer all matters to committees, for “further study and consideration.” Attempt to make the committee as large as possible — never less than five.”
Eliezer’s writeup on corrigibility has now been published (the posts below by “Iarwain”, embedded within his new story Mad Investor Chaos). Although, you might not want to look at it if you’re still writing your own version and don’t want to be anchored by his ideas.
Would be curious to hear more about what kinds of discussion you think are net negative—clearly some types of discussion between some people are positive.
Thanks for writing this! I think it’s a great list; it’s orthogonal to some other lists, which I think also have important stuff this doesn’t include, but in this case orthogonality is super valuable because that way you’re less likely for all lists to miss something.
This is an awesome comment, I think it would be great to make it a top-level post. There’s a Facebook group called “Information Security in Effective Altruism” that might also be interested
I hadn’t seen that, great paper!
Fantastic post! I agree with most of it, but I notice that Eliezer’s post has a strong tone of “this is really actually important, the modal scenario is that we literally all die, people aren’t taking this seriously and I need more help”. More measured or academic writing, even when it agrees in principle, doesn’t have the same tone or feeling of urgency. This has good effects (shaking people awake) and bad effects (panic/despair), but it’s a critical difference and my guess is the effects are net positive right now.
I edited the MNIST bit to clarify, but a big point here is that there are tasks where 99.9% is “pretty much 100%” and tasks where it’s really really not (eg. operating heavy machinery); and right now, most models, datasets, systems and evaluation metrics are designed around the first scenario, rather than the second.
Intentional murder seems analogous to misalignment, not error. If you count random suicides as bugs, you get a big numerator but an even bigger denominator; the overall US suicide rate is ~1:7,000 per year, and that includes lots of people who have awful chronic health problems. If you assume a 1:20,000 random suicide rate and that 40% of people can kill themselves in a minute (roughly, the US gun ownership rate), then the rate of not doing it per decision is ~20,000 * 60 * 16 * 365 * 0.4 = 1:3,000,000,000, or ~99.99999997%.
You say “yet again”, but random pilot suicides are incredibly rare! Wikipedia counts eight on commercial flights in the last fifty years, out of a billion or so total flights, and some of those cases are ambiguous and it’s not clear what happened: https://en.wikipedia.org/wiki/Suicide_by_pilot
Crazy idea: LessWrong and EA have been really successful in forming student groups at elite universities. But in the US, elite university admissions select on some cool traits (eg. IQ, conscientiousness), don’t select on others, and anti-select on some (eg. selection against non-conformists). To find capable people who didn’t get into an elite school, what if someone offered moderate cash bounties (say, $1,000-$5,000 range) to anyone who could solve some hard problem (eg. an IMO gold medal problem, or something like https://microcorruption.com/), without already being a “recognized expert” (say, under age 25, not at an elite school, not already working in the field, etc.). This would be similar to the old “Quixey Challenge” (https://venturebeat.com/2012/06/09/quixey-gamifies-job-hunting-in-an-entirely-new-way/), where coders were offered $100 to find a bug in one minute, but at a slightly broader scale and for different skills. This would select from a broader range of people, and could be adapted to whatever types of skills are most in demand, eg. hacking challenges for trying to recruit computer security people (https://forum.effectivealtruism.org/posts/ZJiCfwTy5dC4CoxqA/information-security-careers-for-gcr-reduction).
I think it’s true, and really important, that the salience of AI risk will increase as the technology advances. People will take it more seriously, which they haven’t before; I see that all the time in random personal conversations. But being more concerned about a problem doesn’t imply the ability to solve it. It won’t increase your base intelligence stats, or suddenly give your group new abilities or plans that it didn’t have last month. I’ll elide the details because it’s a political debate, but just last week, I saw a study that whenever one problem got lots of media attention, the “solutions” people tried wound up making the problem worse the next year. High salience is an important tool, but nowhere near sufficient, and can even be outright counterproductive.
On the state level, the correlation between urbanization and homelessness is small (R^2 = 0.13) and disappears to zero when you control for housing costs, while the reverse is not true (R^2 of the residual = 0.56). States like New Jersey, Rhode Island, Maryland, Illinois, Florida, Connecticut, Texas, and Pennsylvania are among the most urbanized but have relatively low homelessness rates, while Alaska, Vermont, and Maine have higher homelessness despite being very rural. There’s also, like, an obvious mechanism where expensive housing causes homelessness (poor people can’t afford rent).
The correlation between homelessness and black population in a state is actually slightly negative (R^2 = 0.09). Mississippi, Louisiana and Alabama are some of the blackest states and have the lowest homelessness rates in the US; Hawaii, Oregon, Alaska and Vermont have some of the highest despite being <5% black.
Data from 538′s Urbanization Index: https://fivethirtyeight.com/features/how-urban-or-rural-is-your-state-and-what-does-that-mean-for-the-2020-election/
Do you have links handy?
FWIW I’m not convinced by the article on Haley, having bad conservative policies != being an anti-democratic nut job who wants to rig elections and put all your opponents in jail. She’s super unlikely to win, though.