RHollerith

Karma: 2,817

Richard Hollerith. 15 miles north of San Francisco. hruvulum@gmail.com

My probability that AI research will end all human life is .92. It went up drastically when Eliezer started going public with his pessimistic assessment in April 2022. Till then my confidence in MIRI (and knowing that MIRI has enough funding to employ many researchers) was keeping my probability down to about .4. (I am glad I found out about Eliezer’s assessment.)

Currently I am willing to meet with almost anyone on the subject of AI extinction risk.

Last updated 26 Sep 2023.

RHollerith Feb 4, 2025, 12:08 AM
5 points
−2
in reply to: Knight Lee’s comment on: Mikhail Samin’s Shortform

we’re probably doomed in that case anyways, even without increasing alignment research.

I believe we’re probably doomed anyways.

I think even you would agree what P(1) > P(2)

Sorry to disappoint you, but I do not agree.

Although I don’t consider it quite impossible that we will figure out alignment, most of my hope for our survival is in other things, such as a group taking over the world and then using their power to ban AI research. (Note that that is in direct contradiction to your final sentence.) So for example, if Putin or Xi were dictator of the world, my guess is that there is a good chance he would choose to ban all AI research. Why? It has unpredictable consequences. We Westerners (particularly Americans) are comfortable with drastic change, even if that change has drastic unpredictable effects on society; non-Westerners are much more skeptical: there have been too many invasions, revolutions and peasant rebellions that have killed millions in their countries. I tend to think that the main reason Xi supports China’s AI industry is to prevent the US and the West from superseding China and if that consideration were removed (because for example he had gained dictatorial control over the whole world) he’d choose to just shut it down (and he wouldn’t feel that need to have a very strong argument for that shutting it down like Western decision-makers would: non-Western leader shut important things down all the time or at least they would if the governments they led had the funding and the administrative capacity to do so).

Of course Xi’s acquiring dictatorial control over the whole world is extremely unlikely, but the magnitude of the technological changes and societal changes that are coming will tend to present opportunities for certain coalitions to gain and to keep enough power to shut AI research down worldwide. (Having power in all countries hosting leading-edge fabs is probably enough.) I don’t think this ruling coalition necessarily need to believe that AI presents a potent risk of human extinction for them to choose to shut it down.

I am aware that some reading this will react to “some coalition manages to gain power over the whole world” even more negatively than to “AI research causes the extinction of the entire human race”. I guess my response is that I needed an example of a process that could save us and that would feel plausible—i.e., something that might actually happen. I hasten add that there might be other processes that save us that don’t elicit such a negative reaction—including processes the nature of which we cannot even currently imagine.

I’m very skeptical of any intervention that reduces the amount of time we have left in the hopes that this AI juggernaut is not really as potent a threat to us as it currently appears. I was much much less skeptical of alignment research 20 years ago, but since then a research organization has been exploring the solution space and the leader of that organization (Nate Soares) and its most senior researcher (Eliezer) are reporting that the alignment project is almost completely hopeless. Yes, this organization (MIRI) is kind of small, but it has been funded well enough to keep about a dozen top-notch researchers on the payroll and it has been competently led. Also, for research efforts like this, how many years the team had to work on the problem is more important than the size of the team, and 22 years is a pretty long time to end up with almost no progress other than some initial insights (around the orthogonality thesis, the fragility of value, convergent instrumental values, CEV as a solution to if the problem were solvable by the current generation of human beings.

OK, if I’m being fair and balanced, then I have to concede that it was probably only in 2006 (when Eliezer figured out how to write a long intellectually-dense blog post every day) or even only in 2008 (when Anna Salamon join the organization—she was very good at recruiting and had a lot of energy to travel and to meet people) that Eliezer’s research organization could start to pick and choose among a broad pool of very talented people, but still between 2008 and now is 17 years, which again is a long time for a strong team to fail to make even a decent fraction of the progress humanity would seem to need to make on the alignment problem if in fact the alignment problem is solvable by spending more money on it. It does not appear to me to be the sort of problem than can be solved with 1 or 2 additional insights; it seems a lot more like the kind of problem where insight 1 is needed, but before any mere human can find insight 1, all the researchers need to have already known insight 2, and to have any hope of finding insight 2, they all would have had to know insight 3, and so on.

RHollerith Feb 3, 2025, 12:13 AM
3 points
−10
in reply to: Knight Lee’s comment on: Mikhail Samin’s Shortform

AI safety spending is only $0.1 billion while AI capabilities spending is $200 billion. A company which adds a comparable amount of effort on both AI alignment and AI capabilities should speed up the former more than the latter

There is very little hope IMHO in increasing spending on technical AI alignment because (as far as we can tell based on how slow progress has been on it over the last 22 years) it is a much thornier problem than AI capability research and because most people doing AI alignment research don’t have a viable story about how they are going to stop any insights / progress they achieve from helping with AI capability research. I mean, if you have a specific plan that avoids these problems, then let’s hear it, I am all ears, but advocacy in general of increasing work on technical alignment is counterproductive IMHO.

RHollerith Feb 1, 2025, 5:58 PM
3 points
0
in reply to: RussellThor’s comment on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing
Good reply. The big difference is that in the Cold War, there was no entity with the ability to stop the 2 parties engaged in the nuclear arms race whereas instead of hoping for the leading labs to come to an agreement, in the current situation, we can lobby the governments of the US and the UK to shut the leading labs down or at least nationalize them. Yes, the still leaves an AI race between the developed nations, but the US and the UK are democracies, and most voters don’t like AI whereas the main concern of the leaders of China and Russia is to avoid rebellions of their respective populations and they understand correctly that AI is a revolutionary technology with unpredictable societal effects that might easily empower rebels, so as long as Beijing and Moscow have access to the AI technology useful for effective surveillance of the people, they might stop racing if the US and the UK stop racing (and AI-extinction-risk activists should IMHO probably be helping Beijing and Moscow obtain the AI tech they need to effectively surveil their populations to reduce the incentive for Beijing and Moscow to support native AI research efforts).

To clarify: there is only a tiny shred of hope in the plan I outlined, but it is a bigger shred IMHO than hoping for the AI labs to suddenly start acting responsibly.

RHollerith Feb 1, 2025, 2:42 AM
8 points
1
in reply to: jimrandomh’s comment on: The Failed Strategy of Artificial Intelligence Doomers
Eliezer thinks (as do I) that technical progress in alignment is hopeless without first improving the pool of prospective human alignment researchers (e.g., via human cognitive augmentation).

RHollerith Jan 30, 2025, 1:48 PM
2 points
0
on: Who’s track record of AI predictions would you like to see evaluated?

Who’s track record of AI predictions would you like to see evaluated?

Whoever has the best track record :)

RHollerith Jan 30, 2025, 12:39 AM
5 points
1
in reply to: RussellThor’s comment on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing
Your specific action places most of its hope for human survival on the entities that have done the most to increase extinction risk.

RHollerith Jan 23, 2025, 7:51 PM
2 points
0
in reply to: Donald Hobson’s comment on: How useful would alien alignment research be?

But I’m more thinking about what work remains.

It depends on how they did it. If they did it by formalizing the notion of “the values and preferences (coherently extrapolated) of (the living members of) the species that created the AI”, then even just blindly copying their design without any attempt to understand it has a very high probability of getting a very good outcome here on Earth.

The AI of course has to inquire into and correctly learn about our values and preferences before it can start intervening on our behalf, so one way such a blind copying might fail is if the method the aliens used to achieve this correct learning depended on specifics of the situation on the alien planet that don’t obtain here on Earth.

RHollerith Jan 22, 2025, 3:48 PM
2 points
0
in reply to: Lao Mein’s comment on: Alignment Megaprojects: You’re Not Even Trying to Have Ideas

I’ve also experienced someone discouraging me from acquiring technical AI skills for the purpose of pursuing a career in technical alignment because they don’t want me to contribute to capabilities down the line. They noted that most people who skill up to work on alignment end up working in capabilities instead.

I agree with this. I.e., although it is possible for individual careers in technical alignment to help the situation, most such careers have the negative effect of speeding up the AI juggernaut without any offsetting positive effects. I.e., the fewer people trained in technical alignment, the better.

RHollerith Jan 17, 2025, 10:08 PM
5 points
0
on: Patent Trolling to Save the World
Patenting an invention necessarily discloses the invention to the public. (In fact, incentivizing disclosure was the rationale for the creation of the patent system.) That is a little worrying because the main way the non-AI tech titans (large corporations) have protected themselves from patent suits has been to obtain their own patent portfolios, then entering into cross-licensing agreements with the holders of the other patent portfolios. Ergo, the patent-trolling project you propose could incentivize the major labs to disclose inventions that they are currently keeping secret, which of course would be bad.

Although patent trolling is a better use of money IMHO than most of things designed to reduce AI extinction risk that have been funded so far, lobbying the US and UK governments to nationalize their AI labs is probably better because a government program can be a lot more strict than a private organization can be in prohibiting researchers and engineers from sharing insights and knowledge outside the program. In particular, if the nationalized program is sufficiently like top-secret defense programs, long prison sentences are the penalty for a worker to share knowledge outside of the program.

Also, note that the patent troll needs to be well-funded (with a warchest of 100s of millions of dollars preferably) before it starts to influence the behavior of the AI labs although if it starts out with less and gets lucky, it might in time manage to get the needed warchest by winning court cases or arriving at settlements with the labs.

The non-AI parts of the tech industry have been dealing with patent trolls since around 1980 and although technologists certainly complain about these patent trolls, I’ve never seen any indication that anyone has been completely demoralized by them and we do not see young talented people proclaim that they’re not going to go into tech because of the patent trolls. Nationalization of AI research in contrast strikes me as having the potential to completely demoralize a significant fraction of AI researchers and put many talented young people off of the career path although I admit it will also tend to make the career path more appealing to some young people.

Both interventions (patent trolling and lobbying for nationalization) are just ways to slow down the AI juggernaut, buying us a little time to luck into some more permanent solution.

RHollerith Jan 13, 2025, 6:40 PM
2 points
0
in reply to: bokov’s comment on: What to do if a nuclear weapon is used in Ukraine?
With a specific deadline and a specific threat of a nuclear attack on the US.

RHollerith Jan 11, 2025, 7:24 PM
2 points
0
in reply to: Vasco Grilo’s comment on: AI Timelines

The transfer should be made in January 2029

I think you mean in January 2029 or earlier if the question resolves before the end of 2028 otherwise there would be no need to introduce the CPI into the bet to keep things fair (or predictable).

RHollerith Dec 27, 2024, 12:47 AM
3 points
4
in reply to: Nate Showell’s comment on: The Field of AI Alignment: A Postmortem, and What To Do About It
You do realize that by “alignment”, the OP (John) is not talking about techniques that prevent an AI that is less generally capable than a capable person from insulting the user or expressing racist sentiments?

We seek a methodology for constructing an AI that either ensures that the AI turns out not to be able to easily outsmart us or (if it does turn out to be able to easily outsmart us) ensures (or makes it unlikely) that it won’t kill us all or do something other terrible thing. (The former is not researched much compared to the latter, but I felt the need to include it for completeness.)

The way it is now, it is not even clear whether you and the OP (John) are talking about the same thing (because “alignment” has come to have a broad meaning).

If you want to continue the conversation, it would help to know whether you see a pressing need for a methodology of the type I describe above. (Many AI researchers do not: they think that outcomes like human extinction are quite unlikely or at least easy to avoid.)

RHollerith Dec 16, 2024, 9:48 PM
7 points
8
on: Do infinite alternatives make AI alignment impossible?
Although the argument you outline might be an argument against ever fully trusting tests (usually called “evals” on this site) that this or that AI is aligned, alignment researchers have other tools in their toolbox besides running tests or evals.

It would take a long time to explain these tools, particularly to someone unfamiliar with software development or a related field like digital-electronics design. People make careers in studying tools to make reliable software systems (and reliable digital designs).

The space shuttle is steered by changing the direction in which the rocket nozzles point relative to the entire shuttle. If at any point in flight, one of the rocket nozzles had pointed in a direction a few degrees different from the one it should point in, the shuttle would have been lost and all aboard would have died. The pointing or aiming of these rocket nozzles is under software control. How did the programmers at NASA make this software reliable? Not merely through testing!

These programmers at NASA relied on their usual tools (basically engineering-change-order culture) and did not need a more advanced tool called “formal verification”, which Intel turned to to make sure their microprocessors did not have any flaw necessitating another expensive recall after they spent 475 million dollars in 1994 in a famous product recall to fix the so-called Pentium FDIV bug.

Note that FDIV refers to division of (floating-point) numbers and that it is not possible in one human lifetime to test all possible dividends and divisors to ensure that a divider circuit is operating correctly. I.e., the “impossible even theoretically” argument you outline would have predicted that it is impossible to ensure the correct operation of even something as simple as a divider implemented in silicon, and yet Intel has during the 30 years since the 1994 recall avoided another major recall of any of their microprocessors for reasons of a mistake in their digital design.

“Memory allocation” errors (e.g., use-after-free errors) are an important source of bugs and security vulnerabilities in software, and testing has for decades been an important way to find and eliminate these errors (Valgrind probably being the most well-known framework for doing the testing) but the celebrated new programming language Rust completely eliminates the need for testing for this class of programming errors. Rust replaces testing with a more reliable methodology making use of a tool called a “borrow checker”. I am not asserting that a borrow checker will help people create an aligned super-intelligence: I am merely pointing at Rust and its borrow checker as an example of a methodology that is superior to testing for ensuring some desirable property (e.g., the absence of use-after-free errors that an attacker might be able to exploit) of a complex software artifact.

In summary, aligning a superhuman AI is humanly possible given sufficiently careful and talented people. The argument for stopping frontier AI research (or pausing it for 100 years) depends on considerations other than the “impossible even theoretically” argument you outline.

Methodologies that are superior to testing take time to develop. For example, the need for a better methodology to prevent “memory allocation” errors was recognized in the 1970s. Rust and its borrow checker are the result of a line of investigation inspired by a seminal paper published in 1987. But Rust has become a realistic option for most programming projects only within the last 10 years or less. And an alignment methodology that continue to be reliable even when the AI becomes super-humanly capable is a much taller order than a methodology to prevent use-after-free errors and related memory-allocation errors.

RHollerith Dec 13, 2024, 3:57 AM
3 points
1
in reply to: bokov’s comment on: What to do if a nuclear weapon is used in Ukraine?
The strongest sign an attack is coming that I know of is firm evidence that Russia or China is evacuating her cities.

Another sign that would get me to flee immediately (to a rural area of the US: I would not try to leave the country) is a threat by Moscow that Moscow will launch an attack unless Washington takes action A (or stops engaging in activity B) before specific time T.

RHollerith Dec 13, 2024, 1:46 AM
5 points
0
in reply to: jefftk’s comment on: What to do if a nuclear weapon is used in Ukraine?
Western Montana is separated from the missile fields by mountain ranges and the prevailing wind direction and is in fact considered the best place in the continental US to ride out a nuclear attack by Joel Skousen. Being too far away from population centers to be walkable by refugees is the main consideration for Skousen.

Skousen also likes the Cumberland Plateau because refugees are unlikely to opt to walk up the escarpment that separates the Plateau from the population centers to its south.

RHollerith Dec 8, 2024, 6:39 PM
2 points
0
in reply to: Nathan Helm-Burger’s comment on: Mask and Respirator Intelligibility Comparison
The overhead is mainly the “fixed cost” of engineering something that works well, which suggest re-using some of the engineering costs already incurred in making it possible for a person to make a hands-free phone call on a smartphone.

RHollerith Dec 8, 2024, 6:29 PM
3 points
0
on: Mask and Respirator Intelligibility Comparison
Off-topic: most things (e.g., dust particles) that land in an eye end up in the nasal cavity, so I would naively expect that protecting the eyes would be necessary to protect oneself fully from respiratory viruses:

https://www.ranelle.com/wp-content/uploads/2016/08/Tear-Dust-Obstruction-1024x485.jpg

Does anyone care to try to estimate how much the odds ratio of getting covid (O(covid)) decreases when we intervene by switching a “half-mask” respirator such as the ones pictured here for a “full-face” respirator (which protects the eyes)?

RHollerith Dec 7, 2024, 1:16 AM
5 points
3
in reply to: lemonhope’s comment on: lcmgcd’s Shortform
The way it is now, when one lab has an insight, the insight will probably spread quickly to all the other labs. If we could somehow “drive capability development into secrecy,” that would drastically slow down capability development.

RHollerith Dec 2, 2024, 6:29 PM
8 points
2
on: Is malice a real emotion?
Malice is a real emotion, and it is a bad sign (but not a particularly strong sign) if a person has never felt it.

Yes, letting malice have a large influence on your behavior is a severe character flaw, that is true, but that does not mean that never having felt malice or being incapable of acting out of malice is healthy.

Actually, it is probably rare for a person never to act out of malice: it is probably much more common for a person to just be unaware of his or her malicious motivations.

The healthy organization is to be tempted to act maliciously now and then, but to be good at perceiving when that is happening and to ignore or to choose to resist the temptation most of the time (out of a desire to be a good person).

I expect people to disagree with this answer.

RHollerith Dec 1, 2024, 3:52 PM
5 points
−3
in reply to: Seth Herd’s comment on: Seth Herd’s Shortform

their >90% doom disagrees with almost everyone else who thinks seriously about AGI risk.

The fact that your next sentence refers to Rohin Shah and Paul Christiano, but no one else, makes me worry that for you, only alignment researchers are serious thinkers about AGI risk. Please consider that anyone whose P(doom) is over 90% is extremely unlikely to become an alignment researcher (or to remain one if their P(doom) became high when they were an alignment researcher) because their model will tend predict that alignment research is futile or that it actually increases P(doom).

There is a comment here (which I probably cannot find again) by someone who was in AI research in the 1990s, then he realized that the AI project is actually quite dangerous, so he changed careers to something else. I worry that you are not counting people like him as people who have thought seriously about AGI risk.