Wei Dai comments on AI #25: Inflection Point

[Wei Dai]

Washington Post’s Parmy Olsen complains There’s Too Much Money Going to AI Doomers, opening with the argument that in the Industrial Revolution we shouldn’t have spent a lot of money ensuring the machines did not rise up against us, because in hindsight they did not do that.

I wonder what would happen if we “amplified” reasoning like this, as in HCH, IDA, Debate, etc.

Do we understand reasoning well enough to ensure that this class of errors can avoided in AI alignment schemes that depend on human reasoning, or to ensure that this class of errors will be reliably self-corrected as the AI scales up?

[Rudi C]

This is not an “error” per se. It’s a baseline, outside-view argument presented in lay terms.