“to make predictions with probability 1 (which, you must concede, is a plausible reading of “no danger whatsoever”)”
I don’t know what Eliezer will concede, but in fact it is more than a plausible reading. It’s what the phrase means. Actually, “no danger” means “probability 1 of no conversion”, and “whatsoever” means “for real, folks, I mean probability 1 exactly”. And I don’t see anything religious about it. Hyperbolic perhaps, but not religious.
Nice post, Eliezer.
“It’s a whole lot easier to modify the source code in the interior of your own mind, then to take that modification, and sell it to a friend who happens to be written on different source code”
If I understand the sentence correctly, “then” should be “than”.