Hmmm. Yeah, cancer. The analogy would be “sufficiently advanced tools tend to be a short edit distance away from agents”, which would mean that a typo in the source code or a cosmic ray striking a CPU at the wrong place and time could have pretty bad consequences.
A1987dM comments on Thoughts on the Singularity Institute (SI)