the blind idiot god did ‘accidentally’ make tools into agents when making humans, so… But after all that only happened once in hundreds of millions of years of ‘attempts’.
This seems like a very valuable point. In that direction, we also have the tens of thousands of cancers that form every day, military coups, strikes, slave revolts, cases of regulatory capture, etc.
Hmmm. Yeah, cancer. The analogy would be “sufficiently advanced tools tend to be a short edit distance away from agents”, which would mean that a typo in the source code or a cosmic ray striking a CPU at the wrong place and time could have pretty bad consequences.
This seems like a very valuable point. In that direction, we also have the tens of thousands of cancers that form every day, military coups, strikes, slave revolts, cases of regulatory capture, etc.