just because an idea is, at a high level, some kind of X, doesn’t mean the idea is anything like the fully-fledged, generally applicable version of X that one imagines when describing X
I haven’t heard a response / counterargument to this yet, and many people keep making this logic mistake, including AFAICT you.
As I mentioned, my response is here https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce#_We_just_need_X__intuitions:
I haven’t heard a response / counterargument to this yet, and many people keep making this logic mistake, including AFAICT you.