I feel like an important question is: how far does this generalize? We can estimate the IQ gap between the dumbest person who successfully uses the internet (probably in the 80′s) and the smartest malware author (got to be at least 150+). Is that the limit somehow, or does this knack extend across even more orders of magnitude?
If imagine a world where 100 IQ humans are using an internet that contains malware written by 1000 IQ AGI, do humans just “avoid the bad parts”?
For reactive threats, the upper bound is probably at most “people capable of introspection who can detect they are not sure some action will be to net benefit, and therefore refuse to take it”. For active threatening factors, that’s an arms race (>=40% this race is not to infinity—basically, if more-cooperating DT strategies are any good).
Maybe the subject is researched more in biology? Example topic: eating unknown food (berries, nuts) in forest, and balance of lifetime adaptation vs evolutionary adaptation (which involves generations passing).
For reactive threats, the upper bound is probably at most “people capable of introspection who can detect they are not sure some action will be to net benefit, and therefore refuse to take it”. For active threatening factors, that’s an arms race (>=40% this race is not to infinity—basically, if more-cooperating DT strategies are any good).
Maybe the subject is researched more in biology? Example topic: eating unknown food (berries, nuts) in forest, and balance of lifetime adaptation vs evolutionary adaptation (which involves generations passing).