You make an excellent point. The evolutionary argument is not as strong as I presented it.
Given that recorded history has no record of successful Xanatos gambits (TVTropes lingo), the case is strong that the intelligence limit is not medium distance from human average (i.e. not 20-50 std. dev. from average).
That leaves the possibility that (A) the limit is far (>50 std dev.) or (B) very near (the 8-12 range I mentioned above).
It seems to me that our ability to understand and prove certain results about computational difficulty (and the power of self-reference) that would apply even if super-human intelligence was possible is evidence that (B) is more likely than (A).
You make an excellent point. The evolutionary argument is not as strong as I presented it.
Given that recorded history has no record of successful Xanatos gambits (TVTropes lingo), the case is strong that the intelligence limit is not medium distance from human average (i.e. not 20-50 std. dev. from average).
That leaves the possibility that (A) the limit is far (>50 std dev.) or (B) very near (the 8-12 range I mentioned above).
It seems to me that our ability to understand and prove certain results about computational difficulty (and the power of self-reference) that would apply even if super-human intelligence was possible is evidence that (B) is more likely than (A).