Yes; good points! Do note that my original comment was made eight years ago! (At least – it was probably migrated from Overcoming Bias if this post is as early as it seems to be.)
So I have had some time to think along these lines a little more :)
But I don’t think intelligence itself can lead one to conclude as you have:
If it is intelligent, it will make the distinction.
It’s not obvious to me now that any particular distinction will be made by any particular intelligence. There’s maybe not literally infinite, but still a VAST number of possible ontologies with which to make distinctions. The general class of ‘intelligent systems’ is almost certainly WAY more alien than we can reasonably imagine. I don’t assume that even a ‘super-intelligence’ would definitely ever “differentiate between smiley-faces and happy people”.
But I don’t remember this post that well, and I was going to re-read before I remembered that I didn’t even know what I was originally replying to (as it didn’t seem to be the post itself), and re-constructing the entire context to write a better reply which my temporal margin “is too narrow to contain” at the moment.
But I think I still disagree with whatever Shane wrote!
Yes; good points! Do note that my original comment was made eight years ago! (At least – it was probably migrated from Overcoming Bias if this post is as early as it seems to be.)
So I have had some time to think along these lines a little more :)
But I don’t think intelligence itself can lead one to conclude as you have:
It’s not obvious to me now that any particular distinction will be made by any particular intelligence. There’s maybe not literally infinite, but still a VAST number of possible ontologies with which to make distinctions. The general class of ‘intelligent systems’ is almost certainly WAY more alien than we can reasonably imagine. I don’t assume that even a ‘super-intelligence’ would definitely ever “differentiate between smiley-faces and happy people”.
But I don’t remember this post that well, and I was going to re-read before I remembered that I didn’t even know what I was originally replying to (as it didn’t seem to be the post itself), and re-constructing the entire context to write a better reply which my temporal margin “is too narrow to contain” at the moment.
But I think I still disagree with whatever Shane wrote!