I think even with humans, IQ isn’t the best measure to quantify what we call intelligence. The way I tend to think of it is that high general intelligence correlates with higher IQ test scores, but just optimizing performance on IQ tests doesn’t necessarily mean that you become more intelligent in general outside of that task.
But I’m okay with the idea of using IQ scores in the context of this post because it seems useful to capture the change in capabilities of these models.
This reminds me of this LessWrong post.
If It’s Worth Doing, It’s Worth Doing With Made-Up Statistics
https://www.lesswrong.com/posts/9Tw5RqnEzqEtaoEkq/if-it-s-worth-doing-it-s-worth-doing-with-made-up-statistics