Stupidity is also hard
Everyone today talks about Intelligence, but no one talks about the inverse problem: Stupidity. How stupid can you make a system? And what are the (mis)alignment properties of such a system?
If intelligence is the ability to efficiently achieve goals in a wide range of domains, then stupidity is the lack of such ability. Give a stupid being a goal and it will with great probability fail to achieve it.
Let’s look at specific examples. What about a random agent? How stupid is it? Not as stupid as you would think. Give it a multiple choice test and it will average 50%. Let it try out the stock market and watch it beat troves of traders. In a chess game, with some small percentage it will play a perfect game.
What about a superintelligence who likes to fail? Given any task, it will try to fail it. It will always get 0% on multiple choice tests and lose all its money in the stock market. It will lose each chess game in record time. Does true stupidity require intelligence?
What about the average human? It will get above 50% on many multiple choice tests. It will lose money in the market. And it will lose most chess games.
What about a rock? It does literally nothing. Give it a multiple choice test and it will average 0% and it will not lose a cent in the stock market. It will lose every chess game on time. The strategy a rock implements is nontrivial. For instance, most humans would rather make things worse than do nothing.
Who is stupider? Certainly, the one who performs worst is the failure-seeking superintelligence. But most people would not consider it “stupid”, because it has the “potential” to achieve all the goals it failed given a relatively minor modification to its utility function.
So what if I take GPT-5 and replace the last layer with a zero matrix? It now outputs a zero vector every time, so it is the zero function, but it is within a small edit-distance from GPT-5 proper. Is this over-engineered zero function intelligent or stupid?
What if I take a very large sequence of matrix computations, such that probabilistically it contains a section isomorphic to GPT-5, is this intelligent or stupid? Here, the key is that although it contains sufficient complexity that seems “intelligence”, intelligence is also required to prune the non-intelligent parts out.
The hallmark of intelligence is the capacity to direct the world toward certain states. (Is a really large black hole intelligent?) Perhaps stupidity of a system N can be defined as the inverse of the intelligence of The World Except for N. A stupid system is that whose state can be easily altered by the rest of the world. What about a rock made of a really really hard material?
It would seem that sufficient stupidity requires a degree of intelligence. What I mean is it could be that the stupid system itself is intelligent or that it is produced as a result of an intelligent process. It is confusing to use “intelligent” to describe properties of a stupid system, so we should have a different word that describes this common thing between stupid and intelligent systems. It is similar to how Shannon Entropy measures the “surprise” of a random variable, and all outcomes being equal probability leads to maximum entropy. Except now, we want a quantity capturing a certain direction of deviation from the random agent, that is high for things that are very stupid and very smart. We could call it Interestingness.
Inverted stupidity is not intelligence, but understanding stupidity is a hard problem too.
See Reversed Stupidity Is Not Intelligence
Reversed Stupidity Is Not Intelligence
[didn’t downvote—it’s already negative. but I’d like to explain why I think it should be negative. ]
I don’t think “stupid” is a useful descriptor in this context, and this post does nothing to explain or understand what elements of decision or intent we should be looking at. I can’t tell what is being said, nor what definitions would become apparent if you taboo’d “stupid” and “intelligent”.
Thanks Dagon, I appreciate the concrete feedback.
I’m trying to express that typically people think about systems with very high cognitive power (relative to humans), but it could be interesting/useful to consider what very low cognitive power systems are like. Looking at such extreme cases can inform things such as the type signature of intelligence/agency/etc. The post is me trying to think through this and noticing that low cognitive power systems are hard to characterize, e.g. give a superintelligence the right goal and it can behave like a low cognitive power system.