Human information throughput is allegedly only about 10-50 bits per second. This implies an interesting upper bound, in that the information throughput of biological humanity as a whole can’t be higher than around 50 * 10^10 = 500Gbit/s. I.e., if all distinguishable actions made by humans were perfectly independent, biological humanity as a whole would have at most 500Gbit/s of “steering power”.
I need to think more about the idea of “steering power” (e.g. some obvious rough edges around amplifying your steering power using external information processing / decision systems), but I have some intuition that one might actually be able to come up with a not-totally-useless concept that lets us say something like “humanity can’t stay in ‘meaningful control’ if we have an unaligned artificial agent with more steering power than humanity, expressed in bits/s”.
Human information throughput is allegedly only about 10-50 bits per second. This implies an interesting upper bound, in that the information throughput of biological humanity as a whole can’t be higher than around 50 * 10^10 = 500Gbit/s. I.e., if all distinguishable actions made by humans were perfectly independent, biological humanity as a whole would have at most 500Gbit/s of “steering power”.
I need to think more about the idea of “steering power” (e.g. some obvious rough edges around amplifying your steering power using external information processing / decision systems), but I have some intuition that one might actually be able to come up with a not-totally-useless concept that lets us say something like “humanity can’t stay in ‘meaningful control’ if we have an unaligned artificial agent with more steering power than humanity, expressed in bits/s”.
I asked Deep Research to see if there are existing treatments of this basic idea in the literature. It seems most closely related to the concept of “empowerment” in RL, which I’m surprised I hadn’t heard of: https://en.m.wikipedia.org/wiki/Empowerment_(artificial_intelligence)
The Wikipedia article makes it seem like this might also be how RL people think about instrumental convergence?