If we focus on the bounded subspace of mind design space which contains all those minds whose makeup can be specified in a trillion bits or less, then every universal generalization that you make has two to the trillionth power chances to be falsified.
Conversely, every existential generalization—“there exists at least one mind such that X”—has two to the trillionth power chances to be true.
So you want to resist the temptation to say either that all minds do something, or that no minds do something.
This is fine where X is a property which has a one-to-one correspondence with a particular bit in the mind’s specification. For higher-level properties (perhaps emergent ones—yes, I said it) this probabilistic argument is not convincing.
Consider the minds of specification-size 1 trillion. We can happily make the generalisation that none of them will be able to predict whether a given Turing machine halts. Yes, there are 2^trillion chances for this generalisation to be falsified, but we know it never will be.
But this generalisation is true of everything, not just “minds”, so we haven’t added to our knowledge. Well, let’s try this generalisation instead: no mind’s state will remain unchanged by a non-null input. This is not true of rocks, but is true of minds. Perhaps there are some other, more useful, things we can say about minds.
Apologies for resurrecting a months-old post. I’m new here.
Consider the minds of specification-size 1 trillion. We can happily make the generalisation that none of them will be able to predict whether a given Turing machine halts. Yes, there are 2^trillion chances for this generalisation to be falsified, but we know it never will be.
Eliezer did qualify those statements:
Somewhere in mind design space is at least one mind with almost any kind of logically consistent property you care to imagine.
If we focus on the bounded subspace of mind design space which contains all those minds whose makeup can be specified in a trillion bits or less, then every universal generalization that you make has two to the trillionth power chances to be falsified.
Conversely, every existential generalization—“there exists at least one mind such that X”—has two to the trillionth power chances to be true.
So you want to resist the temptation to say either that all minds do something, or that no minds do something.
This is fine where X is a property which has a one-to-one correspondence with a particular bit in the mind’s specification. For higher-level properties (perhaps emergent ones—yes, I said it) this probabilistic argument is not convincing.
Consider the minds of specification-size 1 trillion. We can happily make the generalisation that none of them will be able to predict whether a given Turing machine halts. Yes, there are 2^trillion chances for this generalisation to be falsified, but we know it never will be.
But this generalisation is true of everything, not just “minds”, so we haven’t added to our knowledge. Well, let’s try this generalisation instead: no mind’s state will remain unchanged by a non-null input. This is not true of rocks, but is true of minds. Perhaps there are some other, more useful, things we can say about minds.
Apologies for resurrecting a months-old post. I’m new here.
Eliezer did qualify those statements: