[Link] Is the Orthogonality Thesis Defensible? (Qualia Computing)
Full title: Is the Orthogonality Thesis Defensible if We Assume Both Valence Realism and Open Individualism?
An excerpt:
The cleanest typology for metaphysics I can offer is: some theories focus on computations as the thing that’s ‘real’, the thing that ethically matters – we should pay attention to what the *bits* are doing. Others focus on physical states – we should pay attention to what the *atoms* are doing. I’m on team atoms, as I note here: Against Functionalism.
My suggested takeaway: an open individualist who assumes computationalism is true (team bits) will have a hard time coordinating with an open individualist who assumes physicalism is true (team atoms) — they’re essentially running incompatible versions of OI and will compete for resources.
As a first approximation, instead of three theories of personal identity – Closed Individualism, Empty Individualism, Open Individualism – we’d have six. CI-bits, CI-atoms, EI-bits, EI-atoms, OI-bits, OI-atoms.
Whether the future is positive will be substantially determined by how widely and deeply we can build positive-sum moral trades between these six frames.
This argument also proves that quantum computers can’t offer useful speedups, because otherwise evolution would’ve found them.
This argument also proves that Genghis Khan couldn’t have happened, because intelligence and power converge on caring about positive valence for all beings.
I think the orthogonality thesis is in good shape, if these are the strongest arguments we could find against it in almost a decade.
Genghis KhanOpen individualism seems to either fail the monday tuesday test (if it was true on monday, false on tuesday, is there any experiment that would come up with different results?), or be blatantly false. Firstly, open idividualism is defined by Wikipedia as
What sort of philosophical object might this be, Might it be a predictive model? Quantum mechanics predicts everything in principle, so this would make open individualism an approximation to quantum mechanics in the same way that the ideal gas law is. However, for us to gain confidence that it was correct, we would either need sufficiently many correct predictions that we had good evidence for it, or a mathematical derivation of it from other trusted principles. This blog uses it to predict that any sufficiently advanced AI will be friendly. This Blog predicts that agents that believe in open individualism will always cooperate in prisoners dilemmas. And that
With this quote implying that science would be harder if open individualism were true.
There doesn’t seem to be many, obviously correct predictions, or any kind of mathematical derivation. The predictions stated here seem to be coming from mental black boxes rather than formulaic theories. If it is a theory, it seems to be a poor one.
Could it be a definition? It looks like someone, faced with an inability co cut reality at its joints, refused to cut at all. There is a sense in which my mind today is more similar to my mind yesterday, (in a high level behavioral sense) than either is to the mind of Genghis Khan. In circumstances that don’t involve person duplication or other weird brain manipulation techs, asking “is the person who smashed my window yesterday the same person that stole my car today?” is a meaningful question. Declaring everyone to be the same person is like answering “Are there any apples at the shops?” by saying that every possible arrangement of mass was an apple.
In real world situations, we don’t use literal exactly equal equality, we use a close enough for practical purposes. The fact that in english, there are many different ways of asking if things are the same with the word “equals”, “same”, “is”… only confuses things further. Its hard to notice you are redefining a word that isn’t even a word.
Understanding intelligence without the concept.
Either way, it seems that there is a sufficient confusion around the concept, that any deductions made from it are suspect. Try to taboo the concept of open individualism and explain why any arrangement of transistors that can be approximated as maximizing some function over world states must be maximizing a function that looks at other approximate function maximisers and computes the value of that function given the universe as input. Try to explain why this theory still allows alpha go to play go against a copy of itself? Is it not sufficiently smart or thoughtful? If Deep mind trained it for even longer, would they get an algorithm that recognized it was fighting a copy of itself, and instead of playing, agreed to a tie?
Hi Donald- author of opentheory.net here. Really appreciate the thoughtful comment. A few quick notes:
I definitely (and very strongly) do not “predict that agents that believe in open individualism will always cooperate in prisoners dilemmas”—as I said in the OP, “an open individualist who assumes computationalism is true (team bits) will have a hard time coordinating with an open individualist who assumes physicalism is true (team atoms) — they’re essentially running incompatible versions of OI and will compete for resources.” I would say OI implies certain Schelling points, but I don’t think an agent that believes in OI has to always cooperate (largely due to the ambiguity in what a ‘belief’ may be- there’s a lot of wiggle-room here. Best to look at the implementation).
I think the overall purpose of discussing these definitions of personal identity is first, dissolving confusion (and perhaps seeing how tangled up the ‘Closed Individualism’ cluster is); second, trying to decipher Schelling points for each theory of identity. We only get predictions indirectly from this latter factor; mostly this is a definitional exercise.
Doesn’t the efficacy of open individualism depend on whether the number of intelligent agents is high enough to “interfere” with the resources of one another (overpopulation, and overpopulation is a basis for some fears for AI risk)
Metaphysical claims like OI are typically argued on the basis of abduction, ie as a better explanation for existing data.
I say typically, because I am not sure that the claims in question work that way.