I was a downvoter. I think the first three sections are mysterious in a way that invalidates the rest of the post. I like how he thinks in the next few sections, but it reminds me of the urban legend of the PhD student who found a lot of interesting results about a new class of mathematical objects, only to show up to their dissertation and realize only the trivial case existed. It generally isn’t productive to ask a lot of “what if?” questions before pinning down what you mean when you talk about consciousness or morality. If you can’t pin it down exactly, then go with a working definition based on a few examples that should fit that category. At the very least, apply some known examples to the “what if?” that follows. I think kbear’s comment does a pretty good job of this.
James Camacho
Disclaimer: This comment hasn’t really been edited for clarity, cohesion, or politeness. I do think it’s useful, but it’ll definitely be spicy.
Trying to derive all of morality from physics alone – say, if someone is crazy enough to derive an entirely ethical philosophy and ideological movement based on maximizing entropy — would strike most people as deeply confused.
I think if most people consider this philosophy to be deeply confused, it is actually the case that most people are deeply confused. When I read this sentence, I was pleasantly surprised that someone else had figured it out, and even more surprised it was the leader of e/acc (unrelated to the previous surprisal).
I believe you are being serious in your post, but there’s this niggling suspicion in the back of my mind that, if I were satirizing how philosophists talk about consciousness/subjective experience/morality, this is how it would come out. Statements like,
“The world of consciousness. Subjective experience. What it feels like to see red.”
that you see exclaimed everywhere with an undertone of wonder and confusion, and no attempt to really pin down what is meant mathematically. Then a section called “pinpointing the ineffable” saying, “this probably sounds too abstract. Let’s try to make it more concrete,” without actually trying to make it more concrete (mathematically)—just make explicit the wonder and confusion.
The rest of the post builds off of this in a constructive way, so I believe you are being serious here. I just don’t get the confusion around consciousness. As someone else said, the laws of mathematics are enough to explain the phenomenon (though they qualified their statement more). It isn’t a separate world. Subjective experience? Simply a reference to a compressed copy of the self. Ontologies? They’re a little harder to figure out, but I’m pretty sure it’s the significant bits of autoencoding.
And let’s not forget the central question, what about moral goods? Here’s a question for you: is soft actor-critic maxxing energy under entropy regularization, or entropy under energy regularization? They’re the same thing! But if you dig down into the two terms, entropy definitely exists, while energy always feels like a placeholder for something else. Like, “does this policy get the results I want, so I’m going to let it stick around and further evolve?” But that’s just maxxing entropy when you consider part of the game is for the researcher to keep using the policy.
If we continue to pursue the ‘decapitation’ theory of warfare, or the ‘kingpin strategy,’ then I do not believe that goes in good places. So far this hasn’t been flipped around against the leadership of democracies so much, but how long will that last?
Can you explain this further? To me it seems to good from a humanitarian, utilitarian, and game theory perspective.
It seems worse to kill millions of rank-and-file soldiers than hundreds of generals/political leaders.
Those leaders are usually coercing the rank-and-file to fight in the first place by threatening their life or liberty. Furthermore, those leaders are usually the ones making the decision to go to war at all.
If you have the capability, you should punish the people imposing negative externalities on you, which sure includes the rank-and-file soldiers, but I think it’s better to model them like you model natural disasters. A lot of military training is spent teaching them to not think and just be a tool the higher-ups can use. They’re the real source of negative externalities here, so the appropriate people to punish.
I get how this kind of warfare changes the decision making process among the generals/political leaders, for example, it is difficult to elect politicians in Mexico that promise to get rid of the drug cartels (at least, difficult to elect them for more than a few days). And maybe this leads to more stupid suffering than WWI, but it seems really hard to be worse 10% of a generation getting conscripted and killed.
I am moderately interested in joining the Discord, at least just to see what has worked for others. I also got Long COVID ~1.5 years ago, and it’s rough.
I was talking with Joseph, and I think I like his SharkBot more because it fails more gracefully. Suppose (1) “proof” has an upper bound in computation cycles, and (2) people occasionally make mistakes in their logic. A good prover might spend more computation cycles in error correction. What happens if they do not have enough time to prove cooperation leads to cooperation, or defection leads to defection?
If Joseph’s bot stalls out on the second half of its computation, it concludes, “I couldn’t prove they would cooperate if I’m caught (provably) defecting,” and cooperates. If your bot stalls out on the second half of its computation, it concludes, “I couldn’t prove being caught (provably) defecting would lead to them also defecting,” and thinks it can get away with defection.
Another way of putting it: dumber bots are more likely to think they can get away with defection when they really can’t, and defect against bots smarter than them. If a bot is going to try to take advantage of rocks [1] , it better well make sure it is actually playing against a rock, and not just making a stupid mistake that hurts everyone.
Also, as an aside, I think making mistakes (logical bit flips some percent of the time) naturally penalizes high-complexity policies. This is why you might expect societies to begin with mostly cooperate/defect bots, then transition to citizen/police bots, and slowly build complexity where each individual’s policy is relatively simple, but society as a whole gets more complex interactions. I think this would be an interesting area of research.
- ↩︎
From the phrase, “how do you play a Prisoner’s Dilemma against a rock?” Rocks are bots that cooperate even if it is proven you are going to defect.
- ↩︎
“You, not all of you but most of you, should not be working on AI safety.”
So, it seems you endorse a utility function that puts more weight on others than your actual preferences. Wouldn’t you prefer to endorse a different utility function?
Well do you care about the rest of humanity enough to send yourself to hell? Or adopting policies where you only get sent to hell in
How do you determine which beings ought to be in a utilitarian’s utility function? I think it’s generally the utilitarian decides for themselves and the rest of society beats them over the head until the utilitarian includes them too.
Perhaps here is where the controversy comes in. The utilitarian comes along and says, “I want to maximize utility!” And everyone thinks, “great! she wants to help everyone out!” The selfish egoist comes along and says, “I am just going to fulfill whatever selfish desires I have!” And everyone thinks, “wow, that’s scary! what stops you from murdering people?”
I think, also, there is a sense in which utilitarians work to maximize the same utility function. This is also true for selfish egoists, but they’re both better and worse at negotiating (they are more prone to negotiate, but utilitarians make mistakes that are biased towards reaching a consensus just because they solve the problem from different directions).
Yes.
I don’t understand what you don’t understand. I heard a remark once about a philosopher who really tried to steelman other people’s arguments, but so that they made sense according to the philosopher, not in the mental frame of the other person. It led to some pretty wacky arguments on the steelman side. I think here, you should assume when I say, “mathematically equivalent,” that’s what I mean. Like, any math you use in utilitarianism is the same as that of selfish egoism. Or, if you tried to put the two philosophies in mathematical terms, you get the exact same equations. So, it extends to logical beings or irrational beings. The words “selfish egoism” and “utilitarianism” are synonyms.
Not just egoism, selfish egoism. Every utility function people choose is a selfish one or they wouldn’t choose it. The claim isn’t, “selfish egoism is a subset of utilitarianism” but “selfish egoism is identically the same as utilitarianism.”
It should be consistent with any decision theory.
“People often submit incredibly epistemically rude and short-sighted comments on forums, but they deceive people into upvoting them by putting on a veneer of politeness. ‘John, I feel like you’ve got a nail in your head.’ they say. ‘Your conclusion is wrong so you must not have thought of this thing you explicitly mentioned in your post.’”
“The rationality scene is a little culty.”
“Utilitarianism and selfish egoism are mathematically the same [EDIT: i.e. they could be used as synonyms except for their different connotations].”
EDIT: Separated into multiple comments.
I think the problem is not that, “we don’t have a good mathematical definition of phenomenological consciousness,” it’s that there isn’t a definition at all! My theory is that this babble is useful to survival, because babble can still be used as a justification. On a species level, you still see religious people today saying, “it’s okay to kill and eat animals, but not humans, because humans have souls.” On an individual level, you’re going to fight harder if your brain insists me. On a memetic level, proclaiming to be realer than real, “the thing that is redness”, will get more people talking about it. Doesn’t mean there’s anything there.