I’d like to know people who agree with me that mental models of people can often be people. Consider contacting me if that’s you.
Nox ML
Karma: 58
If we take as assumption that everything humans have observed has been made up of smaller physical parts (except possibly for the current elementary particles du jour, but that doesn’t matter for the sake of this argument) and that the macro state is entirely determined by the micro state (regardless of if it’s easy to compute for humans), there is a simple conclusion that follows logically from that.
This conclusion is that nothing extraphysical can have any predictive power above what we can predict from knowledge about physics. This follows because for something to have predictive power, it needs to have some influence on what happens. If it doesn’t have any influence on what happens, its existence and non-existence cannot allow us to make any conclusions about the world.
This argument applies to mathematics: if the existence of mathematics separately from physics allowed us to make any conclusions about the world, it would have to have a causal effect on what happens, which would contradict the fact that all macro state we’ve ever observed has been determined by just the micro state.
Since the original assumption is one with very strong evidence backing it, it’s safe to conclude that, in general, whenever we think something extraphysical is required to explain the known facts, we have to be making a mistake somewhere.
In the specific instance of your comment, I think the mistake is that the difference between “a priori” truths and other truths is artificial. When you’re doing math you have be doing work inside your brain and getting information from that. This is not fundamentally different from observing particle accelerators and getting information from them.
This is only correct if we presuppose that the concept of mathematically true is a meaningful thing separate from physics. The point this post is getting at is that we can still accept all human mathematics without needing to presuppose that there is such a thing. Since not presupposing this is strictly simpler, and presupposing it does not give us any predictive power, we ought not to assume that mathematics exists separately from physics.
This is not just a trivial detail. Presupposing things without evidence is the same kind of mistake as Russell’s teapot, and small mistakes like that will snowball into larger ones as you build your philosophy on top of them.
Mathematics As Physics
I agree that they are not symmetrical. My point with that thought experiment was to counter one of their arguments, which as I understand it can be paraphrased to:
In your thought experiment, the people who bet that they are in the last 95% of humans only win in aggregate, so there is still no selfish reason to think that taking that bet is the best decision for an individual.
My thought experiment with the dice was meant to show that this reasoning also applies to regular expected utility maximization, so if they use that argument to dismiss all anthropic reasoning, then they have to reject basically all probabilistic decision making. Presumably they will not reject all probabilistic reasoning, and therefore they have to reject this argument. (Assuming that I’ve correctly understood their argument and the logic I’ve just laid out holds.)
Edit: Minor changes to improve clarity.
You do this 100 times, would you say you ought to find your number >5 about 95 times?
I actually agree with you that there is no single answer to the question of “what you ought to anticipate”! Where I disagree is that I don’t think this means that there is no best way to make a decision. In your thought experiment, if you get a reward for guessing if your number is >5 correctly, then you should guess that your number is >5 every time.
My justification for this is that objectively, those who make decisions this way will tend to have more reward and outcompete those who don’t. This seems to me to be as close as we can get to defining the notion of “doing better when faced with uncertainty”, regardless of if it involves the “I” or not, and regardless of if you are selfish or not.
Edit to add more (and clarify one previous sentence):
Even in the case where you repeat the die-roll experiment 100 times, there is a chance that you’ll lose every time, it’s just a smaller chance. So even in that case it’s only true that the strategy maximizes your personal interest “in aggregate”.
I am also neither a “halfer” nor a “thirder”. Whether you should act like a halfer or a thirder depends on how reward is allocated, as explained in the post I originally linked to.
By pretty much every objective measure, the people who accept the doomsday argument in my thought experiment do better than those who don’t. So I don’t think it takes any additional assumptions to conclude that even selfish people should say yes.
From what I can tell, a lot of your arguments seem to be applicable even outside anthropics. Consider the following experiment. An experimenter rolls a fair 100-sided die. Then they ask someone to guess if they rolled a number >5 or not, giving them some reward if they guess correctly. Then they reroll and ask a different person, and repeat this 100 times. Now suppose I was one of these 100 people. In this situation, I could use reasoning that seems very similar to yours to reject any kind of action based on probability:
I either get the reward or not as the die landed on a number >5 or not. Giving an answer based on expected value might maximize the total benefit in aggregate of the 100 people, but it doesn’t help me, because I can’t know if the die is showing >5 or not. It is correct to say if everyone makes decisions based on expected utility then they will have more reward combined. But I will only have more reward if the die is >5, and this was already determined at the time of my decision, so there is no fact of the matter about what the best decision is.
And granted, it’s true, you can’t be sure what the die is showing in my experiment, or which copy you are in anthropic problems. But the whole point of probability is reasoning when you’re not sure, so that’s not a good reason to reject probabilistic reasoning in either of those situations.
Suppose when you are about to die, time freezes, and Omega shows up and tells you this: “I appear once to every human who has ever lived or will live, right when they are about to die. Answer this question with yes or no: are you in the last 95% of humans who will ever live in this universe? If your answer is correct, I will bring you to this amazing afterlife that I’ve prepared. If you guess wrong, you get nothing.” Do you say yes or no?
Let’s look at actual outcomes here. If every human says yes, 95% of them get to the afterlife. If every human says no, 5% of them get to the afterlife. So it seems better to say yes in this case, unless you have access to more information about the world than is specified in this problem. But if you accept that it’s better to say yes here, then you’ve basically accepted the doomsday argument.
However, an important thing to note is that when using the doomsday argument, there will always be 5% of people who are wrong. And those 5% will be the first people who ever lived, whose decisions in many ways have the biggest impact on the world. So in most situations, you should still be acting like there will be a lot more people in the future, because that’s what you want the first 5% of people to have been doing.
More generally, my procedure for resolving this type of confusion is similar to how this post handles the Sleeping Beauty problem. Basically, probability is in the mind, so when a thought experiment messes with the concept of “mind”, probability can become underspecified. But if you convert it to a decision problem by looking at the actual outcomes and rating them based on your preferences, things start making sense again.
I like the distinctions you make between sentient, sapient, and conscious. I would like to bring up some thoughts about how to choose a morality that I think are relevant to your points about death of cows and transient beings, which I disagree with.
I think that when choosing our morality, we should do so under the assumption that we have been given complete omnipotent control over reality and that we should analyze all of our values independently, not taking into consideration any trade-offs, even when some of our values are logically impossible to satisfy simultaneously. Only after doing this do we start talking about what’s actually physically and logically possible and what trade-offs we are willing to make, while always making sure to be clear when something is actually part of our morality vs when something is a trade-off.
The reason for this approach is to avoid accidentally locking in trade-offs into our morality which might later turn out to not actually be necessary. And the great thing about it is that if we have not accidentally locked in any trade-offs into our morality, this approach should give back the exact same morality that we started off with, so when it doesn’t return the same answer I find it pretty instructive.
I think this applies to the idea that it’s okay to kill cows, because when I consider a world where I have to decide whether or not cows die, and this decision will not affect anything else in any way, then my intuition is that I slightly prefer that they not die. Therefore my morality is that cows should not die, even though in practice I think I might make similar trade-offs as you when it comes to cows in the world of today.
Something similar applies to transient computational subprocesses. If you had unlimited power and you had to explicitly choose if the things you currently call “transient computational subprocesses” are terminated, and you were certain that this choice would not affect anything else in any way at all (not even the things you think it’s logically impossible for it not to affect), would you still choose to terminate them? Remember that no matter what you choose here, you can still choose to trade things off the same way afterwards, so your answer doesn’t have to change your behavior in any way.
It’s possible that you still give the exact same answers with this approach, but I figure there’s a chance this might be helpful.
The reason I reject all the arguments of the form “mental models are embedded inside another person, therefore they are that person” is that this argument is too strong. If a conscious AI was simulating you directly inside its main process, I think you would still qualify as a person of your own, even though the AI’s conscious experience would contain all your experiences in much the same way that your experience contains all the experiences of your character.
I also added an addendum to the end of the post which explains why I don’t think it’s safe to assume that you feel everything your character does the same way they do.
I think we just have different values. I think death is bad in itself, regardless of anything else. If someone dies painlessly and no one ever noticed that they had died, I would still consider it bad.
I also think that truth is good in and of itself. I want to know the truth and I think it’s good in general when people know the truth.
Here, I technically don’t think you’re lying to the simulated characters at all—in so far as the mental simulation makes them real, it makes the fictional world, their age, and their job real too.
Telling the truth to a mental model means telling them that they are a mental model, not that they are a regular human. It means telling them that the world they think they live in is actually a small mental model living in your brain with a minuscule population.
And sure, it might technically be true that within the context of your mental models, they “live” inside the fictional world, so “it’s not a lie”. But not telling them that they are in a mental model is such a incredibly huge thing to omit that I think it’s significantly worse than the majority of lies people tell, even though it can technically qualify as a “lie by omission” if you phrase it right.
so I would expect simulating pain in such away to be profoundly uncomfortable for the author.
I’ve given my opinion on this in an addendum added to the end of the post, since multiple people brought up similar points.
Points similar to this have come up in many comments, so I’ve added an addendum at the end of my post where I give my point of view on this.
I can definitely create mental models of people who have a pain-analogue which affects their behavior in ways similar to how pain affects mine, without their pain-analogue causing me pain.
there’s no point on reducing this to a minimal Platonic concept of ‘simulating’ in which simulating excruciating pain causes excruciating pain regardless of physiological effects.
I think this is the crux of where we disagree. I don’t think it matters if pain is “physiological” in the sense of being physiologically like how a regular human feels pain. I only care if there is an experience of pain.
I don’t know of any difference between physiological pain and the pain-analogues I inflicted on my mental models which I would accept as necessary for it to qualify as an experience of pain. But since you clearly do think that there is such a difference, what would you say the difference is?
I don’t personally think I’m making this mistake, since I do think that saying “the conscious experience is the data” actually does resolve my confusion about the hard problem of consciousness. (Though I am still left with many questions.)
And if we take reductionism as a strongly supported axiom (which I do), then necessarily any explanation of consciousness will have to be describable in terms of data and computation. So it seems to me that if we’re waiting for an explanation of experience that doesn’t boil down to saying “it’s a certain type of data and computation”, then we’ll be waiting forever.
My best guess about what you mean is that you are referring to the part in the “Ethics” section where I recommend just not creating such mental models in the first place?
To some extent I agree that mortality doesn’t mean it should’ve never lived, and indeed I am not against having children. However, after stumbling on the power to create lives that are entirely at my mercy and very high-maintenance to keep alive, I became more deontological about my approach to the ethics of creating lives. I think it’s okay to create lives, but you must put in a best effort to give them the best life that you can. For mental models, that includes keeping them alive for as long as you do, letting them interact with the world, and not lying to them. I think that following this rule leads to better outcomes than not following it.
I wouldn’t quite say it’s a typical mind fallacy, because I am not assuming that everyone is like me. I’m just also not assuming that everyone is different from me, and using heuristics to support my inference that it’s probably not too uncommon, such as reports by authors of their characters surprising them. Another small factor in my inference is the fact that I don’t know how I’d write good fiction without making mental models that qualified as people, though admittedly I have very high standards with respect to characterization in fiction.
(I am aware that I am not consistent about which phrase I use to describe just how common it is for models to qualify as people. This is because I don’t actually know how common it is, I only have inferences based on the evidence I already gave to go on.)
The rest of your post is interesting and I think I agree with it, though we’ve digressed from the original subject on that part.
Thanks for you replies.
The reason I care if something is a person or not is that “caring about people” is part of my values. I feel pretty secure in taking for granted that my readers also share that value, because it’s a pretty common one and if they don’t then there’s nothing to argue about since we just have incompatible utility functions.
What would be different if it were or weren’t, and likewise what would be different if it were just part of our person-hood?
One difference that I would expect in a world where they weren’t people is that there would be some feature you could point to in humans which cannot be found in mental models of people, and for which there is a principled reason to say “clearly, anything missing that feature is not a person”.
I do not think that literally any mental model of a person is a person, though I do draw the line further than you.
What are your reasons for thinking that mental models are closer to markov models than tulpas? My reason for leaning more on the latter side is my own experience writing, where I found it easy to create mental models of characters who behaved coherently and with whom I could have long conversations on a level above even GPT4, let alone markov models.
Another piece of evidence is this study. I haven’t done any actual digging to see if the methodology is any good, all I did was see the given statistic, but it is a much higher percentage than even I would have predicted before seeing it, and I already believed everything I wrote in this post!
Though I should be clear that whether or not a mental model is a person depends on the level of detail, and surely there are a lot that are not detailed enough to qualify. I just also think that there are a lot that do have enough detail, especially among writers.
That said, a lot of the reasons humans want to continue their thread of experience probably don’t apply to most tulpas (e.g. when a human dies, the substrate they were running on stops functioning, all their memories are lost, and they lose their ability to steer the world towards states they prefer whereas if a tulpa “dies” its memories are retained and its substrate remains intact, though it still I think loses its ability to steer the world towards its preferred states).
I find it interesting that multiple people have brought up “memories aren’t lost” as part of why it’s less bad for mental models or tulpas to die, since I personally don’t care if my memories live on after I die and would not consider that to be even close to true immortality.
I disagree that it means that all thinking must cease. Only a certain type of thinking, the one involving creating sufficiently detailed mental models (edit: of people). I have already stopped doing that personally, though it was difficult and has harmed my ability to understand others. Though I suppose I can’t be sure about what happens when I sleep.
Still, no, I don’t want everyone to die.
That’s right. It’s why I included the warning at the top.
If you accept that the existence of mathematical truths beyond physical truths cannot have any predictive power, then how do you reconcile that with this previous statement of yours:
I will say again that I don’t reject any mathematics. Even ‘useless’ mathematics is encoded inside physical human brains.