But that still requires us to have developed human brain-scanning technology within 5 years, right? That does not seem remotely plausible.
Sweetgum
Karma: 95
Indeed, it is instrumentally useful for instrumental rationalists to portray themselves as epistemic rationalists. And so this is a common pattern in human politics—“[insert political coalition] care only about themselves, while [insert political coalition] are merely trying to spread truth” is one of the great political cliches for a reason. And because believing one’s own lies can be instrumentally useful, falsely believing oneself to have a holy devotion to the truth is a not-uncommon delusion.
I try to dissuade myself of this delusion.
There’s a subtle paradox here. Can you spot it?
He is trying to dissuade himself of the premise[X] that he is committed to the truth over socially useful falsehoods. But that premise[X] is itself socially useful to believe, and he claims it’s false, so disbelieving it would show that he does sometimes value the truth over socially useful falsehoods, contradicting the point.
More specifically, there are three possibilities here:X is broadly true. He’s just wrong about X, but his statement that X is false is not socially motivated.
X is usually false, but his statements about X are a special case for some reason.
X is false, but his statement that X is false doesn’t contradict this because denying X is actually the socially useful thing, rather than affirming X. Lesswrong might be the kind of place where denying X (saying that you are committed to spreading socially useful falsehoods over the truth) actually gets you social credit, because readers interpret affirming X as the thing that gets you social credit, so denying it is interpreted as a signal that you are committed to saying the taboo truth (not-X) over what is socially useful (X), the exact opposite of what was stated. If true, this would be quite ironic. This interpretation is self-refuting in multiple ways, both logically (for not-X to be a “taboo truth”, X has to be false, which already rules out the conclusion of this line of reasoning) and causally (if everyone uses this logic, the premise that affirming X is socially useful becomes false, because denying X becomes the socially useful thing.) But that doesn’t mean readers couldn’t actually be drawing this conclusion without noticing the problems.
It’s more akin to me writing down my thoughts and then rereading them to gather my ideas than the kind of loops I imagine our neurons might have.
In a sense, that is what is happening when you think in words. It’s called the phonological loop.
In this cases it can be helpful to imagine your current self in a bargaining game with your future selves, in a sort of prisoner’s dilema. If your current now defects, your future selves will be more prone to defecting as well. If you coordinate and resist tempation now, future resistance will be more likely. In other words, establishing a Schelling fence.
This is an interesting way of looking at it. To elaborate a bit, one day of working toward a long-term goal is essentially useless, so you will only do it if you believe that your future selves will as well. This is some of where the old “You need to believe in yourself to do it!” advice comes from. But there can be good reasons not to believe in yourself as well.
In the context of the iterated Prisoner’s Dilemma, it’s been investigated what the frequency of random errors (the decision to cooperate or defect being replaced with a random one in x% of instances) can go up to before cooperation breaks down. (I’ll try to find a citation for this later.) This seems similar, but not literally equivalent, to a question we might ask here: What frequency of random motivational lapses can be tolerated before the desire to work towards the goal at all breaks down?
Naturally, the goals that require the most trust are ones that see no benefit until the end, because they require you to trust that your future selves won’t permanently give up on the goal anywhere between now and the end to be worth working towards at all. But most long term goals aren’t really like this. They could be seen to fall on a spectrum between providing no benefit until a certain point and linear benefit the more they are worked towards with the “goal” point being arbitrary. (This is analogous to the concept of a learning curve.) Actions towards a goal may also provide an immediate benefit as well as progress toward the goal, which reduces the need to trust your future selves.
If you don’t trust your future selves very much, you can seek out “half-measure” actions that sacrifice some efficiency toward the goal for immediate benefits, but still contribute some progress toward the goal. You can to some extent set where they are along this spectrum, but you are also limited by the types of actions available to you.
Thanks, this is a great explanation and you changed my mind on this. This is probably the reason why most people have the intuition that legalizing these things makes things worse for everyone. There were many proposed explanations for that intuition in this thread, but none of the others made sense/seemed valid to me, so I was beginning to think the intuition was erroneous.
Looks like your comment got truncated: “what is good if they were just”
Edited to fix.
Roman values aren’t stable under reflection; the CEV of Rome doesn’t have the same values as ancient Rome.
I’m not exactly sure what you’re saying here, but if you’re saying that the fact of modern Roman values being different than Ancient Roman values shows that Ancient Roman values aren’t stable under reflection, then I totally disagree. History playing out is a not-at-all similar process to an individual person reflecting on their values, so the fact that Roman values changed as history played out from Ancient Rome to modern Rome does not imply that an individual Ancient Roman’s values are not stable under reflection.
As an example, Country A conquering Country B could lead the descendants of Country B’s population to have the values of Country A 100 years hence, but this information has nothing to do with whether a pre-conquest Country B citizen would come to have Country A’s values on reflection.
Locking in extrapolated Roman values sounds great to me because I don’t expect that to be significantly different than a broader extrapolation.
I guess I just have very different intuitions from you on this. I expect expect people from different historical time periods and cultures to have quite different extrapolated values. I think the concept that all peoples throughout history would come into near agreement about what is good if they just reflected on it long enough is unrealistic.
(unless, of course, we snuck a bit of motivated reasoning into the design of our Value Extrapolator so that it just happens to always output values similar to our 21st century Western liberal values...)
Skimming the Nick Bostrom and Effective Altruism Wikipedia pages, there doesn’t seem to be anything particularly wrong with them, certainly not anything that I would consider vandalism. What do you see as wrong with those articles?
Could you explain how allowing sex for rent or kidney sale would lead to an arms race that makes everyone worse off? Or is this just meant to be an argument for why allowing extra options isn’t necessarily good, that doesn’t apply to the specific examples in the post?
Slavery and theft harm others, so they are not relevant here. Age limits would be the most relevant. We have age limits on certain things because we believe that regardless of whether they want to, underage people deciding to do those things is usually not in their best interest. Similarly, bans on sex for rent and kidney sale could be justified by the belief that regardless of whether they want to, people doing these things is usually not in their best interest. However, this is somewhat hard to back up: It’s pretty unclear whether prostitution or homelessness is worse, and it’s easy to think of situations where selling a kidney definitely would be worth it (like the one given in the post).
I don’t want to live in a world where women have to prostitute themselves to afford rent.
I don’t want to live in that world either, but banning sex for rent doesn’t resolve the issue. It just means we’ve gone from a world where women have to prostitute themselves to afford rent to a world where women just can’t afford rent, period.What I said here is wrong, see this comment
Have them be homeless until the homelessness situation becomes severe enough that we resolve it. Otherwise, IMO, we are just boiling the frog. There will be no protests, no riots, because selling our kidneys and having sex for rent is just enough for us to get by.
You don’t think having to sell your kidneys and have sex for rent to get by is bad enough to get people to protest/riot?
Also, it seems like you’ve implicitly changed your position here. Previously, you said that when someone sells a kidney/trades sex for rent it would usually not be in their best interest, and that those options would usually only be taken under the influence of addiction or mental illness. Now, when you say that people would do those things “to get by” it sounds like you’re implying that these are rational choices that would be in peoples’ best interest given the bad situation, and would be taken by ordinary people. Which of these do you agree with?
Could you give some examples? I understand you may not want to talk about culture war topics on lesswrong, so it’s fine if you decline, but without examples I unfortunately cannot picture what you’re talking about
poor intelligence and especially memory (other people say otherwise), pathetic mathematical abilities (takes longer than the blink of an eye to divide two 100 digit numbers)… …inability to communicate at more than about 0.005 kB/s
What do you consider to be the “normal” level of intelligence/memory/communication bitrate? Why?
Perhaps itsability to verify things, being hampered by its only seeing the world through text, is fatal.
Suggest to change to “its inability” to make this sentence more clear
Discord does have allow you to make named threads that branch off of a channel, and later archive them. It’s not the default mode of conversation on Discord but it is available if you care to use it. Also, I am confused what you mean by “threads are made after the fact” on Discord.
These aren’t like Dennett’s “deepities”—Deepities are statements that sound profound by sneakily having two alternate readings, one mundanely true and one radical or outlandish, sort of like a motte and bailey argument. These answers are just somewhat vague analogies and a relatively normal opinion that uses eloquent language (“because we are”) to gain extra deepness points.
Have you heard of Xiaoice? It’s a Chinese conversational/romantic chatbot similar to Replika. This article from 2021 claimed it already had 660 million users.
Logically, I knew it was all zeros and ones, but they felt so real.
There are various reasons to doubt that LLMs have moral relevance/sentience/personhood, but I don’t think being “all zeros and ones” is one of them. Preemptively categorizing all possible digital computer programs as non-people seems like a bad idea.
Are you sure that “browsing:disabled” refers to browsing the web? If it does refer to browsing the web, I wonder what this functionality would do? Would it be like Siri, where certain prompts cause it to search for answers on the web? But how would that interact with the regular language model functionality?
But the analogy is more like a kid thinking they’re playing a game that’s on autoplay mode.
No. In your analogy, what the kid does has no causal impact on what their character does in the game. In real life, what you(your brain) does is almost always the cause of what your body does. The two situations are not analogous. Remember, determinism does not mean you lack control over your decisions. Also remember, you just are your brain. There’s no separate “you” outside your brain that exists but lacks control because all your actions are caused by your brain instead.
Bayesian probability (which is the kind Yudkowsky is using when he gives the probability of AI doom) is subjective, referring to one’s degree of belief in a proposition, and cannot be 0% or 100%. If you’re using probability to refer to the objective proportion of future Everett branches something occurs in, you are using it in a very different way than most, and probabilities in that system cannot be compared to Yudkowsky’s probabilities.