Shiroe

Karma: 279

Shiroe Jul 24, 2022, 11:23 AM
1 point
0
in reply to: Vladimir_Nesov’s comment on: Which values are stable under ontology shifts?

but the point is that threatening total nonexistence is very hard once you have even an informal understanding of what it is you are talking about, at most you get loss of relevance

True, it’s hard, but it does happen. As evidenced by the many of us who actually have become apostates of our native religions. After becoming convinced that the central thesis of the faith was in error, the jig was up.

EDIT: I think your way of phrasing it is descriptively the most accurate, because it’s psychologically quite possible to resist the act of apostasy despite not actually believing in the truth of your own beliefs. However, for many of us, we would consider such a person a non-believer and hence a de facto apostate, even if they didn’t think of themselves that way.

Shiroe Jul 24, 2022, 11:08 AM
2 points
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?

Shiroe Jul 24, 2022, 10:48 AM
1 point
in reply to: flaritza’s comment on: flaritza’s Shortform
Response to first paragraph: Under the regime of “rationality is winning” (which is possibly being used as a motte) there is no room for being skeptical about rationality if rationality is simply what you would most prefer.
Response to second and third paragraph: such a person will come across as “disorganized” if they actually are so, but hopefully their interlocutor will be charitable after recognizing the goodness of the intent of that person.
EDIT: I prefer Dagon’s comments above. Mine wasn’t particularly helpful.

Shiroe Jul 24, 2022, 10:21 AM
3 points
0
in reply to: Vladimir_Nesov’s comment on: Which values are stable under ontology shifts?
If a deity is revealed to be physically nonexistent, that is perfectly in order because deities are supernatural rather than physical. But if it’s revealed that a deity is totally nonexistent i.e. that there is no entity which is the referent of its name, then that is equivalent to faith in it being shaken.

Shiroe Jul 24, 2022, 10:12 AM
1 point
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
Under your latter definition, could an agent be surprised by learning what its values are?

Shiroe Jul 24, 2022, 9:46 AM
1 point
0
in reply to: Vladimir_Nesov’s comment on: Which values are stable under ontology shifts?
It seems like Thane Ruthenis is claiming rather that even if one accepted that their religious view was refuted, that still wouldn’t destroy the believer’s value system, or to quote directly: “Either way, ontology shifts don’t do anything bad to our values.”

Shiroe Jul 24, 2022, 9:34 AM
8 points
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
Okay. Let me back up a bit. You had said:
Either there are some concrete things corresponding to personal identity/preference satisfaction/etc., or we value not these things but some actually-existing correlates of these things, or acting like we value these things is a heuristic that instrumentally helps us arrive at good outcomes. Either way, ontology shifts don’t do anything bad to our values.
Leading to this claim, which is very alluring to me, even if I disagree with it:
Or, the other way around, perhaps “values” are defined by being robust to ontology shifts.
You gave the example that us learning that our world is physical didn’t make us value humans less, even if we no longer believe that they have immortal souls. That’s true, we still value humans. But the problem is that the first group of agents (medieval “we”) are a different group than the second group (post-industrial “we”). Each group has a different concept of “human”. The two concepts roughly map to the same targets in the territory, but the semantic meaning is different in a way that is crucial to the first group of agents. The first group weren’t convinced that they were wrong. They simply got replaced by a new group who inherited their title.
That might sound dramatic, but ask a truly committed religious person if they can imagine themselves not believing in God. They will not be able to imagine it. This is because, in fact, they would no longer be the same agency in the sense that is crucial to them. Society would legally consider them the same person, and their apostate future-self might claim the same title as them and even deny that the change really mattered, but the original entity who was asked, the theistic agent, would no longer recognize any heir as legitimate at that point. From its point of view, it has simply died. That is why the theist cannot even imagine becoming an apostate, even if they grant that there’s a greater than zero chance of becoming an apostate at all times.
It’s true what you say, if you’re talking about people i.e. biological human organisms. Such entities will always be doing something in the world up until the very moment they physically expire, including having their whole world view shattered and living in the aftermath of that. However, pre-world-view-shattering them would not recognize post-world-view-shattering them as a legitimate heir. The person might be the same, but it’s a different agency in control.
Similar things can be said about populations and governments.

Shiroe Jul 24, 2022, 8:59 AM
1 point
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
A theist who has apostatized may still be the same person afterwards, but by no means are they the same agency. One agent has simply died, being replaced with a new one.

Shiroe Jul 24, 2022, 8:53 AM
1 point
0
in reply to: Noosphere89’s comment on: Which values are stable under ontology shifts?
Only if one interprets “subjective” as meaning “arbitrary”. The second meaning of “subjective”, which it shares with “phenomena”, is quite in accordance with moral realism and CEV-like aspirations.

Shiroe Jul 24, 2022, 8:40 AM
1 point
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
Okay, yes. One interpretation of the consequentialism I mentioned degrades gracefully like you say, giving “all zeroes” to a world of mere automata. (That’s a point in favor of such a consequentialism that it cannot ever become internally inconsistent.) But what you’re saying in the general case doesn’t fit what people usually mean when they talk about their values. A world without a God is not merely a world with lower utility for a theist, because the concept of “world” is already defined in terms of a God for the theist to begin with. You think otherwise perhaps because you have carved the concept-space up in such a way that lets you take apart different ideas like “world”, “God”, “value” and evaluate them separately. But this is not the case for the theist. Otherwise, they would not be a theist.

You’re right that it ought to work that way, though.

Shiroe Jul 24, 2022, 8:17 AM
1 point
0
in reply to: Thane Ruthenis’s comment on: Which values are stable under ontology shifts?
They’d re-define their policies to protect or multiply whatever objects of value actually do exist in the world, or attempt to create the valuable things that turned out not to exist (e. g., the kid baking a cake on his own).
Well, yes, assuming there still are other things of value for them, at least potentially. But if a robot had been a hedonistic consequentialist, then learned that itself and all beings in its universe lacked any phenomenal experiences (and that such experiences were impossible in its universe), that robot wouldn’t have any further set of objects to value while still being a hedonistic consequentialist.
You seem to be saying: well, when people say “all I really value is X” they don’t really mean that literally. But sometimes, for some agents, it is true that all they value is X. In such cases, there is no further set of things to still value. Utilitarians might fit this description of such an agent. This is why NickGabs had mentioned the hard problem of consciousness.

Shiroe Jul 20, 2022, 11:12 AM
2 points
2
in reply to: Ajeya Cotra’s comment on: Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
(By the way, I was delighted to find out you’re working on that ARC project that I had linked! Keep it up! You are like in the top 0.001% of world-helping-people.)

Shiroe Jul 20, 2022, 10:21 AM
3 points
1
on: Peter Singer’s first published piece on AI
Dwelling on issues such as if a moral self-driving car may run over migrating crabs on Christmas Island, this paper is unlikely to find much appeal here. Simply put, unless something entails the total destruction of everything, locals are unlikely to take much notice. Movement building has no appeal to them. This paper will likely be dismissed as academic lobbying, without relevance to the apocalypse.
But thank you very much for sharing it. I enjoyed a quick read, and in particular seeing familiar maneuvers like:
It is important to emphasize that although we ourselves accept the principle of equal consideration of similar interests, the arguments that follow are relevant to everyone who accepts that the interests of animals matter to some extent, even if they do not think that similar human and non-human interests matter equally.
This is what I find so strong in Singer’s argumentation: graceful degradation with weakening assumptions, retaining its conclusion under all cases. No room for respectable indifference. Ethics should bite, bleed, not let go.
This point at the end is great, I think:
In fact, it is evident that some scholars hold complacent and misguided judgments about ethics, and therefore are likely to develop methods of ethics building in AI that are unfriendly to animals. For example, Wu and Lin, … proposed an approach to ethics-building in AI systems that is based on “the assumption that under normal circumstances the majority of humans do behave ethically” … Most humans, however, buy factory-farmed animal products, if they are available and affordable for them. As we have already argued, this causes unnecessary harm to animals, and so is ethically dubious. Therefore, at least with regard to the treatment of animals, we cannot accept Wu and Lin’s assumption.
This is a point that I wish I heard more often on LessWrong. Preserving the world as it is today would not be an achievement but rather a disaster. Preserving the world that most humans want is not obviously better than Clippy, either. An “aligned” AGI may be worse than useless, depending on who does the “alignment”. These are all simple observations for the utilitarian, but I frequently find the instinct for survival has a yet greater hold on people here.

Shiroe Jul 19, 2022, 10:25 AM
10 points
2
on: Help ARC evaluate capabilities of current language models
In case that I get distracted and fail to come back to this, I just want to say that I think this type of project is extremely valuable and should be the main focus of the AI safety/alignment movement (IMHO). From my perspective, most effort seems to be going into writing about arguments for why a future AGI will be a disaster by default, as well as some highly theoretical ideas for preventing this from happening. This type of discourse ignores current ML systems, understandably, because future AGI will be qualitatively different from our current models.

However, the problem with this a priori approach is that it alienates the people working in the industry, who are ultimately the ones we need to be in conversation with if alignment is ever going to become a popular issue. What we really need are experiments like what you’re doing, i.e. actually getting a real AI to do something nasty on camera. This helps us learn to deal with nasty AI, but I think far more importantly it puts the AI safety conversation in the same experimental setting as the rest of the field.

Shiroe Jul 19, 2022, 10:04 AM
4 points
4
in reply to: Flaglandbase’s comment on: Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
Yes, but this century will be the decisive one. The phrase “most important century” isn’t claiming that future centuries lack moral significance, but the contrary.

Shiroe Jul 19, 2022, 8:18 AM
LW: 44 AF: 11
26
AF
on: Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
Reading AI safety articles like this one, I always find myself nodding along in agreement. The conclusions simply follow from the premises, and the premises are so reasonable. Yet by the end, I always feel futility and frustration. Anyone who wanted to argue that AI safety was a hopeless program wouldn’t need to look any further than the AI safety literature! I’m not just referring to “death with dignity”. What fills me with dread and despair is paragraphs like this:
However, optimists often take a very empiricist frame, so they are likely to be interested in what kind of ML experiments or observations about ML models might change my mind, as opposed to what kinds of arguments might change my mind. I agree it would be extremely valuable to understand what we could concretely observe that would constitute major evidence against this view. But unfortunately, it’s difficult to describe simple and realistic near-term empirical experiments that would change my beliefs very much, because models today don’t have the creativity and situational awareness to play the training game. [original emphasis]
Here is the real chasm between the AI safety movement and the ML industry/academia. One field is entirely driven by experimental results; the other is dominated so totally by theory that its own practitioners deny that there can be any meaningful empirical aspect to it, at least, not until the moment when it’s too late to make any difference.
Years ago, I read an article about an RL agent wireheading itself via memory corruption, thereby ignoring its intended task. Either this article exists and I can’t find it now, or I’m misremembering. Either way, it’s exactly the sort of research that the AI safety community should be conducting and publishing right now (i.e. propaganda with epistemic benefits). With things like GPT-3 around nowadays, I bet one could even devise experiments where artificial agents learn to actually deceive humans (via Mechanical Turk, perhaps?). Imagine how much attention such an experiment could generate once journalists pick it up!
EDIT: This post is very close to what I have in mind.

Shiroe Jul 14, 2022, 4:55 AM
4 points
2
on: How to impress students with recent advances in ML?
Are you looking for particular examples of AI doing impressive stuff (like PaLM explaining jokes), or do you already have enough examples of that sort to draw from? One thing I would emphasize to your students is how easy it is to underestimate a system that is near us in certain abilities but not very human-like psychologically: for example, see this recent discussion about GPT-3.

Shiroe Jul 12, 2022, 4:09 AM
4 points
3
in reply to: Dagon’s comment on: How do AI timelines affect how you live your life?
Do you know of any recent arguments from individuals with timelines like you mention, which are long/broad and emphasize uncertainty? At least the pool of people making a convincing case for the >100 year timelines has been drying up, from what I can see. Even Paul Christiano expects “40% on singularity by 2040”.

Shiroe Jul 6, 2022, 3:58 PM
6 points
2
on: Should you write under a blog or your own name?
Both types of names are forms of branding. Your real name is your career’s brand, while a pseudonym brands an idea. Pseudonyms attract more attention because ideas are more interesting than somebody else’s career. These purposes don’t exclude each other, but they signal different priorities.

Shiroe Jul 5, 2022, 10:14 PM
1 point
in reply to: Vladimir_Nesov’s comment on: wrapper-minds are the enemy
Complexity in case of humans is not evidence for the distinction, the standard position is that the stuff you are describing is complexity of extrapolated human wrapper-mind goal, not different kind of value, a paperclipper whose goals are much more detailed. From that point of view, the response to your post is “Huh?”, it doesn’t engage the crux of the disagreement.
“Huh?” was exactly my reaction. My values don’t vary depending on any environmental input; after all, they are the “ground truths” that give meaning to everything else.