quila

Karma: 911

suffering-focused-altruist/longtermist. currently looking for a value alignment solution.

my pgp public key, partly to prevent future LLM impersonation; you can always ask me to sign a dated message. [the private key is currently stored in plaintext within an encrypted drive, so is vulnerable to being read by local programs]:

I have signed no contracts or agreements whose existence I cannot mention.

quila 14 Feb 2025 20:01 UTC
3 points
2
in reply to: Noosphere89’s comment on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise
My answer is values don’t have to converge, and human value conflicts are more severe than people like to think, for many reasons.
That doesn’t seem related to the question (e.g., it’s not an evolutionary explanation, but it’s also not about the particular frame in the OP). I’m also confused why you seem to write that under many posts, but that’s tangential.
There doesn’t have to be a nice/simple reason for why some people are pleased by the idea of others coming to harm.
Of course, but this shouldn’t stop us from looking for particular reasons. OP’s analysis was interesting, so I wondered if they have another interesting one for the opposite case.

quila 14 Feb 2025 8:43 UTC
1 point
0
on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise
I wonder how you would explain actually evil people. In the framing of this post, I mean ones who don’t just exploit ‘less-conditional cooperators’ for own gain, but are intrinsically pleased by the idea of others coming to harm. At least some such people exist^[1].
In the evolutionary environment, humans were a source of free energy, so even such a person—who makes an offer favoring themself, out of ‘irrational misanthropy’ - could be positive (to the trader) to trade with (modulo how their existence acausally depends on them actually being traded with).
1. ^
  I’m reminded of this text which I unfortunately came across today (ctrl+f “biodiesel” if choose to read it, also consider not opening it at all, if you don’t want to become more cynical)

quila 13 Feb 2025 12:36 UTC
1 point
0
on: Why you maybe should lift weights, and How to.
Lifting weights, statistically, is extremelly safe compared to any other sport. In the same vein as above, if someone tells you “you need to do X on top of your lifting to make sure you don’t get injured”, they’re probably wrong.
Known fallacy: the statistical safety could be conditional on participants taking special precaution, like in skydiving.
(Though the part after suggests there’s a true adjacent thing, if {it’s true those median participants don’t actually use special (even if ‘lay’) safety caution}, which I’m doubtful of)

quila 13 Feb 2025 7:41 UTC
1 point
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
goodharting to change it’s definition of what consitutes a paperclip that is easier for it to maximize
Same thing applies. “Does that fulfill the current goal-definition?” (Note this is not a single question; we can ask this about each possible goal-definition)
Why is it most likely that it [...]
This was about an abstract definition of an agent (not itself a prediction, but does say something about a space of math, that we might end up in). There are surely possible programs which would exhibit any behavior, although some look harder to program (or ‘less natural’): for example, “an entity that is a paperclip maximizer for 100 years, then suddenly switches to maximizing stamps” looks harder to program (if an embedded agent) because you’d need to find a method where it won’t just self-modify to never turn into a stamp maximizer (as turning into one would prevent if from maximizing paperclips), or to not unleash a true paperclip maximizer and shut itself down if you rule out just self-modification (and so on if you were to additionally rule out just that).^[1]
1. ^
  (though very tangentially there is a simple way to do that)

quila 10 Feb 2025 7:42 UTC
1 point
0
in reply to: TsviBT’s comment on: TsviBT’s Shortform
Also,
Background music: −6 to −2 CQ points.
That might generalize to “minimizing sound good”, in which case I’d suggest trying these earplugs.
Generalizing to sensory deprivation in general, an easy way to do that is to lay in bed with eyes closed and lights off (not to sleep). (I’ve found this helpful, but maybe it’s a side effect of not being distracted by a computer)

quila 9 Feb 2025 8:27 UTC
1 point
0
in reply to: TsviBT’s comment on: TsviBT’s Shortform
I notice the potential for combo between these two:
Sleep deprived a little, like stay up really late but without sleep debt: +5 CQ point
Oversleep a lot, like 11 hours: +6 CQ points.
(One can stay up till sleep deprived and oversleep on each sleep/wake cycle, though they’ll end up with a non-24-hour schedule)

quila 8 Feb 2025 4:27 UTC
3 points
1
in reply to: Davey Morse’s comment on: nikola’s Shortform
If there’s no constraints on when they have to name a system “GPT-5”, they can make the statement true by only naming a system “GPT-5″ if it is smart enough. (cf. Not Technically a Lie)
Edit: though “and those two [...] I’d like to hear back from you in a little bit of time” implies a system named GPT-5 will be released ‘in a little bit of time’
Edit 2: “Internally we’ve gone all the way to about a maybe like a 4.5”

quila 8 Feb 2025 1:19 UTC
3 points
0
in reply to: Nisan’s comment on: Nisan’s Shortform
The version of this that OpenAI did wasn’t unexpected to me, but this one was, I wasn’t sure about DeepMind. (Wait, now I’m remembering that “Google AI” is a separate group to “DeepMind”? Web says they were merged in 2023. I wonder which members on the inside this change reflects).

quila 7 Feb 2025 11:20 UTC
2 points
0
in reply to: Logan Riggs’s comment on: Logan Riggs’s Shortform
Oh, I’ve thought a lot about something similar that I call “background processing”—I think it happens during sleep, but also when awake. I think for me it works better when something is salient to my mind / my mind cares about it. According to this theory, if I was being forced to learn music theory but really wanted to think about video games, I’d get less new ideas about music theory from background processing, and maybe it’d be less entered into my long term memory from sleep.
I’m not sure how this effects more ‘automatic’ (‘muscle memory’) things (like playing the piano correctly in response to reading sheet music).
I’m unsure if you [practice piano] in the morning, you’ll need to remind yourself about the experience before you fall asleep, also implying that you can only really consolidate 1 thing a day
I’m not sure about this either. It could also be formulated as there being some set amount of consolidation you do each night, and you can divide them between topics, but it’s theoretically (disregarding other factors like motivation; not practical advice) most efficient if you do one area per day (because of stuff in the same topic having more potential to relate to each other and be efficiently compressed or generalized from or something. Alternatively, studying multiple different areas in a day could lead to creative generalization between them).

quila 7 Feb 2025 10:17 UTC
17 points
7
in reply to: JenniferRM’s comment on: Wild Animal Suffering Is The Worst Thing In The World
Pollywogs (the larval form of frogs, after eggs, and before growing legs) are an example where huge numbers of them are produced, and many die before they ever grow into frogs, but from their perspective, they probably have many many minutes of happy growth, having been born into a time and place where quick growth is easy: watery and full of food
Consider an alien species which requires oxygen, but for whom it was scarce during evolution, and so they were selected to use it very slowly and seek it ruthlessly, and feel happy when they manage to find some. It would be wrong for one of that species to conclude that a species on earth must be happy all the time because there’s so much oxygen; because oxygen is abundant, we are neutral to it.
Most of the animal-life-moments I’ve seen in nature have exhibited a profound “OK-ness”. Just chilling. Just being. Just finding some food. Just happy to exist.
Emotional status must be inferred. There are some cases we can easily infer happiness or suffering in nonhumans, by similarity to how we express emotions: a dog jumping around and excited to go outside, a pig squealing and struggling as they are lowered into a gas chamber. There are others we cannot easily know the experience of: a duck sitting in a lake, a caterpillar crawling across a leaf. To call them happy without an analysis from first-principles of what evolutionary pressures might lead to the experience or not of happiness in such moments is to project happiness onto them.
I think most of the rest of your comment is similarly projective.
I wonder if you have the opposite sort of bias to that which you say the OP might have; maybe you have a very happy outlook which causes you to project happiness-to-exist onto life forms you do not understand.
Certainly when “life itself” becomes questionable for animals in nature, they seem to do everything they can to keep their life… escaping danger, seeking safety, eating food (rather than engaging in a hunger strike or trying to forgo water and thirst to death to bring their own end on faster). Every indication we have is that animals are NOT suicidal, but rather they seem to “net value” their own lives. Presumably because these are good lives according to their own subjectivities.
A human with chronic depression will still flee if under attack. Evolution imbues beings with a drive to survive even when in great pain. It does not care whether we suffer, indeed it uses suffering to motivate us; the counter that we would choose to kill ourselves if we suffered enough does not hold, evolution gets around that by then also selecting for us to not kill ourselves.
(I note these things without implying a side on the question of what kinds of mental states are most common in wild animal lives, which looks difficult to speculate about with confidence.)

quila 4 Feb 2025 9:40 UTC
1 point
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
consider a system which is capable of self-modification and changing it’s own goals, now the difference between an instrumental goal and a terminal goal erodes.
If an entity’s terminal goal is to maximize paperclips, it would not self-modify into a stamp maximizer, because that would not satisfy the goal (except in contrived cases where doing that is the choice that maximizes paperclips). A terminal goal is a case of criteria according to which actions are chosen; “self-modify to change my terminal goal” is an action.

quila 31 Jan 2025 5:35 UTC
3 points
0
in reply to: plex’s comment on: ete’s Shortform
I sent the following offer (lightly edited) to discuss metaphysics to plex. I extend the same to anyone else.^[1]
(Note: I don’t do psychedelics or participate in the mentioned communities^[2], and I’m deeply suspicious of intuitions/deep-human-default-assumptions. I notice unquestioned intuitions across views on this, and the primary thing I’d like to do in discussing is try to get the other to see theirs.)
Hi, I saw [this shortform] by you
Wondering if you’d want to discuss metaphysics itself / let me socratic-question^[3] you. I lean towards the view that “metaphysics is extra-mathematical; it contains non-mathematical-objects, such as phenomenal qualia” but I’d like to mostly see if I can socratic-question you into noticing incoherence. I’m too uncertain about the true metaphysics to put forth a totalizing theory.
It might help if you’ve read this post first.
My starting questions:
1. Do you believe phenomenal qualia exist? If you do, are they mathematical objects? If you don’t, can you watch this short video on a thought experiment?
  (Commentary: someone in the comments of the linked post told me that video was what changed their position from ‘camp 1’ (~illusionist, e.g. “qualia is an intuition from evolution”) to ‘camp 2’ (“qualia is metaphysically fundamental”))
2. You mentioned ‘matter’ and ‘math’ as candidates for ‘fundamental to reality’. My question depends on which is your view.
  If you believe matter is fundamental, what do you mean by matter; how does it differ from math?
  (Commentary: “matter” seems like an intuitive concept; one has an intuition that reality is “made of something”, and calls that “thing” (undefined) “matter”, and refers to “matter” without knowing “what it is” (also undefined); there being a word for this intuition prevents them from noticing their lack of knowledge, or the circular self-reference between “matter” and “what the world is made of”)
  If you believe math is fundamental, what distinguishes this particular mathematical universe from other ones; what qualifies this world as “real”, if anything; what ‘breathes fire into the equations and creates a world for them to describe’?
  (Commentary: one self-consistent position answers “nothing”—that this world is just one of the infinitely many possible mathematical functions / programs. That ‘real’ secretly means ‘the program(s?) we are part of’. Though I observe this position to be rare; most have a strong intuition that there is something which “makes reality real”.)
  If you believe math is fundamental, what is math?^[4]
1. ^
  Though I’ll likely lose interest if it seems like we’re talking past each other / won’t resolve any cruxy disagreements.
2. ^
  (except arguably the qualia research institute’s discord server, which might count because it has psychedelics users in it)
3. ^
  (Questioning with the goal of causing the questioned one to notice specific assumptions or intuitions to their beliefs, as a result of trying to generate a coherent answer)
4. ^
  From an unposted text:
  The paradox of recursive explanation applies to metaphysics too
  In Explain/Worship/Ignore (~500 words), Eliezer describes an apparent paradox: if you ask for some physical phenomena to be explained (shown to have a more fundamental cause), then ask the same of the explanation, and so on, the only conceivable outcomes seem paradoxical: infinite recurse, circular self-reference, or a ‘special’ first cause that does not itself need to be explained.
  This is sometimes called the paradox of why. It’s usually applied to physics; this text applies it to logic/math-structure and metaphysics/ontology too. In short, you can continually ask “but what is x” for any aspect of logic or ontology.
  Here’s a hypothetical Socratic dialogue.
  Author: “What is reality?”
  Interlocutor: “Reality is <complete description of [very large mathematical structure that perfectly reflects the behavior of the universe]>.”
  Author: “Let’s suppose that is true. But this ‘math’ you just used; what is it?”
  Interlocutor: “Mathematics is a program which assigns, based on a few simple rules, ‘T’ or ‘F’ to inputs formatted in a certain way.”
  Author: “I think you just moved the hard part of the question to be one question away: What is a program?”
  Interlocutor: “Hmm. I can’t define a program in terms of math, because I just did the reverse. Wikipedia says a program is ‘a sequence of instructions for a computer to execute’. If I just wrote that, Author would ask what it means for a computer to execute an instruction. What else could I write?
  A computer is a part of physics arranged into a localized structure, and for this structure to ‘execute an instruction’ is for physical law to flow through (operate on, apply to) it, such that the structure’s observed behavior matches that of a simpler-than-physics abstract system for transforming inputs to outputs. Unfortunately, I’ve already defined physics in terms of math, so defining a program in terms of physics would be circular. I think I’m at a dead end.”
  Author: “Do you want to revise one of your previous definitions?”
  Interlocutor: “Maybe I could define math as some more fundamental thing instead of as a certain kind of program. But I just failed to find such a more fundamental thing for ‘program’. Let’s check Wikipedia again...
  WP:Mathematics: ‘Mathematics involves the description and manipulation of abstract objects that consist of either abstractions from nature or—in modern mathematics—purely abstract entities that are stipulated to have certain properties, called axioms’
  WP:Mathematical_Object ‘Typically, a mathematical object can be a value that can be assigned to a symbol, and therefore can be involved in formulas. Commonly encountered mathematical objects include numbers, expressions, shapes, functions, and sets.’
  “I can’t use the ‘abstractions from nature’ part, because I’ve already defined nature to be mathematical, so that would be circular. Saying math is made of numbers, expressions, symbols, etc, isn’t helpful either, though; Author will just ask what those are, metaphysically. Okay, I concede for now, though maybe a commenter will propose such a more-fundamental-thing that I’m not aware of, though it would invite the same question.”
  [End of dialogue]

quila 29 Jan 2025 12:33 UTC
3 points
1
in reply to: Satron’s comment on: [Link] A community alert about Ziz
(It’s not actually the case that I don’t value their well-being; I don’t want them to suffer and if they were tortured, imagining that would make me sad; I’d prefer beings who don’t care about some subset of other beings / are value-orthogonal to just be prevented from hurting them. I just think Richard, in the current world, probably causes more tragedy, based on the comment, so yes I think the current world would be better if it did not have any such people.)
Agreed about the double standard part, that’s something I was hoping to highlight.

quila 29 Jan 2025 0:21 UTC
5 points
0
in reply to: FeepingCreature’s comment on: [Link] A community alert about Ziz
A community that can accept “nazis” (in the vegan sense) cannot also accept “resistance fighters” (in the vegan sense). Either the “nazi” deserves to exist or he doesn’t. But to test this dichotomy, somebody has to out themselves as a “nazi.”
This doesn’t seem true. It seems like it’s saying that the directly opposing views on this cannot both exist in a “community” (to the extent LW is a community), but they evidently do both exist here (which is to be expected with enough users).
(quoting the comment by Richard_Kennaway that started this thread since it’s 2 years old, and plausibly some will see my comment from the ‘new’ section and be confused by what I write next otherwise)
I eat meat and wear leather and wool. I do think that animals, the larger ones at least, can suffer. But I don’t much care. I don’t care about animal farming, nor the (non-human) animal suffering resulting from carnivores and parasites. I’d rather people not torture their pets, and I’d rather preserve the beauty and variety of nature, but that is the limit of my caring. If I found myself on the surface of a planet on which the evolution of life was just beginning, I would let it go ahead even though it mean all the suffering that the last billion years of this planet have seen.
Bring on the death threats.
[Even if one thinks, in a utilitarian sense, the world would be better without such a person in it], killing them would still be a waste of one’s opportunity to effect the world, given there are much more effective ways to improve the future (e.g., donating $1k to an animal charity does more good IIUC; more ambitiously, helping solve alignment saves all the animals in one go, if we don’t die to unaligned ASI first).
(I feel like this comment would be incomplete without also mentioning that I guess most but not all people stating they’re indifferent to and cause non-human suffering now would reproach the view and behavior eventually, and that relative to future beings who have augmented their thinking ability and lived for thousands of years, all current beings are like children, some hurting others very badly in confusion.)

quila 26 Jan 2025 7:10 UTC
2 points
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
Which I took to mean that some they overlap in some instrumental goals. That is what you meant right?
No. I was trying to explain that: any agent that can be predicted by thinking of them as having two separate values for two different things, can also be predicted by thinking of them as maximizing some single value which internally references both things.
For example: “I value paperclips. I also value stamps, but one stamp is only half as valuable as a paperclip to me” → “I have the single value of maximizing this function over the world: {paperclip-amount×2 + stamp-amount}”. (It’s fine to think of it in either way)
can you be explicit (to be be honest, use layman’s terms)
If you want, it would help me learn to write better, for you to list off all the words (or sentences) that confused you.

quila 26 Jan 2025 4:08 UTC
1 point
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
are you saying that superintelligent AGIs won’t necessary converge in values because even a single superintelligent agent may have multiple terminal goals?
No, I was responding to your claim that I consider unrelated. Like I wrote at the top: “That [meaning your claim that humans have multiple terminal goals] is not relevant to whether there are convergent terminal values”^[1]
some terminal goals may overlap in they share certain instrumental goals?
I don’t know what this is asking / what ‘overlap’ means. That most terminal goals share instrumental subgoals is called instrumental convergence.
1. ^
  Which in other words means, even if it were true, “humans have multiple terminal goals” would not be a step of the argument for it

quila 26 Jan 2025 2:53 UTC
1 point
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
Don’t people usually have several terminal goals at any given time?
That is not relevant to whether there are convergent terminal values^[1].
To answer it anyways, people are not well-modeled as idealized terminal-goal-pursuers. More broadly, programs/minds don’t have to be idealized terminal-goal-pursuers, so humans as a particular case of programs/minds-in-general present no paradox. “What is the true terminal goal” has a false premise that there must be some true terminal goal.
As for the case of idealized terminal-goal-pursuers, any two terminal goals can be combined into one, e.g. {paperclip-amount×2 + stamp-amount} or {if can create a black hole with p>20%, do so, else maximize stamps}, etc.
what distinguishes a very high level instrumental goal form a terminal goal
it being instrumental to some top-level goal
1. ^
  (or ‘mind-independent moral facts’, as the idea has been called in philosophy. https://plato.stanford.edu/entries/moral-anti-realism/)

quila 24 Jan 2025 23:20 UTC
1 point
1
in reply to: trevor’s comment on: Habryka’s Shortform Feed
A better way is to download it. See also Preserving and continuing alignment research through a severe global catastrophe

quila 24 Jan 2025 4:01 UTC
2 points
0
in reply to: CstineSublime’s comment on: Siebe’s Shortform
I’ll raise you an even stupider question: surely once an A.I. becomes sufficiently super-intelligent, all superintelligent systems will converge on certain values rather than be biased towards their initial training data?
video introducing the orthogonality thesis

quila 24 Jan 2025 0:48 UTC
7 points
0
on: Announcement: Learning Theory Online Course
To get started right away, see the entry questions
What set of textbooks could I (or, to simplify the question, a past version of you with only HS-level math knowledge) read to be able to understand the meaning of these?