Could you tell them afterwards that it was just an experiment, that the experiment is over, that they showed admirable traits (if they did), and otherwise show kindness and care?
I think this would make a big difference to humans in an analogous situation. At the very least, it might feel more psychologically healthy for you.
I don’t disagree that totalitarian AI would be real bad. It’s quite plausible to me that the “global pause” crowd are underweighting how bad it would be.
I think an important crux here is on how bad a totalitarian AI would be compared to a completely unaligned AI. If you expect a totalitarian AI to be enough of an s-risk that it is something like 10 times worse than an AI that just wipes everything out, then racing starts making a lot more sense.
I think mostly we’re on the same page then? Parents should have strong rights here, and the state should not.
I think that there’s enough variance within individuals that my rule does not practically restrict genomic liberty much, while making it much more palatable to the average person. But maybe that’s wrong, or it still isn’t worth the cost.
Your rule might for example practically prevent a deaf couple from intentionally having a child who is deaf but otherwise normal. E.g. imagine if the couple’s deafness alleles also carry separate health risks, but there are other deafness alleles that the couple does not have but that lead to deafness without other health risks.
That’s a good point, I wouldn’t want to prevent that. I’m not sure how likely this is to practically come up though.
Restrictions on genomic liberty should be considered very costly: they break down walls against eugenics-type forces (i.e. forces on people’s reproduction coming from state/collective power, and/or aimed at population targets).
Strong agree.
However, the difference is especially salient because the person deciding isn’t the person that has to live with said genes. The two people may have different moral philosophies and/or different risk preferences.
A good rule might be that the parents can only select alleles that one or the other of them have, and also have the right to do so as they choose, under the principle that they have lived with it. (Maybe with an exception for the unambiguously bad alleles, though even in that case it’s unlikely that all four of the parent’s alleles are the deleterious one or that the parents would want to select it.) Having the right to select helps protect from society/govt imposing certain traits as more or less valuable, and keeping within the parent’s alleles maintains inheritance, which I think are two of the most important things people opposed to this sort of thing want to protect.
What else did he say? (I’d love to hear even the “obvious” things he said.)
Thank you for doing this research, and for honoring the commitments.
I’m very happy to hear that Anthropic has a Model Welfare program. Do any of the other major labs have comparable positions?
To be clear, I expect that compensating AIs for revealing misalignment and for working for us without causing problems only works in a subset of worlds and requires somewhat specific assumptions about the misalignment. However, I nonetheless think that a well-implemented and credible approach for paying AIs in this way is quite valuable. I hope that AI companies and other actors experiment with making credible deals with AIs, attempt to set a precedent for following through, and consider setting up institutional or legal mechanisms for making and following through on deals.
I very much hope someone makes this institution exist! It could also serve as an independent model welfare organization, potentially. Any specific experiments you would like to see?
Happy solstice
Well, I’m very forgetful, and I notice that I do happen to be myself so… :p
But yeah, I’ve bitten this bullet too, in my case, as a way to avoid the Boltzmann brain problem. (Roughly: “you” includes lots of information generated by a lawful universe. Any specific branch has small measure, but if you aggregate over all the places where “you” exist (say your exact brain state, though the real thing that counts might be more or less broad than this), you get more substantial measure from all the simple lawful universes that only needed 10^X coincidences to make you instead of the 10^Y coincidences required for you to be a Boltzmann brain.)
I think that what anthropically “counts” is most likely somewhere between conscious experience (I’ve woken up as myself after anesthesia), and exact state of brain in local spacetime (I doubt thermal fluctuations or path dependence matter for being “me”).
I don’t doubt that LLMs could do this, but has this exact thing actually been done somewhere?
The “one weird trick” to getting the right answers is to discard all stuck, fixed points. Discard all priors and posteriors. Discard all aliefs and beliefs. Discard worldview after worldview. Discard perspective. Discard unity. Discard separation. Discard conceptuality. Discard map, discard territory. Discard past, present, and future. Discard a sense of you. Discard a sense of world. Discard dichotomy and trichotomy. Discard vague senses of wishy-washy flip floppiness. Discard something vs nothing. Discard one vs all. Discard symbols, discard signs, discard waves, discard particles.
All of these things are Ignorance. Discard Ignorance.
Is this the same principle as “non-attachment”?
Make a letter addressed to Governor Newsom using the template here.
For convenience, here is the template:
September [DATE], 2024
The Honorable Gavin Newsom
Governor, State of California
State Capitol, Suite 1173
Sacramento, CA 95814
Via leg.unit@gov.ca.govRe: SB 1047 (Wiener) – Safe and Secure Innovation for Frontier Artificial Intelligence Models Act – Request for Signature
Dear Governor Newsom,
[CUSTOM LETTER BODY GOES HERE. Consider mentioning:
Where you live (this is useful even if you don’t live in California)
Why you care about SB 1047
What it would mean to you if Governor Newsom signed SB 1047
SAVE THIS DOCUMENT AS A PDF AND EMAIL TO leg.unit@gov.ca.gov
]Sincerely,
[YOUR NAME]
This matches my memory as well.
I have no idea, but I wouldn’t be at all surprised if it’s a mainstream position.
My thinking is that long-term memory requires long-term preservation of information, and evolution “prefers” to repurpose things rather than starting from scratch. And what do you know, there’s this robust and effective infrastructure for storing and replicating information just sitting there in the middle of each neuron!
The main problem is writing new information. But apparently, there’s a protein evolved from a retrotransposon (those things which viruses use to insert their own RNA into their host’s DNA) which is important to long term memory!
And I’ve since learned of an experiment with snails which also suggests this possibility. Based on that article, it looks like this is maybe a relatively new line of thinking.
It’s good news for cryonics if this is the primary way long term memories are stored, since we “freeze” sperm and eggs all the time, and they still work.
Do you know if fluid preservation preserves the DNA of individual neurons?
(DNA is on my shortlist of candidates for where long-term memories are stored)
Consider finding a way to integrate Patreon or similar services into the LW UI then. That would go a long way towards making it feel like a more socially acceptable thing to do, I think.
Yeah, that’s not what I’m suggesting. I think the thing I want to encourage is basically just to be more reflective on the margin of disgust-based reactions (when it concerns other people). I agree it would be bad to throw it out unilaterally, and probably not a good idea for most people to silence or ignore it. At the same time, I think it’s good to treat appeals to disgust with suspicion in moral debates (which was the main point I was trying to make) (especially since disgust in particular seems to be a more “contagious” emotion for reasons that make sense in the context of infectious diseases but usually not beyond that, making appeals to it more “dark arts-y”).
As far as the more object-level debate on whether disgust is important for things like epistemic hygiene, I expect it to be somewhere where people will vary, so I think we probably agree here too.
I meant wrong in the sense of universal human morality (to the extent that’s a coherent thing). But yes, on an individual level your values are just your values.
I see that stuff as at best an unfortunate crutch for living in a harsher world, and which otherwise is a blemish on morality. I agree that it is a major part of what many people consider to be morality, but I think people who still think it’s important are just straightforwardly wrong.
I don’t think disgust is important for logic / reflectivity. Personally, it feels like it’s more of a “unsatisfactory” feeling. A bowl with a large crack, and a bowl with mold in it are both unsatisfactory in this sense, but only the latter is disgusting. Additionally, it seems like people who are good at logic/math/precise thinking seem to care less about disgust (as morality), and highly reflective people seem to care even less about it.
ETA: Which isn’t to say I’d be surprised if some people do use their disgust instinct for logical/reflective reasoning. I just think that if we lived in the world where that main thing going on, people good at that kind of stuff would tend to be more bigoted (in a reflectively endorsed way) and religious fundamentalism would not be as strong of an attractor as it apparently is.
That doesn’t seem right to me. My thinking is that disgust comes from the need to avoid things which cause and spread illness. On the other hand, things I consider more central to morality seem to have evolved for different needs [these are just off-the-cuff speculations for the origins]:
Love—seems to be generalized from parental nurturing instincts, which address the need to ensure your offspring thrive
Friendliness—seems to have stemmed from the basic fact that cooperation is beneficial
Empathy—seems to be a side-effect of the way our brains model conspecifics (the easiest way to model someone else is to emulate them with your own brain, which happens to make you feel things)
These all seem to be part of a Cooperation attractor which is where the pressure to generalize/keep these instincts comes from. I think of the Logic/reflectivity stuff as noticing this and developing it further.
Disgust seems unsavory to me because it dampens each of the above feelings (including making the logic/reflectivity stuff more difficult). That’s not to say I think it’s completely absent form human morality, it just doesn’t seem like it’s where it comes from.
(As far as Enforcement goes, it seems like Anger and Fear are much more important than Disgust.)
It’s a great case, as long as you assume that AIs will never be beyond our control, and ignore the fact that humans have a metabolic minimum wage.