Hm. This rings true… but also I think that selecting [vibes, in this sense] for attention also selects against [things that the other person is really committed to]. So in practice you’re just giving up on finding shared commitments. I’ve been updating that stuff other than shared commitments is less good (healthy, useful, promising, etc.) than it seems.
TsviBT(Tsvi Benson-Tilsen)
Karma: 4,637
Ok but how do you deal with the tragedy of the high dimensionality of context-space? People worth thinking with have wildly divergent goals—and even if you share goals, you won’t share background information.
Are you claiming that this would help significantly with conceptual thinking? E.g., doing original higher math research, or solving difficult philosophical problems? If so, how would it help significantly? (Keep in mind that you should be able to explain how it brings something that you can’t already basically get. So, something that just regular old Gippity use doesn’t get you.)
I didn’t read this carefully—but it’s largely irrelevant. Adult editing probably can’t have very large effects because developmental windows have passed; but either way the core difficulty is in editor delivery. Germline engineering does not require better gene targets—the ones we already have are enough to go as far as we want. The core difficulty there is taking a stem cell and making it epigenomically competent to make a baby (i.e. make it like a natural gamete or zygote).
Ben’s responses largely cover what I would have wanted to say. But on a meta note: I wrote specifically
I think a hypothesis that does have to be kept in mind is that some people don’t care.
I do also think the hypothesis is true (and it’s reasonable for this thread to discuss that claim, of course).
But the reason I said it that way, is that it’s a relatively hard hypothesis to evaluate. You’d probably have to have several long conversations with several different people, in which you successfully listen intensely to who they are / what they’re thinking / how they’re processing what you say. Probably only then could you even have a chance at reasonably concluding something like “they actually don’t care about X”, as distinct from “they know something that implies X isn’t so important here” or “they just don’t get that I’m talking about X” or “they do care about X but I wasn’t hearing how” or “they’re defensive in this moment, but will update later” or “they just hadn’t heard why X is important (but would be open to learning that)”, etc.
I agree that it’s a potentially mindkilly hypothesis. And because it’s hard to evaluate, the implicature of assertions about it is awkward—I wanted to acknowledge that it would be difficult to find a consensus belief state, and I wanted to avoid implying that the assertion is something we ought to be able to come to consensus about right now. And, more simply, it would take substantial work to explain the evidence for the hypothesis being true (in large part because I’d have to sort out my thoughts). For these reasons, my implied request is less like “let’s evaluate this hypothesis right now”, and more like “would you please file this hypothesis away in your head, and then if you’re in a long conversation, on the relevant topic with someone in the relevant category, maybe try holding up the hypothesis next to your observations and seeing if it explains things or not”.
In other words, it’s a request for more data and a request for someone to think through the hypothesis more. It’s far from perfectly neutral—if someone follows that request, they are spending their own computational resources and thereby extending some credit to me and/or to the hypothesis.
don’t see the downstream impacts of their choices,
This could be part of it… but I think a hypothesis that does have to be kept in mind is that some people don’t care. They aren’t trying to follow action-policies that lead to good outcomes, they’re doing something else. Primarily, acting on an addiction to Steam. If a recruitment strategy works, that’s a justification in and of itself, full stop. EA is good because it has power, more people in EA means more power to EA, therefore more people in EA is good. Given a choice between recruiting 2 agents and turning them both into zombies, vs recruiting 1 agent and keeping them an agent, you of course choose the first one--2 is more than 1.
The main difficulty, if there is one, is in “getting the function to play the role of the AGI values,” not in getting the AGI to compute the particular function we want in the first place.
Right, that is the problem (and IDK of anyone discussing this who says otherwise).
Another position would be that it’s probably easy to influence a few bits of the AI’s utility function, but not others. For example, it’s conceivable that, by doing capabilities research in different ways, you could increase the probability that the AGI is highly ambitious—e.g. tries to take over the whole lightcone, tries to acausally bargain, etc., rather than being more satisficy. (IDK how to do that, but plausibly it’s qualitatively easier than alignment.) Then you could claim that it’s half a bit more likely that you’ve made an FAI, given that an FAI would probably be ambitious. In this case, it does matter that the utility function is complex.
Here’s an argument that alignment is difficult which uses complexity of value as a subpoint:
-
A1. If you try to manually specify what you want, you fail.
-
A2. Therefore, you want something algorithmically complex.
-
B1. When humanity makes an AGI, the AGI will have gotten values via some process; that process induces some probability distribution over what values the AGI ends up with.
-
B2. We want to affect the values-distribution, somehow, so that it ends up with our values.
-
B3. We don’t understand how to affect the values-distribution toward something specific.
-
B4. If we don’t affect the value-distribution toward something specific, then the values-distribution probably puts large penalties for absolute algorithmic complexity; any specific utility function with higher absolute algorithmic complexity will be less likely to be the one that the AGI ends up with.
-
C1. Because of A2 (our values are algorithmically complex) and B4 (a complex utility function is unlikely to show up in an AGI without us skillfully intervening), an AGI is unlikely to have our values without us skillfully intervening.
-
C2. Because of B3 (we don’t know how to skillfully intervene on an AGI’s values) and C1, an AGI is unlikely to have our values.
I think that you think that the argument under discussion is something like:
-
(same) A1. If you try to manually specify what you want, you fail.
-
(same) A2. Therefore, you want something algorithmically complex.
-
(same) B1. When humanity makes an AGI, the AGI will have gotten values via some process; that process induces some probability distribution over what values the AGI ends up with.
-
(same) B2. We want to affect the values-distribution, somehow, so that it ends up with our values.
-
B′3. The greater the complexity of our values, the harder it is to point at our values.
-
B′4. The harder it is to point at our values, the more work or difficulty is involved in B2.
-
C′1. By B′3 and B′4: the greater the complexity of our values, the more work or difficulty is involved in B2 (determining the AGI’s values).
-
C′2. Because of A2 (our values are algorithmically complex) and C′1, it would take a lot of work to make an AGI pursue our values.
These are different arguments, which make use of the complexity of values in different ways. You dispute B′3 on the grounds that it can be easy to point at complex values. B′3 isn’t used in the first argument though.
-
I am quite interested in how (dangers from) cell division are different in the embryonic stage as compared to at a later stage.
I don’t know much about this, but two things (that don’t directly answer your question):
Generally, cells accumulate damage over time.
This happens both genetically and epigenetically. Genetically, damage accumulates (I think the main cause is cosmic rays hitting DNA that’s exposed for transcription and knocking nucleic acids out? Maybe also other copying errors?), so that adult somatic cells have (I think) several hundred new mutations that they weren’t born with. Epigenetically, I imagine that various markers that should be there get lost over time for some reason (I think this is a major hypothesis about the sort of mechanism behind various forms of aging).
This means that generally, ESCs are more healthy than adult somatic cells.
One major function of the reproductive system is to remove various forms of damage.
You can look up gametogenesis (oogenesis, spermatogenesis). Both processes are complicated, in that they involve many distinct steps, various checks of integrity (I think oocytes + their follicles are especially stringently checked?), and a lot of attrition (a fetus has several million oocytes; an adult woman ovulates at most a few hundred oocytes in her lifetime, without exogenous hormones as in IVF).
So, ESCs (from an actual embryo, rather than from some longer-term culture) will be heavily selected for genetic (and epigenetic?) integrity. Mutations that would have been severely damaging to development will have been weeded out. (Though there will also be many miscarriages.)
That’s a reasonable point… But I don’t think we can just count number of divisions either? For one thing, there are several populations of stem cells in an adult. For another, people who are 50% bigger than other people don’t live 2⁄3 as long (right? though maybe that’s not the prediction?). I think maybe embryonic stem cells protect their telomeres—not sure.
Wouldn’t it age them by at most 1 day (which is about how long mitosis takes)?
I’m not sure how well curated and indexed most information is.
Working memory allows for looking at the whole picture at once better with the full might of human intelligence (which is better at many things than LLMs), while removing frictions that come from delays and effort expended in search for data and making calculations.
How specifically would you use BCIs to improve this situation?
Curated reservoirs of practical and theoretical information, well indexed, would be very useful to super geniuses.
You don’t actually need to hook them up physically. Having multiple people working on different parts of a problem lets them all bounce ideas off each other.
But both of these things are basically available currently, so apparently our current level isn’t enough. LLMs + google (i.e. what Perplexity is trying to be) are already a pretty good index; what would a BCI add?
Overall: The goal should be to create a number of these people, then let them plan out the next round if their intelligence doesn’t do it.
I commented on a similar topic here: https://www.lesswrong.com/posts/jTiSWHKAtnyA723LE/overview-of-strong-human-intelligence-amplification-methods?commentId=uZg9s2FfP7E7TMTcD
That’s incorrect, but more importantly it’s off topic. The topic is “what does the complexity of value have to do with the difficulty of alignment”. Barnett AFAIK in this comment is not saying (though he might agree, and maybe he should be taken as saying so implicitly or something) “we have lots of ideas for getting an AI to care about some given values”. Rather he’s saying “if you have a simple pointer to our values, then the complexity of values no longer implies anything about the difficulty of alignment because values effectively aren’t complex anymore”.
Alice: I want to make a bovine stem cell that can be cultured at scale in vats to make meat-like tissue. I could use directed evolution. But in my alternate universe, genome sequencing costs $1 billion per genome, so I can’t straightforwardly select cells to amplify based on whether their genome looks culturable. Currently the only method I have is to do end-to-end testing: I take a cell line, I try to culture a great big batch, and then see if the result is good quality edible tissue, and see if the cell line can last for a year without mutating beyond repair. This is very expensive, but more importantly, it doesn’t work. I can select for cells that make somewhat more meat-like tissue; but when I do that, I also heavily select for other very bad traits, such as forming cancer-like growths. I estimate that it takes on the order of 500 alleles optimized relative to the wild type to get a cell that can be used for high-quality, culturable-at-scale edible tissue. Because that’s a large complex change, it won’t just happen by accident; something about our process for making the cells has to put those bits there.
Bob: In a recent paper, a polygenic score for culturable meat is given. Since we now have the relevant polygenic score, we actually have a short handle for the target: namely, a pointer to an implementation of this polygenic score as a computer program.
Alice: That seems of limited relevance. It’s definitely relevant in that, if I grant the premise that this is actually the right polygenic score (which I don’t), we now know what exactly we would put in the genome if we could. That’s one part of the problem solved, but it’s not the part I was talking about. I’m talking about the part where I don’t know how to steer the genome precisely enough to get anywhere complex.
Bob: You’ve been bringing up the complexity of the genomic target. I’m saying that actually the target isn’t that complex, because it’s just a function call to the PGS.
Alice: Ok, yes, we’ve greatly decreased the relative algorithmic complexity of the right genome, in some sense. It is indeed the case that if I ran a computer program randomly sampled from strings I could type into a python file, it would be far more likely to output the right genome if I have the PGS file on my computer compared to if I don’t. True. But that’s not very relevant because that’s not the process we’re discussing. We’re discussing the process that creates a cell with its genome, not the process that randomly samples computer programs weighted by [algorithmic complexity in the python language on my computer]. The problem is that I don’t know how to interface with the cell-creation process in a way that lets me push bits of selection into it. Instead, the cell-creation process just mostly does its own thing. Even if I do end-to-end phenotype selection, I’m not really steering the core process of cell-genome-selection.
Bob: I understand, but you were saying that the complexity of the target makes the whole task harder. Now that we have the PGS, the target is not very complex; we just point at the PGS.
Alice: The point about the complexity is to say that cells growing in my lab won’t just spontaneously start having the 500 alleles I want. I’d have to do something to them—I’d have to know how to pump selection power into them. It’s some specific technique I need to have but don’t have, for dealing with cells. It doesn’t matter that the random-program complexity has decreased, because we’re not talking about random programs, we’re talking about cell-genome-selection. Cell-genome-selection is the process where I don’t know how to consistently pump bits into, and it’s the process that doesn’t by chance get the 500 alleles. It’s the process against which I’m measuring complexity.
Protip: You can prevent itchy skin from being itchy for hours by running it under very hot water for 5-30 seconds. (Don’t burn yourself; I use tap water with some cold water, and turn down the cold water until it seems really hot.)
I don’t know enough to evaluate your claims, but more importantly, I can’t even just take your word for everything because I don’t actually know what you’re saying without asking a whole bunch of followup questions. So hopefully we can hash some of this out on the phone.
exogenous driving of a fraction of cortical tissue to result in suffering of the subjects
My reason is that suffering in general seems related to [intentions pushing hard, but with no traction or hope]. A subspecies of that is [multiple drives pushing hard against each other, with nobody pulling the rope sideways]. A new subspecies would be “I’m trying to get my brain tissue to do something, but it’s being externally driven, so I’m just scrabbling my hands futilely against a sheer blank cliff wall.” and “Bits of my mind are being shredded because I create them successfully by living and demanding stuff of my brain, but the the bits are exogenously driven / retrained and forget to do what I made them to do.”.
It is quite impractical. A weird last ditch effort to save the world. It wouldn’t be scalable, you’d be enhancing just a handful of volunteers who would then hopefully make rapid progress on alignment.
Gotcha. Yeah, I think these strategies probably just don’t work.
It seems less problematic to me than a single ordinary pig farm, since you’d be treating these pigs unusually well.
The moral differences are:
Humanized neurons.
Animals with parts of their brains being exogenously driven; this could cause large amounts of suffering.
Animals with humanized thinking patterns (which is part of how the scheme would be helpful in the first place).
Weird that you’d feel good about letting the world get destroyed in order to have one fewer pig farm in it.
Where did you get the impression that I’d feel good about, or choose, that? My list of considerations is a list of considerations.
That said, I think morality matters, and ignoring morality is a big red flag.
Separately, even if you’re pretending to be a ruthless consequentialist, you still want to track morality and ethics and ickyness, because it’s a very strong determiner of whether or not other people will want to work on something, which is a very strong determiner of success or failure.
Closeness is the operating drive, but it’s not the operating telos. The drive is towards some sort of state or feeling—of relating, standing shoulder-to-shoulder looking out at the world, standing back-to-back defending against the world; of knowing each other, of seeing the same things, of making the same meaning; of integrated seeing / thinking. But the telos is tikkun olam (repairing/correcting/reforming the world)--you can’t do that without a shared idea of better.
As an analogy, curiosity is a drive, which is towards confusion, revelation, analogy, memory; but the telos is truth and skill.
In your example, I would say that someone could be struggling with “moral responsibility” while also doing a bunch of research or taking a bunch of action to fix what needs to be fixed; or they could be struggling with “moral responsibility” while eating snacks and playing video games. Vibes are signals and signals are cheap and hacked.