I just want to say that I appreciate this post, and especially the “What it might look like if this gap matters” sections. They were super useful for contextualizing the more abstract arguments, and I often found myself scrolling down to read them before actually reading the corresponding section.
Insub
I’ll definitely agree that most people seem to prefer having their own kids to adopting kids. But is this really demonstrating an intrinsic desire to preserve our actual physical genes, or is it more just a generic desire to “feel like your kids are really yours”?
I think we can distinguish between these cases with a thought experiment: Imagine that genetic engineering techniques become available that give high IQs, strength, height, etc., and that prevent most genetic diseases. But, in order to implement these techniques, lots and lots of genes must be modified. Would parents want to use these techniques?
I myself certainly would, even though I am one of the people who would prefer to have my own kids vs adoption. For me, it seems that the genes themselves are not actually the reason I want my own kids. As long as I feel like the kids are “something I created”, or “really mine”, that’s enough to satisfy my natural tendencies. I suspect that most parents would feel similarly.
More specifically, I think what parents care about is that their kids kind of look like them, share some of their personality traits, “have their mother’s eyes”, etc. But I don’t think that anyone really cares how those things are implemented.
So basically you admit that humans are currently an enormous success according to inclusive fitness, but at some point this will change—because in the future everyone will upload and humanity will go extinct
Not quite—I take issue with the certainty of the word “will” and with the “because” clause in your quote. I would reword your statement the following way:
“Humans are currently an enormous success according to inclusive fitness, but at some point this may change, due to any number of possible reasons which all stem from the fact that humans do not explicitly care about / optimize for our genes”
Uploading is one example of how humans could become misaligned with genetic fitness, but there are plenty of other ways too. We could get really good at genetic engineering and massively reshape the human genome, leaving only very little of Evolution’s original design. Or we could accidentally introduce a technology that causes all humans to go extinct (nuclear war, AI, engineered pandemic).
(Side note: The whole point of being worried about misalignment is that it’s hard to tell in advance exactly how the misalignment is going to manifest. If you knew in advance how it was going to manifest, you could just add a quick fix onto your agent’s utility function, e.g. “and by the way also assign very low utility to uploading”. But I don’t think a quick fix like this is actually very helpful, because as long as the system is not explicitly optimizing for what you want it to, it’s always possible to find other ways the system’s behavior might not be what you want)
My point is that I’m not confident that humans will always be aligned with genetic fitness. So far, giving humans intelligence has seemed like Evolution’s greatest idea yet. If we were explicitly using our intelligence to maximize our genes’ prevalence, then that would probably always remain true. But instead we do things like create weapons arsenals that actually pose a significant risk to the continued existence of our genes. This is not what a well-aligned intelligence that is robust to future capability gains looks like.
I see your point, and I think it’s true right at this moment, but what if humans just haven’t yet taken the treacherous turn?
Say that humans figure out brain uploading, and it turns out that brain uploading does not require explicitly encoding genes/DNA, and humans collectively decide that uploading is better than remaining in our physical bodies, and so we all upload ourselves and begin reproducing digitally instead of thru genes. There is a sense in which we have just destroyed all value in the world, from the anthropomorphized Evolution’s perspective.
If we say that “evolutions goal” is to maximize the number of human genes that exist, then it has NOT done a good job at aligning humans in the limit as human capabilities go to infinity. We just havent reached the point yet where “humans following our own desires” starts to diverge with evolution’s goals. But given that humans do not care about our genes implicitly, there’s a good chance that such a point will come eventually.
Thanks for coming today, everyone! For anyone who is interested in starting a regular Princeton meetup group / email list / discord, shoot me an email at dskumpf@gmail.com, and I’ll set something up!
Princeton, NJ – ACX Meetups Everywhere 2022
I agree. I find myself in an epistemic state somewhat like: “I see some good arguments for X. I can’t think of any particular counter-argument that makes me confident that X is false. If X is true, it implies there are high-value ways of spending my time that I am not currently doing. Plenty of smart people I know/read believe X; but plenty do not”
It sounds like that should maybe be enough to coax me into taking action about X. But the problem is that I don’t think it’s that hard to put me in this kind of epistemic state. Eg, if I were to read the right blogs, I think I could be brought into that state for a bunch of different values of X. A few of the top of my head that seem plausible:
Climate change
Monetary policy/hyperinflation
Animal suffering
So I don’t feel super trusting of my epistemic state. I guess I feel a sort of epistemic learned helplessness, where I am suspicious of smart bloggers’ ability to get me to think an issue is important and worth dedicating my life to.
Not totally sure how to resolve this, though I suppose it would involve some sort of “going off on my own and actually thinking deeply about what it should take to convince me”
A few more instances of cheap screening of large numbers:
I’ve seen people complain about google-style technical interviews, because implementing quicksort in real-time is probably not indicative of what you’ll be doing as a software engineer on the job. But google has enough applicants that it doesn’t matter if the test is noisy; some genuinely good candidates may fail the test, but there are enough candidates that it’s more efficient to just test someone else than to spend more time evaluating any one candidate
Attractive women on dating apps. A man’s dating profile is a noisy signal of his value as a partner, but it’s extremely cheap for an attractive woman to just “swipe left” and try the next one. This strategy will certainly pass up people who would have made good partners, but the cost of evaluating a new profile is so low that it makes sense to just ignore any profiles that aren’t obviously great
I’ll offer up my own fasting advice as well:
I (and the couple of people I know who have also experimented with fasting) have found it to be a highly trainable skill. Doing a raw 36-hour fast after never having fasted before may be miserable; but doing the same fast after two weeks of 16-8 intermittent fasting will probably be no big deal.
Before I started intermittent fasting, I’d done a few 30-hour fasts, and all of them got very difficult towards the end. I would get headaches, feel very fatigued, and not really be able to function from hours 22-30. When I started IF, the first week was quite tough. I’d have similar symptoms as the fasting window was ending: headaches, trouble focusing. But then right around the two week mark, things changed. The symptoms went away, and the hunger became a much more “passive” feeling. Rather than hunger directly causing discomfort, the hunger now feels more like a “notification”. Just my body saying “hey, just so you know, we haven’t eaten for a while”, rather than it saying “you’re going to die if you don’t eat right this moment”. This change has been persistent, even during periods where I’ve stopped IF.
Both of the two others I’ve seen try IF have reported something similar, that the first few weeks are tough, but then the character of hunger itself starts to change. Today, I can go 24 hours without eating fairly trivially, ie without much distraction or performance decreases from hunger.
Going 36 will still be a challenge, but some pre-training may make it easier! Of course you may be specifically trying to test your willpower, in which case making it easier may be counter productive. Either way, this seems like a cool idea for a secular holiday. Best of luck!
I’m in a similar place, and had the exact same thought when I looked at the 80k guide.
Yes that was my reasoning too. The situation presumably goes:
Omicron chooses a random number X, either prime or composite
Omega simulates you, makes its prediction, and decides whether X’s primality is consistent with its prediction
If it is, then:
Omega puts X into the box
Omega teleports you into the room with the boxes and has you make your choice
If it’s not, then...? I think the correct solution depends on what Omega does in this case.
Maybe it just quietly waits until tomorrow and tries again? In which case no one is ever shown a case where the box does not contain Omicron’s number. If this is how Omega is acting, then I think you can act as though your choice affects Omircon’s number, even though that number is technically random on this particular day.
Maybe it just picks its own number, and shows you the problem anyway. I believe this was the assumption in the post.
I remember hearing from what I thought was multiple sources that your run-of-the-mill PCR test had something like a 50-80% sensitivity, and therefore a pretty bad bayes factor for negative tests. But that doesnt seem to square with these results—any idea what Im thinking of?
I agree. It makes me really uncomfortable to think that while Hell doesn’t exist today, we might one day have the technology to create it.
I’m disappointed that a cooperative solution was not reached
I think you would have had to make the total cooperation payoff greater than the total one-side-defects payoff in order to get cooperation as the final result. From a “maximize money to charity” standpoint, defection seems like the best outcome here (I also really like the “pre-commit to flip a coin and nuke” solution). You’d have to believe that the expected utility/$ of the “enemy” charity is less than 1⁄2 of the expected utility/$ of yours; otherwise, you’d be happier with the enemy side defecting than with cooperation. I personally wouldn’t be that confident about the difference between AMF and MIRI.
For those of us who don’t have time to listen to the podcasts, can you give a quick summary of which particular pieces of evidence are strong? I’ve mostly been ignoring the UFO situation due to low priors. Relatedly, when you say the evidence is strong, do you mean that the posterior probability is high? Or just that the evidence causes you to update towards there being aliens? Ie, is the evidence sufficient to outweigh the low priors/complexity penalties that the alien hypothesis seems to have?
FWIW, my current view is something like:
I’ve seen plenty of videos of UFOs that seemed weird at first that turned out to have a totally normal explanation. So I treat “video looks weird” as somewhat weak Bayesian evidence.
As for complexity penalties: If there were aliens, it would have to be explained why they mostly-but-not-always hide themselves. I don’t think it would be incompetence, if they’re the type of civilization that can travel stellar distances.
It would also have to be explained why we haven’t seen evidence of their (presumably pretty advanced) civilization
And it would have to be explained why there hasn’t been any real knock-down evidence, eg HD close-up footage of an obviously alien ship (unless this is the type of evidence you’re referring to?). A bunch of inconclusive, non-repeatable, low-quality data seems to be much more likely in the world where UFOs are not aliens. Essentially there’s a selection effect where any sufficiently weird video will be taken as an example of a UFO. It’s easier for a low-quality video to be weird, because the natural explanations are masked by the low quality. So the set of weird videos will include more low-quality data sources than the overall ratio of existing high/low quality sources would indicate. Whereas, if the weird stuff really did exist, you’d think the incidence of weird videos would match the distribution of high/low quality sources (which I don’t think it does? as video tech has improved, have we seen corresponding improvements in average quality of UFO videos?).
I really like this post for two reasons:
I’ve noticed that when I ask someone “why do you believe X”, they often think that I’m asking them to cite sources or studies or some such. This can put people on the defensive, since we usually don’t have ready-made citations in our heads for every belief. But that’s not what I’m trying to ask; I’m really just trying to understand what process actually caused them to believe X, as a matter of historical fact. That process could be “all the podcasters I listen to take X as a given”, or “my general life experience/intuition has shown X to be true”. You’ve put this concept into words here and solidified the idea for me: that it’s helpful to communicate why you actually believe something, and let others do with that what they will.
The point about uncertainty is really interesting. I’d never realized before that if you present your conclusion first, and then the evidence for it, then it sure looks like you already had that hypothesis for some reason before getting a bunch of confirming evidence. Which implies that you have some sort of evidence/intuition that led you to the hypothesis in addition to the evidence you’re currently presenting.
I’ve wondered why I enjoy reading Scott Alexander so much, and I think that the points you bring up here are a big reason why. He explains his processes really well, and I usually end up feeling that I understand what actually caused him to believe his conclusions.
In a similar vein, there’s a bunch of symphony of science videos. These are basically remixes of random quotes by various scientists, roughly grouped by topic into a bunch of songs.
If, on the other hand, heritability is high, then throwing more effort/money at how we do education currently should not be expected to improve SAT scores
I agree with spkoc that this conclusion doesn’t necessarily follow from high heritability. I think it would follow from high and stable heritability across multiple attempted interventions.
An exaggerated story for the point I’m about to make: imagine you’ve never tried to improve SAT scores, and you measure the heritability. You find that, in this particular environment, genetic variance explains 100% of SAT scores. You can predict someone’s SAT scores perfectly just by looking at their genome. You decide to take the half of the population with the highest predicted scores, and keep the SAT a secret from them until the day they take the test. And for the lower half, you give them dedicated tutors to help them prepare. Given the 100% heritability, you expect scores to stay exactly the same. But wait! What no one told you was that before your intervention, the learning environment had been magically uniform for every student. There had been no environmental variance at all, and so the only thing left to explain test scores was genetics. What you didn’t realize is that your heritability estimate gave you no information at all about how environmental changes would affect scores—because there was no environmental variance at all!
A single heritability measurement only tells you, roughly, the ratio of “[existing environmental variance] times [sensitivity to environmental variance]” to “[existing genetic variance] times [sensitivity to genetic variance]”. But it doesn’t do anything to disentangle the sensitivities-to-variances from the actual variances. What if there’s practically zero variance in the environment, but a high sensitivity of the trait you’re looking at to environmental variance? You’d find heritability is very high, but changes to the environment will cause large decreases to heritability. Same thing with genes: what if your trait is 100% determined by genes, but it just so happens that everyone has the exact same genes? You’d find that genetic variance explains zero percent of your trait, but if you then tried some genetic engineering, you’d find heritability shoot upward.
In order to disentangle the “sensitivity of X to environmental variance” from “the level of environmental variance”, you’d have to run multiple interventions over time, and measure the heritability of X after each one (or be confident that your existing environment has lots of variance).
People get fat eating fruits
Are you implying that there are examples of people like BDay mentioned, who are obese despite only eating fruits/nuts/meat/veggies? Or just that people can get fat while including fruit in the diet? I’d be surprised and intrigued if it were the former.
I’ve tried the whole foods diet, and I’ve personally found it surprisingly hard to overeat, even when I let myself eat as many fruits and nuts as I want. You can only eat so many cashews before they start to feel significantly less appetizing. And after I’ve eaten 500 cal of cashews in one sitting, the next time I’m feeling snacky, those cashews still sound kinda meh. Fruit is certainly easier to eat, but still after the fourth or fifth clementine I feel like “ok that’s enough” (and that’s probably only ~300 calories). Whereas I could easily eat 500 cal of candy without breaking a sweat.
I think one major roadblock to overeating with fruit is that it takes effort to eat. You have to peel an orange, or cut up a kiwi or melon, or bite off the green part of a strawberry. There’s a lot more work involved in eating 500 cal of fruit than there is in unwrapping a candy bar or opening a party size bag of chips.
So all of this rambling is just to say that I’m somewhat skeptical of the claims that “fruit (nuts) are mostly sugar (fat) and are calorie dense, and you can overeat them just like you can with junk food”. I think it’s surprisingly hard in practice to do so (and it’s much less enjoyable than overeating junk food).
Well, really every second that you remain alive is a little bit of bayesian evidence for quantum immortality: the likelihood of death during that second according to quantum immortality is ~0, whereas the likelihood of death if quantum immortality is false is >0. So there is a skewed likelihood ratio in favor of quantum immortality each time you survive one extra second (though of course the bayesian update is very small until you get pretty old, because both hypotheses assign very low probaility to death when young)