6. Spreading throughout the galaxy would certainly be harder if nothing like mind uploading (which I’ll discuss in a future piece, and which is part of why I think future space settlements could have “value lock-in” as discussed above) can ever be done. I would find a view that “mind uploading is impossible” to be “wild” in its own way, because it implies that human brains are so special that there is simply no way, ever, to digitally replicate what they’re doing. (Thanks to David Roodman for this point.)
4. A logarithmic version doesn’t look any less weird, because the distances between the “middle” milestones are tiny compared to both the stretches of time before and after these milestones. More fundamentally, I’m talking about how remarkable it is to be in the most important [small number] of years out of [big number] of years—that’s best displayed using a linear axis. It’s often the case that weird-looking charts look more reasonable with logarithmic axes, but in this case I think the chart looks weird because the situation is weird. Probably the least weird-looking version of this chart would have the x-axis be something like the logged distance from the year 2100, but that would be a heck of a premise for a chart—it would basically bake in my argument that this appears to be a very special time period.
2. If we are able to create mind uploads, or detailed computer simulations of people that are as conscious as we are, it could be possible to put them in virtual environments that automatically reset, or otherwise “correct” the environment, whenever the society would otherwise change in certain ways (for example, if a certain religion became dominant or lost dominance). This could give the designers of these “virtual environments” the ability to “lock in” particular religions, rulers, etc. I’ll discuss this more in a future piece.
3. I’ve focused on the “galaxy” somewhat arbitrarily. Spreading throughout all of the accessible universe would take a lot longer than spreading throughout the galaxy, and until we do it’s still imaginable that some species from outside our galaxy will disrupt the “stable galaxy-scale civilization,” but I think accounting for this correctly would add a fair amount of complexity without changing the big picture. I may address that in some future piece, though.
5. This is exactly the kind of thought that kept me skeptical for many years of the arguments I’ll be laying out in the rest of this series about the potential impacts, and timing, of advanced technologies. Grappling directly with how “wild” our situation seems to ~undeniably be has been key for me.
Footnotes Container
This comment is a container for our temporary “footnotes-as-comments” implementation that gives us hover-over-footnotes.
6. Spreading throughout the galaxy would certainly be harder if nothing like mind uploading (which I’ll discuss in a future piece, and which is part of why I think future space settlements could have “value lock-in” as discussed above) can ever be done. I would find a view that “mind uploading is impossible” to be “wild” in its own way, because it implies that human brains are so special that there is simply no way, ever, to digitally replicate what they’re doing. (Thanks to David Roodman for this point.)
1. or Kardashev Type III.
4. A logarithmic version doesn’t look any less weird, because the distances between the “middle” milestones are tiny compared to both the stretches of time before and after these milestones. More fundamentally, I’m talking about how remarkable it is to be in the most important [small number] of years out of [big number] of years—that’s best displayed using a linear axis. It’s often the case that weird-looking charts look more reasonable with logarithmic axes, but in this case I think the chart looks weird because the situation is weird. Probably the least weird-looking version of this chart would have the x-axis be something like the logged distance from the year 2100, but that would be a heck of a premise for a chart—it would basically bake in my argument that this appears to be a very special time period.
2. If we are able to create mind uploads, or detailed computer simulations of people that are as conscious as we are, it could be possible to put them in virtual environments that automatically reset, or otherwise “correct” the environment, whenever the society would otherwise change in certain ways (for example, if a certain religion became dominant or lost dominance). This could give the designers of these “virtual environments” the ability to “lock in” particular religions, rulers, etc. I’ll discuss this more in a future piece.
3. I’ve focused on the “galaxy” somewhat arbitrarily. Spreading throughout all of the accessible universe would take a lot longer than spreading throughout the galaxy, and until we do it’s still imaginable that some species from outside our galaxy will disrupt the “stable galaxy-scale civilization,” but I think accounting for this correctly would add a fair amount of complexity without changing the big picture. I may address that in some future piece, though.
5. This is exactly the kind of thought that kept me skeptical for many years of the arguments I’ll be laying out in the rest of this series about the potential impacts, and timing, of advanced technologies. Grappling directly with how “wild” our situation seems to ~undeniably be has been key for me.
7. That is, advanced AI that pursues objectives of its own, which aren’t compatible with human existence. I’ll be writing more about this idea. Existing discussions of it include the books Superintelligence, Human Compatible, Life 3.0, and The Alignment Problem. The shortest, most accessible presentation I know of is The case for taking AI seriously as a threat to humanity (Vox article by Kelsey Piper). This report on existential risk from power-seeking AI, by Open Philanthropy’s Joe Carlsmith, lays out a detailed set of premises that would collectively imply the problem is a serious one.
9. See https://arxiv.org/pdf/1806.02404.pdf
8. Thanks to Carl Shulman for this point.
I’m impressed with your footnotes container.