Without making any normative arguments: if you’re in a position (industrially and technologically) to disassemble the sun at all, or build something like a Dyson swarm, then it’s probably not too difficult to build an artificial system to light the Earth in such a way as to mimic the sun, and make it look and feel nearly identical to biological humans living on the surface, using less than a billionth of the sun’s normal total light output. The details of tides might be tricky, but probably not out of reach.
But most people on Earth don’t want “an artificial system to light the Earth in such a way as to mimic the sun”, they want the actual sun to go on existing.
This point doesn’t make sense to me. It sounds similar to saying “Most people don’t like it when companies develop more dense housing in cities, therefore a good democracy should not have it” or “Most people don’t like it when their horse-drawn carriages are replaced by cars, therefore a good democracy should not have it”.
The cost-benefit calculations on these things work out and it’s good if most uninformed people who haven’t spent much time on it are not able to get in the way of companies that are building goods and services in this regard.
There are many many examples (e.g. GMOs, nuclear power, coal power, privatized toll roads, fracking, etc), and I expect if I researched for a few hours I would find even clearer examples for which it is currently consensus that it is a good idea, but at the time the majority disliked it.
More generally:
People’s mass preferences are sometimes dumb, and sometimes quite reasonable, and you should have a decision rule that distinguishes between the two, and when things are orders of magnitude more cost effective than other things, this is a good argument against arguments based on simple preference / aesthetics, and this comment does nothing to show that this isn’t stupidity rather than wisdom.
Just because a lot of people in a democracy disapproves of things does not mean that market forces shouldn’t be able to disagree with them and be correct about that. Analogous to the Luddites who had little concept of how technological and economic progress lifts everyone out of poverty, most people today do not appreciate that future computational-life forms will be just as meaningful as the meat-based ones today, and should not sacrifice orders of magnitudes more life-years than will be lived on Earth[1] for the difference between a big ball of plasma and something else that recreates the same quality of light.
Majoritarian vote on everything is a terrible way to make decisions; most decisions should be given to as small a group as possible (ideally an individual) who is held accountable for the outcome being good, and is given the resources to make the decision well. We do it for political leaders due to the low levels trust and high levels of adversarial action, but this should not be extended to whether to take the sun apart for parts.
I am claiming that people when informed will want the sun to continuing being the sun. I also think that most people when informed will not really care that much about creating new people, will continue to believe in the act-omission distinction, etc. And that this is a coherent view that will add up to a large set of people wanting things in the solar system to remain conservatively the same. I seperately claim that if this is true, then other people should just respect this preference, and use the other stars that people don’t care about for energy.
As I mentioned in the other thread, it seems right to me that some people will want the sun to continue being the sun, but my sense is that within the set of people who don’t want to leave the solar system, don’t want to be uploads, don’t want to be cryogenically shipped to other solar systems, or otherwise for some reason will have strong preferences over what happens with this specific solar system, this will be a much less important preference than using the sun for things that people care about more.
I think the majority of humans probably won’t want to be uploads, leave the solar system permanently, etc. Maybe this is where we disagree? I don’t really think there’s going to be a thing that most people care about more.
Sorry, that’s literally what I am saying. If many people don’t want to leave the solar system, and don’t want to be uploads, then using the matter and energy available in the solar system effectively is a decision with a huge stake to many people.
I think if everyone or really almost everyone would want to be an upload, I think this would make it more likely that we should keep the sun intact, because then the sun could belong to just the few humans who don’t have better alternatives in other solar systems. But if there is anything above 10% of humanity who don’t want to be uploaded, or go on long-distance spaceship journeys in their biological bodies, then you better make sure you make the solar system great for this substantial fraction of humanity, and I think that will likely involve disassembling the sun.
I agree with you that many people don’t want to be uploads, etc. I disagree that the majority of people who don’t want to be uploads have attachments to the specific celestial bodies in our solar system. I think they just want to have a good life in their biological bodies, doing nice human things. Those goals would be non-trivially hampered if they couldn’t disassemble the sun. That’s like 99.9% of the energy and matter by which they could achieve those goals, and while I do think this subset of the population will be selected for less scope-sensitivity, I think there will be enough scope-sensitivity to make leaving the sun intact a bad choice.
(To be clear, I disagree that the majority of humanity would not want to be uploads over the course of multiple generations, but it seems plausible to me that like 10%-20% of humanity don’t want to be uploads, even over multiple generations)
Uploads have 10,000x life expectancy due to running faster, regardless of what global circumstance eventually destroys them (I’m expecting distributed backups for biological humans as well, but by definition they remain much slower).
Analogously: “I am claiming that people when informed will want horses to continue being the primary mode of transportation. I also think that most people when informed will not really care that much about economic growth, will continue to believe that you’re more responsible for changing things than for maintaining the status quo, etc. And that this is a coherent view that will add up to a large set of people wanting things in cities to remain conservatively the same. I separately claim that if this is true, then other people should just respect this preference, and go find new continents / planets on which to build cars that people in the cities don’t care about.”
Sometimes it’s good to be conservative when you’re changing things, like if you’re changing lots of social norms or social institutions, but I don’t get it at all in this case. The sun is not a complicated social institution, it’s primarily a source of heat and light and much of what we need can be easily replicated especially when you have nanobots. I am much more likely to grant that we should be slow to change things like democracy and the legal system than I am that we should change exactly how and where we should get heat and light. Would you have wanted conservatism around moving from candles to lightbulbs? Installing heaters and cookers in the house instead of fire pits? I don’t think so.
I don’t think that’s a very good analogy, but I will say that is is basically true for the Amish. And I do think that we should respect their preferences. (I seperately think cars are not that good, and that people would infact prefer to bicycle around or ride house drawn carriges or whatever if civilization was conducive to that, although that’s kinda besides the point.)
I’m not arguing that we should be conservative about changing the sun. I’m just claiming that people like the sun and won’t want to see it eaten/fundamentally transformed, and that we should respect this preference. This is reason why it’s different from candles → lightbulbs, because people very obviously wanted lightbulbs when offered. But I don’t think the marginal increase in well-being from eating the sun will be nearly enough to make balance against the desire that the sun remain the same, so I don’t think most people will on net want the sun to be eaten. To be clear, this is an empirical claim about what people want that might very well be false.
I am not sure what point you are making with “respect their preferences”, I am not proposing one country go to war with other countries to take the sun. For instance, one way it might go down is someone will just offer to buy it from Earth, and the price will be many orders of magnitude more resources than Earth has, so Earth will accept, and replace it with an artificial source of light & heat.
I may be wrong about the estimates of the value of the energy, neither of us have specified how the rest of the stars in the universe will get distributed. For concreteness, I am here imagining something like: the universe is not a whole singleton but made of many separate enclaves that have their own governance and engage in trade with one another, and that Earth is a special one that keeps a lot of its lineage with present-day Earth, and is generally outcompeted by all the others ones that are smarter/faster and primarily run by computational-minds rather than biological ones.
I think I expect Earth in this case to just say no and not sell the sun? But I was confused at like 2 points in your paragraph so I don’t think I understand what you’re saying that well. I also think we’re probably on mostly the same page, and am not that interested in hashing out further potential disagreements.
Also, mostly unrelated, maybe a hot take, but if you’re able to get outcompeted because you don’t upload, then the future you’re in is not very good.
As I explain in more detail in my other comment, I expect market based approaches to not dismantle the Sun anytime soon. I’m interested if you know of any governance structure that you support that you think will probably lead to dismantling the Sun within the next few centuries.
most people today do not appreciate that future computational-life forms will be just as meaningful as the meat-based ones today, and should not sacrifice orders of magnitudes more life-years than will be lived on Earth for the difference between a big ball of plasma and something else that recreates the same quality of light.
huh i agree with this, but i’ve been imagining that earth and many galaxies will be in the domain of preferences which want to “live on actual planets” because of this chain-of-logic:
the lightcone is very much larger than just earth. (wikipedia says “[there are] an estimated 100 billion [galaxies] in all of the observable universe”). we’d want to give up earth (and some surrounding many galaxies) in return for more good possible futures, because of the good which can be derived from the non-earth parts of those ones.
some beings care disproportionately about what happens to earth and its sun. some of them are alignment researchers (e.g. some comments in this thread), or otherwise influencing the trajectory
it’s better for collaboration if those preferences determine the fate of earth and its sun. i.e. this prevents some values from being incentivized to compete for alignment to them in particular (to the detriment of general success rates).
i can write more about this part if wanted. notably (unless i’ve made a mistake) it doesn’t rely on trust, but that’s probably not very clear with just this.
actually, this case is less interesting than the general case i had in mind. because in this case one side has non-linearity in value of this sun versus others, both sides want to make this trade even without a greater chance of both dying if they don’t.
I agree that not all decisions about the cosmos should be made on a majoritarian democratic way, but I don’t see how replacing the Sun with artificial light can be done by market forces under normal property rights. I think you are currently would not be allowed to build a giant glass dome around someone’s pot of land, and this feels at least that strong.
I’m broadly sympathetic to having property rights and markets in the post-Singularity future, and probably the people will scope-sensitive and longtermist preferences will be able to buy out the future control of far-away things from the normal people who don’t care about these too much. But these trades will almost certainly result the solar system being owned by a coalition of normal people, except if they start with basically zero capital. I don’t know how you imagine the initial capital allocation to look like in your market-based post-Singularity world, but if the vast majority of the population doesn’t have enough control to even save the Sun, then probably something went deeply wrong.
People don’t generally have strong preferences about celestial objects. I really don’t understand why you think most people care about the sun qua the sun, as opposed to the things the sun provides.
Most people when faced with the choice to be more than twice as rich in new-earth, which they get to visualize and explore using the best of digital VR and sensory technology, with a fake sun indistinguishable for all intends and purposes from the real sun, will of course choose that over the attachment to maintaining that specific ball of plasma in the sky.
Side-note: Just registering that I personally aspire to always taboo ‘normal people’ and instead name to specific populations. I think it tends to sneak in a lot of assumptions to call people ‘normal’ – I’ve seen it used to mean “most people on Twitter” or “most people in developed countries” or “most working class people” or “most people alive today” – the latter of which is not at all normal by historical standards!
I expect non-positional material goods to be basically saturated for Earth people in a good post-Singularity world, so I don’t think you can promise them to become twice as rich. And also, people dislike drastic change and new things they don’t understand. 20% of the US population refused the potentially life-saving covid vaccine out of distrust of new things they don’t understand. Do you think they would happily move to a new planet with artificial sky maintained by supposedly benevolent robots? Maybe you could buy off some percentage of the population if material goods weren’t saturated, but surely not more than you could convince to get the vaccine? Also, don’t some religions (Islam?) have specific laws about what to do at sunrise and sunset and so on? Do you think all the imams would go along with moving to the new artificial Earth? I really think you are out of touch with the average person on this one, but we can go out to the streets and interview some people on the matter, though Berkeley is maybe not the most representative place for this.
(Again, if you are talking about cultural drift over millennia, that’s more plausible, though I’m below 50% they would dismantle the Sun. But I’m primarily arguing against dismantling the Sun within twenty years of the Singularity.)
Twenty years seems indeed probably too short, though it’s hard to say how post-singularity technology will affect things like public deliberation timelines.
My best guess is 200 years will very likely be enough.
I agree with you that there exist some small minority of people who will have a specific attachment to the sun, but most people just want to live good and fulfilling lives, and don’t have strong preferences about whether the sun in the sky is exactly 1 AU away and feels exactly like the sun of 3 generations past. Also, people will already experience extremely drastic change in the 20 years after the singularity, and my sense is marginal cost of change is decreasing, and this isn’t the kind of change that would most affect people’s lived experience.
To be clear, for me it’s a crux whether not dismantling the sun is basically committing everyone who doesn’t want to be uploaded to relative cosmic poverty. It would really suck if all remaining biological humans would be unable to take advantage of the vast majority of the energy in the solar system.
I am not at present compelled that the marginal galaxies are worth destroying the sun and earth for (though I am also not confident it isn’t, I feel confused about it, and also don’t know where most people would end up after having been made available post-singularity intelligence enhancing drugs and deliberation technologies, which to be clear not everyone would use, but most people probably would).
I maintain that biological humans will need to do population control at some point. If they decide that enacting the population control in the solar system at a later population leve is worth it for them to dismantle the Sun, then they can go for it. My guess is that they won’t, and will have population control earlier.
I want the Sun to keep existing, I am not “uninformed”, and I think it would be good if I am able to get in the way of people who want to dismantle the Sun, and bad if I were not able to do so.
when things are orders of magnitude more cost effective than other things, this is a good argument against arguments based on simple preference / aesthetics
I strongly disagree. This is not any kind of argument against arguments based on simple preferences / aesthetics, much less a good one. In fact, it’s not clear to me that there are any such arguments at all (except ones based on [within-agent] competing preferences / aesthetics).
Just because a lot of people in a democracy disapproves of things does not mean that market forces shouldn’t be able to disagree with them and be correct about that.
You are perhaps missing the point of democracy.
(Now, if your view is “actually democracy is bad, we ought to have some other system of government”, fair enough, but then you should say so explicitly.)
the Luddites who had little concept of how technological and economic progress lifts everyone out of poverty
The Luddites had a completely correct expectation about how “technological and economic progress” would put them, personally and collectively, out of jobs, which it in fact did. They were not “lifted out of poverty” by mechanization—they were driven into poverty by it.
future computational-life forms will be just as meaningful as the meat-based ones today
You neither have nor can have any certainty about this, or even high confidence. Neither is it relevant—future people do not exist; existing people do.
should not sacrifice orders of magnitudes more life-years than will be lived on Earth
Declining to create people is not analogous to destroying existing people. To claim otherwise is tendentious and misleading. There is no “sacrificing” involved in what we are discussing.
most decisions should be given to as small a group as possible (ideally an individual) who is held accountable for the outcome being good, and is given the resources to make the decision well
Decisions should “be given”—by whom? The people—or else your position is nonsense. Well, I say that we should not give decision-making power to people who will dismantle the Sun. You speak of being “held accountable”—once again, by whom? Surely, again: the people. And that means that the people may evaluate the decisions that the one has made. Well, I say we should evaluate the decision to dismantle the Sun, pre-emptively—and judge it unacceptable. (Why wait until after the fact, when it will be too late? How, indeed, could someone possibly be “held accountable” for dismantling the Sun, after the deed is done? Absurdity!)
I think that’s very probably true, yes. I’m not certain that will continue to be true indefinitely, or that it will or should continue to be the deciding factor for future decision making. I’m just pointing out that we’re actually discussing a small subset of a very large space of options, that there are ways of “eating the sun” that allow life to continue unaltered on Earth, and so on. TBH even if we don’t do anything like this, I wouldn’t be terribly surprised if future humans end up someday building a full or partial shell around Earth anyway, long before there’s anything like large-scale starlifting under discussion. Living space, power generation, asteroid deflection, off-world industry, just to name a few reasons we might do something like that. It could end up being easier and cheaper to increase available surface area and energy by orders of magnitude doing something like this than by colonizing other planets and moons.
(This seems false and in as much as someone is willing to take bets that resolve after superintelligence, I would bet that most people do not care any appreciable amount about the actual sun existing by the time humanity is capable of actually providing a real alternative)
Without making any normative arguments: if you’re in a position (industrially and technologically) to disassemble the sun at all, or build something like a Dyson swarm, then it’s probably not too difficult to build an artificial system to light the Earth in such a way as to mimic the sun, and make it look and feel nearly identical to biological humans living on the surface, using less than a billionth of the sun’s normal total light output. The details of tides might be tricky, but probably not out of reach.
But most people on Earth don’t want “an artificial system to light the Earth in such a way as to mimic the sun”, they want the actual sun to go on existing.
This point doesn’t make sense to me. It sounds similar to saying “Most people don’t like it when companies develop more dense housing in cities, therefore a good democracy should not have it” or “Most people don’t like it when their horse-drawn carriages are replaced by cars, therefore a good democracy should not have it”.
The cost-benefit calculations on these things work out and it’s good if most uninformed people who haven’t spent much time on it are not able to get in the way of companies that are building goods and services in this regard.
There are many many examples (e.g. GMOs, nuclear power, coal power, privatized toll roads, fracking, etc), and I expect if I researched for a few hours I would find even clearer examples for which it is currently consensus that it is a good idea, but at the time the majority disliked it.
More generally:
People’s mass preferences are sometimes dumb, and sometimes quite reasonable, and you should have a decision rule that distinguishes between the two, and when things are orders of magnitude more cost effective than other things, this is a good argument against arguments based on simple preference / aesthetics, and this comment does nothing to show that this isn’t stupidity rather than wisdom.
Just because a lot of people in a democracy disapproves of things does not mean that market forces shouldn’t be able to disagree with them and be correct about that. Analogous to the Luddites who had little concept of how technological and economic progress lifts everyone out of poverty, most people today do not appreciate that future computational-life forms will be just as meaningful as the meat-based ones today, and should not sacrifice orders of magnitudes more life-years than will be lived on Earth[1] for the difference between a big ball of plasma and something else that recreates the same quality of light.
Majoritarian vote on everything is a terrible way to make decisions; most decisions should be given to as small a group as possible (ideally an individual) who is held accountable for the outcome being good, and is given the resources to make the decision well. We do it for political leaders due to the low levels trust and high levels of adversarial action, but this should not be extended to whether to take the sun apart for parts.
I’d quickly guess that the energy difference supports 10n life-years where n is somewhere between 10 and 30.
I am claiming that people when informed will want the sun to continuing being the sun. I also think that most people when informed will not really care that much about creating new people, will continue to believe in the act-omission distinction, etc. And that this is a coherent view that will add up to a large set of people wanting things in the solar system to remain conservatively the same. I seperately claim that if this is true, then other people should just respect this preference, and use the other stars that people don’t care about for energy.
As I mentioned in the other thread, it seems right to me that some people will want the sun to continue being the sun, but my sense is that within the set of people who don’t want to leave the solar system, don’t want to be uploads, don’t want to be cryogenically shipped to other solar systems, or otherwise for some reason will have strong preferences over what happens with this specific solar system, this will be a much less important preference than using the sun for things that people care about more.
I think the majority of humans probably won’t want to be uploads, leave the solar system permanently, etc. Maybe this is where we disagree? I don’t really think there’s going to be a thing that most people care about more.
Sorry, that’s literally what I am saying. If many people don’t want to leave the solar system, and don’t want to be uploads, then using the matter and energy available in the solar system effectively is a decision with a huge stake to many people.
I think if everyone or really almost everyone would want to be an upload, I think this would make it more likely that we should keep the sun intact, because then the sun could belong to just the few humans who don’t have better alternatives in other solar systems. But if there is anything above 10% of humanity who don’t want to be uploaded, or go on long-distance spaceship journeys in their biological bodies, then you better make sure you make the solar system great for this substantial fraction of humanity, and I think that will likely involve disassembling the sun.
I agree with you that many people don’t want to be uploads, etc. I disagree that the majority of people who don’t want to be uploads have attachments to the specific celestial bodies in our solar system. I think they just want to have a good life in their biological bodies, doing nice human things. Those goals would be non-trivially hampered if they couldn’t disassemble the sun. That’s like 99.9% of the energy and matter by which they could achieve those goals, and while I do think this subset of the population will be selected for less scope-sensitivity, I think there will be enough scope-sensitivity to make leaving the sun intact a bad choice.
(To be clear, I disagree that the majority of humanity would not want to be uploads over the course of multiple generations, but it seems plausible to me that like 10%-20% of humanity don’t want to be uploads, even over multiple generations)
Cool. I misinterpreted your previous comment and think we’re basically on the same page.
Uploads have 10,000x life expectancy due to running faster, regardless of what global circumstance eventually destroys them (I’m expecting distributed backups for biological humans as well, but by definition they remain much slower).
Analogously: “I am claiming that people when informed will want horses to continue being the primary mode of transportation. I also think that most people when informed will not really care that much about economic growth, will continue to believe that you’re more responsible for changing things than for maintaining the status quo, etc. And that this is a coherent view that will add up to a large set of people wanting things in cities to remain conservatively the same. I separately claim that if this is true, then other people should just respect this preference, and go find new continents / planets on which to build cars that people in the cities don’t care about.”
Sometimes it’s good to be conservative when you’re changing things, like if you’re changing lots of social norms or social institutions, but I don’t get it at all in this case. The sun is not a complicated social institution, it’s primarily a source of heat and light and much of what we need can be easily replicated especially when you have nanobots. I am much more likely to grant that we should be slow to change things like democracy and the legal system than I am that we should change exactly how and where we should get heat and light. Would you have wanted conservatism around moving from candles to lightbulbs? Installing heaters and cookers in the house instead of fire pits? I don’t think so.
I don’t think that’s a very good analogy, but I will say that is is basically true for the Amish. And I do think that we should respect their preferences. (I seperately think cars are not that good, and that people would infact prefer to bicycle around or ride house drawn carriges or whatever if civilization was conducive to that, although that’s kinda besides the point.)
I’m not arguing that we should be conservative about changing the sun. I’m just claiming that people like the sun and won’t want to see it eaten/fundamentally transformed, and that we should respect this preference. This is reason why it’s different from candles → lightbulbs, because people very obviously wanted lightbulbs when offered. But I don’t think the marginal increase in well-being from eating the sun will be nearly enough to make balance against the desire that the sun remain the same, so I don’t think most people will on net want the sun to be eaten. To be clear, this is an empirical claim about what people want that might very well be false.
I am not sure what point you are making with “respect their preferences”, I am not proposing one country go to war with other countries to take the sun. For instance, one way it might go down is someone will just offer to buy it from Earth, and the price will be many orders of magnitude more resources than Earth has, so Earth will accept, and replace it with an artificial source of light & heat.
I may be wrong about the estimates of the value of the energy, neither of us have specified how the rest of the stars in the universe will get distributed. For concreteness, I am here imagining something like: the universe is not a whole singleton but made of many separate enclaves that have their own governance and engage in trade with one another, and that Earth is a special one that keeps a lot of its lineage with present-day Earth, and is generally outcompeted by all the others ones that are smarter/faster and primarily run by computational-minds rather than biological ones.
I think I expect Earth in this case to just say no and not sell the sun? But I was confused at like 2 points in your paragraph so I don’t think I understand what you’re saying that well. I also think we’re probably on mostly the same page, and am not that interested in hashing out further potential disagreements.
Also, mostly unrelated, maybe a hot take, but if you’re able to get outcompeted because you don’t upload, then the future you’re in is not very good.
As I explain in more detail in my other comment, I expect market based approaches to not dismantle the Sun anytime soon. I’m interested if you know of any governance structure that you support that you think will probably lead to dismantling the Sun within the next few centuries.
huh i agree with this, but i’ve been imagining that earth and many galaxies will be in the domain of preferences which want to “live on actual planets” because of this chain-of-logic:
the lightcone is very much larger than just earth. (wikipedia says “[there are] an estimated 100 billion [galaxies] in all of the observable universe”). we’d want to give up earth (and some surrounding many galaxies) in return for more good possible futures, because of the good which can be derived from the non-earth parts of those ones.
some beings care disproportionately about what happens to earth and its sun. some of them are alignment researchers (e.g. some comments in this thread), or otherwise influencing the trajectory
it’s better for collaboration if those preferences determine the fate of earth and its sun. i.e. this prevents some values from being incentivized to compete for alignment to them in particular (to the detriment of general success rates).
i can write more about this part if wanted. notably (unless i’ve made a mistake) it doesn’t rely on trust, but that’s probably not very clear with just this.
actually, this case is less interesting than the general case i had in mind. because in this case one side has non-linearity in value of this sun versus others, both sides want to make this trade even without a greater chance of both dying if they don’t.
I agree that not all decisions about the cosmos should be made on a majoritarian democratic way, but I don’t see how replacing the Sun with artificial light can be done by market forces under normal property rights. I think you are currently would not be allowed to build a giant glass dome around someone’s pot of land, and this feels at least that strong.
I’m broadly sympathetic to having property rights and markets in the post-Singularity future, and probably the people will scope-sensitive and longtermist preferences will be able to buy out the future control of far-away things from the normal people who don’t care about these too much. But these trades will almost certainly result the solar system being owned by a coalition of normal people, except if they start with basically zero capital. I don’t know how you imagine the initial capital allocation to look like in your market-based post-Singularity world, but if the vast majority of the population doesn’t have enough control to even save the Sun, then probably something went deeply wrong.
People don’t generally have strong preferences about celestial objects. I really don’t understand why you think most people care about the sun qua the sun, as opposed to the things the sun provides.
Most people when faced with the choice to be more than twice as rich in new-earth, which they get to visualize and explore using the best of digital VR and sensory technology, with a fake sun indistinguishable for all intends and purposes from the real sun, will of course choose that over the attachment to maintaining that specific ball of plasma in the sky.
Side-note: Just registering that I personally aspire to always taboo ‘normal people’ and instead name to specific populations. I think it tends to sneak in a lot of assumptions to call people ‘normal’ – I’ve seen it used to mean “most people on Twitter” or “most people in developed countries” or “most working class people” or “most people alive today” – the latter of which is not at all normal by historical standards!
Seems right, I used the language of the thread, but edited it since I agree.
I expect non-positional material goods to be basically saturated for Earth people in a good post-Singularity world, so I don’t think you can promise them to become twice as rich. And also, people dislike drastic change and new things they don’t understand. 20% of the US population refused the potentially life-saving covid vaccine out of distrust of new things they don’t understand. Do you think they would happily move to a new planet with artificial sky maintained by supposedly benevolent robots? Maybe you could buy off some percentage of the population if material goods weren’t saturated, but surely not more than you could convince to get the vaccine? Also, don’t some religions (Islam?) have specific laws about what to do at sunrise and sunset and so on? Do you think all the imams would go along with moving to the new artificial Earth? I really think you are out of touch with the average person on this one, but we can go out to the streets and interview some people on the matter, though Berkeley is maybe not the most representative place for this.
(Again, if you are talking about cultural drift over millennia, that’s more plausible, though I’m below 50% they would dismantle the Sun. But I’m primarily arguing against dismantling the Sun within twenty years of the Singularity.)
Twenty years seems indeed probably too short, though it’s hard to say how post-singularity technology will affect things like public deliberation timelines.
My best guess is 200 years will very likely be enough.
I agree with you that there exist some small minority of people who will have a specific attachment to the sun, but most people just want to live good and fulfilling lives, and don’t have strong preferences about whether the sun in the sky is exactly 1 AU away and feels exactly like the sun of 3 generations past. Also, people will already experience extremely drastic change in the 20 years after the singularity, and my sense is marginal cost of change is decreasing, and this isn’t the kind of change that would most affect people’s lived experience.
To be clear, for me it’s a crux whether not dismantling the sun is basically committing everyone who doesn’t want to be uploaded to relative cosmic poverty. It would really suck if all remaining biological humans would be unable to take advantage of the vast majority of the energy in the solar system.
I am not at present compelled that the marginal galaxies are worth destroying the sun and earth for (though I am also not confident it isn’t, I feel confused about it, and also don’t know where most people would end up after having been made available post-singularity intelligence enhancing drugs and deliberation technologies, which to be clear not everyone would use, but most people probably would).
I maintain that biological humans will need to do population control at some point. If they decide that enacting the population control in the solar system at a later population leve is worth it for them to dismantle the Sun, then they can go for it. My guess is that they won’t, and will have population control earlier.
I want the Sun to keep existing, I am not “uninformed”, and I think it would be good if I am able to get in the way of people who want to dismantle the Sun, and bad if I were not able to do so.
I strongly disagree. This is not any kind of argument against arguments based on simple preferences / aesthetics, much less a good one. In fact, it’s not clear to me that there are any such arguments at all (except ones based on [within-agent] competing preferences / aesthetics).
You are perhaps missing the point of democracy.
(Now, if your view is “actually democracy is bad, we ought to have some other system of government”, fair enough, but then you should say so explicitly.)
The Luddites had a completely correct expectation about how “technological and economic progress” would put them, personally and collectively, out of jobs, which it in fact did. They were not “lifted out of poverty” by mechanization—they were driven into poverty by it.
You neither have nor can have any certainty about this, or even high confidence. Neither is it relevant—future people do not exist; existing people do.
Declining to create people is not analogous to destroying existing people. To claim otherwise is tendentious and misleading. There is no “sacrificing” involved in what we are discussing.
Decisions should “be given”—by whom? The people—or else your position is nonsense. Well, I say that we should not give decision-making power to people who will dismantle the Sun. You speak of being “held accountable”—once again, by whom? Surely, again: the people. And that means that the people may evaluate the decisions that the one has made. Well, I say we should evaluate the decision to dismantle the Sun, pre-emptively—and judge it unacceptable. (Why wait until after the fact, when it will be too late? How, indeed, could someone possibly be “held accountable” for dismantling the Sun, after the deed is done? Absurdity!)
I think that’s very probably true, yes. I’m not certain that will continue to be true indefinitely, or that it will or should continue to be the deciding factor for future decision making. I’m just pointing out that we’re actually discussing a small subset of a very large space of options, that there are ways of “eating the sun” that allow life to continue unaltered on Earth, and so on. TBH even if we don’t do anything like this, I wouldn’t be terribly surprised if future humans end up someday building a full or partial shell around Earth anyway, long before there’s anything like large-scale starlifting under discussion. Living space, power generation, asteroid deflection, off-world industry, just to name a few reasons we might do something like that. It could end up being easier and cheaper to increase available surface area and energy by orders of magnitude doing something like this than by colonizing other planets and moons.
(This seems false and in as much as someone is willing to take bets that resolve after superintelligence, I would bet that most people do not care any appreciable amount about the actual sun existing by the time humanity is capable of actually providing a real alternative)