I agree with this part too. But I’d add that the people who “control” AIs won’t necessarily be the people who build them.
I agree, I used the general term to avoid implying necessarily that OpenAI et. al. will get to decide, though I think the implicit goal of most AGI developers is to get as much control over the lightcone as possible and that deliberately working towards that particular goal counts for a lot.
I think the implicit goal of most AGI developers is to get as much control over the lightcone as possible and that deliberately working towards that particular goal counts for a lot.
That seems right. I’d broaden this claim a bit: most people in general, want to be rich, i.e. “get control over the lightcone”. People vary greatly in their degree of rapaciousness, and how hard they work to become rich, but to a first approximation, people really do care a lot about earning a high income. For example, most people are willing to work ~40 hours a week for ~40 years of their life even though a modern wage in a developed country is perfectly capable of sustaining life at a fraction of the cost in time.
I agree, I used the general term to avoid implying necessarily that OpenAI et. al. will get to decide, though I think the implicit goal of most AGI developers is to get as much control over the lightcone as possible and that deliberately working towards that particular goal counts for a lot.
That seems right. I’d broaden this claim a bit: most people in general, want to be rich, i.e. “get control over the lightcone”. People vary greatly in their degree of rapaciousness, and how hard they work to become rich, but to a first approximation, people really do care a lot about earning a high income. For example, most people are willing to work ~40 hours a week for ~40 years of their life even though a modern wage in a developed country is perfectly capable of sustaining life at a fraction of the cost in time.