I’ve been skimming some of my setting-idea notes, such as ‘algorithms replacing middle-managers’ and have realized that, for a certain point of the planned setting, you’ve highlighted an approach that is likely to be common among many other people. However, one of the main reasons for my protagonist’s choice to try relying on himselves is that AIs which optimize for various easy-to-check metrics, such as profitability, tend not to take into account that human values are complex.
So there are likely going to be all manner of hyper-efficient, software-managed organizations who, in a straight fight, could out-organize my protagonist’s little personal co-op. Various copies of my protagonist, seeing the data, will conclude that the costs are worth the benefits, and leave the co-op to gain the benefits of said organizational methods. However, this will cause a sort of social ‘evaporative cooling’, so that the copies who remain in the co-op will tend to be the ones most dedicated to working towards the full complexity of their values. As long as they can avoid going completely bankrupt—in other words, as long as there’s enough income to pay for the hardware to run at least one copy that remains a member—then the co-op will be able to quietly chug along doing its own thing while wider society changes in various values-simplifying ways around it.
… That is, if I can do everything that such a story needs to get done right.
I’ve been skimming some of my setting-idea notes, such as ‘algorithms replacing middle-managers’ and have realized that, for a certain point of the planned setting, you’ve highlighted an approach that is likely to be common among many other people. However, one of the main reasons for my protagonist’s choice to try relying on himselves is that AIs which optimize for various easy-to-check metrics, such as profitability, tend not to take into account that human values are complex.
So there are likely going to be all manner of hyper-efficient, software-managed organizations who, in a straight fight, could out-organize my protagonist’s little personal co-op. Various copies of my protagonist, seeing the data, will conclude that the costs are worth the benefits, and leave the co-op to gain the benefits of said organizational methods. However, this will cause a sort of social ‘evaporative cooling’, so that the copies who remain in the co-op will tend to be the ones most dedicated to working towards the full complexity of their values. As long as they can avoid going completely bankrupt—in other words, as long as there’s enough income to pay for the hardware to run at least one copy that remains a member—then the co-op will be able to quietly chug along doing its own thing while wider society changes in various values-simplifying ways around it.
… That is, if I can do everything that such a story needs to get done right.