I’ll add to the above that I run this risk in the no-telepathy scenario as well. I run that risk every day in the real world.. I am not a well-designed intelligence with fixed values; I am a human being and my brain experiences value drift in response to various inputs. This is perhaps a bad thing, but it’s nevertheless true.
Yes, the risk is intensified as the number of interactions increase, and as the bandwidth of those interactions increases… it’s easier to preserve the values I had as a child if I live in a small town with people who mostly share my values than if I move to a large heterogenous city, and the risk would intensify still further in a scenario like a telepathic collective.
But I choose to live in a heterogenous city rather than a homogenous small town. I choose to read blogs written by people who don’t share my values. Why would I not choose to live in a telepathic collective if that option were available? Is there some absolute threshold of acceptable risk of value drift I should avoid crossing?
I’ll add to the above that I run this risk in the no-telepathy scenario as well. I run that risk every day in the real world.. I am not a well-designed intelligence with fixed values; I am a human being and my brain experiences value drift in response to various inputs. This is perhaps a bad thing, but it’s nevertheless true.
Yes, the risk is intensified as the number of interactions increase, and as the bandwidth of those interactions increases… it’s easier to preserve the values I had as a child if I live in a small town with people who mostly share my values than if I move to a large heterogenous city, and the risk would intensify still further in a scenario like a telepathic collective.
But I choose to live in a heterogenous city rather than a homogenous small town. I choose to read blogs written by people who don’t share my values. Why would I not choose to live in a telepathic collective if that option were available? Is there some absolute threshold of acceptable risk of value drift I should avoid crossing?