the earring provides a kind of slow mind-uploading
This is only true if whatever (hyper)computation the earring is using to make recommendations contains a model of the wearer. Such a model could be interpreted as a true upload, in which case it would be true that the wearer’s mind is not actually destroyed.
However, if the earring’s predictions are made by some other means (which I don’t think is impossible even in real life—predictions are often made without consulting a detailed, one-to-one model of the thing being predicted), then there is no upload, and the user has simply been taken over like a mindless puppet.
This wades deep into the problem of what makes something feel conscious. I believe (and Scott Aaronson also wrote about it), that to have such a detailed understanding of a consciousness, one must also have a consciousness-generating process in it. That is, to fully understand a mind, it’s necessary to recreate the mind.
If the Earring merely does the most satisfactory decisions according to some easy-to-compute universal standards (like to act morally according to some computationally efficient system), then the takeover makes sense to me, but otherwise it seems like a refusal to admit multiple realizations of a mind.
Part of the story is that !> it tells you you are better off taking it off. Given that it’s always as good/better than you at making decisions, leaving it on is a bad idea. <!
it seems like a refusal to admit multiple realizations of a mind.
I think it admits the possibility that such a thing may be to your detriment. (Perhaps it only contains one model (a human mind?), and uses that knowledge to destroy, rather than upload, human minds.)
After reading the story, I don’t believe that it is a bad idea to leave on the earring, and I just think the author made an inconsistency in the story.
This is only true if whatever (hyper)computation the earring is using to make recommendations contains a model of the wearer. Such a model could be interpreted as a true upload, in which case it would be true that the wearer’s mind is not actually destroyed.
However, if the earring’s predictions are made by some other means (which I don’t think is impossible even in real life—predictions are often made without consulting a detailed, one-to-one model of the thing being predicted), then there is no upload, and the user has simply been taken over like a mindless puppet.
This wades deep into the problem of what makes something feel conscious. I believe (and Scott Aaronson also wrote about it), that to have such a detailed understanding of a consciousness, one must also have a consciousness-generating process in it. That is, to fully understand a mind, it’s necessary to recreate the mind.
If the Earring merely does the most satisfactory decisions according to some easy-to-compute universal standards (like to act morally according to some computationally efficient system), then the takeover makes sense to me, but otherwise it seems like a refusal to admit multiple realizations of a mind.
Part of the story is that !> it tells you you are better off taking it off. Given that it’s always as good/better than you at making decisions, leaving it on is a bad idea. <!
I think it admits the possibility that such a thing may be to your detriment. (Perhaps it only contains one model (a human mind?), and uses that knowledge to destroy, rather than upload, human minds.)
EDIT: How does one add spoilers here?
Re spoiler tags: https://www.lesswrong.com/posts/xWrihbjp2a46KBTDe/editor-mini-guide
After reading the story, I don’t believe that it is a bad idea to leave on the earring, and I just think the author made an inconsistency in the story.