This wades deep into the problem of what makes something feel conscious. I believe (and Scott Aaronson also wrote about it), that to have such a detailed understanding of a consciousness, one must also have a consciousness-generating process in it. That is, to fully understand a mind, it’s necessary to recreate the mind.
If the Earring merely does the most satisfactory decisions according to some easy-to-compute universal standards (like to act morally according to some computationally efficient system), then the takeover makes sense to me, but otherwise it seems like a refusal to admit multiple realizations of a mind.
Part of the story is that !> it tells you you are better off taking it off. Given that it’s always as good/better than you at making decisions, leaving it on is a bad idea. <!
it seems like a refusal to admit multiple realizations of a mind.
I think it admits the possibility that such a thing may be to your detriment. (Perhaps it only contains one model (a human mind?), and uses that knowledge to destroy, rather than upload, human minds.)
After reading the story, I don’t believe that it is a bad idea to leave on the earring, and I just think the author made an inconsistency in the story.
This wades deep into the problem of what makes something feel conscious. I believe (and Scott Aaronson also wrote about it), that to have such a detailed understanding of a consciousness, one must also have a consciousness-generating process in it. That is, to fully understand a mind, it’s necessary to recreate the mind.
If the Earring merely does the most satisfactory decisions according to some easy-to-compute universal standards (like to act morally according to some computationally efficient system), then the takeover makes sense to me, but otherwise it seems like a refusal to admit multiple realizations of a mind.
Part of the story is that !> it tells you you are better off taking it off. Given that it’s always as good/better than you at making decisions, leaving it on is a bad idea. <!
I think it admits the possibility that such a thing may be to your detriment. (Perhaps it only contains one model (a human mind?), and uses that knowledge to destroy, rather than upload, human minds.)
EDIT: How does one add spoilers here?
Re spoiler tags: https://www.lesswrong.com/posts/xWrihbjp2a46KBTDe/editor-mini-guide
After reading the story, I don’t believe that it is a bad idea to leave on the earring, and I just think the author made an inconsistency in the story.