If I were evil, I would have people repeat Epiphany’s slogan and they would think they were practicing dutiful nonconformity while actually their brain was thinking “Go Eliezer” all the time...
Or you might pretend to be good, and push on us in nefarious ways we wouldn’t even notice so that we end up saying things like “Go Eliezer”.
How would you tell the difference between him pretending to be good and being good, whatever you mean by good? (This is the basic question of rationality.)
That would be very hard to do in a public forum where he could read our methods of distinguish between him pretending to be good and actually being good. You could try to figure out what unconscious signals of “goodness” are, but again that’s hard through text where the person in question knows what you’re testing and can optimize the writing to score well on the test.
Or you could just go with your priors. Or message privately, though that runs into the problem of sampling people who are likely to agree with you.
I’ve heard you are in the possession of fully general mind-hacks.
If so he must have been using them on us in order to make his argument that such a thing is not theoretically possible so compelling. ie. Eliezer needs to be operating from outside our physical reality in order to have fully general mind-hacks and in that case the mind hacks are the least of our worries. (Unless that is what he wants us to think.)
If I were evil, I would have people repeat Epiphany’s slogan and they would think they were practicing dutiful nonconformity while actually their brain was thinking “Go Eliezer” all the time...
Or you might pretend to be good, and push on us in nefarious ways we wouldn’t even notice so that we end up saying things like “Go Eliezer”.
I’ve heard you are in the possession of fully general mind-hacks. Just like painting asteroids and poisoning tigers, right?
Go Eliezer! Save us from the Forces of Bad. (ha ha only serious).
How would you tell the difference between him pretending to be good and being good, whatever you mean by good? (This is the basic question of rationality.)
That would be very hard to do in a public forum where he could read our methods of distinguish between him pretending to be good and actually being good. You could try to figure out what unconscious signals of “goodness” are, but again that’s hard through text where the person in question knows what you’re testing and can optimize the writing to score well on the test.
Or you could just go with your priors. Or message privately, though that runs into the problem of sampling people who are likely to agree with you.
in principle, you can’t.
A rational evil would pretend to be good until it didn’t matter anymore
If so he must have been using them on us in order to make his argument that such a thing is not theoretically possible so compelling. ie. Eliezer needs to be operating from outside our physical reality in order to have fully general mind-hacks and in that case the mind hacks are the least of our worries. (Unless that is what he wants us to think.)