I just so happened to read Coherent Extrapolated Volition today. Insofar as this post is supposed to be about “what an FAI should do” (rather than just about your general feeling that objections to wire-heading are irrational), it seems to me that this post all really boils down to navel-gazing once you take CEV into account. Or in other words, this post isn’t really about FAI at all.
I just so happened to read Coherent Extrapolated Volition today. Insofar as this post is supposed to be about “what an FAI should do” (rather than just about your general feeling that objections to wire-heading are irrational), it seems to me that this post all really boils down to navel-gazing once you take CEV into account. Or in other words, this post isn’t really about FAI at all.