A non-conscious intent? What on earth does this mean?
A brain can run computations optimizing for an outcome without running the additional computations needed to represent this optimization target to itself in explicit self-models available to reflective cognition.
Robin Hanson’s developed extensive, detailed sociological models that include this component. I think that the entire Overcoming Bias archives, not just Eliezer’s Sequences, ought to be canonical here, both because of their intellectual merit and because most of the Sequences were originally written on the Overcoming Bias blog in dialogue with Robin and his other co-bloggers there. A Theory of Identity seems particularly relevant here, and develops a directly relevant claim:
Our conscious minds are the public relations department of our larger minds, presenting and managing a story of ourselves to others.
A brain can run computations optimizing for an outcome without running the additional computations needed to represent this optimization target to itself in explicit self-models available to reflective cognition.
Robin Hanson’s developed extensive, detailed sociological models that include this component. I think that the entire Overcoming Bias archives, not just Eliezer’s Sequences, ought to be canonical here, both because of their intellectual merit and because most of the Sequences were originally written on the Overcoming Bias blog in dialogue with Robin and his other co-bloggers there. A Theory of Identity seems particularly relevant here, and develops a directly relevant claim: