I’m really confused; I want to say that I feel as if my brain has wandered into a slightly different state of consciousness that I’m not used to. I thought this rationality stuff was cool and all, and I thought I understood it pretty well—couldn’t I speak fluently about the same things everyone else was talking about?---but suddenly over the past several days, as I’ve tried to apply the intelligence-as-optimization viewpoint to my personal life problems (q.v. the parent and my comment on lying), it starts to feel as if I’m actually starting to sort of get it. I seem to feel reluctant to report this (notice all the hedging words: “seem,” “as if,” “want to say,” &c.), because introspection is unreliable, and verbal self-reports are unreliable, and I seem to have this thing where I feel reluctant to endorse statements that could be construed to imply that I should have higher status, and there have certainly been occasions in the past where I thought I had a life-altering epiphany and I turned out to be mistaken. So maybe you shouldn’t believe me … but that’s just the thing: this entire idea of believing or not-believing natural language propositions can’t be how intelligence actually works, and maybe the reason I feel the need to use all these hedging words is because it’s becoming more salient to me that I really don’t know what’s actually going on when I think; I’m writing these words, but I no longer feel sure what it means to believe them.
I want to say that I ought to be scared about the whole AI existential risk thing, but I’m not—and, come to think of it, as a matter of cause and effect, my being scared won’t actually help except insofar as it motivates me to do something helpful. I’d really rather just not think about it at all anymore. Of course, not-thinking about a risk doesn’t make it go away, but we should make a distinction between not-thinking-about something as a way of denying reality, and not-thinking-about something as a reallocation of cognitive resources: if I spend my own thinking time on fun, safe, selfish ideas, but learn how to make some money, and use some of the money to help fund people who are better at thinking than me to work on the scary confusing world-destroying problems, isn’t that good enough? Isn’t that morally acceptable? Of course, these ideas of “enough” and “morally acceptable” don’t exist in decision theory, either, but I doubt it’s psychologically realistic to function without them, and I don’t think I actually want to.
I’m really confused; I want to say that I feel as if my brain has wandered into a slightly different state of consciousness that I’m not used to. I thought this rationality stuff was cool and all, and I thought I understood it pretty well—couldn’t I speak fluently about the same things everyone else was talking about?---but suddenly over the past several days, as I’ve tried to apply the intelligence-as-optimization viewpoint to my personal life problems (q.v. the parent and my comment on lying), it starts to feel as if I’m actually starting to sort of get it. I seem to feel reluctant to report this (notice all the hedging words: “seem,” “as if,” “want to say,” &c.), because introspection is unreliable, and verbal self-reports are unreliable, and I seem to have this thing where I feel reluctant to endorse statements that could be construed to imply that I should have higher status, and there have certainly been occasions in the past where I thought I had a life-altering epiphany and I turned out to be mistaken. So maybe you shouldn’t believe me … but that’s just the thing: this entire idea of believing or not-believing natural language propositions can’t be how intelligence actually works, and maybe the reason I feel the need to use all these hedging words is because it’s becoming more salient to me that I really don’t know what’s actually going on when I think; I’m writing these words, but I no longer feel sure what it means to believe them.
I want to say that I ought to be scared about the whole AI existential risk thing, but I’m not—and, come to think of it, as a matter of cause and effect, my being scared won’t actually help except insofar as it motivates me to do something helpful. I’d really rather just not think about it at all anymore. Of course, not-thinking about a risk doesn’t make it go away, but we should make a distinction between not-thinking-about something as a way of denying reality, and not-thinking-about something as a reallocation of cognitive resources: if I spend my own thinking time on fun, safe, selfish ideas, but learn how to make some money, and use some of the money to help fund people who are better at thinking than me to work on the scary confusing world-destroying problems, isn’t that good enough? Isn’t that morally acceptable? Of course, these ideas of “enough” and “morally acceptable” don’t exist in decision theory, either, but I doubt it’s psychologically realistic to function without them, and I don’t think I actually want to.