Eliezer, I still don’t think the definition of doubt you posit in your comment accounts well for its usage in your post. There are a number of topics that I have only cursory knowledge on, and positions on ideas in those topics that I judge as having only a limited probability of truth. Most of these topics are on the periphery of my awareness most of the time, and only have a very limited usefulness to me, and are not very interesting. While I could easily investigate these matters in more depth and come to a more well-considered position on many ideas, I have no motivation to. So do I have doubts about these? According to your definition, yes, but that doesn’t quite work. If the topics were to become important to me for some reason (whether because they were necessary to solve some problem I were working on or just because I became interested in them) then I would have doubts, and I would fill out my knowledge to get rid of them.
Maybe all your definition needs is a proviso that the utility of the increased certainty about a topic (which would be determined by the goal system of the intelligence, and might be technically arational) must outweigh the cost of the line of inquiry suggested by the doubt. More precisely, that the utility of resolving the doubt be higher than the utility of saving resources for other pursuits.
Eliezer, I still don’t think the definition of doubt you posit in your comment accounts well for its usage in your post. There are a number of topics that I have only cursory knowledge on, and positions on ideas in those topics that I judge as having only a limited probability of truth. Most of these topics are on the periphery of my awareness most of the time, and only have a very limited usefulness to me, and are not very interesting. While I could easily investigate these matters in more depth and come to a more well-considered position on many ideas, I have no motivation to. So do I have doubts about these? According to your definition, yes, but that doesn’t quite work. If the topics were to become important to me for some reason (whether because they were necessary to solve some problem I were working on or just because I became interested in them) then I would have doubts, and I would fill out my knowledge to get rid of them.
Maybe all your definition needs is a proviso that the utility of the increased certainty about a topic (which would be determined by the goal system of the intelligence, and might be technically arational) must outweigh the cost of the line of inquiry suggested by the doubt. More precisely, that the utility of resolving the doubt be higher than the utility of saving resources for other pursuits.