The ideas coming into your awareness are very strongly pre-filtered; creativity is far from random noise.
I agree but I don’t think that changes my conclusions. In teaching humans to be more creative, they are taught to pay more attention for a longer time to at least some of the outlier ideas. Indeed, a lot of times I think the difference between the intellectually curious and creative people I like to interact with and the rest is that the rest have predecided a lot of things, turned their thresholds for “unreal” ideas coming in to consciousness up higher than I have turned mine. Maybe they are right more often than I am, but the real measure of why they do this is that their ancestors who outsurvived a lot of other people trying a lot of other things did that same level of filtering, and it resulted in winjning more wars, having more children that survived, killing more competitors, or some combination of these and other results that constitute selection pressures.
An AI in the process of FOOMing, which necessarily has the capacity to consider a lot more ideas in a lot more detail than we do, what makes you think that AI will constrain itself by the values it used to have? Unless you think we have the same values as the first self-replicating molecules that began life on earth, the FOOMing of Natural Intelligence (which has taken billions of years) has been accompanied by value changes.
I agree but I don’t think that changes my conclusions. In teaching humans to be more creative, they are taught to pay more attention for a longer time to at least some of the outlier ideas. Indeed, a lot of times I think the difference between the intellectually curious and creative people I like to interact with and the rest is that the rest have predecided a lot of things, turned their thresholds for “unreal” ideas coming in to consciousness up higher than I have turned mine. Maybe they are right more often than I am, but the real measure of why they do this is that their ancestors who outsurvived a lot of other people trying a lot of other things did that same level of filtering, and it resulted in winjning more wars, having more children that survived, killing more competitors, or some combination of these and other results that constitute selection pressures.
An AI in the process of FOOMing, which necessarily has the capacity to consider a lot more ideas in a lot more detail than we do, what makes you think that AI will constrain itself by the values it used to have? Unless you think we have the same values as the first self-replicating molecules that began life on earth, the FOOMing of Natural Intelligence (which has taken billions of years) has been accompanied by value changes.