Yeah I’m open to improvements upon the use of the word “knowledge” because you’re right that what I’m describing here isn’t quite what either philosophers or cognitive scientists refer to as knowledge.
Yes knowledge-accumulating systems do seem to be a special case of optimizing systems. It may be that among all optimizing systems, it is precisely the ones that accumulate knowledge in the process of optimization that are of most interest to us from an alignment perspective, because knowledge-accumulating optimizing systems are (perhaps) the most powerful of all optimizing systems.
Thank you for the kind words Jemist.
Yeah I’m open to improvements upon the use of the word “knowledge” because you’re right that what I’m describing here isn’t quite what either philosophers or cognitive scientists refer to as knowledge.
Yes knowledge-accumulating systems do seem to be a special case of optimizing systems. It may be that among all optimizing systems, it is precisely the ones that accumulate knowledge in the process of optimization that are of most interest to us from an alignment perspective, because knowledge-accumulating optimizing systems are (perhaps) the most powerful of all optimizing systems.