I assume we all agree that the system can understand the human ontology, though?
This, however likely, is not certain. A possible way for this assumption to fail is when a system allocates minimal cognitive capacity to its internal ontology and remaining power to selecting best actions; this may be a viable strategy if system’s world model is still enough descriptive but does not have extra space to represent human ontology fully.
This, however likely, is not certain. A possible way for this assumption to fail is when a system allocates minimal cognitive capacity to its internal ontology and remaining power to selecting best actions; this may be a viable strategy if system’s world model is still enough descriptive but does not have extra space to represent human ontology fully.