We know distributional shift can make an AI system perform poorly, if the new data is different from the old data. But as AI gets more advanced, quickly, we could see the same thing happen in reverse. The AI gets good, to an unfamiliar degree, at our explicit goal, resulting in problems when we look at our implicit goal. (Loosely adapted from here). (Machine learning researchers)
We know distributional shift can make an AI system perform poorly, if the new data is different from the old data. But as AI gets more advanced, quickly, we could see the same thing happen in reverse. The AI gets good, to an unfamiliar degree, at our explicit goal, resulting in problems when we look at our implicit goal. (Loosely adapted from here). (Machine learning researchers)