. Even as the AI safety / control / Friendliness field gets more attention and funding, it seems easy to foresee a future where mainstream AI researchers continue to ignore such work because it does not contribute to the tenths of a percent that they are seeking but instead can only hinder their efforts.
Or even worse that AI research overall takes the form of a race to the precipice. Think self-opimizing AI being employed to shave of some more percentage points.
Machine learning is already increasingly about automatic optimization. But it all depends on the task and training. Many layers of self-optimization on an image benchmark just leads to a good vision system, not AGI. Self-optimization is probably necessary for AGI, but it is not sufficient.
Or even worse that AI research overall takes the form of a race to the precipice. Think self-opimizing AI being employed to shave of some more percentage points.
Machine learning is already increasingly about automatic optimization. But it all depends on the task and training. Many layers of self-optimization on an image benchmark just leads to a good vision system, not AGI. Self-optimization is probably necessary for AGI, but it is not sufficient.