Good summary and good points. I agree this is an advantage of truly corrigible agents over merely shutdownable agents. I’m still concerned that CAST training doesn’t get us truly corrigible agents with high probability. I think we’re better off using IPP training to get shutdownable agents with high probability, and then aiming for full alignment or true corrigibility from there (perhaps by training agents to have preferences between same-length trajectories that deliver full alignment or true corrigibility).
Good summary and good points. I agree this is an advantage of truly corrigible agents over merely shutdownable agents. I’m still concerned that CAST training doesn’t get us truly corrigible agents with high probability. I think we’re better off using IPP training to get shutdownable agents with high probability, and then aiming for full alignment or true corrigibility from there (perhaps by training agents to have preferences between same-length trajectories that deliver full alignment or true corrigibility).