use an L-infinity norm for deviations (across every moment of time as well).
The future 10^8 years later is going to look very different, even if things go right (FAI style or whatever), simply because we’ll have used the AI for something. This is going to push your L-infinity norm very high, regardless of it’s actions now, which is obviously very bad. As such, I think you want to weigh it be e^-t or something.
My other concern is that the AI will note that dedicating lots of resources to learning how to obey (game) the system will result in a really low score.
The future 10^8 years later is going to look very different, even if things go right (FAI style or whatever), simply because we’ll have used the AI for something. This is going to push your L-infinity norm very high, regardless of it’s actions now, which is obviously very bad. As such, I think you want to weigh it be e^-t or something.
My other concern is that the AI will note that dedicating lots of resources to learning how to obey (game) the system will result in a really low score.