it will always be possible to find such an experiment for any action, even desirable ones, because the AI will have defended the diamond in a way the human didn’t understand or the AI will have deduced some property of diamonds that humans thought they didn’t have
or there will be some tampering for which it’s impossible to find an experiment, because in order to avoid the above problem, you will have to restrict the space of experiments
My point is either that:
it will always be possible to find such an experiment for any action, even desirable ones, because the AI will have defended the diamond in a way the human didn’t understand or the AI will have deduced some property of diamonds that humans thought they didn’t have
or there will be some tampering for which it’s impossible to find an experiment, because in order to avoid the above problem, you will have to restrict the space of experiments