Maybe part of the point of this question was, much like the injunction to write our own commandments, an exercise in determining what we would do in a God-like position- such as we might be if we were to design the first successful fAI and tell it what it should want to do.
Maybe part of the point of this question was, much like the injunction to write our own commandments, an exercise in determining what we would do in a God-like position- such as we might be if we were to design the first successful fAI and tell it what it should want to do.