Ah, another point about maximising. What if the AI uses CEV of the programmers or the corporation? In other words, it’s programmed to maximise their wealth in a way they would actually want? Solving that problem is a subset of Friendliness.
Suppose we get the funding, find the people, pull together the project, solve the technical problems of AI and Friendly AI, carry out the required teaching, and finally set a superintelligence in motion, all before anyone else throws something together that only does recursive self-improvement. It is still pointless if we do not have a nice task for that optimization process to carry out.
Ah, another point about maximising. What if the AI uses CEV of the programmers or the corporation? In other words, it’s programmed to maximise their wealth in a way they would actually want? Solving that problem is a subset of Friendliness.
That’s not how the term is used here. Friendliness is prior to and separate from CEV, if I understand it correctly.
From the CEV document: