Yes, I agree that getting the right tests is probably hard. What you need is to achieve the point where the FAI’s utility function + the utility function that fits the test cases compresses better than the unfriendly AI’s utility function + the utility function that fits the test cases.
Yes, I agree that getting the right tests is probably hard. What you need is to achieve the point where the FAI’s utility function + the utility function that fits the test cases compresses better than the unfriendly AI’s utility function + the utility function that fits the test cases.