Well, if the PSFAI was the AI the Pebblesorters would have wanted to build, it would generally prevent murder of PS’s, because murder reduces the abilities of PS’s to sort pebbles. It would also sort pebbles into more and larger piles than ever before, because that is the core value that PS’s would want maximized. It would be able to see outside the algorithm that the PS run, and see that it was a primality-test function.
Well, if the PSFAI was the AI the Pebblesorters would have wanted to build, it would generally prevent murder of PS’s, because murder reduces the abilities of PS’s to sort pebbles. It would also sort pebbles into more and larger piles than ever before, because that is the core value that PS’s would want maximized. It would be able to see outside the algorithm that the PS run, and see that it was a primality-test function.