I claim the bet is fair if both players expect to make the same profit on average.
I like this idea. As you say, it’s not the only way to define it but it does seem like a very reasonable way. The two players have come upon a situation which seems profitable to both of them and they simply agree to “split the profit”.
Thank you. I didn’t phrase my question very well but what I was trying to get at was whether making a friendly AGI might be, by some measurement, orders of magnitude more difficult than making a non-friendly one.