I find Roko on-point. The psychological unity of humankind is important, but it can be over-stated. While human beings may occupy a very small area in the space of all possible minds, it is still an area and not a single point. When we shut up and multiply by sufficiently large numbers, very small differences in the starting point are very meaningful. If we are talking about a difference valued at 0.0000001% of a human life, and you extrapolate it over a billion lives, we are talking about life and death matters. Successful AI will affect more than a billion lives.
If you are the advocate of 3^^^3 specks, you must take seriously the notion that 0.0000001% differences in morality will be meaningful when multiplied over all minds into the indefinite future.
I find Roko on-point. The psychological unity of humankind is important, but it can be over-stated. While human beings may occupy a very small area in the space of all possible minds, it is still an area and not a single point. When we shut up and multiply by sufficiently large numbers, very small differences in the starting point are very meaningful. If we are talking about a difference valued at 0.0000001% of a human life, and you extrapolate it over a billion lives, we are talking about life and death matters. Successful AI will affect more than a billion lives.
If you are the advocate of 3^^^3 specks, you must take seriously the notion that 0.0000001% differences in morality will be meaningful when multiplied over all minds into the indefinite future.